Privacy preserving machine learning labeling

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for identifying labels for a data set without disclosing the data set to any individual computing system. A method can include receiving, by a first computing system in a multi-party computation (MPC) system, a query including a first share and a second share of a given user profile. The second share is encrypted with a key that prevents the first computing system from accessing the second share. The second share is transmitted to a second computing system in the MPC system. The first computing system and the second computing system generate a machine learning model and identify respective first and second labels. The first computing system receives the second label from the second computing system as a response. The first computing system responds to the query with a response including the first label and the second label.

Description

Background Technology

[0001] This manual covers data processing and machine learning models.

[0002] Client devices can use applications (e.g., web browsers, native applications) to access content platforms (e.g., search platforms, social media platforms, or another platform hosting or aggregating content). Content platforms can then display digital components (discrete units of digital content or information, such as video clips, audio clips, multimedia clips, images, text, or another unit of content) that may be provided by one or more content sources different from the content platform, within an application launched on the client device. Summary of the Invention

[0003] Typically, an innovative aspect of the subject matter described in this specification can be embodied in a method comprising: receiving a query by a first computing system in a multi-party computation (MPC) system, comprising a first share of a given user profile and a second share of the given user profile, wherein the second share is encrypted using a key that prevents the first computing system from accessing the second share; transmitting the second share by the first computing system to a second computing system in the MPC; determining a first label by the first computing system of a first cluster having a centroid closest to the first share, wherein the first cluster is one of a plurality of clusters generated by a machine learning model trained by the first and second computing systems; receiving a response by the first computing system from the second computing system in the MPC, comprising a second label of the second cluster; and responding to the query using the response comprising the first label and the second label.

[0004] Other embodiments of this aspect include corresponding apparatus, systems, and computer programs configured to perform methods encoded on a computer storage device. These and other embodiments may each optionally include one or more of the following features.

[0005] The method may also include: receiving first plurality of partial shares of a user profile from a digital component distribution system different from the MPC system by a first computing system; receiving second plurality of partial shares of the user profile from the digital component distribution system by a second computing system, wherein neither the first plurality of shares nor the second plurality of shares are shared for individual users, wherein the first plurality of shares and the second plurality of shares are secret shares including all dimensions of the individual user's user profile; and training a machine learning model by the first computing system and the second computing system using the first plurality of partial shares and the second plurality of partial shares.

[0006] The method can include training a clustering model to create multiple clusters of a user profile based on a first plurality of partial shares and a second plurality of partial shares.

[0007] The method can include: generating centroid feature vectors for each cluster from multiple clusters by the MPC system; modeling the cluster by the MPC system using the probability distribution of user profiles in each cluster; generating new centroid feature vectors for each cluster based on the probability distribution and the centroid feature vectors of the corresponding cluster by the MPC system; and sharing the new centroid feature vectors with the digital component distribution system by the MPC computing system.

[0008] The method may include: splitting a given user profile into a first share and a second share by a client device; generating a query as a request for labels for a cluster corresponding to the given user profile and transmitting it to a first computing system; receiving a response including the first label and the second label by the client device; and storing a final device label generated based on the first label and the second label by the client device.

[0009] The method can include generating a final label, which further includes: modeling user profiles of a first cluster and a second cluster as normal distributions by a first computing system and a second computing system; determining parameters of the normal distribution, including the centroid and covariance matrix, by the first computing system and the second computing system; generating a first share and a second share of the final label by both the first computing system and the second computing system; transmitting the first share and the second share of the final label to a client device by an MPC system; and reconstructing the final label by the client device using the first share and the second share of the final label.

[0010] The method can include determining a covariance matrix by a first computing system and a second computing system, wherein determining the covariance matrix by the first computing system and the second computing system includes determining an integer matrix by the first computing system and the second computing system, such that the matrix generates a covariance matrix when multiplied by its transpose.

[0011] Specific embodiments of the subject matter described in this specification can be implemented to achieve one or more of the following advantages. The techniques described in this document enable the creation of user groups with similar interests and the expansion of user group membership while preserving user privacy, for example, without sharing users' online activities outside the browser. This limits access to sensitive user information, protects user privacy relative to such platforms, and safeguards the security of data generated by violations during transmission to or from the platform. Encryption techniques such as secure multi-party computation (MPC) enable the expansion of user groups based on similarity in user profiles without the use of third-party cookies. This protects user privacy without negatively impacting the ability to expand user groups and, in some cases, provides better user group expansion based on more complete profiles than that achievable using third-party cookies (i.e., cookies from a domain different from the domain of the resource accessed by the client device, e.g., eTLD+1). Furthermore, even when browsers (or other applications) block the use of third-party cookies, the techniques discussed herein still enable the creation of user groups, thus solving the technical problem of how to group data about accessing multiple different websites into datasets when third-party cookies cannot be used. MPC technology guarantees that user data is protected from plaintext disclosure as long as one computing system within an MPC system does not collude with other computing systems. Therefore, the techniques discussed in this paper also address the technical problem of enabling different systems to use specific datasets while preventing any individual system from accessing those datasets in plaintext (e.g., in unencrypted form). These techniques also allow for the secure identification, grouping, and transmission of user data without the need for third-party cookies to determine any relationship between user data corresponding to accesses at multiple different sites located at different eTLD+1 (the effective top-level domain plus the portion of the domain immediately preceding it). This is a different approach from and an improvement upon existing methods that require third-party cookies to determine relationships between data collected from different sites (e.g., eTLD+1). Grouping user data in this way improves the efficiency of delivering data content to user devices because irrelevant data content does not need to be transmitted. In particular, the elimination of third-party cookies avoids their storage, improving memory utilization. Exponential decay techniques can be used to build user profiles at the client device, reducing the size of the original data required to build the user profile and thus reducing data storage requirements.

[0012] Details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from the specification, drawings, and claims. Attached Figure Description

[0013] Figure 1 This is a block diagram of an example environment in which digital components are distributed.

[0014] Figure 2 This is a swimlane diagram of an example process for training a machine learning model and using the machine learning model to determine a user's user group.

[0015] Figure 3 This is a flowchart of an example process for adding differential privacy to the centroid of a cluster.

[0016] Figure 4 This is a flowchart illustrating an example process for generating a k-means model.

[0017] Figure 5 This is a flowchart illustrating an example process for processing a query by a computational multi-party computation system.

[0018] Figure 6 This is a block diagram of an example computer system.

[0019] In the various figures, the same reference numerals and names indicate the same elements. Detailed Implementation

[0020] This document discloses methods, systems, apparatus, and computer-readable media that implement techniques for using machine learning models to identify labels for datasets without disclosing the dataset to any individual computing system. For example, the techniques discussed herein provide different computing systems with access to portions of a dataset while preventing each computing system from accessing other portions of the dataset. In some implementations, multiple portions of a dataset are created by splitting the dataset into multiple parts such that each individual part represents an incomplete portion of the dataset and does not disclose anything about the dataset.

[0021] The computing system executes a cryptographic protocol to identify tags used for the entire dataset while restricting any computer system to access only the discrete portion of the dataset it is granted access to, and returns the tags securely such that only devices requesting tags for the complete dataset will have access to those tags. For example, a computing system provided with a first discrete portion of the dataset can identify a first information share of the resulting tag, encrypt the first information identifying the first secret share of the tag using a key known only to devices requesting tags for that dataset, and pass the encrypted version of the first information to a second computing system that has been provided with a second discrete portion of the dataset. The second computing system can similarly identify a second tag, encrypt the second information identifying the second tag using a key known only to devices requesting tags for the complete dataset, and, if other computing systems are processing other discrete portions of the complete dataset, pass the encrypted version of the second secret share of the tag along with the encrypted version of the first secret share of the tag to yet another computing system, or pass the encrypted information to the device requesting the tag for the complete dataset.

[0022] The device requesting the tags for the complete dataset is then able to decrypt the received information, combine all the secret shares of the tags to obtain the final tags in plaintext form for the dataset. As described above, and in more detail throughout this document, this technique solves the technical problem of how to generate tags for a complete dataset without providing access to the complete dataset, representing an improvement in data access technology and data security.

[0023] The techniques discussed in this document can be used in many data processing environments. One environment that can benefit from using these techniques is one where user data constitutes a dataset (or is included in a dataset), because these techniques prevent access to the complete user dataset while still allowing the user data to be labeled within the aggregate. For example, as described in more detail below, these techniques enable the complete user dataset to remain stored in a single trusted location (e.g., at the user's device), while allowing the user data to be processed and / or labeled by remote systems capable of running more complex algorithms (e.g., machine learning algorithms) that can be executed on the user's device (e.g., mobile phone, tablet, wearable, voice assistant, gaming device, or laptop). As described in detail below, machine learning models used to determine labels for discrete portions of data can also be trained using user data, which is also protected in a similar manner to the discrete portions of data labeled by the machine learning model.

[0024] Figure 1This is a block diagram of an example environment 100 in which digital components are distributed for presentation (e.g., along with electronic documents). Example environment 100 includes a network 102, such as a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof. Network 102 connects a content server 104, client devices 106, a digital component provider 108, and a digital component distribution system 110 (also referred to as a component distribution system (CDS)).

[0025] Client device 106 is an electronic device capable of requesting and receiving resources via network 102. Example client devices 106 include personal computers, mobile communication devices, wearable devices, personal digital assistants (PDAs), tablets, gaming devices, media streaming devices, IoT devices (e.g., thermostats, home control units, appliances, and various sensors), and other devices capable of sending and receiving data via network 102. Client device 106 typically includes a user application 107, such as a web browser, to facilitate sending and receiving data via network 102; however, native applications executed by client device 106 can also facilitate sending and receiving data via network 102. Client device 106, and specifically a PDA, may include hardware and / or software enabling voice interaction with client device 106. For example, client device 106 may include a microphone through which a user can submit audio (e.g., voice) input, such as commands, search queries, browsing instructions, smart home commands, and / or other information. Additionally, client device 106 may include a speaker through which audio (e.g., voice) output can be provided to the user. The personal digital assistant can be implemented in any client device 106, examples of which include wearable devices, smart speakers, home appliances, automobiles, tablets or other client devices 106.

[0026] An electronic document is data that presents a set of content at client device 106. Examples of electronic documents include web pages, word processing documents, portable document format (PDF) documents, audio, images, video, search results pages, and feeds. Native applications (e.g., "apps"), such as those installed on mobile devices, tablets, or desktop computing devices, are also examples of electronic documents. Electronic documents can be provided to client device 106 by content server 104. For example, content server 104 can include a server hosting a publisher's website. In this example, client device 106 can initiate a request for a given publisher's web page, and content server 104, including the web server hosting the given publisher's web page, can respond to the request by sending machine-executable instructions to present the given web page at client device 106.

[0027] In another example, content server 104 may include an app server from which client device 106 can download an app. In this example, client device 106 can download the files required to install the app on client device 106 and then execute the downloaded app locally. The downloaded app can be configured to present a combination of native content as part of the application itself and one or more digital components (e.g., content created / distributed by a third party) obtained from digital component server 108 and inserted into the app while the app is executing on client device 106.

[0028] Electronic documents can include a variety of content. For example, an electronic document can include static content (e.g., text or other specified content) that does not change over time and is inherent to the document itself. Electronic documents can also include dynamic content that can change over time or based on each request. For example, the publisher of a given electronic document can maintain a data source for populating portions of the electronic document. In this example, a given electronic document can include tags or scripts that, when processed (e.g., rendered or executed) by client device 106, cause client device 106 to request content from the data source. Client device 106 integrates the content obtained from the data source into the given electronic document to create a composite electronic document that includes the content obtained from the data source.

[0029] In some cases, a given electronic document may include a digital component tag or digital component script referencing the digital component distribution system 110. In these cases, the digital component tag or digital component script is executed by the client device 106 when the given electronic document is processed by the client device 106. The execution of the digital component tag or digital component script configures the client device 106 to generate a request for digital component 112 (referred to as a “component request”), which is transmitted to the digital component distribution system 110 via network 102. For example, the digital component tag or digital component script may enable the client device 106 to generate a packetized data request that includes header and payload data. The digital component request 112 may include event data specifying characteristics such as the name (or network location) of the server from which the requested media originates, the name (or network location) of the requesting device (e.g., client device 106), and / or information that the digital component distribution system 110 can use to select one or more digital components provided in response to the request. The component request 112 is transmitted by the client device 106 to the server of the digital component distribution system 110 via network 102 (e.g., a telecommunications network).

[0030] Digital component request 112 can include event data specifying other event characteristics, such as the characteristics of the requested electronic document and the location of the electronic document in which the digital component can be presented. For example, event data specifying a reference (e.g., a Uniform Resource Locator (URL)) to an electronic document in which the digital component will be presented, the available locations of the electronic document in which the digital component can be presented, the size of the available locations, and / or the media types eligible for presentation at these locations can be provided to digital component distribution system 110. Similarly, event data specifying keywords (“document keywords”) associated with the electronic document or entities (e.g., people, places, or things) referenced by the electronic document can also be included in component request 112 (e.g., as payload data) and provided to digital component distribution system 110 to identify digital components eligible for presentation with the electronic document. Event data can also include search queries submitted from client device 106 to obtain a search results page and / or specify search results and / or data containing text, audio, or other visual content included in the search results.

[0031] Component request 112 may also include event data related to other information, such as information already provided by the user of the client device, geographic information indicating the status or region of submitting the component request, or other information providing context for the environment in which the digital component will be displayed (e.g., the time of day of the component request, the day of the week of the component request, the type of device displaying the digital component, such as a mobile device or a tablet device). For example, component request 112 can be transmitted over a packetized network, and component request 112 itself can be formatted as packetized data with a header and payload data. The header can specify the destination of the packet, and the payload data can include any of the information discussed above.

[0032] A digital component distribution system 110, including one or more digital component distribution servers, selects a digital component to be presented along with a given electronic document in response to receiving a component request 112 and / or using information included in the component request 112. In some implementations, the digital component is selected in less than one second to avoid errors that could be caused by delayed selection of the digital component. For example, a delay in providing the digital component in response to component request 112 could cause a page load error at client device 106, or cause portions of the electronic document to remain unfilled, even after other portions of the electronic document have been presented at client device 106. Moreover, with increased delay in providing the digital component to client device 106, it is more likely that the electronic document will no longer be presented at client device 106 when the digital component is delivered, thus negatively impacting the user's experience with the electronic document and wasting system bandwidth and other resources. Furthermore, a delay in providing the digital component could cause delivery failure of the digital component, for example, if the electronic document is no longer presented at client device 106 when the digital component is provided.

[0033] To facilitate searching electronic documents, environment 100 can include a search system 150 that identifies electronic documents by crawling and indexing (e.g., indexing based on the content of the crawled electronic documents). Data about electronic documents can be indexed based on the electronic documents associated with them. Indexed and optionally cached copies of the electronic documents are stored in a search index 152 (e.g., a hardware memory device). The data associated with the electronic documents is data representing the content included in the electronic documents and / or the metadata of the electronic documents.

[0034] Client device 106 can submit a search query to search system 150 via network 102. In response, search system 150 accesses search index 152 to identify electronic documents related to the search query. Search system 150 identifies the electronic documents in the form of search results and returns the search results to client device 106 in a search results page. Search results are data generated by search system 150 that identifies electronic documents in response to a specific search query (e.g., related to it) and includes active links (e.g., hypertext links) that cause the client device to request data from a specified location in response to user interaction with the search results. Example search results may include a webpage title, an excerpt of text extracted from the webpage or a portion of an image, and the URL of the webpage. Another example search result may include the title of a downloadable application, an excerpt of text describing the downloadable application, an image depicting the user interface of the downloadable application, and / or a URL indicating from which the application can be downloaded to client device 106. Another example search result may include the title of a streaming media, an excerpt of text describing the streaming media, an image depicting the content of the streaming media, and / or a URL indicating from which the streaming media can be downloaded to client device 106. Like other electronic documents, search results pages can include one or more slots that can display digital components (such as advertisements, video clips, audio clips, images, or other digital components).

[0035] In some implementations, the digital component distribution system 110 is implemented in a distributed computing system that includes, for example, a server and multiple sets of computing devices 114 interconnected and identifying and distributing digital components in response to a component request 112. The multiple sets of computing devices 114 operate together to identify a set of digital components eligible for presentation in an electronic document from a corpus of millions of available digital components.

[0036] In some implementations, the digital component distribution system 110 implements various techniques for selecting and distributing digital components. For example, a digital component can include corresponding distribution parameters that facilitate (e.g., modulate or limit) the selection / distribution / transmission of the corresponding digital component. For example, distribution parameters can facilitate the transmission of a digital component by requiring the component to request at least one criterion that matches (e.g., precisely or at a pre-specified level of similarity) one of the distribution parameters of the digital component.

[0037] In another example, the distribution parameters for a particular digital component can include distribution keywords that must be matched (e.g., via electronic documents, document keywords, or items specified in component request 112) for the digital component to be eligible for presentation. The distribution parameters can also require component request 112 to include information specifying a particular geographic region (e.g., country or state) and / or information specifying that component request 112 originates from a particular type of client device 106 (e.g., mobile device or tablet device) in order to qualify the component item for presentation. The distribution parameters can also specify qualification values ​​(e.g., ranking, score, or some other specified value) used (e.g., among other available digital components) to evaluate the component item's eligibility for selection / distribution / transmission, as discussed in more detail below. In some cases, the eligibility value can be based on the amount submitted when a particular event is attributed to the digital component item (e.g., the presentation of the digital component).

[0038] The identification of eligible digital components can be segmented into multiple tasks 117a-117c, which are then assigned to computing devices within a plurality of computing device sets 114. For example, different computing devices in set 114 can each analyze different digital components to identify various digital components having distribution parameters that match the information included in component request 112. In some implementations, each given computing device in set 114 can analyze different data dimensions (or sets of dimensions) and pass (e.g., transmit) the results of the analysis (results 1-3) 118a-118c back to digital component distribution system 110. For example, results 118a-118c provided by each computing device in set 114 can identify a subset of digital component items eligible for distribution in response to a component request and / or a subset of digital components having certain distribution parameters. Identification of a subset of digital components can include, for example, comparing event data with distribution parameters and identifying a subset of digital components having distribution parameters that match at least some characteristics of the event data.

[0039] The digital component distribution system 110 aggregates results 118a-118c received from a plurality of computing device sets 114 and uses information associated with the aggregated results to select one or more digital components to be provided in response to component request 112. For example, the digital component distribution system 110 can select a winning set of digital components (one or more digital components) based on the results of one or more digital component evaluation processes. Furthermore, the digital component distribution system 110 can generate and transmit response data 120 (e.g., digital data representing a response) via network 102, which enables the client device 106 to integrate the winning set of digital components into a given electronic document, such that the winning set of digital components and the content of the electronic document are presented together on the display of the client device 106.

[0040] In some implementations, client device 106 executes instructions included in response data 120 that configure and enable client device 106 to obtain a set of winning digital components from one or more digital component servers 108. For example, the instructions in response data 120 may include a network location (e.g., a URL) and a script that causes client device 106 to send a server request (SR) 121 to digital component server 108 to obtain a given winning digital component from digital component server 108. In response to server request 121, digital component server 108 identifies the given winning digital component specified in server request 121 and sends digital component data 122 (DC data) of the given winning digital component in an electronic document presented at client device 106.

[0041] In some cases, receiving digital components related to web pages, application pages, or other electronic resources previously visited and / or interacted with by the user is beneficial to the user. To distribute such digital components to users, it is possible to assign users to user groups based on the digital content they access, such as user interest groups, groups of similar users, or other group types involving similar user data. For example, when a user visits a specific website and interacts with a specific item presented on the website or adds an item to a virtual shopping cart, the user can be assigned to a user group that has already visited the same website or other websites with similar context, or that is interested in the same item. To illustrate, if a user of client device 106 searches for shoes and visits multiple web pages from different shoe manufacturers, the user can be assigned to the user group "shoes," which can include identifiers of all users who have visited shoe-related websites.

[0042] In some implementations, a user's group membership can be maintained at the user's client device 106, for example, through a browser-based application rather than through a digital component provider, content platform 104, or another party. User groups can be designated by corresponding user group tags. User group tags can describe the group (e.g., a gardening group) or represent the group using a code (e.g., not a descriptive alphanumeric sequence). User group tags can be stored in secure storage at the client device 106 and / or can be encrypted during storage to prevent access to the list by others.

[0043] Digital component servers can use a user's user group membership to select digital components or other content that the user may be interested in or that may otherwise benefit the user / user device (e.g., help the user complete a task). For example, such digital components or other content may include data that improves user experience, enhances the operation of the user device, or otherwise benefits the user or user device. However, user privacy can be protected when using user group membership data to select digital components by preventing the content server from associating user group identifiers with specific users by providing user group labels. This document uses user group membership data and user data as examples of data that should be protected from unauthorized access by unauthorized parties (or computing systems), but the techniques discussed in this document are not limited to such applications and can be used with respect to any dataset that needs to be protected from unauthorized access.

[0044] Application 107 is able to provide user group labels to a trusted computing system that interacts with a digital component server in order to prevent the content platform or any other entity other than the user from knowing the user's full user group membership in order to select the digital component to be presented at the client device 107 based on the user group membership.

[0045] In some implementations, a user is assigned to only one user group at a time, and this assignment is temporary, as a user's group membership can change relative to their browsing activity. For example, when a user begins a web browsing session and visits a specific website and interacts with a specific item presented on the website or adds an item to a virtual shopping cart, the user can be assigned to a user group that has already visited the same website or other websites with similar context, or that is interested in the same item.

[0046] However, if a user visits another website and interacts with a different type of item presented on that website, the user is assigned to another user group that has already visited that other website, other websites with similar context, or is interested in that other item. For example, if a user starts a browsing session by searching for shoes and visiting multiple web pages from different shoe manufacturers, the user can be assigned to the user group "Shoes," which includes all users who have visited shoe-related websites.

[0047] Suppose there are 100 users who have previously visited shoe-related websites. When a user is assigned to the user group "Shoes," the total number of users in that group increases to 101. However, after a user searches for hotels and visits multiple pages from different hotels or travel agencies, the user can be removed from the previously assigned user group "Shoes" and reassigned to a different user group, "Hotels" or "Travel." In this case, assuming no other users are added to or removed from that specific user group, the number of users in the "Shoes" group decreases back to 100.

[0048] The number and type of user groups are managed and / or controlled by the system (or administrator). For example, the system may implement algorithms and / or machine learning methods to oversee the management of user groups. Typically, the number of user groups and the number of users in each user group change over time because the traffic of users participating in active browser sessions changes over time, and because each individual user is responsible for their respective browsing activities.

[0049] In some implementations, component distribution system 110 includes a multi-party computation (MPC) system 130 that implements a machine learning model to supervise the management of user groups. MPC system 130 is capable of training a machine learning model that suggests user groups to users (or their applications 107) based on user profiles, or can be used to generate user group suggestions to users (or their applications 107) based on user profiles. MPC system 130 includes two computation systems, MPC1 132 and MPC2 134, that perform security and privacy protection techniques to train the machine learning model. Although the example MPC system 130 includes two computation systems, more computation systems can be used whenever MPC system 130 includes more than one.

[0050] Computing systems MPC1 132 and MPC2 134 can be operated by different entities, which prevents each entity from accessing the complete user profile in plaintext when implementing the techniques described in this document. Plaintext is text that is not computationally marked, specially formatted, or written in code or data (including binary files) in a form that can be viewed or used without keys or other decryption devices or other decryption processes. For example, one of the computing systems MPC1 132 or MPC2 134 can be operated by a trusted party different from the user's client device 106, content platform 104, and digital component server 108. For example, an industry group, a government group, or a browser developer can maintain and operate one of the computing systems MPC1 132 and MPC2 134. Other computing systems can be operated by different groups within these groups, such that different trusted parties operate each computing system MPC1 132 and MPC2 134. Preferably, the different parties operating the different computing systems MPC1 132 and MPC2 134 have no incentive to collude to compromise user privacy. In some implementations, computing systems MPC1 132 and MPC2 134 are architecturally separate and are monitored to prevent them from communicating with each other outside of performing the secure MPC process described in this document.

[0051] In some implementations, a user profile can be in the form of a feature vector. For example, a user profile can be an n-dimensional feature vector. Each of the n dimensions can correspond to a specific feature, and the value of each dimension can be a feature value of the user. For example, one dimension could be whether a particular numeric component has been presented to the user (or interacted with by the user). In this example, if the numeric component has been presented to the user (or interacted with by the user), the value of that feature could be "1", or if the numeric component has not been presented to the user (or interacted with by the user), the value of that feature could be "0".

[0052] A user's profile may include data related to events initiated by the user and / or events the user may have initiated regarding electronic resources (e.g., web pages or application content). Events may include views of electronic resources, views of digital components, user interactions with electronic resources or digital components (e.g., selection of electronic resources or digital components), or the absence of user interactions with electronic resources or digital components (e.g., selection of electronic resources or digital components), transitions that occur after a user interacts with an electronic resource, and / or other appropriate events related to the user and the electronic resource.

[0053] In some implementations, application 107 can generate different user profiles based on requests from the content server and for different machine learning models owned by the content server. Depending on the design goals, different machine learning models may require different training data. For example, the first model could be a k-NN model used to determine whether to add a user to a user group.

[0054] When an event occurs, the content server can provide event-related data to application 107 running on the client device to generate a user profile. In some implementations, to protect the event data during transmission, the content server encrypts the event data before sending it to application 107. For example, the content server can use application 107's public encryption key (e.g., PubKeyEnc(event_data, application_public_key)) to encrypt the event data.

[0055] In some implementations, event data may include the following items as shown in Table 1 below.

[0056]

[0057] Table 1

[0058] Referring to Table 1, the model identifier identifies the machine learning model, such as the k-NN model, that the user profile will be used to train and predict user group membership, and to generate corresponding labels for the predicted user groups. The profile record is an n-dimensional feature vector that includes event-specific data, such as the type of event, the electronic resource or digital component, the temporal context of the electronic resource or digital component when the event occurred, and / or other appropriate event data that the content server wants to use in training the machine learning model and making user group inferences.

[0059] Upon receiving the event data, application 107 can decrypt the event data using its private key, which corresponds to the public encryption key used to encrypt the event data. Application 107 can verify the event data by (i) using the public verification key of the content server corresponding to the private key used to generate the digital signature, and (ii) ensuring that the event data creation timestamp is not stale, for example, that the time indicated by the timestamp is within a threshold amount of time that the verification is currently taking place. If the event data is valid, application 107 can store the event data, for example, by storing an n-dimensional profile record. If any verification fails, application 107 can ignore the event data, for example, by not storing the n-dimensional profile record.

[0060] In some implementations, application 107 can compute a user profile by aggregating n-dimensional feature vectors (i.e., profile records). For example, a user profile could be the average of n-dimensional feature vectors of multiple events associated with a user. The result is an n-dimensional feature vector representing the user in the profile space. Alternatively, application 112 can, for example, use L2 normalization to normalize the n-dimensional feature vectors to a unit length.

[0061] In some implementations, application 107 can use the following equation to calculate the user profile (P):

[0062]

[0063] Where parameter F i It includes k feature vectors, and each vector has n-dimensional features representing an event (e.g., user interaction with content or another event attributable to the user), record_age_in_seconds i It is the amount of time, in seconds, that the profile record has been stored on the client device, and the parameter decay_rate_in_seconds is the decay rate of the profile record, in seconds.

[0064] In some implementations, application 107 can update the user profile (P) as an event occurs and when an event occurs. In this case, the application can use the following equation to update the user profile:

[0065]

[0066] Where P′ is the updated user profile and F is the n-dimensional feature vector of the new event, while P is the n-dimensional feature vector of the existing user profile generated by user_profile_time.

[0067] Figure 2 This is a swimlane diagram of an example process 200 used to train a k-means machine learning model to predict user groups. The operation of process 200 can be implemented, for example, by client device 110, computing systems MPC1 132 and MPC2 134 of MPC system 130, and a content provider. The operation of process 200 can also be implemented as instructions stored on one or more computer-readable media that may be non-transitory, and execution of the instructions by one or more data processing devices can cause one or more data processing devices to perform the operation of process 200.

[0068] The content server can initiate the training and / or updating of one of its machine learning models by requesting the application 107 running on client device 106 to generate user profiles for their respective users and upload a secret share and / or encrypted version of the user profile to MPC system 130. For the purposes of this document, the secret share of the user profile can be considered an encrypted version of the user profile, as the secret share is not plaintext. Typically, each application 107 is able to store the user profile data and generate updated user profiles in response to requests received from the content platform.

[0069] Application 107 running on client device 106 constructs a user profile for the user of client device 106 (202). The user profile may include data related to events initiated by the user and / or events that the user may have initiated regarding electronic resources (e.g., web pages or application content). Events may include the context of the electronic resource, a view of the electronic resource, a view of a digital component, user interaction with the electronic resource or digital component (e.g., selection of the electronic resource or digital component), or the absence of user interaction with the electronic resource or digital component (e.g., selection of the electronic resource or digital component), transitions that occur after the user interacts with the electronic resource, and / or other appropriate events related to the user and the electronic resource.

[0070] A user's profile can be in the form of a feature vector. For example, a user profile can be an n-dimensional feature vector. Each of the n dimensions can correspond to a specific feature, and the value of each dimension can be a feature value of the user. For example, one dimension could be whether a particular digital component has been presented to the user (or interacted with). In this example, if the digital component has been presented to the user (or interacted with), the value of that feature could be "1", or if the digital component has not been presented to the user (or interacted with), the value of that feature could be "0".

[0071] Application 107 generates shares (204) of a user profile for the user. In this example, application 107 generates two shares of the user profile, each share used by one computing system of MPC system 130. Note that each share can be a pseudo-random variable itself, which does not disclose any information about the user profile. Both shares will need to be combined to obtain the user profile. If MPC system 130 includes more computing systems involved in training the machine learning model, application 107 will generate more shares, each used by one computing system. In some implementations, to protect user privacy, application 107 can use a pseudo-random function to split the user profile into shares. That is, application 107 can use a pseudo-random function to generate two shares {[P i,1 ], [P i,2The precise splitting can depend on the secret-sharing algorithm and cryptographic library used by application 107.

[0072] The share of the 107 encrypted user profile [P] i,1 ] and [P i,2 (206). In some implementations, application 107 uses the public encryption key of computing system MPC1 132 to encrypt the first share [P]. i,1 Similarly, application 107 uses the public encryption key of computing system MPC2 134 to encrypt the second share of the user profile message [P]. i,2 These functions can be represented as PubKeyEncrypt([P i,1 ], MPC1) and PubKeyEncrypt([P i,2 [MPC1 132, MPC2 134], where PubKeyEncrypt represents a public-key encryption algorithm using the corresponding public encryption key of MPC1 132 or MPC2 134. In some implementations, the second share is encrypted using a key that prevents MPC1 132 from accessing the second share, thereby protecting the data included in the second share from being disclosed in plaintext by MPC1 132. This enhances the security of the second share by preventing MPC1 132 from being able to recreate the complete dataset representing the complete user profile.

[0073] Application 107, running on client device 106, uploads an encrypted share of the user profile to computing system MPC1 (208). For example, application 107 uploads the first share of the user profile (e.g., PubKeyEncrypt([P...)...)...)... i,1 ], MPC1)) and the second share (e.g., PubKeyEncrypt([P i,2 The first share of the user profile is uploaded to MPC1 by MPC2. MPC1 uses its private key to decrypt the first share of the user profile and transmits the second secret share of the user profile to MPC2 (210). MPC2 uses its private key to decrypt the second share of the user profile.

[0074] In some implementations, application 107 must simultaneously upload multiple shares of a user profile to the corresponding MPC system so that the computing system can correctly match all shares of the same user profile. In some implementations, application 107 can explicitly assign the same pseudo-random or sequentially generated identifier to multiple shares of the same user profile to facilitate matching. While some MPC techniques can rely on random perturbations of inputs or intermediate results, the MPC techniques described in this document may not include such random perturbations and may instead rely on the upload order for matching.

[0075] In some implementations, operations 208 and 210 can be replaced by an alternative process in which application 107 uploads multiple shares of a user profile to a content server, and the content server uploads the multiple shares to the MPC system. This alternative process increases the infrastructure cost for the content server to support operations 208 and 210. It also increases the latency of starting to train or update machine learning models in the MPC system. However, this alternative process allows the content server to store and manage user data without disclosing any user details to the content server, thereby maintaining user privacy.

[0076] In some implementations, the content server is capable of collecting shares of multiple distinct user profiles (or other datasets), and each share can be individually encrypted as described above (e.g., in a way that a secret share intended for a particular MPC server can only be accessed by that particular MPC server). Using a content server as an aggregator of shares of multiple distinct user profiles makes it possible to collect and upload many different encrypted user profiles that can be used to train one or more machine learning models. While training of the machine learning model typically occurs before a request for tags, the machine learning model can continue to be updated using newly collected data even after it has been used to generate tags. The following paragraphs discuss the training of a model used to generate tags using the encrypted shares of the aforementioned user profiles.

[0077] Computational systems MPC1 132 and MPC2 134 generate a machine learning model (212). In some implementations, the machine learning model implemented by the MPC1 132 and MPC2 134 systems within MPC system 130 is a k-means model. Typically, the k-means algorithm is an algorithm that attempts to divide a dataset into k distinct non-overlapping groups (clusters), where each data point belongs to only one group (cluster). Computational systems MPC1 132 and MPC2 134 are capable of training the k-means model using MPC techniques based on an encrypted portion of a user profile received from application 107.

[0078] To minimize or at least reduce the computational burden on the computing systems MPC1 132 and MPC2 134, and thus protect user privacy and data during model training and inference, MPC system 130 is able to use random projection techniques, such as SimHash, to quickly, securely, and probabilistically quantize two user profiles P. i and P j The similarity between the two user profiles. i and P j The similarity between them can be determined by identifying the user profiles P of the two users. i and P jThe Hamming distance between two bit vectors is used to determine the similarity, which is proportional to the cosine similarity between the two user profiles with a high probability.

[0079] Conceptually, for each training session, m random projected hyperplanes U = {U1, U2, ..., U} can be generated. m The random projection hyperplane can also be called the random projection plane. One purpose of the multi-step computation between the computation systems MPC1132 and MPC2134 is to calculate the value of each user profile P used in training the k-means model. i Create a bit vector B of length m. i In the bit vector B i In the context of ⊙, which represents the dot product of two vectors of equal length, each bit B... i,j Represents the projection plane U j One of the user profiles P i The sign of the dot product, i.e., B. i,j =sign(U j ⊙P i In other words, each bit represents the user profile P. i Located in plane U j Which side? A bit value of one indicates a positive sign, while a bit value of zero indicates a negative sign.

[0080] At the end of the multi-step computation, each of the two computation systems, MPC1 132 and MPC2 134, generates an intermediate result comprising a bit vector of each user profile in plaintext and a share of each user profile. For example, the intermediate result of computation system MPC1 132 could be the data shown in Table 2 below. Computation system MPC2 134 will have a similar intermediate result, but with a different share of each user profile. To add additional privacy protection, each of the two servers in MPC system 130 is only able to obtain half of the m-dimensional bit vector in plaintext; for example, computation system MPC1 132 obtains the first m / 2 dimension of all m-dimensional bit vectors, and computation system MPC2 134 obtains the second m / 2 dimension of all m-dimensional bit vectors.

[0081] Bit vector in plaintext form <![CDATA[For the MPC1 132 share of P i > … … <![CDATA[B i ]]> … <![CDATA[B i+1 ]]> … … …

[0082] Table 2

[0083] Given two arbitrary user profile vectors P of unit length i ≠ j i and P j As has been shown, assuming the number of random projections m is sufficiently large, then the two user profile vectors P i and P j Bit vector B i and B jThe Hamming distance between them has a high probability of being related to the user profile vector P. i and P j The cosine similarity between them is proportional.

[0084] Based on the above intermediate results, and because of the bit vector B i It is in plaintext form, so each computing system MPC1132 and MPC2 134 can be created independently, for example, by training the corresponding k-means model using the k-means algorithm.

[0085] In some implementations, the number of clusters k in the k-means model is chosen according to the following equation:

[0086] k = z * 2 -x

[0087] Where z is the number of applications (10⁷), and x is the number of entropy bits. For example, suppose a total of 256 applications must be grouped (or clustered) together such that each group includes the same number of applications, and the number of entropy bits (x) is 5. In such a scenario, the k-means model generates k = 8 clusters, where each cluster includes 32 applications. (Reference) Figure 4 The illustration shows an example process for training a k-means model.

[0088] In some implementations, after clustering user profiles using a k-means model, each cluster is assigned a unique identifier (called a label). For example, if there are 10 clusters, they can be labeled using numbers 1 to 10. In another implementation, labels can be assigned to clusters generated by the k-means model based on the previous labels of the majority of user profiles in the respective cluster. For example, suppose a cluster contains 32 user profiles. Also suppose that 20 of the 32 user profiles have the same previous label “id_x”. In such a scenario, the cluster is assigned the label “id_x”. However, it should be noted that to implement this labeling technique, application 107 must upload the corresponding previous label along with the user profile share to the corresponding MPC system 130. In this case, each encrypted share of the user profile includes the user profile share and the previous label.

[0089] Application 107 transmits a query for user group tags to MPC system 130 (214). In this example, application 107 transmits the query for user group tags to computing system MPC1 132, the query including a first encrypted share and a second encrypted share of the user profile. In other examples, application 107 may transmit the query for user group tags to computing system MPC2. Application 107 may submit a query for user group tags in response to a request from a content server to provide the tags of the user group to which application 107 is assigned. For example, the content server may request application 107 to query a k-means model to determine the user group tags of application 107 on client device 110.

[0090] In order to initiate a query for user group tags, the content server can send a token M to application 107 for querying user group tags. infer Token M infer This enables the server in MPC system 130 to verify that application 107 is authorized to query the k-means model of the content server implemented by MPC system 130. If model access control is optional, then token M... infer It is optional.

[0091] In some implementations, token M infer It can include digital signatures of token-based content and token creation time using the content server's private key.

[0092] To query user group tags for a specific user, the content server can generate a token M for querying user group tags. infer The content server then sends the token to application 107 running on the user's client device 106. In some implementations, the content server uses the public encryption key of application 107 to encrypt the token M. infer This allows only the application 107 to decrypt token M using its secret private key corresponding to the public encryption key. infer In other words, the content platform can send PubKeyEnc(M) to application 107. infer ,application_public_key).

[0093] Application 107 is able to decrypt and verify token M infer Application 107 is able to use its private key to decrypt the encrypted token M. infer Application 107 is able to verify the digital signature by (i) using the public cryptographic key of the content server corresponding to the private key of the content server used to generate the digital signature, and (ii) ensuring that the token creation timestamp is not stale, for example, that the time indicated by the timestamp is within a threshold amount of time for the verification to take place. inferIf token M infer If valid, application 107 can query MPC system 130.

[0094] Conceptually, a query for user group tags can include a model identifier used to identify a specific machine learning model from among multiple machine learning models that can be implemented to predict user groups and corresponding tags. The query can also include the current user profile P. i However, to prevent the plaintext user profile P... i The application 107 can disclose user profiles to computing systems MPC1 132 or MPC2 134, thereby protecting user privacy. i Split into two shares for MPC1 132 and MPC2 respectively [P i,1 ] and [P i,2 Application 107 can then select, for example, randomly or pseudo-randomly, one of two computing systems, MPC1 132 or MPC2 134, for querying. If application 107 selects computing system MPC1, then application 107 can send a message with a first share [P] to computing system MPC1 132. i,1 The second share of the encrypted version (e.g., PubKeyEncrypt([P i,2 A single query of [MPC2]. In this example, application 107 uses the public encryption key of computing system MPC2 134 to encrypt the second share [P]. i,2 To prevent the computing system MPC1 132 from accessing [P] i,2 This will enable the computing system MPC1 132 to [P] i,1 ] and [P i,2 To reconstruct the user profile P i .

[0095] MPC system 130 determines a tag (216) for a user profile. In some implementations, each computing system within MPC system 130 determines a corresponding tag (or partial tag) based on a share of the user profile received from application 107. Each computing system within MPC system 130 performs an operation similar to that mentioned in step 212 after receiving the corresponding share of the user profile and converts the corresponding share into a bit vector. After converting the share into a bit vector, each computing system in MPC system 130 determines a cluster with the centroid closest to the corresponding bit vector. For example, as mentioned above, application 107 can, for example, randomly or pseudo-randomly select one of two computing systems, MPC1 132 or MPC2, for a query. If application 107 selects computing system MPC1, application 107 can send a first share [P] to computing system MPC1 132. i,1The second share of the encrypted version (e.g., PubKeyEncrypt([P i,2 A single query of MPC2). The second share of the user profile [P i,2 The encrypted version of [P] is transmitted to computing system MPC2 134. MPC2 uses the private key of computing system MPC2 to decrypt the second share [P]. i,2 MPC1 132 and MPC2 134 perform the cryptographic operations described in step 212 to determine the first share [P] secretly held by MPC1. i,1 ] and the second share [P] secretly held by MPC2 i,2 The first bit vector is generated. The MPC1 132 determines the cluster with the centroid closest to the first bit vector (referred to as the first cluster). An identifier for this cluster can be selected as the label (or partial label) for the first share.

[0096] The computing system MPC1 132 uses parameters x to N. n The users of the first cluster are modeled using an n-dimensional normal distribution of (μ1, ∑1), where μ1 is the n-dimensional centroid of the first cluster and ∑1 is the covariance matrix of dimension n x n. In this example, MPC1 132 calculates μ1 = ∑ i∈ID [P i，1 Similarly, MPC2 134 calculates μ2 = ∑ i∈ID [P i，2 ], where [μ1] and [μ2] are secret shares of kx μ, and k is a known number in plaintext form.

[0097] To calculate the covariance matrix ∑, MPC1 132 calculates [∑1] = ∑ i∈ID (k*[P i，i ]-μ1) T *(k*[P i，1 ]-μ1), where k*[P i，1 ]-μ1 is a matrix with 1 x n secret shares, and (k*[P i，1 ]-μ1) T It is a matrix k*[P] with dimension n x 1. i，1 The transpose of ]-μ1. This results in the covariance matrix having n x n dimensions. Similarly, MPC2 134 is able to compute [∑2]=∑ i∈ID (k*[P i，2 ]-μ2) T *(k*[P i，2 ]-μ2).

[0098] MPC1 132 and MPC2 134 are able to use two secret shares [∑1] and [∑2] and follow the equation To construct the covariance matrix ∑,

[0099] The function `reconstruct()` generates a secret in plaintext based on two secret shares. Both MPC1 132 and MPC2 134 can compute an integer matrix A such that A*A through Cholesky decomposition. T =∑. After calculating matrix A, matrix A is shared with other computing systems of MPC system 130.

[0100] In this example, when using parameters x to N n After modeling the users of the first cluster using an n-dimensional normal distribution of (μ,∑), MPC1 132 can use the Box-Muller transform to generate random vectors z = (z1,...,z) randomly drawn from a standard normal distribution. n ) T MPC1 132 then splits z into two shares, [z1] and [z2]. MPC1 132 then shares [z2] with MPC2. Similarly, MPC2 134 is able to use the Box-Muller transform to generate a random vector z′ = (z′1,...,z′) randomly drawn from a standard normal distribution. n ) T MPC2 134 then splits z into two shares, [z′1] and [z′2]. MPC2 134 then shares [z′1] with MPC1. MPC1 132 then calculates the first label. Similarly, MPC2 134 then calculates the second label.

[0101] The MPC system transmits user group tags to application 107 (218). Computation system MPC2 134 is able to provide computation system MPC1 with an encrypted version of the second tag (i.e., [result2]), wherein the second tag is encrypted using the public encryption key of application 107. Computation system MPC1 132 is able to provide application 107 with a first tag for the result cluster, i.e., [result1], and an encrypted version of the second tag for the second cluster determined by computation system MPC2. Application 107 is able to decrypt the second tag for the result cluster determined by computation system MPC2 in conjunction with MPC1. In some embodiments, to prevent computation system MPC1 132 from forging the results of computation system MPC2, computation system MPC2 134 digitally signs its results before or after encrypting them using the public encryption key of application 107. Application 107 uses the public encryption key of MPC2 to verify the digital signature of computation system MPC2.

[0102] Application 107 updates and stores the user group label (220). After receiving the first and second labels from MPC system 130, the application is able to calculate the final label as... The tag is then stored on the client device 106. In this implementation, the FLoC ID is an n-dimensional vector randomly generated for each user group to which the user is a member.

[0103] As mentioned earlier, user group tags are merely identifiers of the user group a user belongs to, and lack the contextual meaning that a content platform, such as a digital component provider, can utilize to select digital components for application 107. As a solution, MPC system 130 can share certain information, such as the centroids of clusters from a k-means machine learning model, with the digital component provider. The centroids of clusters from the k-means machine learning model implemented by MPC system 130 have information related to the user profile (P... i The same dimensions. Sharing the centroid with the content platform will allow the platform to provide digital components based on previous events that occurred due to user activity.

[0104] For example, suppose application 107 updates an n-dimensional user profile based on events that occur as a result of user actions. The MPC 130 system determines the cluster to which application 107 belongs based on the distance between the n-dimensional feature vector user profile provided by application 107 and the n-dimensional feature vector of the centroid of a cluster from a k-means model. After receiving a user's tag, application 107 stores that tag in client device 106. Assume that MPC system 130 has already shared the n-dimensional centroid feature vector with a digital component provider. When an application loads a resource that includes one or more digital component slots, the client device 106 providing the resource or the content server generates a request for a digital component that includes the tag from application 107. Upon receiving a request for a digital component, the digital component provider can provide the digital component based on the n-dimensional centroid of the cluster to which the application belongs.

[0105] However, sharing the centroids of clusters with content platforms raises privacy concerns. To overcome this, MPC system 130 utilizes differential privacy technology and generates new centroids for clusters by adding random noise to the centroids. See details for further information. Figure 3 Further explanation.

[0106] Figure 3This is a flowchart of an example process 300 for generating new centroids for clustering user profiles using differential privacy techniques. The operation of process 300 can be implemented, for example, by computing systems MPC1 132 and MPC2 134 of MPC system 130. The operation of process 300 can also be implemented as instructions stored on one or more computer-readable media that may be non-transitory, and execution of the instructions by one or more data processing devices can cause one or more data processing devices to perform the operation of process 300.

[0107] MPC system 130 generates centroid feature vectors for each cluster (302). For example, the computational system of MPC system 130 uses the k-means clustering algorithm to cluster user profiles into k clusters. In this particular example, MPC1 132 and MPC2 134 train k-means machine learning models as described in step 212 of process 200. Training the k-means model requires computing the centroids of the clusters. Since user profiles are n-dimensional feature vectors, the k-means clustering algorithm forms clusters in an n-dimensional feature space and generates an n-dimensional centroid for each cluster.

[0108] MPC system 130 uses the probability distribution of user profiles in the clusters to model each cluster (304). For example, the computational system of MPC system 130 uses step 218 of process 200 to model the users of each cluster of the k-means machine learning model as a normal distribution.

[0109] MPC system 130 generates a new centroid feature vector (306) for each cluster. For example, the computational system of MPC system 130 can generate random feature vectors for each of the multiple clusters in the k-means machine learning model by randomly sampling from a standard normal distribution using a Box-Muller transformation. In this example, each computational system MPC1 132 and MPC2 134 generates corresponding random feature vectors for the centroids of the clusters in the k-means machine learning model. In some implementations, to provide stronger privacy protection, computational systems MPC1 132 and MPC2 134 perform a cryptographic protocol to collaboratively generate corresponding random feature vectors for the centroids of the clusters in the k-means machine learning model, and the generated centroids are in the form of secret shares.

[0110] The MPC system 130 shares a new centroid feature vector (308) with the digital component provider. For example, instead of sharing the actual centroid of the clusters of the k-means machine learning model, the MPC system 130 shares a random feature vector with the digital component provider.

[0111] Figure 4This is a flowchart illustrating an example process 400 for generating a k-means machine learning model. The operation of process 400 can be performed, for example, by... Figure 1 The MPC system 130 is implemented. The operation of process 400 can also be implemented as instructions stored on one or more computer-readable media that may be non-transitory, and instructions to the instructions by one or more data processing devices can cause one or more data processing devices to execute the operation of process 400.

[0112] MPC system 130 obtains a share of the user profile (402). The content server can request application 107 to update and / or obtain the tags of the user group to which the application belongs. In response to the request, application 107 can upload a share of the user profile to MPC system 130 to train a k-means machine learning model.

[0113] For example, application 107 can transmit user profile P to computing system MPC1. i The first encrypted share of the user profile (e.g., PubKeyEncrypt([P i,1 Similarly, application 107 can transmit user profile P to computing system MPC2. i The encrypted second share of the user profile (e.g., PubKeyEncrypt([P i,2 ],MPC2)).

[0114] Computational systems MPC1 132 and MPC2 134 create random projection planes (404). Computational systems MPC1 132 and MPC2 134 are able to collaboratively create m random projection planes U = {U1, U2…U3}. m These random projection planes should be kept as a secret share between the two computing systems MPC1 132 and MPC2. In some implementations, computing systems MPC1 132 and MPC2 134 create random projection planes and use Diffie-Hellman key exchange techniques to maintain their secrecy.

[0115] As described in more detail below, computation systems MPC1 132 and MPC2 134 project the share of each user profile onto each random projection plane, and for each random projection plane, determine whether the share of the user profile lies on one side of the random projection plane. Then, each computation system MPC1 132 and MPC2 134 is able to construct a bit vector in the secret share from the secret share of the user profile based on the result of each random projection. Partial knowledge of the user's bit vector, for example, user profile P... i Is it on the projection plane U? kOn one side, the computing system MPC1 132 or MPC2 134 is allowed to obtain information about P. i Some knowledge about the distribution of P, relative to the user profile P i The existing knowledge of unit length is incremental. To prevent computing systems MPC1 132 and MPC2 134 from gaining access to this information (e.g., in embodiments where this is necessary or preferred for user privacy and / or data security), in some embodiments, the random projection plane is in a secret share, so neither computing systems MPC1 132 nor MPC2 134 can access the random projection plane in plaintext. In other embodiments, a secret sharing algorithm can be used to apply a random bit-flipping pattern to the random projection result, as described in optional operations 406-408.

[0116] To illustrate how bits can be flipped via secret shares, assume there are two secrets, x and y, whose values ​​are either zero or one with equal probability. If y == 0, the equation [x] == [y] will flip the bits of x, and if y == 1, the equation [x] == [y] will leave the bits of x intact. This operation requires a remote procedure call (RPC) between two computing systems, MPC1 132 and MPC2 134, and the number of rounds depends on the data size and the secret-sharing algorithm chosen.

[0117] Each computing system MPC1 132 and MPC2 134 creates a secret m-dimensional vector (406). The computing system MPC1 132 is capable of creating a secret m-dimensional vector {S1, S2…S}. m}, where each element S i Each has a value of zero or one with equal probability. The computational system MPC1 132 splits its m-dimensional vector into two shares, the first share {[S 1,1 ], [S 2,1 ], ...[S m,1 ]} and the second share {[S 1,2 ], [S 2,2 ], ...[S m,2 The computational system MPC1 132 is able to keep the first share secret and provide the second share to the computational system MPC2. The computational system MPC1 132 can then discard the m-dimensional vector {S1, S2…S}. m}

[0118] The computing system MPC2 134 is capable of creating a secret m-dimensional vector {T1, T2…T}. m}, where each element Ti has a value of zero or one. The computational system MPC2 134 splits its m-dimensional vector into two shares, the first share {[T 1,1 ],[T 2,1 ],…[Tm,1 ]} and the second share {[T 1,2 ],[T 2,2 ],…[T m,2 The computational system MPC2 can keep the first share secret and provide the second share to the computational system MPC1. The computational system MPC2 can then discard the m-dimensional vector {T1, T2…T}. m}

[0119] Two computing systems, MPC1 132 and MPC2 134, use secure MPC technology to calculate the share of the bit-flipping pattern (408). Computing systems MPC1 132 and MPC2 134 can use a secure share MPC equality test along with multiple round trips between the computing systems MPC1 132 and MPC2 134 to calculate the share of the bit-flipping pattern. The bit-flipping pattern can be based on the above operation [x] == [y]. That is, the bit-flipping pattern can be {S1 == T1, S2 == T2…S…} m ==T m Let each ST... i =(S i ==T i Each ST i It has a value of zero or one. After the MPC operation is completed, the computing system MPC1 132 has the first share of the bit-flipping mode {[ST 1,1 [ST] 2,1 ],…[ST m,1 ]}, and compute the second share {[ST] of system MPC2 134 with bit-flipping mode. 1,2 [ST] 2,2 ],…[ST m,2 Each ST i The share allows the two computing systems, MPC1132 and MPC2134, to flip bits in the bit vector in a manner that is opaque to either of the two computing systems, MPC1132 and MPC2134.

[0120] Each computing system MPC1 132 and MPC2 134 projects the share of each user profile onto each random projection plane (410). That is, for each user profile whose share is received by computing system MPC1 132, computing system MPC1 132 is able to project the share [P] onto each random projection plane. i,1 Projected onto each projection plane U j Above. For each share of the user profile and for each random projection plane U jPerforming this operation yields a z×m matrix R, where z is the number of available user profiles and m is the number of random projection planes. Each element R in matrix R... i,j It is possible to calculate the projection plane U j and share [P i,1 The dot product between ] is used to determine, for example, R i,j =sign(U j P i,1 The operation ⊙ represents the dot product of two vectors of equal length.

[0121] If bit flipping is used, the computing system MPC1 132 can use a bit flipping pattern secretly shared between the computing systems MPC1 132 and MPC2 134 to modify one or more elements R in the matrix. i,j The value of . For each element R in matrix R. i,j The computing system MPC1 132 can calculate [ST] j,1 ]==sign(R i,j ) as element R i,j The value of R. Therefore, element R i,j The symbol will be in its bit flip mode [ST] j,1 The corresponding bit in the [] is flipped if it has a zero value. This calculation can be performed using multiple RPCs of the MPC2 134 computing system.

[0122] Similarly, for each user profile received by the computing system MPC2 134, the computing system MPC2 134 is able to [P] i,2 Projected onto each projection plane U j Above. For each share of the user profile and for each random projection plane U j Performing this operation produces a z×m dimensional matrix R', where z is the number of available user profiles and m is the number of random projection planes. Each element R in matrix R' is... i,j 'Able to calculate the projection plane U j and share [P i,2 The dot product between ] is used to determine, for example, R i，j '=U j ⊙[P i，2 The operation ⊙ represents the dot product of two vectors of equal length.

[0123] If bit flipping is used, the computing system MPC2 134 can use a bit flipping pattern secretly shared between the computing systems MPC1 132 and MPC2 134 to modify one or more elements R in the matrix. i,j The value of . For each element R in matrix R. i,jThe computing system MPC2 134 is capable of calculating [ST] j,2 ]==sign(R i,j ') as element R i,j The value of '. Therefore, element R i,j The symbol ' will be in its bit-flipping mode [ST] j,2 The corresponding bit in the [] is flipped if it has a zero value. This calculation can be performed using multiple RPCs in the computing system MPC1.

[0124] Computation systems MPC1 132 and MPC2 134 reconstruct the bit vector (412). Computation systems MPC1 132 and MPC2 134 are capable of reconstructing the bit vector of a user profile based on matrices R and R' of exactly the same size. For example, computation system MPC1 132 can send a portion of the columns of matrix R, and computation system MPC2 134 can send the remaining columns of matrix R' to MPC1. In a specific example, computation system MPC1 132 can send the first half of the columns of matrix R to computation system MPC2 134, and computation system MPC2 134 can send the second half of the columns of matrix R' to MPC1. Although columns are used for horizontal partitioning in this example and are preferred to protect user privacy, rows can be used for vertical reconstruction in other examples.

[0125] In this example, computing system MPC2 134 is able to combine the first half of matrix R' with the first half of matrix R received from computing system MPC1 132 to reconstruct the first half (i.e., m / 2-dimensional) of the bit vector in plaintext form. Similarly, computing system MPC1 132 is able to combine the second half of matrix R with the second half of matrix R' received from computing system MPC2 134 to reconstruct the second half (i.e., m / 2-dimensional) of the bit vector in plaintext form. Conceptually, computing systems MPC1 132 and MPC2 134 have now combined corresponding shares of the two matrices R and R' to reconstruct the bit matrix B in plaintext form. This bit matrix B will include the bit vector of the projection results (projected onto each projection plane) of each user profile received from application 107 of the machine learning model. Each of the two servers in MPC system 130 possesses half of the bit matrix B in plaintext form.

[0126] However, if bit flipping is used, the computational systems MPC1 132 and MPC2 134 have already flipped the bits of the elements in matrices R and R' using a random pattern fixed for the machine learning model. This random bit flipping pattern is opaque to either of the two computational systems, MPC1 132 and MPC2 134, making it impossible for either system to infer the original user profile from the bit vector of the projection result. The cryptographic design further prevents MPC1 132 or MPC2 134 from inferring the original user profile by horizontally dividing the bit vector; that is, computational system MPC1 132 stores the latter half of the bit vector of the projection result in plaintext, while computational system MPC2 134 stores the former half in plaintext.

[0127] MPC system 130 generates machine learning models (414). Computational systems MPC1 132 and MPC2 134 within MPC system 130 can generate machine learning models using bit vectors corresponding to previously generated user profiles. In some implementations, if the machine learning model is a k-nn model, each of the two MPC computational systems 132 and 134 can generate a separate k-nn model using the corresponding half of the bit vector. For example, MPC system 132 can generate a k-nn model using the latter half of the bit vector. Additionally, computational system MPC2 134 can generate a k-nn model using the first half of the bit vector. Using bit flipping and horizontal partitioning of the matrix to generate the model applies the principle of defense-in-depth to protect the confidentiality of the user profiles used to generate the model.

[0128] In some implementations, if the machine learning model is a k-means model, either computational system MPC1 132 or MPC2 134 can generate a single k-means model using both halves of a bit vector. For example, computational system MPC1 132 uses both halves of a bit vector to generate a k-means model. However, in some implementations, each of computational systems MPC1 132 and MPC2 134 can generate a separate k-means model. Typically, a k-means model represents the cosine similarity (or distance) between user profiles of a set of users. The k-means model generated by either computational system MPC1 132 or MPC2 134 represents the similarity between bit vectors.

[0129] A k-means model generated by computing systems MPC1 132 or MPC2 134 can be referred to as a k-means model, which has a unique model identifier as described above. Computing system MPC130 can store the model and the share of labels for each user profile used to generate the model. Application 107 can then query the model to infer labels for the user group to which application 107 belongs.

[0130] Figure 5 This is a flowchart illustrating an example process 500 for training and querying the MPC system 130. The operation of process 500 can be performed, for example, by... Figure 1 The MPC system 130 is implemented. The operation of process 500 can also be implemented as instructions stored on one or more computer-readable media that may be non-transitory, and the execution of the instructions by one or more data processing devices can cause one or more data processing devices to perform the operation of process 500.

[0131] The first computing system in the multi-party computation (MPC) system 130 receives a query (502) including a first share of a given user profile and a second share of a given user profile. For example, application 107 generates two shares of the user profile (e.g., [P i,1 ]、[P i,2 Each share is used by one computing system of MPC system 130. Application 107 uses the public encryption key of computing system MPC1 132 to encrypt the first share [P]. i,1 Similarly, application 107 uses the public encryption key of computing system MPC2 134 to encrypt the second share of the user profile [P]. i,1 The application 107 executing on client device 106 will use the first encrypted share (e.g., PubKeyEncrypt([P...)). i,1 ], MPC1)) and the second encrypted share (e.g., PubKeyEncrypt([P i,2 ], MPC2)) is uploaded to the computing system MPC1 132.

[0132] The first computing system in MPC system 130 transmits the second share to the second computing system (504) in MPC system 130. For example, application 107 executing on client device 106 uploads the encrypted share of the user profile to computing system MPC1. Application 107 transmits the first share of the user profile (e.g., PubKeyEncrypt([P i,1 ], MPC1)) and the second share (e.g., PubKeyEncrypt([P i,2The first share of the user profile is uploaded to MPC1 by MPC2. MPC1 uses its private key to decrypt the first share of the user profile and transmits the second share of the user profile to MPC2 (210). MPC2 uses its private key to decrypt the second share of the user profile.

[0133] The first computing system of MPC system 130 determines a first label (506) for a first cluster with a centroid closest to a first share. For example, after training a k-means machine learning model by MPC1 and MPC2, application 107 transmits a query for user group labels to computing system MPC1132, the query including a first encrypted share and a second encrypted share of the user profile. In other examples, application 107 may transmit a query for user group labels to computing system MPC2134. Application 107 may submit a query for user group labels in response to a request from a content server to provide the labels for the user groups to which application 107 is assigned. For example, the content server may request application 107 to query the k-means model to determine the user group labels for application 107 on client device 110.

[0134] The cryptographic protocol described in step 212 of the execution process 200 of MPC1 132 and MPC2 134 is based on the first part [P] secretly stored by MPC1. i,1 ] and the second part [P] secretly stored by MPC2 i,2 To generate the first bit vector. In addition, MPC1 132 and MPC2 134 perform cryptographic protocols to determine the first cluster and the first label.

[0135] The first computing system receives a response (508) including a second label for the selected cluster. For example, computing system MPC2 performs the operation mentioned in step 212 of procedure 200 and determines the cluster and the second label. After determining the second label, computing system MPC2 134 is able to provide an encrypted version of the second label to computing system MPC1, wherein the second label is encrypted using the public encryption key of application 107.

[0136] MPC system 130 responds to queries (510) using a response that includes a first label and a second label. For example, computing system MPC1 132 can provide application 107 with encrypted versions of the first label and the second label of the selected cluster determined by computing system MPC2. Application 107 can decrypt the second label of the second cluster determined by computing system MPC2. After receiving the first label and the second label from MPC system 130, the application can reconstruct the final label in plaintext based on two secret shares received from two servers in MPC system 130 and store the label on client device 106.

[0137] Figure 6 This is a block diagram of an example computer system 600 capable of performing the operations described above. System 600 includes a processor 610, memory 620, storage device 630, and input / output device 640. Each of components 610, 620, 630, and 640 can be interconnected, for example, using a system bus 650. Processor 610 is capable of processing instructions for execution within system 600. In some embodiments, processor 610 is a single-threaded processor. In another embodiment, processor 610 is a multi-threaded processor. Processor 610 is capable of processing instructions stored in memory 620 or storage device 630.

[0138] Memory 620 stores information within system 600. In one embodiment, memory 620 is a computer-readable medium. In some embodiments, memory 620 is a volatile memory cell. In another embodiment, memory 620 is a non-volatile memory cell.

[0139] Storage device 630 provides high-capacity storage for system 600. In some embodiments, storage device 630 is a computer-readable medium. In various embodiments, storage device 630 may include, for example, a hard disk drive, an optical disk drive, a storage device shared by multiple computing devices over a network (e.g., a cloud storage device), or some other high-capacity storage device.

[0140] Input / output device 640 provides input / output operations for system 600. In some embodiments, input / output device 640 may include one or more of the following: a network interface device, such as an Ethernet card; a serial communication device, such as an RS-232 port; and / or a wireless interface device, such as an 802.11 card. In another embodiment, input / output device may include a driver configured to receive input data and send output data to external device 660 (e.g., a keyboard, printer, and display device). However, other embodiments, such as mobile computing devices, mobile communication devices, set-top box television client devices, etc., may also be used.

[0141] Although already Figure 6 An example processing system is described herein, but implementations of the subject matter and functional operations described herein can be implemented in other types of digital electronic circuits, or in computer software, firmware, or hardware (including the structures disclosed herein and their equivalents), or in a combination of one or more of them.

[0142] Embodiments of the subject matter and operations described in this specification can be implemented in digital electronic circuits, or in computer software, firmware, or hardware (including the structures disclosed in this specification and their equivalents), or in a combination of one or more of these. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a computer storage medium (or medium) for execution by or control of the operation of a data processing apparatus. Alternatively or additionally, program instructions can be encoded on artificially generated propagated signals, such as machine-generated electrical, optical, or electromagnetic signals, generated to encode information for transmission to a suitable receiver device for execution by the data processing apparatus. The computer storage medium can be a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of these, or be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of these. Furthermore, although the computer storage medium is not a propagated signal, it can be a source or destination of computer program instructions encoded in artificially generated propagated signals. Computer storage media can also be one or more separate physical components or media (e.g., multiple CDs, discs or other storage devices) or included in one or more separate physical components or media (e.g., multiple CDs, discs or other storage devices).

[0143] The operations described in this specification can be implemented as operations performed by a data processing device on data stored on one or more computer-readable storage devices or received from other sources.

[0144] The term "data processing apparatus" encompasses all kinds of devices, apparatuses, and machines for processing data, including, for example, programmable processors, computers, systems-on-a-chip, or a combination thereof. The apparatus can include special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, the apparatus can also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, cross-platform runtime environments, virtual machines, or combinations thereof. The apparatus and execution environment can implement a variety of different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0145] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but does not need to, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple collaborative files (e.g., a file storing one or more modules, subroutines, or code portions). A computer program can be deployed to execute on a single computer or on multiple computers located at a single site or distributed across multiple sites and interconnected through a communication network.

[0146] The processes and logic flows described in this specification can be executed by one or more programmable processors that execute one or more computer programs to perform actions by manipulating input data and generating outputs. The processes and logic flows can also be executed by dedicated logic circuits, and the devices can be implemented as dedicated logic circuits, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits).

[0147] For example, processors suitable for executing computer programs include both general-purpose microprocessors and special-purpose microprocessors. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for performing actions according to instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to, one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, to receive data from or transfer data to, or both. However, a computer does not need to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), and so on. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM discs. Processors and memory can be supplemented by or incorporated into dedicated logic circuits.

[0148] To provide interaction with the user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, and a keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Furthermore, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser on the user's client device in response to a request received from a web browser.

[0149] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes back-end components (e.g., as a data server), middleware components (e.g., an application server), or front-end components (e.g., a client computer having a graphical user interface or web browser that a user can interact with through embodiments of the subject matter described in this specification), or any combination of one or more such back-end components, middleware components, or front-end components. The components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., self-organizing peer-to-peer networks).

[0150] A computing system can include clients and servers. Clients and servers are typically geographically separated and usually interact via a communication network. The client-server relationship arises from computer programs running on their respective computers and involves a client-server relationship. In some embodiments, the server sends data (e.g., HTML pages) to the client device (e.g., to display data to a user interacting with the client device and to receive user input from the user). Data generated at the client device (e.g., the result of user interaction) can be received from the client device at the server.

[0151] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope that may be claimed, but rather as descriptions of features specific to particular embodiments of a particular invention. Certain features described in the context of individual embodiments in this specification can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in certain combinations, and even initially claimed in this way, one or more features from a claimed combination can be removed from that combination in some cases, and the claimed combination may involve sub-combinations or variations thereof.

[0152] Similarly, although operations are described in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed in the specific order or sequence shown, or requiring all illustrated operations to be performed to obtain the desired result. In some cases, multitasking and parallel processing can be advantageous. Furthermore, the separation of various system components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0153] Therefore, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific order or sequence shown to achieve the desired result. In some embodiments, multitasking and parallel processing can be advantageous.

Claims

1. A computer-implemented method, comprising: A first computing system of a multi-party computation (MPC) system receives a query including a first share of a given user profile and a second share of the given user profile, wherein the second share is encrypted using a key that prevents the first computing system from accessing the second share before being received by the first computing system. The first computing system transmits the second share to the second computing system of the MPC system; The first label of the first cluster with the centroid closest to the first share is determined by the first computing system, wherein the first cluster is one of a plurality of clusters generated by a machine learning model trained by the first computing system and the second computing system; The first computing system receives a response, including a second label of a second cluster, from the second computing system of the MPC system; The query is responded to using a response that includes both the first tag and the second tag.

2. The computer-implemented method according to claim 1 further includes: The first computing system receives first multiple partial shares of a user profile from a digital component distribution system different from the MPC system; The second computing system receives a second plurality of partial shares of a user profile from the digital component distribution system, wherein, for an individual user, the first plurality of partial shares and the second plurality of partial shares are secret shares that include all dimensions of the individual user's user profile; The machine learning model is trained by the first computing system and the second computing system using the first plurality of partial shares and the second plurality of partial shares.

3. The computer-implemented method according to claim 2, wherein, Training the machine learning model includes training a clustering model to create multiple clusters of user profiles based on the first plurality of partial shares and the second plurality of partial shares.

4. The computer-implemented method according to claim 3 further includes: The MPC system generates a centroid feature vector for each of the multiple clusters. The MPC system models the clusters using the probability distribution of the user profiles in each cluster; The MPC system generates a new centroid feature vector for each cluster based on the probability distribution and the centroid feature vector of the corresponding cluster; The new centroid feature vector is shared by the MPC system to the digital component distribution system.

5. The computer-implemented method according to claim 1, further comprising: The client device splits the given user profile into the first share and the second share; Generate the query as a request for labels of clusters corresponding to the given user profile and transmit the query to the first computing system; The client device receives the response including the first tag and the second tag; The client device stores the final device tag generated based on the first tag and the second tag.

6. The computer-implemented method according to claim 5, wherein, The final tag generated by the client device includes: The user profiles of the first cluster and the second cluster are modeled as normal distributions by the first computing system and the second computing system; The parameters of the normal distribution, including the centroid and covariance matrix, are determined by the first computing system and the second computing system. The first and second shares of the final label are generated by both the first and second computing systems. The MPC system transmits the first and second shares of the final tag to the client device; The client device reconstructs the final tag using the first share and the second share of the final tag.

7. The computer-implemented method according to claim 6, wherein, Determining the covariance matrix by the first computing system and the second computing system includes determining an integer matrix by the first computing system and the second computing system, such that the covariance matrix is ​​generated when the integer matrix is ​​multiplied by its transpose.

8. A multi-party computation (MPC) system, comprising: The first computing system of the MPC system receives a query including a first share of a given user profile and a second share of the given user profile, wherein the second share is encrypted using a key that prevents the first computing system from accessing the second share before being received by the first computing system; The first computing system transmits the second share to the second computing system of the MPC system; The first label of the first cluster with the centroid closest to the first share is determined by the first computing system, wherein the first cluster is one of a plurality of clusters generated by a machine learning model trained by the first computing system and the second computing system; The first computing system receives a response, including a second label of a second cluster, from the second computing system of the MPC system; The query is responded to using a response that includes both the first tag and the second tag.

9. The MPC system according to claim 8, further comprising: The first computing system receives first multiple partial shares of a user profile from a digital component distribution system different from the MPC system; The second computing system receives a second plurality of partial shares of a user profile from the digital component distribution system, wherein, for an individual user, the first plurality of partial shares and the second plurality of partial shares are secret shares that include all dimensions of the individual user's user profile; The machine learning model is trained by the first computing system and the second computing system using the first plurality of partial shares and the second plurality of partial shares.

10. The MPC system according to claim 9, wherein, Training the machine learning model includes training a clustering model to create multiple clusters of user profiles based on the first plurality of partial shares and the second plurality of partial shares.

11. The MPC system according to claim 10, further comprising: The MPC system generates a centroid feature vector for each of the multiple clusters. The MPC system models the clusters using the probability distribution of the user profiles in each cluster; The MPC system generates a new centroid feature vector for each cluster based on the probability distribution and the centroid feature vector of the corresponding cluster; The new centroid feature vector is shared by the MPC system to the digital component distribution system.

12. The MPC system according to claim 8, further comprising: The client device splits the given user profile into the first share and the second share; Generate the query as a request for labels of clusters corresponding to the given user profile and transmit the query to the first computing system; The client device receives the response including the first tag and the second tag; The client device stores the final device tag generated based on the first tag and the second tag.

13. The MPC system according to claim 12, wherein, The final tag generated by the client device includes: The user profiles of the first cluster and the second cluster are modeled as normal distributions by the first computing system and the second computing system; The parameters of the normal distribution, including the centroid and covariance matrix, are determined by the first computing system and the second computing system. The first and second shares of the final label are generated by both the first and second computing systems. The MPC system transmits the first and second shares of the final tag to the client device; The client device reconstructs the final tag using the first share and the second share of the final tag.

14. The MPC system according to claim 13, wherein, Determining the covariance matrix by the first computing system and the second computing system includes determining an integer matrix by the first computing system and the second computing system, such that the covariance matrix is ​​generated when the integer matrix is ​​multiplied by its transpose.

15. A non-transitory computer-readable medium storing instructions that, when executed by one or more data processing means, cause the one or more data processing means to perform operations, the operations including: A first computing system of a multi-party computation (MPC) system receives a query including a first share of a given user profile and a second share of the given user profile, wherein the second share is encrypted using a key that prevents the first computing system from accessing the second share before being received by the first computing system. The first computing system transmits the second share to the second computing system of the MPC system; The first label of the first cluster with the centroid closest to the first share is determined by the first computing system, wherein the first cluster is one of a plurality of clusters generated by a machine learning model trained by the first computing system and the second computing system; The first computing system receives a response, including a second label of a second cluster, from the second computing system of the MPC system; The query is responded to using a response that includes both the first tag and the second tag.

16. The non-transitory computer-readable medium of claim 15, further comprising: The first computing system receives first multiple partial shares of a user profile from a digital component distribution system different from the MPC system; The second computing system receives a second plurality of partial shares of a user profile from the digital component distribution system, wherein, for an individual user, the first plurality of partial shares and the second plurality of partial shares are secret shares that include all dimensions of the individual user's user profile; The machine learning model is trained by the first computing system and the second computing system using the first plurality of partial shares and the second plurality of partial shares.

17. The non-transitory computer-readable medium according to claim 16, wherein, Training the machine learning model includes training a clustering model to create multiple clusters of user profiles based on the first plurality of partial shares and the second plurality of partial shares.

18. The non-transitory computer-readable medium of claim 17, further comprising: The MPC system generates a centroid feature vector for each of the multiple clusters. The MPC system models the clusters using the probability distribution of the user profiles in each cluster; The MPC system generates a new centroid feature vector for each cluster based on the probability distribution and the centroid feature vector of the corresponding cluster; The new centroid feature vector is shared by the MPC system to the digital component distribution system.

19. The non-transitory computer-readable medium of claim 15, further comprising: The client device splits the given user profile into the first share and the second share; Generate the query as a request for labels of clusters corresponding to the given user profile and transmit the query to the first computing system; The client device receives the response including the first tag and the second tag; The client device stores the final device tag generated based on the first tag and the second tag.

20. The non-transitory computer-readable medium according to claim 19, wherein, The final tag generated by the client device includes: The user profiles of the first cluster and the second cluster are modeled as normal distributions by the first computing system and the second computing system; The parameters of the normal distribution, including the centroid and covariance matrix, are determined by the first computing system and the second computing system. The first and second shares of the final label are generated by both the first and second computing systems. The MPC system transmits the first and second shares of the final tag to the client device; The client device reconstructs the final tag using the first share and the second share of the final tag.

Images