System for designing and implementing privacy preserving AI with privacy regulations within intelligent pipeline

Abstract

Data may be received that includes information corresponding to a set of users. A privacy protection protocol applicable to the data may be identified. A subset of the data may be identified as Personally Identifiable Information (PII) data, where the subset includes a set of PII attributes. The PII attributes may be split into categories based on a format of a data field in the PII attributes. The processed PII data may be combined with the non-PII data to create processed client data. It may be determined to add noise to a portion of the processed PII data. An amount of noise may be determined based on a privacy protection protocol. An amount of noise may be added to a portion of the processed PII data to produce protected data. A machine learning model may be trained using the protected data.

Description

[0001] Priority claim

[0002] This application claims priority to U.S. Patent Application No. 17 / 932,938, filed on September 16, 2022, entitled “SYSTEMS FOR DESIGN AND IMPLEMENTATION OF PRIVACY PRESERVING AI WITH PRIVACY REGULATIONS WITH ININTELLIGENCE PIPELINES,” which is incorporated by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to data privacy preserving protocols, and more particularly (but not necessarily exclusively) to the implementation of privacy preserving artificial intelligence. Background Art

[0004] The client's data platform may provide tools for identifying users or user devices that are likely to receive a particular type of communication in multiple user experience flows. Such identification of users or user devices may include processing historical and / or current data corresponding to multiple users or user devices based on specific circumstances corresponding to the communication type(s). As data from multiple systems is parsed in the data platform, privacy protection may become important (e.g., due to applicable privacy regulations and / or privacy goals determined by the client).

[0005] For example, privacy regulations and / or goals can define the composition of personally identifiable information (PII) so that the data can be obfuscated before being stored, processed, used and / or transmitted. PII data can include any information that can be used to distinguish one user from another, and can be used to de-anonymize previously anonymous data. Examples of PII data can include social security numbers, passport numbers, telephone numbers, credit card numbers, street addresses, patient identification numbers, email addresses, etc. However, the composition of PII may vary from case to case. For example, assigning a given user to an income level may not be sufficient to identify the user in most zip codes, although it may help identify a specific person in one or more specific zip codes. In addition, the regulations and / or priorities of various entities related to data privacy may change over time and vary from region to region (e.g., privacy regulations may change over time and from region to region).

[0006] If an entity collects PII, then it may be important to implement protocols to ensure that the storage and transmission of PII is controlled to prevent unauthorized access. However, it is also important to control the use of PII. For example, sometimes, if a machine learning (ML) model is trained using a dataset containing PII, the model may be interrogated in a way that allows an attacker to predict the PII data (by carefully examining the outputs generated in response to specific inputs). Summary of the invention

[0007] Techniques (e.g., methods, systems, non-transitory computer-readable media storing codes or instructions executable by one or more processors) for designing and implementing privacy-preserving AI with privacy provisions within smart pipelines are provided. Various embodiments are described herein, including methods, systems, non-transitory computer-readable storage media storing programs, codes, or instructions executable by one or more processors, and the like.

[0008] One embodiment relates to a computer-implemented method, the method comprising: receiving client data including information corresponding to a set of users and / or a set of user devices from a client system; identifying a privacy protection protocol applicable to the client data; identifying a specific subset of the client data as personally identifiable information (PII) data, wherein the specific subset includes a set of PII attributes; for each predefined category in a plurality of predefined categories, splitting the PII attributes into a plurality of predefined categories based on the format of a data field in the PII attributes; selecting a processing technique from a plurality of processing techniques based on the predefined categories and the privacy protection protocol; and processing the PII data corresponding to the predefined categories using the selected processing technique; combining the processed PII data corresponding to the plurality of predefined categories with non-PII data to create processed client data; determining to add noise to at least a portion of the processed PII data based on previous input from the client system and / or based on the processed PII data; determining an amount of noise based on the privacy protection protocol; adding the amount of noise to at least a portion of the processed PII data to generate protected training data; and training a machine learning model using the protected training data.

[0009] In some embodiments, a system is provided that includes one or more data processors and a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform part or all of one or more methods disclosed herein.

[0010] In some embodiments, a computer program product is provided, which is tangibly embodied in a non-transitory machine-readable storage medium and includes instructions configured to cause one or more data processors to perform part or all of one or more methods disclosed herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The present disclosure is described in conjunction with the accompanying drawings:

[0012] Figure 1 Depicted is a flow diagram illustrating an example process for implementing privacy-preserving AI according to some embodiments of the present invention.

[0013] Figure 2 A database environment for implementing privacy-preserving AI in accordance with at least some embodiments of the present invention is described.

[0014] Figure 3 is a diagram depicting the flow of client data through a privacy-preserving protocol according to at least some embodiments.

[0015] Figure 4 is a flow chart illustrating an example process for maintaining privacy of client data by implementing privacy rules according to some embodiments of the invention.

[0016] Figure 5 is a block diagram depicting separation of PII data and non-PII data by a PII identifier module according to an embodiment.

[0017] Figure 6 is a graph illustrating the dependence of a root mean square error of a recommendation algorithm when noise is added to ML model training data according to an embodiment.

[0018] Figure 7 is a graph showing the relationship between the mean absolute error of the recommendation algorithm and the epsilon value when the privacy budget is applied to the training data of the ML model.

[0019] Figure 8 is a graph depicting the accuracy of an ML model as a function of epsilon when an algorithmic noise addition method is used in an ML model privacy protocol according to an embodiment.

[0020] Fig. 9 A simplified diagram of a distributed system 900 for implementing an embodiment is depicted.

[0021] Fig.10 is a simplified block diagram of a cloud-based system environment in which the functionality described herein may be provided as a cloud service, according to some embodiments.

[0022] Fig.11 An exemplary computer system 1100 is illustrated that may be used to implement certain embodiments.

[0023] In the accompanying drawings, similar components and / or features may have the same reference label. In addition, various components of the same type may be distinguished by adding a dash and a second label to distinguish the similar components after the reference label. If only the first reference label is used in the specification, the description applies to any one of the similar components having the same first reference label regardless of the second reference label. DETAILED DESCRIPTION

[0024] In the following description, for the purpose of explanation, specific details are set forth to provide a thorough understanding of certain embodiments. However, it is apparent that various embodiments may be practiced without these specific details. The drawings and descriptions are not intended to be limiting. The word "exemplary" as used herein means "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs.

[0025] Some embodiments of the present invention relate to systems and techniques for generating and implementing privacy-preserving protocols through privacy-preserving artificial intelligence (AI). Specifically, incoming data can be parsed to distinguish data corresponding to multiple predefined categories (e.g., categorized text data, categorized numbers, high cardinality text, high cardinality numbers, continuous numeric data, and text). In some cases, different types of privacy-preserving protocols are implemented for each category of data. For each type of data, a technique can be selected to transform the PII in that category (e.g., if the corresponding PII transformation criteria are met).

[0026] Figure 1 A flow diagram illustrating an example process 100 for implementing privacy preserving AI according to some embodiments of the present invention is depicted. Block 102 includes receiving client data from a client system via a data platform. The client system may include at least one of a mobile device, a cloud storage platform, a personal computer, a laptop, a computer cluster, a server, a server cluster, etc. The client data may be received via an interface generated by the data platform. In some examples, the interface may identify the region of the client data and the client privacy regulations.

[0027] At box 104, a privacy protection protocol for PII data within the client data is defined. In some examples, the client may use a configuration screen to define the privacy protection protocol. Box 104 may include allowing the client to switch to enable masking, obfuscation, or hiding of PII data using various privacy options. The client may choose to implement different privacy protection protocols for different categories of PII data. In some examples, the privacy protection protocol may be determined by a server operator associated with the data platform, such as when the client selects the default settings for the privacy protection protocol or provides limited input to the configuration screen. The privacy protection protocol may be based on the regional or client privacy provisions detected by the interface when receiving client data. For example, a server operator may confirm that the privacy protection protocol selected by the client complies with the regional privacy provisions.

[0028] Since the ML model training data may include PII data, the client may choose to include the privacy settings of the ML model associated with the data as part of the privacy protection protocol. In some examples, the preprocessing of the ML model training data may be configured based on the ML privacy budget, default values, etc., which may be set by the client. By adjusting the ML privacy budget, the client may define the extent to which the PII data is blurred before entering the ML model. For example, noise may be added to the PII data. A small amount of noise may correspond to high accuracy in the ML model, but may not provide adequate privacy protection for the PII data input into the ML model. If the amount of noise increases, privacy protection may be enhanced, but model accuracy may decrease. In some examples, the interface may notify the client of the expected level of accuracy in the ML model based on the selected ML privacy budget. In some examples, noise may be added to the output of the algorithm or ML model according to the client's selection of the privacy budget. The client may adjust the ML privacy budget to adjust the model accuracy and / or privacy. In some examples, the output of the ML model may include PII data. The client may include privacy settings for the output in the privacy protection protocol.

[0029] At box 106, the process includes identifying PII data within the client data. In some examples, the client data is transmitted to a PII identifier module. The PII identifier module can identify which attributes contain PII and which are non-PII attributes. For example, the module can identify that a nine-digit set indicates a social security number, or a seven-digit set or a ten-digit set indicates a U.S. telephone number including an area code. The PII identifier module can use a PII identifier program and an imported PII information type identification library to identify PII attributes. The data type splitter module can split the PII attributes into several categories. These categories include classified text, classified numbers, high cardinality text, high cardinality numbers, continuous text, etc.

[0030] At block 108, the processing includes processing the PII data using the selected processing method. In some examples, the data type splitter module can process the PII attributes based on the category and the privacy rules and heuristics associated with the privacy protection protocol supplied by the client. There are several processing techniques for processing PII attributes, including, for example, adding noise to the PII data in the ML model training data. In some examples, block 110 can include built-in masking, hashing, pseudonymization, or anonymization of the identified PII data based on the PII information type.

[0031] At block 110, the process includes identifying an amount of noise based on a privacy protection protocol and an ML privacy budget. While adding noise to ML training data can provide a certain degree of protection for the ML model, it can also reduce the accuracy of the ML model. Therefore, the amount of noise can be determined based on the ML privacy budget identified by the client, so that the client can control this balance.

[0032] At block 112, the process includes adding the amount of noise to a subset of the combined PII and non-PII client data to generate protected training data. If the client has selected data privacy for the PII data category, random, Gaussian, or Laplace noise may be added to the data category for the PII attributes. Noise may not be added to the non-PII client data included in the protected training data.

[0033] Block 114 of the process includes inputting the protected training data into the ML model. Once trained, the ML model can generate an output. In some examples, the output is returned to the client device.

[0034] Figure 2 A database environment 200 for implementing privacy-preserving AI according to at least some embodiments of the present invention is depicted. The database environment 200 includes a client data platform 202 and a plurality of client systems 208. Figure 2 216-i, user device J 216-j, user device K 216-k, and user device L 216-l. Although three client systems are illustrated in FIG. 216 , more or fewer client systems may be present in database environment 200. Each of the plurality of client systems 208 may receive client data from user devices 216. For example, client system A may receive data from user device A 216-a, user device B 216-b, user device C 216-c, and user device D 216-d. Client system B may receive data from user device E 216-e, user device F 216-f, and user device G 216-g. Client system C may receive data from user device H 216-a, user device I 216-i, user device J 216-j, user device K 216-k, and user device L 216-l. Although Figure 22, but there may be more or fewer user devices 216 in the database environment 200. The client data received from the user devices 216 may include PII data.

[0035] Each of the plurality of client systems 208 can store client data in a user data repository 212. Each of the plurality of client systems 208 can communicate with a client data platform 202. Each client system can transmit client-specific privacy rules 210 to the client data platform 202. In some examples, for each client system, the client data platform 202 can determine the regional privacy rules 204 applicable to the client system by detecting the location of a user associated with the client. Detecting the location of the user can involve extracting information fields from data involved in communicating with the user device 216. In some examples, the location of the user can be extracted from home address data contained in the client data. In some examples, the client data platform can receive information about the regional privacy rules 204 from the client.

[0036] The client data platform 202 can communicate with each client system in the plurality of client systems 208 to define a privacy protection protocol for PII data within each client system. In some examples, the client can use a configuration screen to define the privacy protection protocol. The client can choose to implement different privacy protection protocols for different categories of PII data. In some examples, the client can define a privacy budget to provide privacy settings for client data associated with the client's ML model.

[0037] In some examples, the client data platform 202 may receive user data from a client, and the client data platform 202 may implement a client-specific privacy preservation program. The client data platform 202 may modify the privacy preservation program 206 to create a client-specific privacy preservation program that includes a privacy protection protocol and is based on client-specific privacy rules, applicable regional privacy rules, and a privacy budget. In some examples, a client system may receive the privacy preservation program 206 from the client data platform 202, and the client system may implement the privacy preservation program. Each client system may use the client-specific preservation program to implement the privacy protection protocol and maintain data security measures that comply with regional and client-specific privacy rules.

[0038] Figure 3300 is a diagram depicting the flow of client data through a privacy protection protocol according to at least some embodiments. Client data includes data collected from users associated with the client. The data collected from the users may include PII data. Users associated with the client may live in various regions. Regions may include various sets of countries, single countries, states, provinces, etc. The privacy protection policies for users' PII data may vary from region to region and industry to industry. For example, a country may require clients to implement and maintain specific data security measures to protect the financial information of users who are citizens of that country. Client data may originate from user devices, which may include mobile devices, personal computers, laptops, smart watches, etc. Client data may be merged into a data repository associated with the client system, referred to as a data warehouse 302. Client data may include PII attributes and non-PII attributes.

[0039] In some examples, client data from data warehouse 302 can be exported to client data platform 202. Figure 3 In the example shown in , the client data platform 202 is shown to include cloud storage 304 or object storage (OS) or simple storage service (S3) storage buckets. A PII identifier module 306 associated with the client data platform 202 can analyze the client data in the cloud storage 304. The PII identifier module 306 can use techniques such as a PII identifier program, a PII information type identification library, or a PII definition identified by the client to identify which attributes contain PII attributes and which attributes contain non-PII attributes. The identification of PII attributes may vary from organization to organization, domain to domain, and country to country, and this specificity can be reflected by (for example) program functions, libraries, or client-specified definitions. The PII identifier module 306 can be configured to incorporate various public or private PII information type libraries. The PII information type library can be stored within the client data platform 202, or the PII information type library can be received from the client and associated with the client. Examples of PII information type libraries include Presidio (Microsoft SDK), AWSDetect PII, and open source libraries.

[0040] The data type splitter module can split the PII attributes into several categories. These categories include categorical text, categorical numbers, high cardinality text, high cardinality numbers, continuous text, etc. For example, the data type splitter module can separate PII data with digital attributes (digital data) from PII data with categorical attributes (categorical data). Examples of digital data can include zip code, salary, age, social security number, date of birth, etc. Examples of categorical data can include name, gender, city of residence, race, place of birth, etc. Rule-based algorithms can be used to identify categorical data, such as entity recognition algorithms, such as named entity recognition (NER) algorithms. Pattern recognition techniques can be used to identify certain types of numbers (such as bank account data) or certain types of categorical data (such as national identity).

[0041] The privacy protection protocol 308 can be implemented into the attributes within the client data. Since the privacy protection policy may vary from region to region, the privacy protection protocol 308 may depend on the nationality of the user associated with the client data. In some examples, identifying the privacy protection protocol 308 may include determining one or more geographic regions corresponding to the client system and retrieving one or more data protection restrictions corresponding to the one or more geographic regions. The privacy protection protocol 308 may also depend on the privacy protection policy specified by the client. The decision to implement the privacy protection protocol 308 may depend on whether the attribute of the client data is a PII attribute or a non-PII attribute. The privacy protection protocol 308 implemented on the PII attribute may depend on the category of the PII attribute. For example, when the digital data is subject to the regional privacy protection policy, noise (e.g., Gaussian noise) can be added to the digital data as data privacy (DP) preprocessing 316. In other examples, the digital data is not subject to the privacy protection policy, and the privacy protection protocol is not implemented for the digital data. In other words, when the digital data is not subject to the privacy protection policy, noise is not added to the digital data. If the privacy protection protocol 308 is not implemented for the first data set (eg, the first data set contains only non-PII attributes), then the first data set may be used to train the ML model without (DP) pre-processing 316 .

[0042] If a dataset with PII is used to train a machine learning (ML) model, then the model may be interrogated in a way that allows an attacker to predict PII data (by carefully examining the output generated in response to a specific input). Preprocessing of ML model training data can be configured based on an ML privacy budget, which can be set by the client, can be a default value, etc. By adjusting the ML privacy budget, the client can define the extent to which PII data (or non-PII data) is obfuscated before entering the ML model. For example, noise can be added to PII data. A small amount of noise can correspond to high accuracy in the ML model, but may not provide sufficient privacy protection for the PII data input into the ML model. If the amount of noise increases, then privacy protection can be enhanced, but model accuracy may decrease.

[0043] Even if the privacy preserving protocol 308 is not implemented for the first data set, the ML privacy protocol 310 may be implemented to process the first data set. If the ML privacy protocol 310 is not selected for the first data set, the training data for the ML model using the standard ML algorithm may include the first data set at block 312. Once trained, ML scoring may be performed, and the results of the standard ML model may be returned to the client via the data warehouse 302. If the ML privacy protocol 310 is selected for the first data set, the training data for the ML model using the privacy preserving ML algorithm may include the first data set at block 314. Once trained, ML scoring may be performed, and the results of the privacy preserving ML model may be returned to the client via the data warehouse 302.

[0044] If a privacy protection protocol 308 is to be implemented for the second data set (e.g., the second data set includes at least one category of PII attributes subject to a privacy protection policy), then the second data set can be subjected to DP preprocessing 316. DP preprocessing 316 can include built-in masking, hashing, pseudonymization, or anonymization of the second data set based on the type of PII information. The type of preprocessing can depend on the category of PII attributes included in the second data set. For example, random, Gaussian, or Laplace noise can be added to digital attributes, and categorical attributes can be encrypted.

[0045] The ML privacy protocol 318 may be implemented to the second data set. If the ML privacy protocol 318 is not selected for the second data set, then the training data for the ML model using the standard ML algorithm may include the second data set at block 320. Once trained, ML scoring may be performed, and the results of the standard ML model may be returned to the client via the data warehouse 302. If the ML privacy protocol 318 is selected for the second data set, then the training data for the ML model using the privacy-preserving ML algorithm may include the first data set at block 322. Once trained, ML scoring may be performed, and the results of the privacy-preserving ML model may be returned to the client via the data warehouse 302.

[0046] Figure 4 4 is a flow chart illustrating an example process 400 for maintaining privacy of client data by implementing privacy rules according to some embodiments of the present invention. Block 402 includes data cleaning of client data. Data cleaning may involve repairing incorrect, incomplete, duplicate, or otherwise erroneous data in a data set of client data. For example, data cleaning may involve correcting grammatical errors or errors involving formatting. As an example, a recorded date of birth of 13.03.92 may be changed to 03 / 13 / 92 to conform to or match other data in a data set.

[0047] Box 404 includes transmitting the client data to the PII identifier module 306 to identify the PII attribute data and non-PII attribute data within the client data. The PII identifier module 306 can use the PII identifier program and the imported PII information type identification library to identify the PII attributes within the client data. The information type identification library can determine the patterns in the data set, which can be used to identify the PII attributes. For example, in the United States, a data set with 5 digits can refer to a zip code, or data in the format of (XXX)-XXX-XXXX (where X is any digit) can be identified as a telephone number. The PII identifier module 306 can be configured to support various public or private PII information type libraries. The PII information type library supported by the PII identifier module 306 can be updated regularly.

[0048] Box 406 involves transmitting the PII attribute data to a data type splitter module. The data type splitter module can split the PII attributes into several categories. The categories can include categorical text, categorical numbers, high cardinality text, high cardinality numbers, continuous text, etc. For example, the data type splitter module can separate PII data with digital attributes (digital data) from PII data with categorical attributes (categorical data). Examples of digital data can include zip code, salary, age, social security number, date of birth, etc. In some examples, digital data in a region can be identified by comparing the population distribution in the region. For example, if a PII data set shows a large overlap with the population distribution (e.g., more than 50% overlap), then the PII data can be identified as digital data. Examples of categorical data can include name, gender, city of residence, race, place of birth, etc. Entity recognition algorithms (such as NER algorithms) can be used to identify categorical data. Pattern recognition techniques can be used to identify certain types of numbers (such as bank account data) or certain types of categorical data (such as national identity).

[0049] Box 408 involves applying privacy rules to digital data and classified data. Since privacy regulations may vary from country to country, the source of client data may affect the privacy rules. In some examples, applying privacy rules to digital data involves converting digital data into noise data by adding noise to the digital data. Noise can include random, Gaussian or Laplace noise. The privacy rules applied to PII data can depend on the type. For example, classified digital data containing a limited number of values ​​to represent gender may require less stringent privacy steps than the privacy protocol for PII data containing user bank account numbers or social security numbers. In some examples, applying privacy rules to classified data involves converting classified data into encrypted data by adding encryption to the classified data. Noise data and encrypted data can be combined to form a privacy data set.

[0050] Box 410 involves training a machine model using protected training data. For example, noisy data, encrypted data, and non-PII attribute data may be combined to form protected training data. Once trained, the ML model may generate an output. In some examples, the output may correspond to a client-defined prediction, and the determined prediction may be returned to the client device. If there is PII attribute data within the output, then privacy rules may be applied to the PII attribute output data.

[0051] Figure 55 is a block diagram depicting the separation of PII data and non-PII data by PII identifier module 306 according to an embodiment. The pre-processing module of PII identifier module 306 can separate client data into categories based on attribute types. Attribute types can be PII attributes and non-PII attributes. Categories based on attribute types can include text 502, classified text 504, high cardinality text 506, continuous text 508, high cardinality numbers 510, and classified numbers 512.

[0052] Classified text 504 may include text associated with a fixed or limited number of possible values. An example of classified text 504 may include the brand name of a vehicle driven by a user. Another example of classified text is the user's state of residence. High cardinality data may include very unique or uncommon data that may be used to identify a user. Examples of high cardinality text 506 include email addresses, user names, etc. Examples of high cardinality numbers 510 include bank account numbers, social security numbers, phone numbers, etc. Classified numbers 512 may include numerical values ​​assigned to a limited number of possible values. Examples of classified numbers may be limited numerical values ​​assigned to each possible gender or each possible blood type. Continuous text 508 data may include text assembled into sentences, which are compiled into paragraphs, which are organized into chapters, etc.

[0053] Various techniques can be used to identify PII data within a category based on attribute type. ML model methods 514 can be used to identify PII data within a text 502 category. For example, ML model methods 514 can be applied to text columns to search for PII attributes, such as personal names, organization names, and locations. If the ML model method finds PII attributes within rows in at least a threshold percentage (e.g., at least 80%) of the column, then the column is identified as PII data. In addition, privacy rule-based techniques 516 can be applied to identify PII data. For example, a high cardinality number 520 category can be compared with a distribution of PII attributes (such as age, location population, or salary). If a distribution set of high cardinality number 520 data shows a statistically significant overlap with the distribution, then the set can be identified as PII data. Pattern recognition 518 is another technique for identifying PII data (such as passport numbers, tax identification numbers, national ID numbers, bank account numbers, etc.).

[0054] Figure 6is a graph illustrating the dependence of the root mean square error of the recommendation algorithm when noise is added to the ML model training data according to an embodiment. The ML model algorithm may require strict privacy techniques to protect the PII data used to train the ML model. A list of potential threats to the ML system may include de-anonymization (re-identification) attacks, reconstruction attacks, parameter interference attacks, model inversion attacks, membership inference attacks, and the like. Even if the data set is anonymous, an attacker can use background knowledge to infer the data through de-anonymization or re-identification attacks. An attacker can obtain external knowledge of the feature vectors of the ML model training data in a reconstruction attack. In a model inversion attack, an attacker can discover new test samples submitted by a user to the ML model and the responses generated by the ML model for the new test samples. In a membership inference attack, the attacker gains access to the output of the ML model and can identify whether a specific PII data set is used to train the ML model based on the identified output.

[0055] Table 1 includes various methods by which PII data involved in ML models can be protected. Protection of data incorporated into ML models can be configured based on an ML privacy budget (which can be set by the client, a default value, etc.). By adjusting the ML privacy budget, the client can define the extent to which PII data is involved in the ML model. Noise can be added at various stages associated with the ML model. In order to protect the PII data used to train the ML model, in some examples, differential privacy methods can be employed. Differential privacy refers to a class of techniques that are implemented to ensure that the amount of information that can be leaked due to participation in a database is bounded. Basically, differential privacy limits the amount of information that an attacker can access.

[0056]

[0057]

[0058] Table 1 Description of techniques used to protect PII attributes in ML models.

[0059] The bounds provided by differential privacy can be determined by a variable denoted by the Greek letter epsilon (ε). The value of epsilon is roughly the number of bits of information an analyst may determine about an individual from a database. Epsilon is a parameter that can be linked to an ML privacy budget. The value of epsilon can be chosen based on the ML privacy budget. The smaller the value of epsilon, the greater the amount of noise added (i.e., the larger the privacy budget). Therefore, the smaller the value of epsilon, the stronger the privacy protection, but the lower the accuracy.

[0060] Figure 6The graph in shows the relationship between the RMSE of the recommendation algorithm and the value of epsilon when the privacy budget is applied to the training data of the ML model. For comparison, the non-private value 604 of the RMSE when no noise is added is also shown. In this example, the non-private value 604 is 0.6854. As epsilon increases, the difference between the RMSE of the private data 602 and the non-private value 604 decreases. When the value of epsilon decreases, the difference between the private data 602 and the non-private value 604 increases.

[0061] Figure 7 704 is a graph showing the relationship between the mean absolute error of the recommendation algorithm and the value of epsilon when the privacy budget is applied to the training data of the ML model. For comparison, the non-private value 704 of the mean absolute error when no noise is added is also shown. In this example, the non-private value 704 of the mean absolute error is 0.398. As epsilon increases, the difference between the mean absolute error of the private data 702 and the non-private value 704 of the mean absolute error decreases. When epsilon decreases, the difference between the private data 702 and the non-private value 704 of the mean absolute error increases.

[0062] Figure 8 802 is a graph depicting the accuracy of the ML model as a function of epsilon when an algorithmic noise addition method is used in an ML model privacy protocol according to an embodiment. In the algorithmic noise addition method, the ML model may require several iterations to complete the ML model training phase. During the ML model training phase, the model parameters may be optimized by attempting to reduce noise in each iteration. The various ML algorithms used may include logistic regression, naive Bayes, stochastic gradient descent, or deep neural networks. Figure 8 The graph in depicts the accuracy of the Naive Bayes algorithm as epsilon varies. As mentioned above, the epsilon value selected can affect the amount of noise added to the Naive Bayes algorithm. For comparison, the accuracy value 804 is also shown when no noise is added to the algorithm. The graph shows that lower epsilon values ​​can provide more privacy protection, but with reduced accuracy.

[0063] Fig. 9 A simplified diagram of a distributed system 900 for implementing an embodiment is depicted. In the illustrated embodiment, the distributed system 900 includes one or more client computing devices 902, 904, 906, and 908 coupled to a server 912 via one or more communication networks 910. The client computing devices 902, 904, 906, and 908 may be configured to execute one or more applications.

[0064] In various embodiments, server 912 may be adapted to run one or more services or software applications that enable the processes described in this disclosure.

[0065] In some embodiments, the server 912 may also provide other services or software applications that may include non-virtualized and virtualized environments. In some embodiments, these services may be provided to users of client computing devices 902, 904, 906, and / or 908 as web-based services or cloud services, such as under a software as a service (SaaS) model. Users operating client computing devices 902, 904, 906, and / or 908 may in turn interact with the server 912 using one or more client applications to utilize the services provided by these components.

[0066] exist Fig. 9 In the configuration depicted in FIG. 1 , the server 912 may include one or more components 918, 920, and 922 that implement the functions performed by the server 912. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. It should be appreciated that a variety of different system configurations are possible that may differ from the distributed system 900. Therefore, Fig. 9 The embodiment shown in is an example of a distributed system for implementing the embodiment system and is not intended to be limiting.

[0067] According to the teachings of the present disclosure, a user may interact with server 912 using client computing devices 902, 904, 906, and / or 908. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Fig. 9 Only four client computing devices are depicted, but any number of client computing devices may be supported.

[0068] Computing devices may include various types of computing systems, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computing devices may run various types and versions of software applications and operating systems (e.g., Microsoft Apple or UNIX-like operating systems, Linux or Linux-like operating systems (such as Google Chrome TM OS), including various mobile operating systems (e.g., Microsoft Windows Windows AndroidTM , Palm Portable handheld devices may include cellular phones, smart phones (e.g., ), tablets (e.g. ), personal digital assistants (PDAs), etc. Wearable devices can include Google Head-mounted displays and other devices. Gaming systems can include various handheld gaming devices, Internet-enabled gaming devices (e.g., with or without Gesture Input Devices Microsoft Game consoles, Sony System, by Various game systems provided by and others), etc. The client device can execute various different applications, such as various Internet-related applications, communication applications (e.g., email applications, short message service (SMS) applications), and can use various communication protocols.

[0069] The network(s) 910 may be any type of network familiar to those skilled in the art, and may support data communications using any of a variety of available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internetwork Packet Exchange), The network(s) 910 may be, by way of example only, a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., in any of the Institute of Electrical and Electronics Engineers (IEEE) 1002.11 protocol suites, and / or any other wireless protocol operating under the network) and / or any combination of these and / or other networks.

[0070] The server 912 may be composed of one or more general-purpose computers, dedicated server computers (including, for example, PC (personal computer) servers, The server 912 may be a server, a mid-range server, a mainframe computer, a rack-mounted server, etc.), a server farm, a server cluster, or any other suitable arrangement and / or combination. The server 912 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization, such as one or more flexible pools of logical storage devices that can be virtualized to maintain a virtual storage device of the server. In various embodiments, the server 912 may be suitable for running one or more services or software applications that provide the functionality described in the foregoing disclosure.

[0071] The computing system in the server 912 can run one or more operating systems, including any of the operating systems discussed above and any commercially available server operating system. The server 912 can also run any of a variety of additional server applications and / or middle-tier applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, Servers, database servers, etc. Exemplary database servers include but are not limited to (International Business Machines) and other commercially available database servers.

[0072] In some embodiments, server 912 may include one or more applications to analyze and integrate data feeds and / or event updates received from users of client computing devices 902, 904, 906, and 908. As examples, data feeds and / or event updates may include, but are not limited to, data received from one or more third-party information sources and continuous data streams. feed, Updates or real-time updates, which may include real-time events related to sensor data applications, financial quote machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc. Server 912 may also include one or more applications that display data feeds and / or real-time events via one or more display devices of client computing devices 902, 904, 906, and 908.

[0073] The distributed system 900 may also include one or more data repositories 914, 916. In some embodiments, these data repositories may be used to store data and other information. For example, one or more of the data repositories 914, 916 may be used to store data or information generated by and / or used for the processing described herein. The data repositories 914, 916 may reside in a variety of locations. For example, the data repository used by the server 912 may be local to the server 912 or may be remote from the server 912 and communicate with the server 912 via a network-based connection or a dedicated connection. The data repositories 914, 916 may be of different types. In some embodiments, the data repository used by the server 912 may be a database, for example, a relational database such as a database provided by Databases provided by companies and other suppliers. One or more of these databases may be adapted to respond to SQL formatted commands to enable storage, updating, and retrieval of data in the database.

[0074] In some embodiments, the application may also use one or more of the data repositories 914, 916 to store application data. The data repositories used by the application may be of different types.

[0075] In certain embodiments, the functionality described in this disclosure may be provided as a service via a cloud environment. Fig.10 is a simplified block diagram of a cloud-based system environment in which the functionality described herein may be provided as a cloud service, according to certain embodiments. Fig.10 , cloud infrastructure system 1002 may provide one or more cloud services that may be requested by a user using one or more client computing devices 1004, 1006, and 1008. Cloud infrastructure system 1002 may include one or more computers and / or servers, which may include those described above with respect to server 912. The computers in cloud infrastructure system 1002 may be organized as general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0076] The network(s) 1010 can facilitate communication and data exchange between clients 1004, 1006, and 1008 and the cloud infrastructure system. The network(s) 1010 can include one or more networks. The networks can be of the same or different types. The network(s) 1010 can support one or more communication protocols, including wired and / or wireless protocols, to facilitate communication.

[0077] Fig.10 The embodiment depicted in FIG. 1 is merely one example of a cloud infrastructure system and is not intended to be limiting. It should be appreciated that in some embodiments, the cloud infrastructure system 1002 may have more than Fig.10 The components shown in the figure may be more or fewer components, may be combined with two or more components, or may have a different configuration or arrangement of components. Fig.10 Three client computing devices are depicted, but any number of client computing devices may be supported in alternative embodiments.

[0078] The term "cloud service" is generally used to refer to services that are made available to users on demand by a service provider's system (e.g., cloud infrastructure system 1002) and via a communications network such as the Internet. Typically, in a public cloud environment, the servers and systems that make up the cloud service provider's system are different from the customer's own internal servers and systems. The cloud service provider's system is managed by the cloud service provider. Customers can therefore take advantage of cloud services provided by a cloud service provider without having to purchase separate licenses, support, or hardware and software resources for the services. For example, a cloud service provider's system can host applications, and users can order and use the applications on demand via the Internet without having to purchase infrastructure resources for executing the applications. Cloud services are designed to provide easy, scalable access to applications, resources, and services. Several providers offer cloud services. For example, The company provides several cloud services, such as middleware services, database services, Java cloud services, etc.

[0079] In some embodiments, the cloud infrastructure system 1002 may provide one or more cloud services using different models such as a software as a service (SaaS) model, a platform as a service (PaaS) model, an infrastructure as a service (IaaS) model, and other models including hybrid service models. The cloud infrastructure system 1002 may include a set of applications, middleware, databases, and other resources that enable the provision of various cloud services.

[0080] The SaaS model enables applications or software to be delivered to customers as a service over a communication network such as the Internet, without the customer having to purchase hardware or software for the underlying application. For example, the SaaS model can be used to provide customers with access to on-demand applications hosted by the cloud infrastructure system 1002. Examples of SaaS services provided by the Company include, but are not limited to, various services for human resource / capital management, customer relationship management (CRM), enterprise resource planning (ERP), supply chain management (SCM), enterprise performance management (EPM), analytical services, social applications, and others.

[0081] The IaaS model is often used to provide infrastructure resources (eg, servers, storage devices, hardware, and networking resources) to customers as cloud services to provide elastic computing and storage capabilities. The company provides a variety of IaaS services.

[0082] The PaaS model is typically used to provide platform and environment resources as a service, which enables customers to develop, run, and manage applications and services without having to purchase, build, or maintain such resources. Examples of PaaS services provided by the Company include, but are not limited to, Oracle Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), Data Management Cloud Service, various application development solution services, and other services.

[0083] Cloud services are typically provided on an on-demand self-service, subscription-based, elastically scalable, reliable, highly available, and secure basis. For example, a customer may order one or more services provided by the cloud infrastructure system 1002 via a subscription order. The cloud infrastructure system 1002 then performs processing to provide the services requested in the customer's subscription order. The cloud infrastructure system 1002 may be configured to provide one or even multiple cloud services.

[0084] The cloud infrastructure system 1002 can provide cloud services via different deployment models. In a public cloud model, the cloud infrastructure system 1002 can be owned by a third-party cloud service provider, and the cloud services are supplied to any general public customer, where the customer can be an individual or an enterprise. In certain other embodiments, under a private cloud model, the cloud infrastructure system 1002 can be operated within an organization (e.g., within an enterprise organization) and provide services to customers within the organization. For example, customers can be various departments of an enterprise, such as a human resources department, a payroll department, etc., or even individuals within an enterprise. In certain other embodiments, under a community cloud model, the cloud infrastructure system 1002 and the services provided can be shared by several organizations in a related community. Various other models, such as a hybrid of the models mentioned above, can also be used.

[0085] Client computing devices 1004, 1006, and 1008 may be of different types (such as Fig. 9 1002) and may be capable of operating one or more client applications. A user may use a client device to interact with the cloud infrastructure system 1002, such as requesting services provided by the cloud infrastructure system 1002. For example, a user may use a client device to request a data storage service described in the present disclosure.

[0086] In some embodiments, the processing performed by the cloud infrastructure system 1002 may include big data analysis. The analysis may include using, analyzing, and manipulating large data sets to detect and visualize various trends, behaviors, relationships, etc. within the data. Such analysis may be performed by one or more processors, may process data in parallel, perform simulations using the data, etc. The data used for such analysis may include structured data (e.g., data stored in a database or structured according to a structured model) and / or unstructured data (e.g., data blobs (binary large objects)).

[0087] As in Fig.10As depicted in the embodiment of FIG. 1 , cloud infrastructure system 1002 may include infrastructure resources 1030 for facilitating the provision of various cloud services provided by cloud infrastructure system 1002. Infrastructure resources 1030 may include, for example, processing resources, storage or memory resources, networking resources, and the like.

[0088] In some embodiments, to facilitate efficient provisioning of these resources to support the various cloud services provided by the cloud infrastructure system 1002 for different customers, resources may be bundled into collections of resources or resource modules (also referred to as "pods"). Each resource module or cluster may include a pre-integrated and optimized combination of one or more types of resources. In some embodiments, a different cluster may be provisioned for a database service, a second cluster collection may be provisioned for a Java service that may include a different combination of resources than the pods in the first cluster collection, and so on. For some services, resources allocated to provision the service may be shared between services.

[0089] The cloud infrastructure system 1002 itself may internally use services 1032 that are shared by different components of the cloud infrastructure system 1002 and facilitate the service provisioning of the cloud infrastructure system 1002. These internally shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, services for enabling cloud support, email services, notification services, file transfer services, and the like.

[0090] The cloud infrastructure system 1002 may include multiple subsystems. These subsystems may be implemented in software or hardware or a combination thereof. Fig.10 , the subsystems may include a user interface subsystem 1012 that enables users or customers of cloud infrastructure system 1002 to interact with cloud infrastructure system 1002. User interface subsystem 1012 may include a variety of different interfaces, such as a web interface 1014, an online store interface 1016 (where cloud services provided by cloud infrastructure system 1002 are advertised and available for purchase by customers), and other interfaces 1018. For example, a customer may access an online store, browse cloud services offered by cloud infrastructure system 1002, and place an order for one or more services offered by cloud infrastructure system 1002 that the customer wishes to subscribe to. A service request may include information identifying the customer and one or more services that the customer wishes to subscribe to.

[0091] In some embodiments, such as Fig.10In the embodiment shown in , cloud infrastructure system 1002 may include an order management subsystem (OMS) 1020 configured to process new orders. As part of this process, OMS 1020 may be configured to: create an account for a customer (if not already created); receive billing and / or accounting information from the customer, which will be used to bill the customer for providing the requested service to the customer; verify the customer information; after verification, book an order for the customer; and orchestrate various workflows to prepare the order for supply.

[0092] Once correctly verified, OMS1020 can call order provisioning subsystem (OPS) 1024, which is configured to provision resources for the order, including processing resources, memory resources, and networking resources. Provisioning can include allocating resources to the order, and configuring resources to facilitate the services requested by the customer order. The manner in which resources are provisioned for the order and the type of resources provided can depend on the type of cloud service that the customer has ordered. For example, according to a workflow, OPS1024 can be configured to determine the specific cloud service requested and identify multiple clusters that may have been pre-configured for the specific cloud service. The number of clusters allocated for the order can depend on the size / quantity / level / scope of the requested service. For example, the number of clusters to be allocated can be determined based on the number of users supported by the service, the duration of the service being requested, etc. The allocated clusters can then be customized for the specific requesting customer to provide the requested service.

[0093] Cloud infrastructure system 1002 can send a response or notification 1044 to the requesting customer to indicate when the requested service is ready. In some cases, information (e.g., a link) can be sent to the customer so that the customer can begin to use and utilize the benefits of the requested service.

[0094] The cloud infrastructure system 1002 can provide services to multiple customers. For each customer, the cloud infrastructure system 1002 is responsible for managing information related to one or more subscription orders received from the customer, maintaining customer data related to the order, and providing the requested services to the customer. The cloud infrastructure system 1002 can also collect usage statistics about the customer's use of the subscribed services. For example, statistics can be collected for the amount of storage used, the amount of data transferred, the number of users, and the amount of system uptime and system downtime. This usage information can be used to bill the customer. Billing can be performed, for example, on a monthly cycle.

[0095] The cloud infrastructure system 1002 may provide services to multiple customers in parallel. The cloud infrastructure system 1002 may store information of these customers, including potentially proprietary information. In certain embodiments, the cloud infrastructure system 1002 includes an identity management subsystem (IMS) 1028 that is configured to manage the information of the customers and provide separation of the managed information so that information associated with one customer is not accessible to another customer. The IMS 1028 may be configured to provide various security-related services, such as identity services, such as information access management, authentication and authorization services, services for managing customer identities and roles and related capabilities, and the like.

[0096] Fig.11 An exemplary computer system 1100 that can be used to implement certain embodiments is illustrated. For example, in some embodiments, the computer system 1100 can be used to implement any of the various servers and computer systems described above. Fig.11 As shown in , computer system 1100 includes various subsystems, including a processing subsystem 1104 that communicates with multiple other subsystems via a bus subsystem 1102. These other subsystems may include a processing acceleration unit 1106, an I / O subsystem 1108, a storage subsystem 1118, and a communication subsystem 1124. The storage subsystem 1118 may include non-transitory computer-readable storage media, including storage media 1122 and system memory 1110.

[0097] The bus subsystem 1102 provides a mechanism for enabling the various components and subsystems of the computer system 1100 to communicate with each other as desired. Although the bus subsystem 1102 is schematically shown as a single bus, alternative embodiments of the bus subsystem can utilize multiple buses. The bus subsystem 1102 can be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a local bus using any of a variety of bus architectures, etc. For example, such architectures can include an industrial standard architecture (ISA) bus, a microchannel architecture (MCA) bus, and a peripheral component interconnect (PCI) bus, which can be implemented as a mezzanine bus manufactured according to the IEEE P1386.1 standard, etc.

[0098] The processing subsystem 1104 controls the operation of the computer system 1100 and may include one or more processors, application specific integrated circuits (ASICs), or field programmable gate arrays (FPGAs). The processor may be a single-core or multi-core processor. The processing resources of the computer system 1100 may be organized into one or more processing units 1132, 1134, etc. A processing unit may include one or more processors, one or more cores from the same or different processors, a combination of cores and processors, or other combinations of cores and processors. In some embodiments, the processing subsystem 1104 may include one or more dedicated coprocessors, such as a graphics processor, a digital signal processor (DSP), etc. In some embodiments, some or all of the processing subsystem 1104 may be implemented using custom circuits, such as ASICs or FPGAs.

[0099] In some embodiments, the processing unit in the processing subsystem 1104 can execute instructions stored in the system memory 1110 or on the computer-readable storage medium 1122. In various embodiments, the processing unit can execute various program or code instructions, and can maintain multiple concurrently executed programs or processes. At any given time, some or all of the program code to be executed may reside in the system memory 1110 and / or on the computer-readable storage medium 1122, including possibly on one or more storage devices. Through appropriate programming, the processing subsystem 1104 can provide the various functions described above. In the case where the computer system 1100 is executing one or more virtual machines, one or more processing units can be assigned to each virtual machine.

[0100] In certain embodiments, a processing acceleration unit 1106 may optionally be provided for performing customized processing or for offloading some of the processing performed by the processing subsystem 1104 , thereby speeding up the overall processing performed by the computer system 1100 .

[0101] The I / O subsystem 1108 may include devices and mechanisms for inputting information to the computer system 1100 and / or for outputting information from or via the computer system 1100. In general, use of the term "input device" is intended to include all possible types of devices and mechanisms for inputting information to the computer system 1100. User interface input devices may include, for example, keyboards, pointing devices such as a mouse or trackball, touch pads or touch screens incorporated into a display, scroll wheels, click wheels, dials, buttons, switches, keypads, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may also include software such as Microsoft Windows that enables a user to control and interact with input devices. Motion sensing and / or gesture recognition devices, Microsoft 360 game controllers, devices that provide an interface for receiving input using gestures and spoken commands. User interface input devices may also include eye gesture recognition devices, such as detecting eye activity from a user (e.g., a "wink" when taking a picture and / or making a menu selection) and translating eye gestures to input devices (e.g., Google ) In addition, the user interface input device may include a user interface that enables the user to communicate with the voice recognition system (e.g., Navigator) interactive voice recognition sensing device.

[0102] Other examples of user interface input devices include, but are not limited to, three-dimensional (3D) mice, joysticks or indicator rods, game pads and graphic tablets, and audio / video devices such as speakers, digital cameras, digital video cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode reader 3D scanners, 3D printers, laser rangefinders, and eye tracking devices. In addition, user interface input devices may include, for example, medical imaging input devices such as computed tomography, magnetic resonance imaging, position emission tomography, and medical ultrasound examination equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, etc.

[0103] In general, the use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from the computer system 1100 to a user or other computers. User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices, etc. The display subsystem may be a cathode ray tube (CRT), a flat panel device such as one utilizing a liquid crystal display (LCD) or plasma display, a projection device, a touch screen, etc. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / visual information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0104] The storage subsystem 1118 provides a repository or data repository for storing information and data used by the computer system 1100. The storage subsystem 1118 provides a tangible, non-transitory computer-readable storage medium for storing basic programming and data constructs that provide the functionality of some embodiments. The storage subsystem 1118 can store software (e.g., programs, code modules, instructions) that, when executed by the processing subsystem 1104, provide the functionality described above. The software can be executed by one or more processing units of the processing subsystem 1104. The storage subsystem 1118 can also provide a repository for storing data used in accordance with the teachings of the present disclosure.

[0105] The storage subsystem 1118 may include one or more non-transitory memory devices, including volatile and non-volatile memory devices. Fig.11 As shown in , the storage subsystem 1118 includes a system memory 1110 and a computer-readable storage medium 1122. The system memory 1110 may include multiple memories, including a volatile main random access memory (RAM) for storing instructions and data during program execution and a non-volatile read-only memory (ROM) or flash memory in which fixed instructions are stored. In some embodiments, a basic input / output system (BIOS) may typically be stored in ROM, which contains basic routines that help to transfer information between elements within the computer system 1100, such as during startup. RAM typically contains data and / or program modules currently operated and executed by the processing subsystem 1104. In some embodiments, the system memory 1110 may include multiple different types of memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), etc.

[0106] By way of example and not limitation, Fig.11 As depicted in FIG. 1 , system memory 1110 may load application programs 1112 being executed, which may include various applications (such as a web browser, a middle-tier application, a relational database management system (RDBMS), etc.), program data 1114, and an operating system 1116. As an example, operating system 1116 may include various versions of Microsoft Apple and / or Linux operating system, various commercial or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, Google OS, etc.) and / or mobile operating systems such as iOS, Phone, OS, OS operating system, and other operating systems.

[0107] Computer-readable storage media 1122 may store programming and data structures that provide the functionality of some embodiments. Computer-readable storage media 1122 may provide storage of computer-readable instructions, data structures, program modules, and other data for computer system 1100. When executed by processing subsystem 1104, the software (programs, code modules, instructions) that provide the functionality described above may be stored in storage subsystem 1118. By way of example, computer-readable storage media 1122 may include non-volatile memory such as a hard drive, a disk drive, a hard drive such as a CD ROM, a DVD, a Blu-ray player, or a DVD player. (Blu-ray) disks or other optical media. Computer readable storage media 1122 may include, but are not limited to, Drive, flash memory card, universal serial bus (USB) flash drive, secure digital (SD) card, DVD disk, digital video tape, etc. The computer-readable storage medium 1122 may also include a non-volatile memory-based solid-state device (SSD) (such as a flash memory SSD, an enterprise flash drive, a solid-state ROM, etc.), a volatile memory-based SSD (such as a solid-state RAM, dynamic RAM, static RAM, DRAM-based SSD, magnetoresistive RAM (MRAM) SSD), and a hybrid SSD using a combination of DRAM-based and flash memory-based SSDs.

[0108] In certain embodiments, storage subsystem 1118 may also include a computer-readable storage medium reader 1120, which may also be connected to computer-readable storage medium 1122. Reader 1120 may receive and be configured to read memory from a memory device such as a disk, flash drive, or the like.

[0109] In some embodiments, computer system 1100 can support virtualization technology, including but not limited to virtualization of processing and memory resources. For example, computer system 1100 can provide support for executing one or more virtual machines. In some embodiments, computer system 1100 can execute programs such as hypervisors that facilitate the configuration and management of virtual machines. Memory, computing (e.g., processors, cores), I / O, and networking resources can be allocated to each virtual machine. Each virtual machine usually runs independently of other virtual machines. Virtual machines usually run their own operating system, which can be the same or different from the operating system executed by other virtual machines executed by computer system 1100. Therefore, computer system 1100 can potentially run multiple operating systems simultaneously.

[0110] The communication subsystem 1124 provides an interface to other computer systems and networks. The communication subsystem 1124 serves as an interface for receiving data from the computer system 1100 and transmitting data to other systems. For example, the communication subsystem 1124 can enable the computer system 1100 to establish a communication channel to one or more client devices via the Internet for receiving information from the client devices and sending information to the client devices.

[0111] The communication subsystem 1124 can support both wired and / or wireless communication protocols. For example, in some embodiments, the communication subsystem 1124 may include a radio frequency (RF) transceiver component, a global positioning system (GPS) receiver component, and / or other components for accessing a wireless voice and / or data network (e.g., using cellular telephone technology, advanced data network technology (such as 3G, 4G or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 802.XX family of standards), or other mobile communication technologies, or any combination thereof). In some embodiments, the communication subsystem 1124 may provide a wired network connection (e.g., Ethernet) in addition to or in lieu of a wireless interface.

[0112] The communication subsystem 1124 may receive and transmit data in a variety of forms. For example, in some embodiments, the communication subsystem 1124 may receive incoming communications in the form of structured and / or unstructured data feeds 1126, event streams 1128, event updates 1130, etc., among other forms. For example, the communication subsystem 1124 may be configured to receive, in real time, incoming communications from users of a social media network and / or a user of a website such as feed, The data feed 1126 is received (or sent) by other communication services such as web feeds, Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0113] In certain embodiments, the communication subsystem 1124 may be configured to receive data in the form of a continuous data stream that may be continuous or unbounded in nature and has no clear end, wherein the continuous data stream may include an event stream 1128 of real-time events and / or event updates 1130. Examples of applications that generate continuous data may include, for example, sensor data applications, financial quote machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automotive traffic monitoring, etc.

[0114] The communication subsystem 1124 may also be configured to transmit data from the computer system 1100 to other computer systems or networks. The data may be transmitted in a variety of different forms, such as structured and / or unstructured data feeds 1126, event streams 1128, event updates 1130, etc. to one or more databases that may communicate with one or more streaming data sources coupled to the computer system 1100.

[0115] Computer system 1100 may be one of various types, including a handheld portable device (e.g., Cellular phone, computing tablets, PDAs), wearable devices (e.g., Google head mounted display), personal computer, workstation, mainframe, kiosk, server rack, or any other data processing system. Due to the ever-changing nature of computers and networks, Fig.11 The description of the computer system 1100 depicted in FIG. 1 is intended to be a specific example only. Fig.11 Many other configurations of more or fewer components of the systems depicted are possible.

[0116] Although specific embodiments have been described, various modifications, variations, alternative constructions, and equivalents are possible. Embodiments are not limited to operations within certain specific data processing environments, but can be freely operated within multiple data processing environments. In addition, although certain embodiments have been described using a series of specific transactions and steps, this is not intended to be limiting. Although some flow charts describe operations as sequential processing, many operations can be performed in parallel or simultaneously. In addition, the order of operations can be rearranged. Processing may have other steps not included in the figure. Various features and aspects of the above-described embodiments can be used alone or in combination.

[0117] In addition, although certain embodiments have been described using a particular combination of hardware and software, it should be appreciated that other combinations of hardware and software are possible. Certain embodiments may be implemented using only hardware or only software or a combination thereof. The various processes described herein may be implemented on the same processor or on different processors in any combination.

[0118] Where a device, system, component or module is described as being configured to perform certain operations or functions, such configuration may be achieved, for example, by designing electronic circuits to perform the operations, by programming programmable electronic circuits (such as microprocessors) to perform the operations, such as by executing computer instructions or code, or a processor or core is programmed to execute code or instructions stored on a non-transitory storage medium, or any combination thereof to perform the operations. Processes may communicate using a variety of techniques, including but not limited to conventional techniques for inter-process communication, and different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0119] Specific details are given in this disclosure to provide a thorough understanding of the embodiments. However, embodiments may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques have been shown without unnecessary details to avoid obscuring the embodiments. This description provides only example embodiments and is not intended to limit the scope, applicability, or configuration of other embodiments. On the contrary, the previous description of the embodiments provides an enabling description for implementing various embodiments. Various changes may be made to the functions and arrangements of the elements.

[0120] Therefore, the description and drawings should be regarded as illustrative rather than restrictive. However, it will be apparent that additions, subtractions, deletions and other modifications and changes may be made thereto without departing from the broader spirit and scope set forth in the claims. Therefore, although specific embodiments have been described, these embodiments are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims.

Claims

1. A computer-implemented method comprising: receiving, from a client system, client data including information corresponding to a set of users and / or a set of user devices; Identify privacy protection protocols applicable to client data; Identifying a specific subset of the client data as personally identifiable information (PII) data, wherein the specific subset includes a set of PII attributes; Splitting the PII attributes into predefined categories based on the format of the data fields in the PII attributes; For each predefined category of the plurality of predefined categories: selecting a processing technique from a plurality of processing techniques based on the predefined categories and the privacy protection protocol; and processing the PII data corresponding to the predefined categories using the selected processing technique; combining the processed PII data corresponding to the plurality of predefined categories with the non-PII data to create processed client data; determining, based on prior input from the client system and / or based on the processed PII data, to add noise to at least a portion of the processed PII data; Determine the amount of noise based on a privacy-preserving protocol; adding the amount of noise to the at least a portion of the processed PII data to generate protected training data; as well as Train machine learning models using protected training data.

2. The computer-implemented method of claim 1 , wherein identifying a privacy-preserving protocol comprises: determining one or more geographic regions corresponding to the client system; as well as One or more data protection restrictions corresponding to the one or more geographic regions are retrieved.

3. The computer-implemented method of claim 1 , wherein identifying a privacy-preserving protocol comprises: An input is received from a client system, the input identifying a specification of a privacy protection protocol to be applied to the client system.

4. The computer-implemented method of claim 1 , wherein identifying a privacy-preserving protocol comprises: Receive input from a client system that identifies a degree to which data protection is to be prioritized compared to a performance metric of a machine learning model.

5. The computer-implemented method of claim 1 , wherein identifying a privacy-preserving protocol comprises: Input is received from a client system via an interface that presents a predicted data protection metric and a predicted performance metric of a machine learning model for each of a plurality of potential data protection policies, wherein the plurality of potential data protection policies include a privacy protection protocol.

6. A computer-implemented method as described in any of claims 1-5, wherein the plurality of predefined categories include categorical text data, categorical numeric data, high cardinality text, continuous numeric data, or text.

7. The computer-implemented method of any of claims 1-6, wherein the plurality of processing techniques comprises using another ML model, using a rule-based algorithm, or implementing pattern recognition techniques.

8. A system comprising: one or more data processors; as well as A non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform operations comprising: receiving, from a client system, client data including information corresponding to a set of users and / or a set of user devices; Identify privacy protection protocols applicable to client data; Identifying a specific subset of the client data as personally identifiable information (PII) data, wherein the specific subset includes a set of PII attributes; Splitting the PII attributes into predefined categories based on the format of the data fields in the PII attributes; For each predefined category of the plurality of predefined categories: selecting a processing technique from a plurality of processing techniques based on the predefined categories and the privacy protection protocol; and processing the PII data corresponding to the predefined categories using the selected processing technique; combining the processed PII data corresponding to the plurality of predefined categories with the non-PII data to create processed client data; determining, based on prior input from the client system and / or based on the processed PII data, to add noise to at least a portion of the processed PII data; Determine the amount of noise based on a privacy-preserving protocol; adding the amount of noise to the at least a portion of the processed PII data to generate protected training data; and Train machine learning models using protected training data.

9. The system of claim 8, wherein identifying the privacy protection protocol comprises: determining one or more geographic regions corresponding to the client system; as well as One or more data protection restrictions corresponding to the one or more geographic regions are retrieved.

10. The system of claim 8, wherein identifying the privacy protection protocol comprises: An input is received from a client system, the input identifying a specification of a privacy protection protocol to be applied to the client system.

11. The system of claim 8, wherein identifying a privacy protection protocol comprises: Receive input from a client system that identifies a degree to which data protection is to be prioritized compared to a performance metric of a machine learning model.

12. The system of claim 8, wherein identifying a privacy protection protocol comprises: Input is received from a client system via an interface that presents a predicted data protection metric and a predicted performance metric of a machine learning model for each of a plurality of potential data protection policies, wherein the plurality of potential data protection policies include a privacy protection protocol.

13. The system of any one of claims 8-12, wherein the plurality of predefined categories include categorical text data, categorical numeric data, high cardinality text, continuous numeric data, or text.

14. The system of any of claims 8-13, wherein the plurality of processing techniques comprises using another ML model, using a rule-based algorithm, or implementing a pattern recognition technique.

15. A computer program product tangibly embodied in a non-transitory computer-readable storage medium, comprising instructions configured to cause one or more data processors to perform operations comprising: receiving, from a client system, client data including information corresponding to a set of users and / or a set of user devices; Identify privacy protection protocols applicable to client data; Identifying a specific subset of the client data as personally identifiable information (PII) data, wherein the specific subset includes a set of PII attributes; Splitting the PII attributes into predefined categories based on the format of the data fields in the PII attributes; For each predefined category of the plurality of predefined categories: selecting a processing technique from a plurality of processing techniques based on the predefined categories and the privacy protection protocol; and processing the PII data corresponding to the predefined categories using the selected processing technique; combining the processed PII data corresponding to the plurality of predefined categories with the non-PII data to create processed client data; determining, based on prior input from the client system and / or based on the processed PII data, to add noise to at least a portion of the processed PII data; Determine the amount of noise based on a privacy-preserving protocol; adding the amount of noise to the at least a portion of the processed PII data to generate protected training data; as well as Train machine learning models using protected training data.

Images