Method and system for executing machine learning under data privacy protection

Abstract

The invention provides a method and a system for executing machine learning under data privacy protection. The method comprises the following steps: acquiring a target data set; a migration item about the source data set is obtained, and the migration item is used for migrating knowledge of the source data set to the target data set in a source data privacy protection mode so as to train a target machine learning model on the target data set; and training a target machine learning model based on a target data set in combination with the migration item in a target data privacy protection mode.

Description

[0001] This application is a divisional application of a patent application with an application date of July 10, 2019, application number 201910618274.X, and invention name “Method and system for performing machine learning under data privacy protection”. Technical Field

[0002] The present invention generally relates to data security technology in the field of artificial intelligence, and more specifically, to a method and system for performing machine learning under data privacy protection. Background Art

[0003] As we all know, machine learning often requires a large amount of data to mine valuable potential information from a large amount of data through computational means. Although the development of information technology has generated a huge amount of data, in the current environment, people pay more and more attention to data privacy protection. This means that even if there is a lot of data that can be used for machine learning in theory, different data sources (i.e., data owners, such as banks, insurance companies, hospitals, financial institutions, schools, government departments, etc.) are unwilling or unable to share their data directly with other data users in need due to their privacy protection of their own data. As a result, the data that can actually be used for machine learning may still be insufficient, which leads to the inability to effectively use machine learning to mine information that can create more value based on more relevant data. In addition, even if data containing privacy information has been obtained from other data sources or the institution itself has data containing privacy information, the machine learning model trained based on this data may still leak the privacy information of the data. In other words, it is still difficult to effectively use data from different sources to jointly mine value.

[0004] In view of this, there is a need for technology that can protect the privacy information in the data from being leaked and can effectively use data from different data sources for machine learning. Summary of the invention

[0005] According to an exemplary embodiment of the present disclosure, a method for performing machine learning under data privacy protection is provided, and the method may include: obtaining a target data set; obtaining migration items about a source data set, wherein the migration items are used to migrate knowledge of the source data set to the target data set in a source data set privacy protection manner to train a target machine learning model on the target data set; and training the target machine learning model based on the target data set in combination with the migration items in a target data set privacy protection manner.

[0006] Optionally, the step of obtaining migration items regarding the source data set may include: receiving the migration items from the outside; or, the step of obtaining migration items regarding the source data set may include: obtaining the source data set; performing machine learning-related processing based on the source data set in a source data privacy protection manner; and obtaining migration items regarding the source data set in the process of performing machine learning-related processing based on the source data set.

[0007] Optionally, the source data privacy protection method and / or the target data privacy protection method may be a protection method that complies with the definition of differential privacy.

[0008] Optionally, the migration item may relate to model parameters, objective functions and / or statistical information about the source data obtained in the process of performing machine learning related processing based on the source data set.

[0009] Optionally, the source data privacy protection method may be to add random noise during the process of performing machine learning related processing based on the source data set; and / or, the target data privacy protection method may be to add random noise during the process of training the target machine learning model.

[0010] Optionally, performing machine learning-related processing based on a source data set in a source data privacy protection manner may include: training a source machine learning model based on the source data set in a source data privacy protection manner.

[0011] Optionally, the source machine learning model and the target machine learning model may be based on the same type of machine learning model.

[0012] Optionally, in the source data privacy protection method, the objective function for training the source machine learning model can be constructed to include at least a loss function and a noise term; and / or, in the target data privacy protection method, the objective function for training the target machine learning model can be constructed to include at least a loss function and a noise term.

[0013] Optionally, the machine learning model of the same type may be a logistic regression model, and the migration item is a parameter of the source machine learning model.

[0014] Optionally, in a target data privacy protection mode, the step of training a target machine learning model based on a target data set and in combination with the migration item may include: constructing an objective function for training the target machine learning model to also reflect the difference between the parameters of the target machine learning model and the migration item; and training the target machine learning model based on the target data set by solving the constructed objective function.

[0015] Optionally, the target machine learning model can be used to execute business decisions, wherein the business decisions may involve at least one of transaction anti-fraud, account opening anti-fraud, intelligent marketing, intelligent recommendation, and loan evaluation.

[0016] Optionally, the target machine learning model is used in any of the following scenarios:

[0017] Image processing scenarios;

[0018] Speech recognition scenario;

[0019] Natural language processing scenarios;

[0020] Automatically control the scene;

[0021] Intelligent question-answering scenario;

[0022] Business decision scenarios;

[0023] Recommended business scenarios;

[0024] Search scene;

[0025] Abnormal behavior detection scenario.

[0026] Optionally,

[0027] The image processing scenarios include: optical character recognition (OCR), face recognition, object recognition and image classification;

[0028] The speech recognition scenarios include: products that can perform human-computer interaction through speech;

[0029] The natural speech processing scenarios include: reviewing text, identifying spam content, and classifying text;

[0030] The automatic control scenarios include: prediction of mine group regulation operation, prediction of wind turbine group regulation operation and prediction of air conditioning system regulation operation;

[0031] The intelligent question-answering scenarios include: chatbots and intelligent customer service;

[0032] The business decision-making scenarios include: financial technology, medical and municipal fields. The financial technology field includes: marketing and customer acquisition, anti-fraud, anti-money laundering, underwriting and credit scoring; the medical field includes: disease screening and prevention, personalized health management and auxiliary diagnosis; the municipal field includes: social governance and regulatory enforcement, resource environment and facility management, industrial development and economic analysis, public services and people's livelihood security and smart cities;

[0033] The recommendation business scenarios include: recommendations of news, advertisements, music, consulting, videos and financial products;

[0034] The search scenarios include: web page search, image search, text search, and video search;

[0035] The abnormal behavior detection scenarios include: abnormal behavior detection of power consumption by State Grid customers, network malicious traffic detection, and abnormal behavior detection in operation logs.

[0036] According to another exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions is provided, wherein, when the instructions are executed by at least one computing device, the at least one computing device is prompted to execute the method of performing machine learning under data privacy protection as described above.

[0037] According to another exemplary embodiment of the present disclosure, a system including at least one computing device and at least one storage device storing instructions is provided, wherein the instructions, when executed by the at least one computing device, prompt the at least one computing device to execute the method of performing machine learning under data privacy protection as described above.

[0038] According to another exemplary embodiment of the present disclosure, there is provided a system for performing machine learning under data privacy protection, the system may include: a target data set acquisition device, configured to acquire a target data set; a migration item acquisition device, configured to acquire migration items about a source data set, wherein the migration items are used to migrate knowledge of the source data set to the target data set in a source data set privacy protection manner to train a target machine learning model on the target data set; and a target machine learning model training device, configured to train a target machine learning model based on the target data set in combination with the migration items in a target data set privacy protection manner.

[0039] Optionally, the migration item acquisition device may be configured to receive the migration item from the outside, or the migration item acquisition device may be configured to acquire the migration item by: acquiring a source data set; performing machine learning-related processing based on the source data set in a source data privacy protection mode; and acquiring migration items about the source data set in the process of performing machine learning-related processing based on the source data set.

[0040] Optionally, the source data privacy protection method and / or the target data privacy protection method may be a protection method that complies with the definition of differential privacy.

[0041] Optionally, the migration item may relate to model parameters, objective functions and / or statistical information about the source data obtained in the process of performing machine learning-related processing based on the source data set.

[0042] Optionally, the source data privacy protection method may be to add random noise during the process of performing machine learning related processing based on the source data set; and / or, the target data privacy protection method may be to add random noise during the process of training the target machine learning model.

[0043] Optionally, the operation of performing machine learning-related processing based on the source data set in a source data privacy protection manner may include: training a source machine learning model based on the source data set in a source data privacy protection manner.

[0044] Optionally, the source machine learning model and the target machine learning model may be based on the same type of machine learning model.

[0045] Optionally, in the source data privacy protection method, the migration item acquisition device may construct the objective function for training the source machine learning model to include at least a loss function and a noise term; and / or, in the target data privacy protection method, the target machine learning model training device may construct the objective function for training the target machine learning model to include at least a loss function and a noise term.

[0046] Optionally, the same type of machine learning model may be a logistic regression model, and the migration term may be a parameter of a source machine learning model. Also, optionally, the target machine learning model training device may be configured to: construct an objective function for training the target machine learning model to also reflect the difference between the parameters of the target machine learning model and the migration term; and train the target machine learning model based on the target data set by solving the constructed objective function.

[0047] Optionally, the target machine learning model is used to execute business decisions, wherein the business decisions involve at least one of transaction anti-fraud, account opening anti-fraud, intelligent marketing, intelligent recommendation, and loan evaluation.

[0048] Optionally, the target machine learning model is used in any of the following scenarios:

[0049] Image processing scenarios;

[0050] Speech recognition scenario;

[0051] Natural language processing scenarios;

[0052] Automatically control the scene;

[0053] Intelligent question-answering scenario;

[0054] Business decision scenarios;

[0055] Recommended business scenarios;

[0056] Search scene;

[0057] Abnormal behavior detection scenario.

[0058] Optionally,

[0059] The image processing scenarios include: optical character recognition (OCR), face recognition, object recognition and image classification;

[0060] The speech recognition scenarios include: products that can perform human-computer interaction through speech;

[0061] The natural speech processing scenarios include: reviewing text, identifying spam content, and classifying text;

[0062] The automatic control scenarios include: prediction of mine group regulation operation, prediction of wind turbine group regulation operation and prediction of air conditioning system regulation operation;

[0063] The intelligent question-answering scenarios include: chatbots and intelligent customer service;

[0064] The business decision-making scenarios include: financial technology, medical and municipal fields. The financial technology field includes: marketing and customer acquisition, anti-fraud, anti-money laundering, underwriting and credit scoring; the medical field includes: disease screening and prevention, personalized health management and auxiliary diagnosis; the municipal field includes: social governance and regulatory enforcement, resource environment and facility management, industrial development and economic analysis, public services and people's livelihood security and smart cities;

[0065] The recommendation business scenarios include: recommendations of news, advertisements, music, consulting, videos and financial products;

[0066] The search scenarios include: web page search, image search, text search, and video search;

[0067] The abnormal behavior detection scenarios include: abnormal power consumption behavior detection of State Grid customers, network malicious traffic detection and abnormal behavior detection in operation logs.

[0068] The method and system for performing machine learning under data privacy protection according to the exemplary embodiments of the present disclosure can not only achieve privacy protection for source data and target data, but also can migrate the knowledge in the source data set to the target data set, thereby being able to train a target machine learning model with better model effect based on the target data set and combined with the migrated knowledge. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] These and / or other aspects and advantages of the present invention will become more clear and easier to understand from the following detailed description of the embodiments of the present invention in conjunction with the accompanying drawings, in which:

[0070] Figure 1is a block diagram illustrating a system for performing machine learning under data privacy protection according to an exemplary embodiment of the present disclosure;

[0071] Figure 2 is a flowchart illustrating a method for performing machine learning in a data privacy protection manner according to an exemplary embodiment of the present disclosure;

[0072] Figure 3 is a schematic diagram illustrating a concept of performing machine learning in a data privacy protection manner according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0073] In order to enable those skilled in the art to better understand the present invention, exemplary embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0074] Figure 1 1 is a block diagram showing a system 100 for performing machine learning under data privacy protection according to an exemplary embodiment of the present disclosure (hereinafter, for convenience of description, it is referred to as a “machine learning system” for short). Figure 1 , the machine learning system 100 may include a target data set acquisition device 110, a migration item acquisition device 120 and a target machine learning model training device 130.

[0075] Specifically, the target data set acquisition device 110 can acquire a target data set. Here, the target data set can be any data set that can be used for training the target machine learning model, and can include multiple target data records and / or the results of target data records after various data processing or feature processing. In addition, optionally, the target data set can also include a label of the target data record about the machine learning target. For example, the target data record may include at least one attribute field reflecting various attributes of an object or event (for example, user ID, age, gender, historical credit record, etc.), and the label of the target data record about the machine learning target can be, for example, whether the user is able to repay the loan, whether the user accepts the recommended content, etc., but is not limited to this. In addition, the target data set may involve various personal privacy information that the user does not want to be known by others (for example, the user's name, ID number, mobile phone number, total property, loan record, etc.), and may also include group-related information that does not involve personal privacy. Here, the target data record can come from different data sources (for example, network operators, banking institutions, medical institutions, etc.), and the target data set can be used by specific institutions or organizations with the authorization of the user, but the user often expects that the information involving personal privacy will no longer be further known by other organizations or individuals. It should be noted that in this disclosure, "privacy" may refer to any attribute related to a single individual.

[0076] As an example, the target data set acquisition device 110 may acquire the target data set from the target data source at one time or in batches, and may acquire the target data set manually, automatically or semi-automatically. In addition, the target data set acquisition device 110 may acquire the target data record and / or the mark about the target data record in the target data set in real time or offline, and the target data set acquisition device 110 may acquire the target data record and the mark about the target data record at the same time, or the time of acquiring the mark about the target data record may lag behind the time of acquiring the target data record. In addition, the target data set acquisition device 110 may acquire the target data set from the target data source in an encrypted form or directly use the target data set that has been stored locally. If the acquired target data set is encrypted data, then optionally, the machine learning system 100 may also include a device for decrypting the target data, and may also include a data processing device to process the target data into a form suitable for the current machine learning. It should be noted that the present disclosure has no restrictions on the type, form, content, acquisition method of the target data set, etc. of the target data record and its mark in the target data set, and any data that can be used for machine learning obtained by any means can be used as the target data set mentioned above.

[0077] However, as described in the background technology of this disclosure, for machine learning that expects to mine more valuable information, in practice, only based on the acquired target data set may not be enough to learn a machine learning model that meets the actual task requirements or achieves the predetermined effect. Therefore, efforts can be made to obtain relevant information from other data sources to migrate the knowledge from other data sources to the target data set, thereby combining the target data set with the knowledge from other data sources to jointly perform machine learning, thereby improving the effect of the machine learning model. However, the premise of migration is to ensure that: the privacy information involved in the data set of other data sources (in this disclosure, it can be referred to as the "source data set") is not leaked, that is, the source data needs to be protected for privacy.

[0078] To this end, the migration item acquisition device 120 can acquire migration items about the source data set. Here, the migration items can be used to migrate the knowledge of the source data set to the target data set in a source data privacy protection manner to train the target machine learning model on the target data set. Specifically, the migration item can be any information related to the knowledge contained in the source data set obtained when the source data is privacy protected (i.e., in a source data privacy protection manner). The present disclosure does not limit the specific content and form of the migration item, as long as it can migrate the knowledge of the source data set to the target data set in a source data privacy protection manner. For example, the migration item may involve samples of the source data set, features of the source data set, models obtained based on the source data set, objective functions for model training, statistical information about the source data, etc.

[0079] As an example, the migration item acquisition device 120 may receive migration items about the source data set from the outside. For example, the migration item acquisition device 120 may obtain the migration items from an entity that owns the source data set, or an entity that is authorized to perform relevant processing on the source data source (for example, a service provider that provides machine learning related services). In this case, the migration items may be obtained by the entity that owns the source data set or the entity that is authorized to perform relevant processing on the source data source by performing machine learning related processing based on the source data set, and these entities may send the obtained migration items to the migration item acquisition device 120. Here, according to an exemplary embodiment of the present invention, the prediction target for performing machine learning related processing based on the source data set and the prediction target for the target machine learning model on the target data set may be the same target (for example, both predict whether a transaction is a fraudulent transaction) or related targets (for example, classification problems with a certain degree of similarity, such as predicting whether a transaction is a fraudulent transaction and predicting whether a transaction is suspected of being illegal).

[0080] Different from directly acquiring migration items from the outside, optionally, the migration item acquisition device 120 can also acquire migration items about the source data set by performing machine learning related processing on the source data set. Here, the acquisition and use of the source data set by the migration item acquisition device 120 can be authorized or protected, so that it can perform corresponding processing on the acquired source data set. Specifically, the migration item acquisition device 120 can first acquire the source data set. Here, the source data set can be any data set related to the target data set. Accordingly, the above descriptions of the composition of the target data set, the acquisition method of the target data set, etc. are all applicable to the source data set, and will not be repeated here. In addition, although for the convenience of description, the source data set is described as being acquired by the migration item acquisition device 120, it should be noted that the operation of acquiring the source data set can also be performed by the target data set acquisition device 110, or the source data set can be acquired by the above two together, and the present disclosure does not limit this. In addition, the acquired target data set, source data set and migration items can all be stored in a storage device (not shown) of the machine learning system. As an optional method, the above stored target data, source data or migration items can be physically or access rights isolated to ensure the safe use of the data.

[0081] When the source data set is obtained, for privacy protection considerations, the machine learning system 100 cannot directly use the obtained source data set together with the target data set for machine learning, but can only use it for machine learning when the source data is guaranteed to be privacy protected. To this end, the migration item acquisition device 120 can perform machine learning-related processing based on the source data set in a source data privacy protection mode, and obtain migration items about the source data set in the process of performing machine learning-related processing based on the source data set. According to an exemplary embodiment, the source data privacy protection mode can be a protection mode that complies with the definition of differential privacy, but is not limited to this, and can be any privacy protection mode that already exists or may appear in the future and can protect the privacy of the source data.

[0082] For ease of understanding, we now briefly describe the protection method that follows the definition of differential privacy. Assume that there is a random mechanism M (for example, M can be a machine learning model). For M, any two input data sets that differ by only one sample are and The probability that the output of is equal to t is and And if the following equation 1 is satisfied (where ∈ is a privacy protection constant or privacy budget), then it can be considered that M satisfies ∈ differential privacy protection for any input.

[0083]

[0084] In the above equation 1, the smaller ∈ is, the better the privacy protection is, and vice versa. The specific value of ∈ can be set accordingly according to the user's requirements for the degree of data privacy protection. Suppose there is a user, for whom it is a question of whether to input his personal data to the mechanism M (assuming that the data set before the personal data is input is The data set after the personal data is entered is and Only the personal data is different), and the impact on the output is small (where the impact is defined by the size of ∈), then it can be considered that M has played a role in protecting his privacy. Assuming ∈=0, whether the user inputs his own data to M has no effect on the output of M, so the user's privacy is completely protected.

[0085] According to an exemplary embodiment, the source data protection method may be to add random noise in the process of performing machine learning-related processing based on the source data set. For example, random noise may be added to comply with the above differential privacy protection definition. However, it should be noted that the definition of privacy protection is not limited to the differential privacy protection definition, but may be other privacy protection definitions such as k-anonymization, I diversity, t-closeness, etc.

[0086] As described above, the migration item can be any information related to the knowledge contained in the source data set obtained in a source data privacy protection manner. Specifically, according to an exemplary embodiment of the present disclosure, the migration item may involve model parameters, objective functions and / or statistical information about the source data obtained in the process of performing machine learning-related processing based on the source data set, but is not limited to this. As an example, the operation of performing machine learning-related processing based on the source data set may include: training a source machine learning model based on the source data set in a source data privacy protection manner, but is not limited to this, and may also include, for example, performing machine learning-related processing such as feature processing or data statistical analysis on the source data set. In addition, it should be noted that the above-mentioned model parameters, objective functions and / or statistical information about the source data can be both the above-mentioned information itself directly obtained in the process of performing machine learning-related processing based on the source data, and can also be information obtained after further transformation or processing of this information, and the present disclosure is not limited to this.

[0087] As an example, the migration item involving model parameters may be the parameters of the source machine learning model, for example, the model parameters of the source machine learning model obtained in the process of training the source machine learning model under the source data protection method that satisfies the definition of differential privacy protection. In addition, it may also be, for example, statistical information of the parameters of the source machine learning model, etc., but is not limited to this. As an example, the objective function involved in the migration item may refer to the objective function constructed for training the source machine learning model. When the parameters of the source machine learning model itself are not migrated, the objective function may not be actually solved separately, but the present disclosure is not limited to this. As an example, the migration item involving statistical information about the source data may be data distribution information and / or data distribution change information about the source data obtained under the source data privacy protection method (for example, a protection method that satisfies the definition of differential privacy protection), but is not limited to this.

[0088] As described above, the migration item acquisition device 120 can train the source machine learning model based on the source data set in the source data privacy protection mode. According to an exemplary embodiment, the source machine learning model can be, for example, a generalized linear model, such as a logistic regression model, but is not limited thereto. In addition, in the source data privacy protection mode, the migration item acquisition device 120 can construct the objective function for training the source machine learning model to include at least a loss function and a noise term. Here, the noise term can be used to add random noise in the process of training the source machine learning model, so that the privacy protection of the source data can be achieved. In addition, in addition to being constructed to include a loss function and a noise term, the objective function for training the source machine learning model can also be constructed to include other constraint terms for constraining model parameters. For example, it can also be constructed to include regular terms for preventing model overfitting or preventing model parameters from being too complex, compensation terms for privacy protection, and the like.

[0089] In order to more intuitively understand the process of training the source machine learning model based on the source data set to obtain the migration items about the source data set under the source data privacy protection mode, the process will be further explained in combination with mathematical representation. For the convenience of description, it is assumed here that the source data privacy protection mode is a protection mode that follows the definition of differential privacy, and the source machine learning model is a generalized linear model.

[0090] Specifically, assuming that the source dataset Among them, x i is a sample, yi is the label of the sample, Where n is the number of samples, d is the dimension of the sample space, is a d-dimensional sample space, then based on the source data set The source machine learning model is trained using the following equation 2 to obtain migration terms about the source data set (in this exemplary embodiment, the parameters of the source machine learning model) that satisfy differential privacy protection.

[0091] Specifically, before using Equation 2 to solve the parameters of the source machine learning model, we can set:

[0092] 1. Source dataset Scale so that for any i, ||x i ||≤1, where ||x i || represents x i The second norm of ;

[0093] 2. Where c and λ are constants, ∈ is the privacy protection constant in equation 1 above;

[0094] 3. If ∈′>0, then Δ=0, otherwise, And ∈′=∈ / 2;

[0095] 4. From the density function Sampling b, specifically, for example, can first be from the Gamma distribution Sample the bi-norm of b ||b||, and then based on the uniform random sampling direction u of b, we can obtain b=||b||u.

[0096] Next, we can use Equation 2 to train the source machine learning model based on the source data set in a source data privacy protection manner. Equation 2 is as follows:

[0097]

[0098] In Equation 2, w is the parameter of the source machine learning model, l(w T x i ,y i ) is the loss function, g(w) is the regularization function, is a noise term used to add random noise in the process of training the source machine learning model to achieve source data privacy protection. is a compensation term for privacy protection, λ is a constant used to control the strength of regularization, is the objective function constructed for training the source machine learning model. According to the above equation 2, the value of w when the value of the objective function is the minimum is the parameter w* of the source machine learning model finally solved.

[0099] In order for w* solved by equation 2 above to satisfy the definition of ∈ differential privacy, the following predetermined conditions need to be met: the regularization function g(w) needs to be a 1-strongly convex function and second-order differentiable. Secondly, for all z, the loss function needs to satisfy |l′(z)|≤1 and |l″(z)|≤c, where l′(z) and l″(z) are the first and second derivatives of the loss function, respectively. In other words, as long as it is a generalized linear model that satisfies the above conditions, the parameters of the source data machine model that satisfies differential privacy protection can be obtained through equation 2 above.

[0100] For example, for the logistic regression model, its loss function is If we set the constant c equal to 1 / 4, the regularization function Then the regularization function g(w) satisfies that it is a 1-strongly convex function and is second-order differentiable, and for all z, the loss function satisfies |l′(z)|≤1 and |l″(z)|≤c. Therefore, when the source machine learning model is a logistic regression model, the above equation 2 can be used to solve the parameters of the source machine learning model, and the parameters of the source machine learning model solved in the above manner not only satisfy the privacy protection of the source data, but also carry the knowledge of the source data set. Subsequently, the parameters of the source machine learning model can be used as migration items to migrate the knowledge of the source data set to the target data set to train the target machine learning model on the target data set.

[0101] It should be noted that although the above process of solving the parameters of the source machine learning model is introduced by taking a generalized linear model (for example, a logistic regression model) as an example, in fact, as long as the linear model satisfies the above-mentioned restrictions on the regularization function and the loss function, Equation 2 can be used to solve the parameters of the source machine learning model as a migration term.

[0102] In an embodiment of the invention, the source data and the target data may be data from any one or more of the following entities:

[0103] Data from banks: such as user registration information, bank transaction flow information, deposit information, financial product purchase information, bill information (images), etc.;

[0104] Data from insurance institutions: such as policyholder information, policy information, insurance claims information, etc.;

[0105] Data from medical institutions: such as medical records, diagnosis information, treatment information, etc.;

[0106] Data from other financial institutions such as securities companies; such as user registration information, financial product transaction information, financial product price fluctuation information, etc.;

[0107] Data from schools: such as student source information, enrollment rate, employment rate, teaching information, teacher information, etc.;

[0108] Data from government departments: such as social security information, human resources information, municipal resource information, municipal project-related information, finance-related information, education-related information, etc.;

[0109] Data from Internet entities: such as user registration information from e-commerce platforms or app operating entities, user network behavior (search, browse, favorite, purchase, click, payment, etc.) information, or online video, audio, image, text and other related data from search engines;

[0110] Data from telecom operators: such as mobile user communication data, fixed network or mobile network traffic-related data, etc.;

[0111] Data from traditional industrial enterprises: industrial control data such as power grid-related operation data, wind turbine control data, air-conditioning system control data, mine group control data, etc.

[0112] In terms of type, the source data and target data involved in the embodiments of the present invention may be video data, image data, voice data, text data, formatted form data, etc.

[0113] After the migration item acquisition device 120 acquires the migration item, the target machine learning model training device 130 can train the target machine learning model based on the target data set and in combination with the migration item in a target data privacy protection manner.

[0114] In an embodiment of the present invention, the target machine learning model can be applied to any of the following scenarios:

[0115] Image processing scenarios include: optical character recognition (OCR), face recognition, object recognition and image classification. For example, OCR can be applied to bill (such as invoice) recognition, handwriting recognition, etc., face recognition can be applied to security and other fields, object recognition can be applied to traffic sign recognition in autonomous driving scenarios, and image classification can be applied to "take a photo and buy" and "find the same item" on e-commerce platforms.

[0116] Voice recognition scenarios, including products that enable human-computer interaction through voice, such as mobile phone voice assistants (such as Siri on Apple phones) and smart speakers;

[0117] Natural language processing scenarios, including: reviewing text (such as contracts, legal documents, and customer service records), identifying spam content (such as identifying spam text messages), and classifying text (emotion, intent, and topic).

[0118] Automatic control scenarios include: prediction of adjustment operations of mine groups, wind turbine groups and air conditioning systems; specifically, for mine groups, a group of adjustment operations with high mining rates can be predicted; for wind turbine groups, a group of adjustment operations with high power generation efficiency can be predicted; for air conditioning systems, a group of adjustment operations that can meet demand while saving energy can be predicted;

[0119] Intelligent question-and-answer scenarios, including chatbots and intelligent customer service;

[0120] Business decision-making scenarios, including: scenarios in the fields of financial technology, medical care, and municipal administration, among which:

[0121] The financial technology field includes: marketing (such as coupon usage prediction, ad click behavior prediction, user portrait mining, etc.) and customer acquisition, anti-fraud, anti-money laundering, underwriting and credit scoring, and commodity price prediction;

[0122] Medical fields include: disease screening and prevention, personalized health management and auxiliary diagnosis;

[0123] The municipal sector includes: social governance and regulatory enforcement, resource environment and facility management, industrial development and economic analysis, public services and livelihood security, and smart cities (deployment and management of various urban resources such as public transportation, online car-hailing, and shared bicycles);

[0124] Recommended business scenarios, including: recommendations of news, advertising, music, consulting, videos, and financial products (such as financial management, insurance, etc.);

[0125] Search scenarios include: web search, image search, text search, video search, etc.;

[0126] Abnormal behavior detection scenarios include: abnormal behavior detection of State Grid customers’ electricity consumption, network malicious traffic detection, abnormal behavior detection in operation logs, etc.

[0127] According to an exemplary embodiment, the target data privacy protection method may be the same as the source data privacy protection method, for example, it may also be a protection method that complies with the definition of differential privacy, but is not limited to this. In addition, the target machine learning model may be a machine learning model based on the same type as the source machine learning model. For example, the target machine learning model may also be a generalized linear model, such as a logistic regression model, but is not limited to this. For example, it may be any linear model that meets predetermined conditions. It should be noted that the target data privacy protection method may also be a privacy protection method different from the source data privacy protection method, and the target machine learning model may also be a machine learning model of a different type from the source machine learning model, and the present application is not limited to this.

[0128] According to an exemplary embodiment, the target data privacy protection method may be to add random noise in the process of training the target machine learning model. For example, the target machine learning model training device 120 may construct the objective function for training the target machine learning model to include at least a loss function and a noise term. Optionally, in addition to constructing the objective function to include at least a loss function and a noise term, when training the target machine learning model based on the target data set combined with the migration term under the target data privacy protection method, the target machine learning model training device 130 may construct the objective function for training the target machine learning model to also reflect the difference between the parameters of the target machine learning model and the migration term, and then, based on the target data set, the target machine learning model may be trained by solving the constructed objective function. By reflecting the difference between the parameters of the target machine learning model and the migration term in the objective function for training the target machine learning model, the knowledge in the source data set can be migrated to the target data set, so that the training process can jointly utilize the knowledge on the source data set and the target data set, so that the trained target machine learning model has a better effect.

[0129] In addition, according to actual needs, the objective function can also be constructed to include regularization terms for preventing overfitting of the trained machine learning model, or can also be constructed to include other constraints according to actual task requirements, such as compensation terms for privacy protection. This application does not limit this, as long as the constructed objective function can effectively achieve privacy protection of the target data and can migrate the knowledge on the source data set to the target data set.

[0130] Below, in order to facilitate a more intuitive understanding of the above content, the above process of training the target machine learning model by the target machine learning model training device 130 will be further explained in combination with mathematical representation.

[0131] Here, for the convenience of description, it is assumed that the source machine learning model is a logistic regression model, the target machine learning model is a generalized linear model, and the target data privacy protection method is a protection method that complies with the definition of differential privacy protection.

[0132] First, let the regularization function of the source machine learning model be In the case of , the parameters of the source machine learning model can be solved using the process described above to solve the parameters of the source machine learning model (Here That is, w* in equation 2 above), where A1 is the solution mechanism as described in equation 2 above, ∈ s ,λ s , g s(·) are the source dataset, the privacy protection constant that the source dataset needs to meet, the constant used to control the regularization strength in the objective function for training the source machine learning model, and the regularization function. After obtaining the parameters of the source machine learning model, the regularization function in the objective function of the target machine learning model can be:

[0133]

[0134] Among them, 0≤η≤1. Since g t (w) is a 1-strongly convex function and second-order differentiable, and the loss function l(w T x i ,y i ) satisfies the requirements of the loss function in the above predetermined conditions. Therefore, g(w) in Equation 2 can be replaced by g t (w), and according to the process of training the source machine learning model described above, use equation 2 to train the target machine learning model based on the target data set in a manner that satisfies the definition of differential privacy protection, so as to solve the parameters of the target machine learning model when the objective function used for training the target machine learning model takes the minimum value in, ∈ t ,λ t , g t (·) are the target dataset, the privacy protection constant that the target dataset needs to meet, the constant controlling the regularization strength in the objective function used to train the target machine learning model, and the regularization function.

[0135] In addition, in Equation 3, due to the inclusion of The objective function used for training the target machine learning model is constructed to reflect the difference between the parameters of the target machine learning model and the migration terms (i.e., the parameters of the source machine learning model), thereby effectively realizing the migration of knowledge on the source data set to the target data set.

[0136] It should be noted that although the above mainly uses the logistic regression model as an example to introduce the process of training the target machine learning model under the target data privacy protection method, those skilled in the art should be aware that the source machine learning model and the target machine learning model in the present disclosure are not limited to the logistic regression model, but can be, for example, any linear model that meets the predetermined conditions as described above, or even any other appropriate model.

[0137] According to an exemplary embodiment, the trained target machine learning model can be used to execute business decisions, wherein the business decisions involve at least one of transaction anti-fraud, account opening anti-fraud, intelligent marketing, intelligent recommendation, and loan evaluation, but are not limited thereto. For example, the trained target machine learning model can also be used for business decisions related to physiological conditions, etc.

[0138] According to the above exemplary embodiments, the target machine learning model training device 130 can successfully migrate the knowledge in the source data set to the target data set while protecting the privacy of both the source data and the target data, thereby making it possible to integrate more knowledge to train a target machine learning model with better model effects for application to corresponding business decisions.

[0139] The above has been referred to Figure 1 The machine learning system 100 according to the exemplary embodiment of the present application is described. It should be noted that although the machine learning system is divided into devices for performing corresponding processing respectively (for example, the target data set acquisition device 110, the migration item acquisition device 120 and the target machine learning model training device 130) when describing the machine learning system above, it is clear to those skilled in the art that the processing performed by the above-mentioned devices can also be performed without any specific device division in the machine learning system or without clear demarcation between the devices. In addition, the above reference Figure 1 The described machine learning system 100 is not limited to including the devices described above, but some other devices (e.g., prediction devices, storage devices and / or model updating devices, etc.) may be added as needed, or the above devices may also be combined.

[0140] In addition, it should be noted that the "machine learning" mentioned in the present disclosure can be implemented in the form of "supervised learning", "unsupervised learning" or "semi-supervised learning", and the exemplary embodiments of the present invention do not impose specific restrictions on the specific form of machine learning.

[0141] Figure 2 1 is a flowchart illustrating a method for performing machine learning in a data privacy protection manner according to an exemplary embodiment of the present disclosure (hereinafter, for convenience of description, it will be referred to as “machine learning method”).

[0142] Here, as an example, Figure 2 The machine learning method shown can be Figure 1 The machine learning system 100 shown in the figure may be implemented in software by computer programs or instructions, or may be implemented by a computing system or computing device with a specific configuration. Figure 2 The method shown is Figure 1 The machine learning system 100 shown in FIG. 1 is used to perform the above operation and assumes that the machine learning system 100 may have Figure 1 Configuration shown.

[0143] Reference Figure 2 In step S210, the target data set acquisition device 110 may acquire the target data set. Figure 1 Any content related to obtaining the target data set described when describing the target data set obtaining device 110 is applicable here, and therefore, it will not be described in detail here.

[0144] After acquiring the target data set, in step S220, the migration item acquisition device 120 may acquire migration items about the source data set. Here, the migration items can be used to migrate the knowledge of the source data set to the target data set in a source data privacy protection manner to train the target machine learning model on the target data set. Specifically, in step S220, the migration item acquisition device 120 may receive the migration items from the outside. Alternatively, the migration item acquisition device 120 may acquire the migration items about the source data set by performing machine learning related processing on the source data set by itself. Specifically, the migration item acquisition device 120 may first acquire the source data set, and then, in a source data privacy protection manner, perform processing related to machine learning based on the source data set, and acquire migration items about the source data set in the process of performing processing related to machine learning based on the source data set.

[0145] Here, as an example, the source data privacy protection method may be a protection method that complies with the definition of differential privacy protection, but is not limited thereto. In addition, the source data privacy protection method may be to add random noise in the process of performing machine learning-related processing based on the source data set to achieve privacy protection of the source data. Here, performing machine learning-related processing based on the source data set may include training a source machine learning model based on the source data set under the source data privacy protection method, but is not limited thereto. For example, it may also be statistical analysis or feature processing of the source data set under the source data privacy protection method. According to an exemplary embodiment, in the source data privacy protection method, the objective function for training the source machine learning model may be constructed to include at least a loss function and a noise term. Here, the noise term is used to add random noise in the process of training the source machine learning model, thereby achieving privacy protection of the source data. In addition, optionally, in the source data privacy protection method, the objective function may also be constructed to include other constraint terms for constraining model parameters.

[0146] According to an exemplary embodiment, the migration item may involve model parameters, objective functions and / or statistical information about the source data obtained in the process of performing machine learning-related processing based on the source data set. As an example, the migration item may be a parameter of the source machine learning model, that is, a parameter of the source machine learning model trained in a source data privacy protection manner. According to an exemplary embodiment, the source machine learning model may be a generalized linear model (e.g., a logistic regression model), but is not limited thereto, and for example, may be any linear model that meets predetermined conditions, or even any appropriate model that meets certain conditions.

[0147] Since the above reference Figure 1 The process of the migration item acquisition device 120 training the source machine learning model based on the source data set to obtain the migration item (i.e., the parameters of the source machine learning model) in the source data privacy protection mode is described in combination with mathematical representation, so it is not repeated here. Figure 1 All descriptions of the source data set, source data privacy protection method, migration item, objective function, etc. mentioned in the description of the migration item acquisition device 120 are applicable to Figure 2 Therefore, it will not be described in detail here, and the same or similar contents can be referenced to each other when describing the migration item obtaining device 120 and step S220.

[0148] After the target data set and the migration items about the source data set are obtained, in step S230, the target machine learning model training device 130 can train the target machine learning model based on the target data set and the migration items in the target data privacy protection mode. Here, as an example, the target data privacy protection mode can also be a protection mode that follows the definition of differential privacy, but is not limited to this, and can be other data privacy protection modes that are the same or different from the source data privacy protection mode. In addition, the target data privacy protection mode can be to add random noise in the process of training the target machine learning model to achieve privacy protection of the target data. Specifically, for example, in the target data privacy protection mode, the objective function for training the target machine learning model can be constructed to include at least a loss function and a noise term, but is not limited to this. For example, the objective function can be constructed to also include other constraint terms for constraining the model, for example, regular terms for limiting the complexity of model parameters or preventing model overfitting, compensation terms for privacy protection, etc. In addition, the target machine learning model can belong to the same type of machine learning model as the source machine learning model. For example, the same type of machine learning model can be a logistic regression, but is not limited to this, and can be, for example, any linear model that meets predetermined conditions. It should be noted that the target machine learning model may also be a machine learning model of a different type from the source machine learning model.

[0149] In addition to constructing the objective function for training the target machine learning model to include at least a loss function and a noise term in the target data privacy protection method, according to an exemplary embodiment, in step S230, the target machine learning model training device 130 may construct the objective function for training the target machine learning model to also reflect the difference between the parameters of the target machine learning model and the migration term, and then, based on the target data set, the target machine learning model may be trained by solving the constructed objective function. The specific process of training the target machine learning model using the constructed objective function has been referred to above. Figure 1 This has been described in conjunction with mathematical representation, so it will not be repeated here.

[0150] The target machine learning model trained in the above manner can be used to execute business decisions. For example, the business decision may involve at least one of transaction anti-fraud, account opening anti-fraud, smart marketing, smart recommendation, and loan evaluation, but is not limited thereto. In fact, the present disclosure does not impose any restrictions on the types of specific business decisions to which the target machine learning model can be applied, as long as it is a business suitable for making decisions using a machine learning model.

[0151] The above-described method of performing machine learning in a data privacy protection manner according to an exemplary embodiment of the present disclosure can not only ensure that the privacy of source data and target data is not leaked, but also can migrate the knowledge of source data to the target data set through migration items, thereby facilitating the use of data from more data sources for machine learning to train the machine learning model, so that the effect of the trained target machine learning model can have a better model effect.

[0152] It should be noted that although the above description Figure 2 When Figure 2 The steps in the method are described, however, it is clear to those skilled in the art that the various steps in the above method are not necessarily executed in sequence, but may be executed in the reverse order or in parallel. For example, the steps S210 and S220 described above may be executed in the reverse order or in parallel, that is, the migration items about the source data set may be acquired before the target data set is acquired, or the target data set and the migration items may be acquired at the same time. In addition, while executing step S130, step S110 or step 120 may also be executed, that is, in the process of training the target machine learning model using the target data set and migration items that have been acquired, a new target data set or migration items may be acquired at the same time, for example, for subsequent target machine learning model update operations, etc.

[0153] Figure 3 is a schematic diagram illustrating a concept of performing machine learning in a data privacy protection manner according to an exemplary embodiment of the present disclosure.

[0154] In order to facilitate a clearer and more intuitive understanding of the concept of the present disclosure, the following Figure 3 Taking the loan review scenario in the financial field as an example (i.e., the target machine learning model will be used for the business decision of loan review), the concept of performing machine learning under data privacy protection according to an exemplary embodiment of the present disclosure is briefly described.

[0155] Nowadays, with the continuous development of machine learning, it has begun to play an increasingly important role in the financial field. From loan approval to asset management, to risk assessment, machine learning plays an indispensable role in many stages of the financial ecosystem. For example, banks can use machine learning to decide whether to approve a loan applicant's loan application. However, the historical financial activity records of the loan applicant that a single bank can obtain may not be sufficient to fully reflect the real credit or loan repayment ability of the loan applicant. In this case, the bank may expect to obtain the historical financial activity records of the loan applicant in other institutions. However, for the sake of customer privacy protection, it is difficult for the bank to use the historical financial activity records of the loan applicant owned by other institutions. However, the concept of the present disclosure can make full use of the data of multiple institutions to help banks more accurately determine whether to approve the loan applicant's loan application while protecting the privacy of user data, thereby reducing financial risks.

[0156] Reference Figure 3 , the target data source 310 (e.g., the first banking institution) may send the target data set it has involving the user's historical financial activities to the machine learning system 330. Here, each target data record in the target data set may include various attribute information such as the user's name, nationality, occupation, salary, property, credit record, historical loan amount, etc. In addition, the target data record may also include, for example, tag information about whether the user has repaid the loan on time.

[0157] Here, the machine learning system 330 may be the above reference Figure 1 The machine learning system 100 described above. As an example, the machine learning system 330 may be provided by an entity that specializes in providing machine learning services (e.g., a machine learning service provider), or may be constructed by the target data source 310 itself. Accordingly, the machine learning system 330 may be set up in the cloud (such as a public cloud, a private cloud, or a hybrid cloud), or may be set up in the local system of a banking institution. Here, for the convenience of description, it is assumed that the machine learning system 330 is set up in a public cloud and constructed by a machine learning service provider.

[0158] In order to more accurately predict the user's loan risk index or the user's loan repayment ability, the first banking institution may, for example, reach an agreement with the source data source 320 (e.g., the second institution) to share data with each other while protecting the privacy of the user's data. In this case, based on the agreement, as an example, under corresponding security measures, the source data source 320 may send its source data set to the machine learning system 330, where the source data set may be, for example, a data set involving user financial activities similar to the target data set described above. Then, the machine learning system 330 may perform the above-mentioned Figure 1 and Figure 2 The machine learning related processing is performed based on the source data set in the source data privacy protection mode, and migration items about the source data set are obtained in the process of performing the machine learning processing to migrate the knowledge on the source data set to the target data set. For example, the machine learning system 330 can train a source machine learning model based on the source data set, and use the parameters of the trained source machine learning model as migration items. Here, the source machine learning model can be, for example, a machine learning model for predicting a user's loan risk index or loan repayment ability, or a machine learning model for other similar prediction targets, or a machine learning model for other prediction targets related to the loan estimation business.

[0159] Alternatively, the machine learning system 330 may also directly obtain the migration items from the source data source 320. In this case, for example, the source data source 320 may obtain the migration items by performing machine learning related processing based on the source data set in a source data privacy protection manner in advance through its own machine learning system or by entrusting other machine learning service providers, and send the migration items to the machine learning system 330. Optionally, the source data source 320 may also choose to send the source data set / migration items to the target data source, and then the target data source provides the source data set / migration items together with the target data set to the machine learning system 330 for machine learning.

[0160] Subsequently, the machine learning system 330 trains the target machine learning model based on the target data set and the obtained migration items in the target data privacy protection mode. The target data machine learning model can be, for example, a machine learning model for predicting a user's loan risk index or loan repayment ability. After the target machine learning model is trained, the target data source 310 can send the data set to be predicted involving at least one loan applicant to the machine learning system 330. The machine learning system 330 can use the trained target machine learning model to provide a loan risk index or loan repayment ability score for each loan applicant for the data set to be predicted, and feed back the prediction results to the target data source 310. Subsequently, the target data source 310 can determine whether to approve the loan application submitted by the loan applicant based on the received prediction results. In the above manner, banking institutions can use machine learning to protect user data privacy while using data from other institutions and their own data to obtain more accurate judgment results, thereby avoiding unnecessary financial risks.

[0161] It should be noted that although the concept of the present disclosure is introduced above by taking the application of machine learning in loan estimation in the financial field as an example, it is clear to those skilled in the art that the method and system for performing machine learning under data privacy protection according to the exemplary embodiment of the present disclosure is not limited to application in the financial field, nor is it limited to being used for business decisions such as performing loan estimation. Instead, it can be applied to any field and business decision involving data security and machine learning. For example, the method and system for performing machine learning under data privacy protection according to the exemplary embodiment of the present disclosure can also be applied to transaction anti-fraud, account opening anti-fraud, intelligent marketing, intelligent recommendation, etc.

[0162] As another example, the method and system for performing machine learning under data privacy protection according to the exemplary embodiments of the present disclosure can also be applied to the field of public health, for example, for performing predictions of physiological data. For example, a medical institution hopes to establish a prediction model for a certain health indicator, but only uses the data of this medical institution for training, so the effect of the prediction model may be poor. In fact, many other medical institutions may have corresponding data. If the data of other medical institutions can be used, the prediction effect of the prediction model of the medical institution for a certain health indicator can be improved. At this point, the concept of the present disclosure can be used to protect the privacy of user data of each medical institution, and the data of each medical structure can be integrated to provide more accurate prediction results by using machine learning.

[0163] Furthermore, the scenarios that the target model in this application can be applied to include but are not limited to the following scenarios: image processing scenarios, speech recognition scenarios, natural language processing scenarios, automatic control scenarios, intelligent question-answering scenarios, business decision-making scenarios, recommended business scenarios, search scenarios, and abnormal behavior detection scenarios. For more specific application scenarios in the above-mentioned scenarios, please refer to the previous description.

[0164] Therefore, the method and system for performing machine learning under data privacy protection of the present application can also be applied to any of the above-mentioned scenarios, and the method and system for performing machine learning under data privacy protection of the present application, when applied to different scenarios, has no difference in the overall execution scheme, except that the data targeted in different scenarios are different. Therefore, technical personnel in this field can apply the scheme of the present application to different scenarios without any obstacles based on the aforementioned scheme disclosure, and therefore there is no need to explain each scenario one by one.

[0165] The above has been referred to Figure 1 and Figure 2 A machine learning method and a machine learning system according to an exemplary embodiment of the present disclosure are described with reference to Figure 3 The concepts of the present disclosure are schematically described. However, it should be understood that the devices and systems shown in the drawings may be configured as software, hardware, firmware, or any combination of the above items to perform specific functions. For example, these systems and devices may correspond to dedicated integrated circuits, pure software codes, or modules that combine software and hardware. In addition, one or more functions implemented by these systems or devices may also be uniformly performed by components in physical entity devices (e.g., processors, clients, or servers, etc.).

[0166] In addition, the above method can be implemented by instructions recorded on a computer-readable storage medium. For example, according to an exemplary embodiment of the present application, a computer-readable storage medium storing instructions may be provided, wherein, when the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the following steps: obtaining a target data set; obtaining migration items about a source data set, wherein the migration items are used to migrate knowledge of the source data set to the target data set in a source data privacy protection manner to train a target machine learning model on the target data set; and training the target machine learning model based on the target data set in combination with the migration items in a target data privacy protection manner.

[0167] The instructions stored in the computer-readable storage medium can be executed in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. It should be noted that the instructions can also be used to perform additional steps in addition to the above steps or perform more specific processing when performing the above steps. The contents of these additional steps and further processing have been described in reference to Figure 2It is mentioned in the description of the related method, so it will not be repeated here to avoid repetition.

[0168] It should be noted that the machine learning system according to the exemplary embodiments of the present disclosure may completely rely on the execution of computer programs or instructions to implement corresponding functions, that is, each device corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a special software package (e.g., lib library) to implement the corresponding functions.

[0169] On the other hand, when Figure 1 When the systems and devices shown are implemented in software, firmware, middleware, or microcode, the program code or code segments for performing the corresponding operations may be stored in a computer-readable medium such as a storage medium, so that at least one processor or at least one computing device may perform the corresponding operations by reading and running the corresponding program code or code segments.

[0170] For example, according to an exemplary embodiment of the present application, a system including at least one computing device and at least one storage device storing instructions may be provided, wherein the instructions, when executed by the at least one computing device, prompt the at least one computing device to perform the following steps: obtaining a target data set; obtaining migration items about a source data set, wherein the migration items are used to migrate knowledge of the source data set to the target data set in a source data set privacy protection manner to train a target machine learning model on the target data set; and training the target machine learning model based on the target data set in combination with the migration items in a target data set privacy protection manner.

[0171] Specifically, the above system can be deployed in a server or client, or on a node in a distributed network environment. In addition, the system can be a PC computer, a tablet device, a personal digital assistant, a smart phone, a web application or other device capable of executing the above instruction set. In addition, the system can also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.). In addition, all components of the system can be connected to each other via a bus and / or a network.

[0172] Here, the system is not necessarily a single system, but may be any collection of devices or circuits that can execute the above instructions (or instruction sets) individually or jointly. The system may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0173] In the system, the at least one computing device may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the at least one computing device may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc. The computing device may run instructions or codes stored in one of the storage devices, wherein the storage device may also store data. Instructions and data may also be sent and received over a network via a network interface device, wherein the network interface device may use any known transmission protocol.

[0174] The storage device may be integrated with the computing device, for example, RAM or flash memory is arranged within an integrated circuit microprocessor, etc. In addition, the storage device may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The storage device and the computing device may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the computing device can read instructions stored in the storage device.

[0175] The above describes the exemplary embodiments of the present application. It should be understood that the above description is only exemplary and not exhaustive, and the present application is not limited to the disclosed exemplary embodiments. Without departing from the scope and spirit of the present application, many modifications and changes are obvious to those of ordinary skill in the art. Therefore, the scope of protection of the present application should be based on the scope of the claims.

Claims

1. A method for performing machine learning under data privacy protection, comprising: Get the target dataset; Obtaining a migration item about a source data set, wherein the migration item is used to migrate knowledge of the source data set to a target data set in a source data set privacy protection manner to train a target machine learning model on the target data set; as well as In a target data privacy protection mode, a target machine learning model is trained based on a target data set in combination with the migration item.

2. The method of claim 1, wherein: The step of obtaining migration items about the source data set includes: receiving the migration items from the outside; or, the step of obtaining migration items about the source data set includes: obtaining the source data set; performing machine learning-related processing based on the source data set under the source data privacy protection mode; and obtaining migration items about the source data set in the process of performing machine learning-related processing based on the source data set.

3. The method of claim 2, wherein: The source data privacy protection method and / or the target data privacy protection method is a protection method that complies with the definition of differential privacy.

4. The method of claim 2, wherein: The migration items relate to model parameters, objective functions and / or statistical information about the source data obtained in the process of performing machine learning-related processing based on the source data set.

5. The method of claim 2, wherein: The source data privacy protection method is to add random noise during the process of performing machine learning related processing based on the source data set; and / or, the target data privacy protection method is to add random noise during the process of training the target machine learning model.

6. The method of claim 5, wherein: Performing machine learning-related processing based on a source data set in a source data privacy protection manner includes: training a source machine learning model based on the source data set in a source data privacy protection manner.

7. The method of claim 6, wherein: The source machine learning model and the target machine learning model are machine learning models based on the same type.

8. A computer-readable storage medium storing instructions, wherein: When the instructions are executed by at least one computing device, the at least one computing device is prompted to execute the method for performing machine learning under data privacy protection as described in any one of claims 1 to 7.

9. A system comprising at least one computing device and at least one storage device storing instructions, wherein: When the instructions are executed by the at least one computing device, the at least one computing device is prompted to perform the method for performing machine learning under data privacy protection as described in any one of claims 1 to 7.

10. A system for performing machine learning under data privacy protection, the system comprising: A target data set acquisition device is configured to acquire a target data set; A migration item acquisition device is configured to acquire a migration item about a source data set, wherein the migration item is used to migrate knowledge of the source data set to a target data set in a source data privacy protection manner to train a target machine learning model on the target data set; as well as The target machine learning model training device is configured to train the target machine learning model based on the target data set and in combination with the migration items under the target data privacy protection mode.

Images