Privacy model training method and device based on small amount of public data

Abstract

The invention provides a privacy model training method based on a small amount of public data and an electronic device. The privacy model training method comprises steps of obtaining N neural networkteacher models by using a plurality of trainings; respectively inputting a small amount of public data xi into the N neural network teacher models to obtain a statistical voting result of each publicdata xi for each label k; noise being added to all statistical voting results, and public data xi and corresponding labels meeting the differential privacy principle being obtained; optimizing an adversarial generative network through a large number of random noise vectors and a pre-trained discriminant neural network, and generating a large number of label-free data; and jointly training a student model through the public data xi meeting the differential privacy principle, the corresponding label and a large amount of unlabeled data pair pre-trained auto-encoders to obtain a private student model. According to the method, only a small amount of public data is needed to train one private student model, physical isolation and network isolation of sensitive data are realized, and a problem of low accuracy of the private student model is solved.

Description

Technical field

[0001] The invention belongs to the Internet field, and specifically relates to a neural network model privacy protection training method and device based on differential privacy, semi-supervised learning, and teacher-student knowledge aggregation.Background technique

[0002] In recent years, deep learning technology has made shocking breakthroughs in many fields, such as computer vision, natural language processing, and reinforcement learning. At the same time, the development of deep learning technology is also inseparable from a large amount of training data. At present, many extremely powerful applications use a large amount of sensitive data to train models, for example, use a large number of patients' medical data to train medical diagnosis systems, and use a large amount of user financial data for financial risk control.

[0003] Although deep learning tools can greatly facilitate industrial production and life, recent studies have shown that deep learning models are v...

Images