Intelligent system privacy evaluation method and system

Abstract

The invention discloses an intelligent system privacy evaluation method and system, and the method comprises the steps: firstly, carrying out the data on-demand preparation according to an application scene of a to-be-evaluated system, and carrying out the dynamic data sampling according to the work requirements of the to-be-evaluated system; then modifying privacy information in the evaluation data through an intelligent semantic editing algorithm, analyzing the incidence relation between the data privacy information and the system inference precision by evaluating the influence of the modified data on the model inference precision, and then conducting quantitative evaluation on the privacy of the to-be-evaluated system. According to the method, privacy evaluation of the intelligent system is realized by establishing the relationship between the user privacy information and the model inference precision in a data-driven manner, so that a universal, efficient and automatic privacy evaluation system for the intelligent system is established, the transparency of the intelligent system for use of the user privacy information is effectively improved, and the right of the user to know the use risk of the personal privacy information is ensured.

Description

technical field [0001] The invention relates to an intelligent system privacy evaluation method and system, and belongs to the technical field of data privacy protection. Background technique [0002] In recent years, with the rapid development of deep learning technology, intelligence has gradually become a new requirement for system design. Intelligent inference services, especially Deep Learning Inference Service (DLIS), use user data as input during work to generate inference results with specific semantics. By introducing deep learning inference services into the system, developers can equip the system with more functions, and can also optimize and improve the operating performance of the system. Here, all systems and applications driven by intelligent inference services, or intelligent inference parts in systems and applications are collectively referred to as intelligent systems. [0003] When using the intelligent system, the user provides data according to the req...

AI Technical Summary

Problems solved by technology

[0008] 1) Theoretical level: The research on the interpretability of deep learning is still developing. At present, it is impossible to directly analyze the structure and parameters of the deep learning model to obtain all the semantic information that the model relies on for correct inference. At the same time, the structure and design of the deep learning model The ideas are also complex and diverse, and it is difficult to establish a general privacy evaluation system centered around the model;

[0009] 2) Scenario level: Under normal circumstances, an intelligent system is a gray box or black box system that only provides a query interface. The system takes user data as input and outputs inference results. In-depth learning model knowledge (such as parameters, structure, training methods, etc.) is poorly understood, and the inference model in the intelligent system will be compressed and optimized, so there is no opportunity to analyze the model using techniques such as back propagation;

Images