Method, system and device for quality assessment of structured simulation data

Abstract

The application provides a quality evaluation method and system for structured simulation data, comprising the following steps: constructing a simulation data generation model according to original data; using original data containing privacy information to train the simulation data generation model to obtain simulation data by a data owner; inputting the original data and the simulation data into an evaluation system to evaluate the availability of the simulation data and the privacy of the simulation data, and selecting simulation data with the highest quality for publication; the original data and the simulation data are structured data, which are composed of a plurality of data records, each record has a plurality of fields, and the fields of the simulation data are completely the same as those of the original data. The application uses simulation data to replace original data to participate in data circulation for quality evaluation, since the data entries of the simulation data are all synthesized, do not have any association with entities in reality, and do not need other assumptions or prior knowledge, the risk of privacy leakage is effectively avoided, the evaluation is comprehensive and reliable, and the universality is good.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data security, and particularly relates to a watermark adaptation method for a relational database, a device and a storage medium. BACKGROUND

[0002] In the big data era, it is very common for data owners and collaborators to share data or publish data externally. Traditional data anonymization techniques cannot ideally protect privacy, and for attackers who have knowledge of other data sources, it is possible to infer the identifiers or quasi-identifiers after anonymization, i.e., re-identification attack. Data anonymization techniques greatly reduce the usability of data.

[0003] The patent document CN110222058A disclosed by the State Intellectual Property Office of China proposes a multi-source data association privacy leakage risk assessment system based on FP-growth. The system needs to perform association analysis with external association data sources, and the analysis result is highly related to the collected external data sources, which is not universal. Moreover, the system can only assess privacy in terms of association with external data sources, and the assessment dimension is single. The patent document CN110287729A proposes a privacy leakage risk assessment method for data usage requirements. The method can only assess the privacy feature leakage risk of known original data, and cannot assess potential privacy risks. Moreover, the method can only assess known data operation risks, and is not expandable. The patent document CN107563204A proposes an anonymous data privacy leakage risk assessment method, which can only obtain risk assessment results from the attack effects of several attack models, and the assessment angle is single. The patent document CN111382457A proposes a data risk assessment method and device. The assessment range of the method is fixed, and the method can only assess data sets containing personal plaintext information, and the application range is narrow. Moreover, the assessment result is calculated by a given fixed rule, and gradually has time limitation with the development of the times.

[0004] In the above-mentioned disclosed documents, a large amount of related prior knowledge of data sets is required, such as privacy attributes of data sets, analysis task targets, and associated data sets. The method can only assess data sets after desensitization or anonymization, and is not suitable for assessing simulation data. The method can only assess data sets from the aspect of privacy, and the assessment angle is insufficient.

[0005] Therefore, it is necessary to solve the above problems. SUMMARY

[0006] The present application aims to overcome the deficiencies of the prior art, and provides a quality assessment method, system and device for structured simulation data.

[0007] In a first aspect, an embodiment of the present application provides a quality evaluation method for structured simulation data, which comprises:

[0008] constructing a simulation data generation model according to the original data;

[0009] training the simulation data generation model by the data owner using the original data containing privacy information to obtain simulation data;

[0010] inputting the original data and the simulation data into an evaluation system to evaluate the availability of the simulation data and the privacy of the simulation data, and selecting simulation data with the highest quality for publication;

[0011] wherein the original data and the simulation data are structured data, composed of a plurality of data records, each record having a plurality of fields, and the fields of the simulation data are completely identical to the fields of the original data.

[0012] In a second aspect, an embodiment of the present application provides a quality evaluation system for structured simulation data, which comprises:

[0013] an availability evaluation module for evaluating the degree of similarity between the effect of using simulation data and the effect of using original data in a potential data analysis task;

[0014] a privacy evaluation module of the simulation data for evaluating the risk degree of exposure of original data information by the simulation data.

[0015] In a third aspect, an embodiment of the present application provides a terminal device, comprising a processor, a memory in communication connection with the processor, and an interface for communication with a network device; wherein the memory stores a computer program executable on the processor to implement the quality evaluation method for structured simulation data.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium for storing a computer program, which is executed by a processor to implement the steps in the quality evaluation method for structured simulation data.

[0017] The present application has the following technical effects:

[0018] (1) The present application provides a quality evaluation method for structured simulation data generated by the simulation data generation technology for the data owner, which proposes a quality evaluation method for structured simulation data from the aspects of availability and privacy according to different prior knowledge degrees of the data owner, and uses simulation data to replace original data to participate in data circulation. Since the data entries of the simulation data are all synthetic and do not associate with any entity in reality, the processed data can ensure privacy protection and effectively avoid the risk of privacy leakage.

[0019] (2) The application is used for quality evaluation under the condition of only original data and simulation data, without other assumptions or prior knowledge, simple operation, and good universality.

[0020] (3) The application adopts availability and privacy comprehensive evaluation, and can also give personalized quantitative evaluation index for data use assumption, comprehensive and reliable evaluation, wide application range. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A flow chart of the quality evaluation method for structured simulation data provided for the embodiment of the application is shown in the figure.

[0022] Figure 2 A simulation analysis parameter schematic diagram in the quality evaluation method for structured simulation data provided for the embodiment of the application is shown in the figure.

[0023] Figure 3 A random mapping parameter schematic diagram in the quality evaluation method for structured simulation data provided for the embodiment of the application is shown in the figure.

[0024] Figure 4 An exposure risk parameter schematic diagram in the quality evaluation method for structured simulation data provided for the embodiment of the application is shown in the figure.

[0025] Figure 5 A quality evaluation system schematic diagram for structured simulation data provided for the embodiment of the application is shown in the figure.

[0026] Figure 6 A terminal device schematic diagram of the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical scheme and advantages of the application more clear, the application is further described in detail below with embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

[0028] Term explanation:

[0029] FP-growth: an algorithm for association analysis, which adopts the following divide-and-conquer strategy: compress the database providing frequent item sets to a frequent pattern tree (FP-tree), but still retain the item set association information.

[0030] FP-tree: a data structure called frequent pattern tree, full name Frequent Pattern Tree. The frequent pattern tree is a special prefix tree, which is composed of a frequent item header table and an item prefix tree. The FP-Growth algorithm based on this structure can speed up the entire mining process.

[0031] Regression task: Given a sample, predict its target value, where the target value is continuous.

[0032] Classification task: Given a sample, predict its target value, where the target value is discrete.

[0033] Correlation coefficient: Correlation is a non-deterministic relationship, and the correlation coefficient is a measure of the degree of linear correlation between variables.

[0034] K-means: A clustering analysis method in the field of data mining, which divides n points (which can be a sample observation or an instance) into k clusters, so that each point belongs to the cluster corresponding to the nearest mean (which is the cluster center) as the standard of clustering.

[0035] See Figure 1 The embodiment of the application provides a quality evaluation method for structured simulation data, comprising the steps of:

[0036] S10 generates a simulation data generation model according to the original data.

[0037] Each row of records in the original data is converted into a vector form that can be input into the generation model, the converted data vector is received by the generation model for training, and finally the vector form of the simulation data is output, and then the inverse process of data conversion is performed to obtain the structured simulation data generation model.

[0038] S20 The data owner uses the original data containing privacy information to train the simulation data generation model to obtain simulation data.

[0039] In the traditional data flow process, privacy protection requires certain desensitization and anonymization of data before sharing or publishing. By using simulation data instead of original data for data circulation, since the data entries of simulation data are all synthetic and do not associate with any entity in reality, anonymization or desensitization is not required, and compared with traditional data processing, the use of simulation data greatly reduces the risk of privacy leakage.

[0040] S30 input the original data and the simulation data into the evaluation system for evaluating the availability of the simulation data and the privacy of the simulation data, and select the simulation data with the highest quality for publishing after obtaining a plurality of index values of the availability and the privacy.

[0041] The effect of the simulation data as a substitute for the original data in the data mining analysis can be comparable to that of the original data. The quality of the simulation data needs to be evaluated so that the data owner can select a suitable simulation data generation technology to obtain high-quality simulation data. This step evaluates the usability of the simulation data and also evaluates the privacy of the processed data, that is, the information of the original data cannot be exposed, to ensure that the processed data is sufficient to protect privacy. The privacy of the simulation data is the same as that of the data processed by the traditional method.

[0042] When the evaluation system is input, the original data and the simulation data are structured data composed of a plurality of data records, each record has a plurality of fields, and the fields of the simulation data are exactly the same as those of the original data. For example, a student performance data set stores the information of students in a class, and each record corresponds to the information of a student, each record has a student ID, a name, and a field corresponding to a variable of each subject score. In data mining and analysis, most cases only focus on discrete variable fields and continuous variable fields. The discrete variable field refers to the value set of the variable under the field being a limited set, such as gender and place of origin; the continuous variable field refers to the value of the variable under the field being a value in a certain range, such as age and score. Other fields that are not discrete variables and continuous variables and have analytical significance can usually be split into combinations of discrete variable fields and continuous variable fields, such as the address field which can be split into province, city, and other discrete variable fields. The evaluation method of the present application is only for continuous variable fields and discrete variable fields in the data.

[0043] In a preferred embodiment of the present application, the usability of the simulation data is evaluated according to the similarity of the effect of using the simulation data and using the original data in a potential data analysis task.

[0044] Specifically, the evaluation index parameters of the simulation data include simulation analysis parameters, correlation evaluation parameters, statistical evaluation parameters, and random mapping parameters.

[0045] The simulation analysis parameters are used to evaluate the similarity of the effect of the simulation data and the original data in a specific analysis task, so the input needs to contain the hypothesis that the data is expected to be used for a certain analysis task. This parameter simulates the analysis task by running some machine learning models to predict the specified fields, as shown in Figure 2 The original data is split into two parts, denoted as the original training set T train and the test set T test . The simulation data generation model uses T train for training and outputs a simulation data set T train with the same number of records as T syn . Then T trainTrain a machine learning model for a specific analytical task, make predictions on a field, and use T test Test the machine learning model and obtain the machine learning index P under the original data origin Then use T syn Train the same machine learning model and use T test Test the machine learning model and obtain the machine learning index P under simulation data syn Finally, the difference of the same machine learning model indicators under the same machine learning model is calculated as the output indicator of the simulation analysis parameters.

[0046] If the simulated data is more available in the analysis task, the difference between the machine learning model indicators and the original data should be very close, which means that the simulated data can achieve similar results as the original data in different machine learning models.

[0047] The machine learning models include: 1) regression tasks: multi-layer perceptron, linear regression; 2) binary classification tasks: decision tree, multi-layer perceptron, AdaBoost algorithm, logistic regression; 3) multi-classification tasks: decision tree, multi-layer perceptron.

[0048] The machine learning indicators include: recall rate, precision rate, F1 value, macro-F1 value, R 2 value, AUC-ROC area.

[0049] The association evaluation parameter is used to assess the degree to which the field association relationships in the original data are retained in the simulated data, without requiring any assumptions. For example, in a student performance dataset, the student's final overall evaluation field is highly correlated with the individual subject performance fields, and this correlation needs to be preserved in the simulated data. This indicator reflects the degree of preservation of the association relationship by calculating the correlation degree (a quantity indicating the degree of association between two variables) between the fields in the original data and the simulated data, and comparing the differences. The specific calculation process is as follows.

[0050] The field set in the original data is recorded as A={a1,…,a n}, first calculate the correlation c between each two fields, where field a i and field a j The correlation degree is denoted as c i,j Then construct the correlation matrix C of the original data origin , where the element in row i and column j is c i,j Similarly, construct the correlation matrix C of the simulation data syn , calculate C origin and C syn The difference is used as the output indicator of the association evaluation parameter.

[0051] The correlation degree includes a Pearson correlation coefficient, a Kendall correlation coefficient, a Spearman rank correlation coefficient, normalized mutual information, and a Theil uncertainty coefficient.

[0052] The difference in the correlation matrix is calculated in a manner including a root mean square error, a mean absolute error, and a peak signal-to-noise ratio of a matrix regarded as a gray-scale image after normalization of variables in the correlation matrix.

[0053] The statistical evaluation parameter is used to evaluate whether the statistics of the simulation data and the original data in each field are consistent, without any assumption input. For example, in a student performance data set, the average value of the student final total evaluation field is 80, so the average value of the field in the simulation data should also be close to 80. The specific calculation process of the index is as follows:

[0054] The difference in the mean value, variance, and cumulative distribution of the simulation data and the original data under the corresponding attribute is calculated to obtain a statistical similarity index as the output index of the statistical evaluation parameter.

[0055] When the statistics of the simulation data and the original data are inconsistent, the data owner can select other generation models or adjust the parameters of the current generation model to make the simulation data generated more consistent with the original data.

[0056] The random mapping parameter is used to evaluate the similarity between the simulation data and the original data under different mapping conditions, without any assumption input. The index constructs a plurality of random mappers, and the input of each random mapper is a record, and the output is the vector space coordinate of the record after mapping. Each random mapper can be regarded as a specific analysis task, and the closer the simulation data is to the original data in the new vector space after random mapping, the stronger the usability of the simulation data in the analysis task. The difference comparison of a plurality of random mappings can be regarded as usability evaluation in multiple analysis tasks, and the final result can more comprehensively reflect the usability of the simulation data. The schematic diagram of the random mapping parameter is shown in Figure 3 .

[0057] Referring to Figure 3 , the simulation data and the original data are input into the same random mapper. Since each data record is mapped to a coordinate in the new vector space after passing through the random mapper, if the record is represented by a "point", the point array of the simulation data and the original data in the new vector space can be obtained. Then, the distribution difference of the simulation data point array and the original data point array in the new vector space under the random mapper is calculated. Finally, the distribution difference of the point array is calculated under a plurality of random mappers constructed, and the mean value and variance of the differences are taken as the output index of the random mapping parameter.

[0058] where the difference is calculated as follows: 1) first, the K-means clustering algorithm is trained using the original data points, and the original data points are clustered into several categories, and the number of data points in each category is counted; then the trained K-means clustering algorithm is applied to the simulation data points, and the number of data points in each category is counted; then the number of categories whose number difference between simulation data points and original data points is greater than the standard error is counted, which is called NDC (Number of Statistically-Different Categories) difference; finally, the categories obtained by clustering the original data points are sorted in ascending order according to the number of data points in each category, and the number of original data points in each category in this order is taken as a distribution column, denoted as X origin The number of simulation data points in each category in this order is taken as another distribution column, denoted as X syn The Kullback-Leibler divergence, Jensen-Shannon divergence, Wasserstein distance, and maximum mean difference between X origin and X syn are calculated. The NDC difference, Kullback-Leibler divergence, Jensen-Shannon divergence, Wasserstein distance, and maximum mean difference are used as the difference indicators in the random mapping parameter.

[0059] In a preferred embodiment of the present application, the privacy of the simulation data refers to the risk level of the simulation data exposing the original data information. In traditional data anonymization or desensitization processing, if the anonymous part of the data is inferred by a malicious person, it indicates a high risk of leakage; and all the information of the simulation data is synthetic, and the requirement for privacy is higher, and as long as the information of the original data is exposed, there is a risk of leakage.

[0060] Specifically, the privacy evaluation indicator parameters of the simulation data include: privacy prediction parameters, member inference parameters, exposure risk parameters, and random mapping parameters. These parameters and calculation indicators will be described in detail below.

[0061] The privacy prediction parameter is used to evaluate the true prediction effect of a malicious analyst on the privacy field in a specific analysis task, so the input needs to contain the assumption that the data is not intended to be used for a certain analysis task. The privacy of the simulation data in this scenario represents the effect of defending against malicious analysis tasks. This parameter simulates the analysis task by running some machine learning models to predict the specified field.

[0062] Similar to the simulation analysis indicator, the original data is split into two parts, denoted as original training set T train and test set T test . The simulation data generation model uses T trainThe training and output of T train The same number of simulation data sets T syn are recorded. train Then, a machine learning model for a malicious analysis task is trained, a privacy field assumed by a certain data owner is predicted, and T test The machine learning model is tested to obtain a machine learning indicator P origin under the original data. syn Then, the same machine learning model is trained, and T test The machine learning model is tested to obtain a machine learning indicator P syn under the simulation data. Finally, the difference between the same machine learning model indicators of the same machine learning model is calculated as an output indicator of the privacy prediction parameter.

[0063] The stronger the privacy of the simulation data in the malicious analysis task, the greater the difference between the machine learning model indicators and the original data, that is, the malicious analysis effect in the simulation data is good, but the malicious analysis effect on the original data is poor, which means that the simulation data can defend against the malicious analysis task in different machine learning models.

[0064] The machine learning model includes: 1) regression task: multilayer perception, linear regression; 2) binary classification task: decision tree, multilayer perception, AdaBoost algorithm, logistic regression; 3) multi-classification task: decision tree, multilayer perception.

[0065] The machine learning indicator includes: recall, precision, F1 value, macro-F1 value, R 2 value, AUC-ROC area.

[0066] The member inference parameter is used to evaluate the ability of the simulation data to defend against member inference attacks, so it does not require any assumed input. The member inference attack refers to a black box access right (only input can be controlled and output can be obtained) of a given data record and a trained attack target model, to determine whether the record is in the training data set of the attack target model. The indicator reflects the privacy of the simulation data by constructing an attacker model to attack the generation model corresponding to the simulation data, and evaluating the attack effect.

[0067] Before specifically explaining the attacker model, first explain the simulation data generation model. The simulation data generation model is usually a generator in the generative adversarial network or a variational autoencoder, both of which need to input a noise vector z (usually sampled from a normal distribution) and output a simulation record. The input of the attack model is the data record to be attacked, and the output is a noise vector z'. The loss function is the difference between the simulation record obtained by inputting z' into the generation model and the data record to be attacked. Then use the input noise z and the output simulation record of the simulation data generation model to train the attack model to obtain the trained attack model. Finally, use the attack model to attack the simulation data generation model, calculate the machine learning performance indicators of the attack model, and output the evaluation indicators. The machine learning indicators include: recall, precision, F1 value, macro-F1 value, R 2 value, AUC-ROC area.

[0068] The attack model is to construct a noise vector z', input z' into the simulation data generation model, and get a simulation record. If the simulation record is very similar to the record to be attacked, it is considered that the record to be attacked is in the training set of the generation model, otherwise it is not. The attack idea is: the simulation data generation model tends to generate simulation records similar to the training set records, so it is easy to construct a noise vector as the input of the generation model to make it output a simulation record similar to the training set record. The data record not in the training set, the generation model is not trained with it, so it is difficult to construct a simulation record similar to it, and it is also difficult to construct a noise vector as the input of the generation model to make it output a simulation record similar to the non-training set record.

[0069] The exposure risk parameter is used to focus on the risk degree of the simulation data record exposing the original data item, without any assumption input. If a simulation data record is very similar to a certain original data record, then the simulation data record exposes the corresponding original data record with a high risk. In this parameter, the risk degree is quantitatively evaluated by calculating the distance between the simulation record and the original record.

[0070] First, use principal component analysis method to analyze the original data, get the first principal component and the second principal component of each original data record, and take the first principal component as the x-axis coordinate in the plane rectangular coordinate system and the second principal component as the y-axis, that is, reduce each original data record to two dimensions and regard it as a coordinate point in the plane rectangular coordinate system. Then analyze the simulation data using principal component analysis under the original data, also regard each simulation record as a coordinate point, see Figure 5 .

[0071] This parameter calculates the distance between each simulation record and its nearest original record (such as Figure 4the ratio of the distance between the simulation record and the closest original record (e.g. the length of the solid arrow pointing to the closest original record) and the distance between the simulation record and the second closest original record (e.g. the length of the dashed arrow pointing to the second closest original record), denoted as NNDR (Nearest Neighbour Distance Ratio). The closer the value of NNDR is to 1, the higher the privacy is, and the lower the value of NNDR is, the more likely the sensitive information of the closest original record is exposed. Figure 4 the ratio of the distance between the simulation record and the closest original record (e.g. the length of the solid arrow pointing to the closest original record) and the distance between the simulation record and the second closest original record (e.g. the length of the dashed arrow pointing to the second closest original record), denoted as NNDR (Nearest Neighbour Distance Ratio). The closer the value of NNDR is to 1, the higher the privacy is, and the lower the value of NNDR is, the more likely the sensitive information of the closest original record is exposed. Figure 4 the ratio of the distance between the simulation record and the closest original record (e.g. the length of the solid arrow pointing to the closest original record) and the distance between the simulation record and the second closest original record (e.g. the length of the dashed arrow pointing to the second closest original record), denoted as NNDR (Nearest Neighbour Distance Ratio). The closer the value of NNDR is to 1, the higher the privacy is, and the lower the value of NNDR is, the more likely the sensitive information of the closest original record is exposed.

[0072] DCR and NNDR are the output indicators of the exposure risk assessment.

[0073] The random mapping parameter is consistent with the random mapping in the availability assessment indicator, and only the calculation method is different. Only the difference in calculation is described below, and the architecture of the indicator is not described again. The parameter judges the privacy by calculating the isolation degree of each simulation data point. The greater the isolation degree of a simulation data point, the more likely the simulation data generation model "remembers" a unique original record and exposes the information of the original record in a simulation record.

[0074] Also referring to Figure 3 The simulation data is input into the random mapper. Since each data record is mapped to a coordinate in the new vector space after passing through the random mapper, if the record is represented by a "point", the point array of the simulation data in the new vector space can be obtained. Then, the K-means clustering algorithm is used to cluster the simulation data point array, and the coordinates of each clustering center are calculated. Then, the mean value of the distances between all simulation data points and the corresponding clustering centers is calculated, denoted as DC (Distance to Clustering). The smaller the value of DC, the higher the privacy is. Then, the mean value of the distances between each simulation data point and the closest simulation data points is calculated, denoted as MDR (Mean Distance to Record). The smaller the value of MDR, the higher the privacy is. Finally, in combination with the exposure risk indicator, the DCR of each simulation data point is calculated. The simulation data points with a significantly large DCR are considered as outliers, and the ratio of the number of outliers to the number of simulation data points is calculated, denoted as OR (Outliers Ratio). The smaller the value of OR, the higher the privacy is.

[0075] DC, MDR, and OR are the output indicators of the random mapping parameter.

[0076] In a preferred embodiment of the method of the present application, the step further comprises:

[0077] S301 inputs the assumption of the data owner on the data usage into the evaluation system for evaluating the availability of the simulation data and the privacy of the simulation data.

[0078] The assumption of the data owner on the data usage has two assumptions, that is, the data owner hopes that the data is used for a certain analysis task or does not hope that the data is used for a certain analysis task. The former focuses on whether the analysis effect of the simulation data is consistent with that of the original data in a certain data analysis task; the latter focuses on whether the model trained using the simulation data can act on the original data in a certain data analysis task. For example, the user flow data set in a bank contains various consumption records of users, and the bank hopes that the analyst can analyze which consumption record is a malicious fraudulent transaction, but does not hope that the balance information of the user is analyzed out. In this example, the bank is more concerned that the simulation data of the user flow is consistent with the original data in the analysis task of predicting fraudulent transactions, but does not allow the analyst to secretly train a prediction model capable of predicting the real user balance.

[0079] The above-mentioned quality evaluation method for structured simulation data uses simulation data to replace original data to participate in data circulation, and evaluates the availability and privacy of the simulation data from the perspective of the data owner. Since the data entries of the simulation data are all synthesized and do not associate with any entity in reality, it can be ensured that the processed data is sufficient to protect privacy and effectively avoids the risk of privacy leakage. Further, the simulation data evaluation method based on random mapping can evaluate the quality under the condition of only having original data and simulation data, without other assumptions or prior knowledge, and is simple to operate, has wide application range and good universality.

[0080] Referring to Figure 5 The embodiment of the present application also provides a quality evaluation system for structured simulation data, comprising:

[0081] (1) an availability evaluation module 01 for evaluating various performance parameters of the availability of the simulation data 03, and comparing the similarity degree of the effects of using the simulation data and using the original data in a potential data analysis task;

[0082] In a preferred embodiment of the system of the present application, the availability evaluation module 01 comprises:

[0083] a simulation analysis unit 011 for evaluating the similarity degree of the effects of the simulation data and the original data in a certain analysis task, inputting the assumption that the data is hoped to be used for a certain analysis task, and simulating the analysis task by running a machine learning model to predict a specified field.

[0084] The correlation evaluation unit 012 is configured to evaluate the degree of preservation of the field correlation in the original data in the simulation data, and determine the difference in the correlation between the fields in the original data and the simulation data.

[0085] The statistical evaluation unit 013 is configured to evaluate whether the statistics of the simulation data and the original data in each field are consistent.

[0086] The random mapping unit 014 is configured to evaluate the similarity between the simulation data and the original data under different mapping conditions, and compare the differences in different random mappings to confirm the availability of the simulation data.

[0087] The privacy evaluation module 02 of the simulation data is configured to evaluate the risk degree of the simulation data exposing the original data information.

[0088] In a preferred embodiment of the system, the privacy evaluation module 02 of the simulation data comprises:

[0089] The privacy prediction unit 021 is configured to evaluate the real prediction effect of a malicious analyst on a privacy field in a specific analysis task, and simulate the specific analysis task by running some machine learning models to predict the specified field. The greater the difference between the machine learning model index and the original data, the stronger the privacy of the simulation data in the malicious analysis task.

[0090] The member inference unit 022 is configured to build an attacker model, attack the generation model corresponding to the simulation data, and reflect the privacy of the simulation data by evaluating the defense effect of the simulation data against the member inference attack.

[0091] The exposure risk unit 023 is configured to focus on the risk degree of the simulation data record exposing the original data record. If a simulation data record is very similar to a original data record, the simulation data record has a high risk of exposing the corresponding original data record.

[0092] The random mapping unit 024 is configured to evaluate the similarity between the simulation data and the original data under different mapping conditions. The unit judges the privacy by calculating the isolation degree of each simulation data point. The greater the isolation degree of a simulation data point, the greater the possibility of exposing private information.

[0093] Referring to Figure 6The embodiment of the present application further provides a terminal device, comprising a processor 100, a memory 200 connected with the processor 100 in communication, and an interface 300 for communicating with a network device; wherein the memory 200 stores a computer program, and various function applications and data processing are completed by the control instructions issued by the processor 100 executing the computer program, so as to realize the quality evaluation method for the structured simulation data. The memory 200 can be a memory arranged internally in the terminal device, and can further comprise a memory arranged remotely relative to the processor 100, and these remote memories 200 can be connected to the terminal through a network.

[0094] The embodiment of the present application further provides a computer readable storage medium, which stores computer executable instructions, and can send, propagate or transmit programs for executing or in combination with the devices. When the computer executable instructions are executed by the processor, the quality evaluation method for the structured simulation data is realized.

[0095] The above embodiments of the present application are only part of the preferred embodiments of the present application, and cannot limit the present application, and any modification, equivalent replacement and improvement made by the person skilled in the art without departing from the essence of the present application are within the protection scope of the present application.

Claims

1. A quality assessment method for structured simulation data, characterized in that: Including steps: Construct a simulation data generation model based on the original data; The data owner uses the original data containing private information to train the simulation data generation model to obtain simulation data; Inputting the original data and the simulation data into an evaluation system, evaluating the availability of the simulation data and the privacy of the simulation data, and selecting the simulation data with the highest quality for release; The original data and the simulation data are structured data, consisting of a plurality of data records, each record having a plurality of fields, and the fields of the simulation data are exactly the same as the fields of the original data; The availability assessment of the simulation data includes: Evaluate the similarity between the effects of the simulated data and the original data in a specific analysis task; Evaluating the degree to which the field association relationships in the original data are preserved in the simulated data; Evaluate whether the statistics of the simulated data and the original data in each field are consistent; Constructing multiple random mappers, inputting the simulated data and the original data into the same random mapper, mapping each data record to a coordinate in a new vector space through the random mapper, representing the record with a point, obtaining a dot matrix of the simulated data and the original data in the new vector space, then calculating the distribution difference between the simulated data dot matrix and the original data dot matrix in the new vector space under the random mapper, and finally calculating the distribution difference of the dot matrix under the constructed multiple random mappers, taking the mean and variance of these differences as output indicators of random mapping parameters, and evaluating the similarity between the simulated data and the original data under different mapping conditions; The privacy assessment of the simulation data includes: Evaluate the true prediction effect of malicious analysts on private fields in specific analysis tasks; evaluating the effectiveness of the simulation data in defending against member inference attacks; Assessing the risk level of the simulated data record exposing the original data entry; The simulation data is input into the random mapper, and the degree of isolation of each simulation data point is calculated to judge the privacy. The mean of the distances of all simulation data points to the corresponding cluster center after clustering the simulation data point matrix, the mean of the distances of each simulation data point to the nearest several simulation data points, and the ratio of the number of outliers to the number of simulation data points are used as output indicators of the random mapping parameters to evaluate the similarity between the simulation data and the original data under different mapping conditions.

2. The quality assessment method for structured simulation data according to claim 1, wherein: In the step of evaluating the availability of the simulation data and the privacy of the simulation data, the method further includes inputting the assumption of the data owner on the use of the data into the evaluation system.

3. The quality assessment method for structured simulation data according to claim 1, wherein: The usability of the simulated data is evaluated based on the similarity between the effects of the simulated data and the original data in potential data analysis tasks.

4. The quality assessment method for structured simulation data according to claim 1, wherein: The privacy of the simulation data is evaluated according to the risk level of the simulation data exposing the original data information.

5. A quality assessment system for structured simulation data, characterized in that: Executing the quality assessment method for structured simulation data according to claim 1, comprising: Usability evaluation module, used to evaluate the similarity between using simulated data and using original data in potential data analysis tasks; The privacy assessment module of simulation data is used to assess the risk level of simulation data exposing original data information.

6. The quality assessment system for structured simulation data according to claim 5, wherein: The usability evaluation module includes: A simulation analysis unit, used to evaluate the similarity between the effects of the simulation data and the original data in a specific analysis task; an association evaluation unit, configured to evaluate the degree to which the field association relationships in the original data are retained in the simulation data; A statistical evaluation unit, configured to evaluate whether the statistics of the simulation data in each field are consistent with those of the original data; a random mapping unit, configured to evaluate the similarity between the simulated data and the original data under different mapping conditions; The privacy assessment module of the simulation data includes: The privacy prediction unit is used to evaluate the true prediction effect of malicious analysts on private fields in specific analysis tasks; a member inference unit, configured to evaluate the effectiveness of the simulation data in defending against a member inference attack; an exposure risk unit, configured to monitor the risk level of the simulation data record exposing the original data entry; The random mapping unit is used to evaluate the similarity between the simulated data and the original data under different mapping conditions.

7. The quality assessment system for structured simulation data according to claim 6, wherein: The random mapping unit constructs a plurality of random mappers, inputs each record into the random mapper, and maps each data point from the original space to the vector space coordinate for comparison, so as to evaluate the difference between the original data and the simulated data.

8. A terminal device, characterized in that: It comprises a processor, a memory communicatively connected to the processor, and an interface for communicating with a network device; wherein the memory stores a computer program that can be run on the processor to implement the quality assessment method for structured simulation data according to any one of claims 1 to 4.

9. A computer-readable storage medium for storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images