Data product publishing method or system

Abstract

A computer-implemented data product release method or system. A data product release is derived from a sensitive dataset using a privacy preserving system such as a differentially private system. Privacy preserving parameters such as noise addition magnitude can be configured as part of the data product release method or system to vary the balance between maintaining privacy of the sensitive dataset and making the data product release useful.

Description

Background of the Invention 1. Field of Invention The field of this invention relates to computer-implemented methods or systems for publishing data products. More particularly, but not exclusively, the invention relates to a computer-implemented process for managing privacy-preserving parameters for publishing data products.

[0001] This patent document disclosure includes copyrighted material. The copyright holder does not object to any fax reproduction of the patent document or patent disclosure in the form it appears in as a patent document or patent disclosure in the U.S. Patent and Trademark Office patent document or record, but otherwise reserves all copyright rights.

[0002] 2. Description of the prior art In some cases, publishing aggregated statistics (e.g., contingency tables) about private datasets can lead to the disclosure of private information about individuals. Often, it is not obvious how aggregated statistics about a group can leak information about individuals, and manual output checks cannot detect all these unintended disclosures. Researchers have devised techniques to mitigate the risk of private information leaks. Two such techniques are suppressing statistics about small groups and adding random noise to the statistics.

[0003] Few techniques exist for measuring the risks associated with publishing aggregated statistics. One approach to assessing risk is to use theoretical privacy models, such as differential privacy. These models provide some measure of the privacy security of statistics, but they suffer from at least two problems. First, the theoretical measure is difficult to map to an intuitive understanding of privacy: what does a value of 0.5 for ε (epsilon) (the primary parameter of differential privacy) actually mean? Second, theoretical models account for worst-case scenarios and therefore tend to be unrealistically pessimistic about the magnitude of risk in data publishing.

[0004] Other methods are needed to measure the privacy risks of aggregated statistics.

[0005] Furthermore, privacy-preserving technologies that prevent the disclosure of private information involve a trade-off between the privacy protection achieved and the loss of data utility. For example, suppressing statistics about small groups prevents the direct disclosure of private attributes, but at the same time reduces the amount of information that can be published. Therefore, it is important to assess the utility of data published under privacy-preserving technologies. However, it is not always clear how best to measure the loss of utility or data distortion. In the absence of a clearly defined utility cost of distortion and data loss beforehand, alternative methods are needed to measure the data utility of privately aggregated statistics.

[0006] This invention addresses the aforementioned vulnerability, as well as other issues not described above. Summary of the Invention

[0007] One aspect of the invention is a computer-implemented method for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein privacy-preserving parameters are configurable as part of the data product publication method or system to alter the balance between maintaining the privacy of the sensitive dataset and making the data product publication useful. Attached Figure Description

[0008] Various aspects of the invention will now be described by way of example with reference to the following accompanying drawings, each illustrating a feature of the invention: Figure 1 A diagram showing the key elements of the system's architecture is provided.

[0009] Figure 2 A graph showing the number of statistics as a function of cumulative distortion is presented.

[0010] Figure 3 A diagram showing an example of a visualization with the applied noise distribution is provided.

[0011] Figure 4 An example is shown, depicting a curve representing the percentage of statistics retained in contrast to failed attacks.

[0012] Figure 5 A vertical bar graph is shown, illustrating failed attacks and retained insights as a function of noise levels.

[0013] Figure 6 A screenshot shows an example with a user interface that enables data owners to create privacy-preserving data products.

[0014] Figure 7 The query summary for the pending release is shown.

[0015] Figure 8 The report shows a detailed overview of the pending data product releases.

[0016] Figure 9 The data product values ​​for a specific query are displayed.

[0017] Figure 10 A map showing retail store transaction details by region is displayed.

[0018] Figure 11 A histogram showing transaction details broken down by garment type is provided.

[0019] Figure 12This shows a histogram of average monthly spending by customers segmented by market.

[0020] Figure 13 The system consists of three components: Abe, Canary, and Eagle.

[0021] Figure 14 An example of a statistical release is shown.

[0022] Figure 15 An example of a row in a COUNT contingency table is shown.

[0023] Figure 16 A diagram of the risk measurement algorithm is shown.

[0024] Figure 17 A diagram illustrating the rules for testing attacks and determining whether an attack is successful is shown.

[0025] Figure 18 A horizontal bar chart showing the results generated by Eagle is shown.

[0026] Figure 19 A horizontal bar chart showing individuals identified as being in danger by Canary is displayed.

[0027] Figure 20 An example of a transaction data pattern is shown. Detailed Implementation

[0028] This detailed description describes one implementation of the invention, referred to as Lens or the Lens platform.

[0029] The Lens platform, used for privacy-preserving data products, is a system that allows data holders (such as hospitals) to publish statistics (e.g., counts, sums, averages, medians) about their private data while protecting the private information of the individual data subjects that make up the private dataset. It ensures that personal information is not accidentally disclosed during statistical publication.

[0030] Data holders store sensitive data and wish to release statistical data once or periodically. Furthermore, statistical data can take many forms: numbers, charts such as histograms or CDFs, or even synthetic data reflecting the desired statistics. These outputs are collectively referred to as 'data products,' 'data product releases,' or 'data releases.'

[0031] Data products involve a bounded or fixed set of statistical data, predefined by the data holder and derived from a sensitive dataset. Data product releases may include one or more of the following: aggregated statistical data reports, visualizations, infographics or dashboards, or any other form of aggregated statistical data summary. Data products can also be machine learning models. Data products may also be released as APIs or synthetic microdata files.

[0032] These data products have economic value; for example, health data statistics can drive faster healthcare research, or payment data statistics can inform better business decisions. Lens distinguishes itself by effectively publishing data products from private datasets such as health or payment datasets while ensuring individual privacy is protected.

[0033] Lens employs a differentiated privacy publishing mechanism to provide robust protection for individuals. Differential privacy is a feature of the data publishing mechanism that ensures that the disclosure of information about any individual is limited. This limit is set by a parameter called 'ε'. The lower ε is, the less information is leaked, and the stronger the privacy guaranteed by differential privacy. More information on differential privacy can be found in Nissim et al.'s 2017 paper, "Differential Privacy: A Primer for a Non-technical Audience".

[0034] The key features of the invention will be described in one of the following sections: Section A: Lens Platform Overview Section B: Detailed description of the Lens platform used to create privacy-preserving data products Section A: Overview of the Lens Platform 1. Toolkit for building data products When publishing statistics, it is often difficult to know how high the privacy protection parameters should be set for security purposes while still being useful. Lens includes several features to calibrate the appropriate amount of noise that needs to be added to prevent privacy leaks.

[0035] refer to Figure 1 This illustrates the key elements of the system's architecture. Lens provides a secure way to query sensitive data while protecting individual privacy. Lens processes sensitive data and stores approved, securely aggregated statistics into a relational database called the 'Secure Insights Store'. The statistical insights stored in the 'Secure Insights Store' support various applications or APIs, such as interactive visualizations, dashboards, or reports.

[0036] Interactive 'data products' or 'data releases' or 'data product launches' allow end users to access insights from sensitive datasets without providing access to the raw data within those datasets.

[0037] Given a basic sensitive dataset, Lens allows for the 'data publishing' of security aggregate statistics, which describes and computes them, and makes them available for use outside of Lens. Data Release It refers to a set of statistical data generated by applying many predefined statistical filters, drill-downs, queries, and aggregations to a sensitive dataset.

[0038] In this context, 'security' means protection by a range of privacy-enhancing techniques, such as, as described in other sections of this specification, the addition of differential private noise.

[0039] This protection makes it difficult to reverse the aggregation and learn any information about any individual data topic in a sensitive dataset.

[0040] To generate data publications, Lens uses a description of the required processing of sensitive data, called a 'data product specification'. This can be generated by the data holder through the Lens user interface and stored by Lens, or it can be generated externally using other tools and imported into the Lens system.

[0041] Lens uses data product specifications to derive data publications from any schema-compatible sensitive dataset. This includes a) repeatedly using a single data product specification on a dataset that evolves over time, or b) using a data product specification on multiple unrelated datasets.

[0042] Data product specifications include: ● A representation of the underlying sensitive data schema. This can be a single table or multiple tables joined using foreign key relationships (as in a relational database).

[0043] ● A set of preprocessing transformations performed on instances of sensitive data patterns, such as (but not limited to): ○ 'Rectangularization': The operation of converting a multi-table schema into a single table, as described in subsection 3.1 of section B. ○ Binning variables into more general variables (e.g., binning 37 into 35 into 40) ● Describe the statistical data required for data publication, based on both the underlying data schema and the preprocessing transformations already performed. This includes (but is not limited to): ○ Sum, average, count, median, minimum / maximum, etc. ○ Linear Regression Model ● Descriptions of the conditions that suppress statistical data, such as: ○ Query set size limit (QSSR), which suppresses queries involving a population size smaller than a configurable threshold (e.g., 5).

[0044] ● Indications of 'priority' or other important metrics in statistics to allow for the expression of the most important statistics for the intended data product use case. This allows Lens to consider 'usefulness' when determining how to add noise to statistics. For example, it can add less noise to more important statistics. Examples are as follows: For gender equality research, average wage statistics based on gender drill-down can be marked as 'important' and therefore receive less noise addition compared to location-based drill-down.

[0045] ● Query sensitivity. See the notes below.

[0046] ● Free-text annotations, descriptions, or other requirements may be used, as appropriate, to allow for a later understanding of the specifications.

[0047] Compared to other privacy-preserving technologies that integrate differential privacy into interactive query interfaces, Lens directly integrates differential privacy into its data product publishing system.

[0048] 2. Sensitivity Lens determines query sensitivity based on examining the original data before adding noise, as shown below: 1. Query the raw data to obtain the distribution of the values ​​you want to query.

[0049] 2. Identify outliers and crop the range or summary values ​​as needed.

[0050] 3. Use a cropped / generalized range as the sensitivity and display this range to the user for confirmation.

[0051] User confirmation is an essential step because the true scope of the data may not exist in the dataset, and external domain knowledge may be required to properly specify sensitivity.

[0052] End users can also configure the range of sensitive variables and may truncate or clamp outliers that exceed a certain range to improve the PUT of data products.

[0053] End users can also configure how sensitive variables are generalized. For example, age can be generalized into 10 bins, or categorical variables can be generalized using a user-defined mapping. Lens then enforces this generalization when generating and publishing data. This, in turn, improves the privacy-utility tradeoff.

[0054] Generalizing a range can protect privacy. For example, narrowing the range to the nearest multiple of 10 can hide information about the actual maximum value (e.g., if a maximum value of 100 is reported, the actual maximum value could be any value between 11 and 100).

[0055] This feature will also be discussed in subsection 4 of section B.

[0056] 3. Generate and publish data. The workflow described below includes the following steps: collecting data product specifications, analyzing the data product specifications, and returning one or more data product specifications along with recommended noise additions and other settings for privacy and / or utility.

[0057] Data product specifications include any user-configurable parameters related to the data product.

[0058] The process is flexible enough to manage different datasets and guide many types of users through a good privacy-utility trade-off.

[0059] Given data product specifications, there are several methods to achieve secure data publishing: 1. Data product specifications can be provided or transmitted to human experts ('Lens experts') who facilitate the process described below, or 2. Automated systems can directly use data product specifications to generate secure data releases.

[0060] In case (1), the process is as follows: 1. Lens experts receive the data product specifications and input them into the Lens tool as part of understanding the required data release.

[0061] 2. Lens experts investigate the feasibility and privacy-utility balance of data product specifications and final data release. Lens experts use Abe for attack and distortion calculations. Lens experts can use the latest versions of these tools without updating the Lens interface itself.

[0062] 3. Lens experts can now optionally decide to propose one or more alternative data product specifications that they believe better meet the required use case. For example, different rectangularization, binning, or QSSRs can be proposed. In some cases, Lens experts may conclude that no good secure data release is sufficient to meet the use case, and therefore, based on their investigation, they may choose to respond negatively to the data product specification, detailing why this is the case.

[0063] 4. Lens experts use the Lens tool to generate detailed reports and perform the privacy transformations described in the data product specifications, and then apply noise addition as indicated by the tool's testing with Abe to produce data releases for each proposed data product specification.

[0064] 5. Lens experts will integrate detailed reports and data into the Lens software.

[0065] 6. Inform Lens users that detailed reports are available.

[0066] 7. Lens users can review detailed reports and decide which reports they deem appropriate, if available.

[0067] 8. Based on the selection, Lens makes the selected data available for later use.

[0068] The changes to the above content are as follows: ● In step (4), Lens experts do not generate data outputs to Lens. They only generate and input detailed reports.

[0069] ● Between steps (7) and (8), based on the choices made by the Lens user in step (7), Lens directly uses the selected detailed report and sensitive dataset to compute the data release without interacting with Lens experts.

[0070] ● Because this processing may take some time, the Lens software indicates to the user that processing is in progress. Meanwhile, if a previous data release of the same data product is being actively used (such as via an API), that previous release will remain available until the new release is approved and activated. In case (2), the process is similar, but the Lens expert is replaced by automation: 1. Lens software analyzes product specifications and can generate a set of recommended alternatives.

[0071] 2. For each of these, Lens generates detailed reports and publishes data by directly processing sensitive datasets. 3. Inform Lens users that detailed reports are available.

[0072] 4. Lens users can review detailed reports and decide which reports they deem appropriate, if available.

[0073] 5. Based on the selection, Lens makes the selected data available for later use.

[0074] 4. Detailed Report According to (1) and (2), the Lens software displays one or more detailed reports to the user based on the data product specifications. This is a rich summary of the effects of differential private noise addition, which allows the user to determine whether noise statistics can be used.

[0075] The report provides a detailed yet understandable picture of the privacy-utility characteristics of the anticipated data release.

[0076] It consists of several parts: ● Privacy Recommendation ● Attack Summary ● Utility Recommendation ● Summary of effects Privacy recommendations are straightforward yes / no indicators presented to the user, showing whether the noise level recommended by Abe satisfactorily prevents attacks. The criteria used for a 'yes' result depend on which attacks were performed and whether the added noise is sufficient to protect the dataset. For example, in the case of a differential attack, a 'yes' result is returned only if all detected attacks are defeated by the added noise. As an example of a solver attack, a 'yes' result is returned only if, for some appropriate pre-configured value of x, more than x% of the dataset cannot be correctly guessed.

[0077] The attack digest contains a summary of the outputs of different types of deterministic and probabilistic attacks performed on the Lens. For example: ● Differential Attack. A list of individuals whose raw data values ​​would be exposed if not protected by adding noise. Each entry in the list contains the raw data value and a summary of the attack that would leak that value.

[0078] ● Solver Attacks. A summary of the impact of noise on an attacker's ability to reconstruct a dataset is presented compared to a known baseline (e.g., if gender is a private variable, it is always guessed to be 'female'. In a sample of the global population, this should succeed approximately 50% of the time, as the male-to-female ratio is known to be roughly 50-50). For example, it may be possible to show that the addition of noise has reduced an attacker's ability to reconstruct 90% of the records to 52%, compared to a baseline of 50%. The change here is a measure of the success rate of Lens's defeat attack.

[0079] The effectiveness of defenses against attacks depends on Lens having a baseline risk model. This means that any enhancements to the protection measures should be understood relative to the background knowledge that attackers might possess.

[0080] The utility recommendation is a straightforward yes / no indicator presented to the user, showing whether the noise level preserves sufficient utility within the statistical data. Lens can support different heuristics to determine whether to display 'yes': ● A thresholding method based on the distribution of distortion in noisy statistics compared to their values ​​before noise was added. The threshold can be expressed as 'no more than x% of the statistics have a distortion greater than y%'.

[0081] ● As described above, the threshold method uses a threshold based on sample error, rather than a simple percentage distortion threshold. This heuristic can be expressed as 'no more than x% of the statistics have distortion greater than z*sample error'. ● One approach respects the statistics most valuable to the user and gives greater weight to these values ​​when calculating overall recommendations. Lower-value statistics tolerate more noise. This depends on which statistics Lens users specified as most valuable during the development of the data product specifications. Lens can provide UI features to allow this to be expressed.

[0082] ● A thresholding method based on advanced insights from statistical data, employing the Eagle system described in subsection 1.5 of section B. Before calculating a detailed report, Lens extracts a list of features of the statistical data before adding noise. This includes overall trends, minimum / maximum values, etc. A similar list of features can also be extracted after adding noise, and utility recommendations can be based on applying a threshold to the proportion of insights still evident in the noisy statistical data.

[0083] The utility summary illustrates the impact of noise addition on utility, measured by calculating the distortion of each statistic relative to its original value and visualizing the distribution of the distortion values.

[0084] Distortions can be visualized using standard techniques, such as: 1. Box plot.

[0085] 2. Histograms. For example, this might allow users to see that 90% of the statistics are distorted between 0-5%, and 10% of the statistics are distorted by more than 5%.

[0086] 3. Cumulative Distribution of Distortion. By plotting distortion cumulatively, users can more easily see the proportion of distortion exceeding a given statistical value. Figure 2 The example shown is where the number of statistics is plotted as a function of cumulative distortion. The curve allows reading the number of statistics whose distortion exceeds a threshold percentage from the y-axis.

[0087] The aim of these methods is to enable users to understand, in general, how noise alters statistical data and thus how to fit the data into the intended data product. Users must then decide whether the final publication is appropriate based on the utility summary and recommendations.

[0088] The detailed report contains all the information, including statistics at noise levels, that users can use to determine whether they want to approve or reject the recommendation.

[0089] If security statistics are approved, they can be published and used in the data product. This is done by placing the aggregated security statistics into a relational database called the 'Security Insights Store'. This utilizes standard database technologies to provide maximum scope for subsequent use of the data.

[0090] 5. Visualization of noise / accuracy Noise can be visualized directly on the chart representing the statistical data itself. This can be shown as error bars, by calculating the confidence interval of the applied noise distribution and applying that confidence interval to a bar chart displaying the original (noise-free) statistical data. Multiple statistical data can be displayed on the same chart, each with error bars, thus allowing for comparisons between noise values.

[0091] Figure 3 A graph showing an example of a visualization with the applied noise distribution is provided. In this graph, the sensitivity value (average wage) and the classification by age range are shown. The original statistics are displayed as a bar chart overlaid with error bars to visualize the amount of noise that may have been added to the corresponding data release.

[0092] Unified visualization and control of privacy and utility: Lens supports visualizing both privacy and utility, and these visualizations can be used interactively to allow users to control the Lens's automatic selection of noise levels and determine their own privacy-utility balance. Two such visualizations are described below: 1. Statistical curves of %failed attacks against % retained data; 2. Insight charts based on the attack and retention of defenses against noise levels.

[0093] The following examples illustrate these points: % Failed attack statistics curve for % Retained like Figure 4 As shown in the curve, Lens displays the impact of various noise levels (in this case, the value of ε) on the statistics of failed attacks and retention ('retention' here means that the distortion does not exceed the threshold amount).

[0094] By selecting a node along the curve, users can specify the amount of noise, at the cost of preserving statistical data. This is an intuitive way for users to understand how the chosen noise level explicitly affects utility.

[0095] Based on the attack and retention insights chart of the defense against noise levels: In this figure, the two vertically placed bars indicate the impact of choosing a certain amount of noise on the number of attacks to be defended against and the number of insights retained.

[0096] The selected noise level is represented by a vertical dashed line. If the display is used as an interactive control, the display slides along the x-axis to control the noise level. As the line moves to the left (lower noise), the user can clearly see that fewer attacks will be defended against because the applied noise is less than the amount needed to defend against each type of noise, as shown by the bars in the bar chart above.

[0097] As the line moves to the right (where there is more noise), less insight is retained after adding noise. 'Insight' here refers to interest features automatically extracted by the Lens, which are measured before and after adding noise to gauge changes in utility. Reference Figure 5 The chart below shows a vertical bar graph used to visualize failed attacks and retained insights as a function of noise levels. More insights are lost as the noise level increases, as shown by the bars in the chart below.

[0098] By selecting the noise level in this way, users can understand the trade-off between protecting against privacy attacks and maintaining the usefulness of the dataset. Users can use this display to set their own trade-off.

[0099] 6. Recommendations for Data Product Improvement Given the data product specifications that have already generated detailed reports, Lens can recommend improvements to these specifications to achieve a better privacy-utility tradeoff. These improvements can be suggested by Lens experts or automatically by Lens itself.

[0100] If users decide to implement some or all of the recommendations, new data product specifications and new detailed reports will be prepared to describe the changes and summarize the new privacy-utility trade-offs.

[0101] Lens guides end-users on how to modify data products to achieve better puts. For example, if a data holder wants to publish data products that don't protect privacy, such as someone wanting to publish square feet per second multiplied by the population per square foot, Lens guides the data holder to try publishing aggregated statistics that are inherently more privacy-friendly. Use Lens or some quick heuristics to determine the privacy utility trade-offs. If the trade-offs don't meet the user's or data holder's requirements, modifications to the data product specifications are recommended, such as reducing table dimensions, decreasing publication frequency, summarizing the data, and suppressing outliers.

[0102] Other recommended examples are as follows: ● Numerical variables can be summarized by dividing them into bins of a certain size.

[0103] ● Generalize categorical variables by grouping them into similar, related, or hierarchical categories. In the hierarchical case, generalization can be performed by using an external hierarchy definition to elevate values ​​to broader categories.

[0104] ● Modify the data product specifications to include histograms of numerical variables instead of averages.

[0105] ● Apply a QSSR threshold to suppress statistics based on low counts.

[0106] ● Suppress or inhibit outliers.

[0107] ● Prohibit the publication of certain unimportant drill-downs. By default, Lens can calculate multidimensional 'cubes' for drill-downs (e.g., age group multiplied by gender multiplied by income group). It is recommended to publish only 2D tables instead of n-dimensional tables. This is an effective way to limit the number of statistics published, which in turn will require less overall noise.

[0108] End users can also configure any parameters of the data product specification via a graphical user interface. The system can then automatically display recommendations based on any updated parameters of the data product specification. For example, an end user can enter a QSSR value that results in fewer compromised statistics, and the system may find the same level of privacy achievable with less noise. As the end user updates different QSSRs, the system displays noisy recommendations for each QSSR. The end user may then automatically discover that publishing statistics with query set sizes below a certain threshold offers no benefit.

[0109] Over time, new technologies for generating recommendations will become available. Lens can provide a universal user interface to review proposed improvements and allow users to apply these improvements to pending data product specifications. In each case, a new detailed report will be prepared to allow for understanding of the effects of applying the recommendations.

[0110] 7. Lens API Once the data publication is approved, it can be used outside of the Lens. There are two methods to retrieve values ​​from the data publication from the security insights store: 1. API Access. Lens exposes an API that can be used by external data products to retrieve values ​​from specific data publications from the Security Insights repository. This API will be expressed according to the corresponding data product specifications, meaning that the values ​​expressed here for drill-down, querying, and filtering will be supplied in the API calls and reflected in the returned values.

[0111] 2. Direct Database Access. To support low-level efficient access to values ​​in data publications, direct access to the security insights store database is also permitted. This will be accomplished using standard database technologies such as JDBC.

[0112] 8. Benchmark based on the organization's specified data. Lens supports a 'benchmarking' use case, in which security aggregate statistics in the security insights store can be compared with certain raw data that facilitates aggregation. Importantly, raw data values ​​are only published under certified models with verified access permissions.

[0113] For example, if a data product has been defined that calculates an average transaction value using data from a set of retail companies, then any of these companies would be interested in comparing their raw value to the secure aggregate. Each company can 'log in' to an authenticated section of the data product, thereby gaining access to their own raw value. The Lens API can then return both the aggregate and the raw value, allowing for visualization where the two can be compared.

[0114] The same process can be applied to drill-down subsets of records, such as comparing raw values ​​of demographic categories or time windows with aggregates.

[0115] 9. Duplicate releases Lens supports the following scenarios: data evolution, and it is appropriate to publish new, updated data based on the new state. This could be due to periodically refreshing sensitive datasets from the 'main' business system, or due to changes in the scope of the dataset, such as the inclusion of more entities.

[0116] Therefore, Lens allows companies to manage regularly refreshed data products while ensuring their privacy is protected.

[0117] During the generation of new data releases via the aforementioned mechanism, existing 'current' data releases can be obtained from the security insights store and via the API. Approving a pending data release results in the current release being 'archived,' making the pending release the new current release. Detailed reports for any archived data release can always be accessed via the Lens UI, allowing you to determine the current date and usage date of any data release and its detailed report.

[0118] Disequilibrium noise related to repeated publication As described in this specification, when multiple data releases are made based on the same entity, these entities may be vulnerable to attack. To mitigate this, Lens can determine the noise level for protecting the entity for a given number of future releases, given a specific data release.

[0119] Lens supports two strategies for distributing noise between the current release and future releases: 1. Quantitative Noise: Based on multiple releases to be protected, a quantification is added to the noise so that the noise added to the current release and each future release is expected to be roughly the same, and is expected to defend against all attacks. When each new data release is needed, the calculation is re-examined using the new data and the quantification is updated. This process is discussed in subsection 1.7.3 of Section B. Each statistic in each data release receives the same amount of budget. In this case, Lens may issue a warning if a release requires significantly more noise than previous releases to achieve the same privacy. This is an important feature of Lens because changes to the data can introduce additional unintended risks.

[0120] 2. Treat each release independently: In this way, each release is protected independently. While simpler, this method does not address attacks that exploit multiple releases. Therefore, method 1 is more secure.

[0121] These strategies can coexist with an equal / weighted distribution of each published budget, which is done to prioritize the utility of more important statistics, as discussed above.

[0122] 10. Understand sampling error Some statistics may be inherently uncertain and generally do not require much attention. However, noise often severely distorts these statistics. In such cases, comparing the distortion with sampling error provides a useful picture of the distortion involved, as sampling error highlights the inherently uncertain statistics.

[0123] The raw data processed by Lens typically represents a sample of a broader population, and therefore, any statistics calculated from this raw data will contain sampling errors. Lens adds differential private noise to such statistics as needed to prevent attacks.

[0124] Given a data product configuration and sample dataset, Lens can compare the magnitudes of noise and sample error and derive interesting conclusions that can be displayed in a utility report.

[0125] If the magnitude of the noise is much smaller than the sample error (expressed as a ratio), it indicates that the decrease in utility caused by the addition of noise is acceptable, because the statistics are largely uncertain due to sampling error. Lens can show this conclusion in a detailed report.

[0126] If the magnitude of the noise is similar to the sampling error, this still indicates a good utility trade-off, because the uncertainty of the statistics does not change significantly compared to the original underlying statistics due to the sampling error. Lens can demonstrate this conclusion in a detailed report.

[0127] If the noise level is significantly greater than the sampling error, the user should use other information presented in the utility report to determine whether the data release is justifiable.

[0128] 11. Example of a use case with aggregated statistics from apparel retail stores Lens provides data holders with an intuitive set of tools to manage the privacy of raw datasets while maintaining the utility of the data and determining appropriate privacy parameters, such as differential privacy parameters.

[0129] The following screenshot shows an example of data releases from aggregated statistics from apparel retail stores.

[0130] Figure 6 A screenshot is shown with an example of a user interface that enables data owners to create privacy-preserving data products.

[0131] Figure 7 The system displays a summary of queries for pending releases, including AVERAGE and SUM queries. It also shows when the data product is ready for release.

[0132] Figure 8 It displays a detailed report on pending data releases.

[0133] Figure 9 An example of the data product specification is displayed as a JSON file.

[0134] Data holders can drill down into multiple dimensions to obtain more details, such as based on demographic or behavioral information, while also protecting their privacy.

[0135] Figure 10 The total transaction value was displayed by region. Figure 11 It is a histogram of average transaction value broken down by clothing category. Figure 12 It is a histogram of average monthly spending by customers segmented by market. Further drill-down information can be obtained by factors such as age, gender, income, or time period.

[0136] Section B: Detailed description of the Lens platform used to create privacy-preserving data products Lens contains the following key innovative features: 1. The process of selecting the appropriate ε strength for data products. This process is driven by automated adversarial testing and analysis.

[0137] 2. Support features from data products that contain multiple private attributes for each individual (e.g., an HR dataset with both sick pay and disciplinary records).

[0138] 3. Supports features of data products from transaction or time series datasets.

[0139] 4. The process of guiding users to set "sensitivity" (an important concept in differential privacy).

[0140] 5. The option to publish aggregated statistics or synthetic data that reflects these statistics.

[0141] 6. Features that provide privacy protection for one or more entities (e.g., people and companies).

[0142] 7. A set of heuristics for quickly (but not with 100% accuracy) determining whether statistical publishing is safe.

[0143] 1. Set "ε" (the amount of noise added to the statistics) through automated adversarial testing and analysis. Lens uses noise addition to ensure that statistical publication does not lead to disclosures about individuals. It employs differential private noise addition mechanisms, such as the Laplace mechanism. When using these mechanisms, the amount of noise is controlled by a parameter called ε.

[0144] Lens includes a system for setting ε through adversarial testing and utility testing. This section describes this adversarial and utility testing system. The system is a principled approach for selecting ε to balance privacy risks and analytical utility.

[0145] The penetration testing engine system automatically launches a set of predefined privacy attacks against a set of statistics tables and identifies privacy risks associated with potential releases of said statistics tables. By automating multiple attacks, comprehensive penetration testing is easily performed. Automated adversarial testing is significantly faster and more repeatable than manual testing. Furthermore, it is more reliable and quantifiable than previous privacy penetration testing systems.

[0146] The penetration engine also manages the privacy parameter ε by estimating the likelihood of multiple attacks succeeding and selecting ε to make all attacks fail.

[0147] Please note that although this section primarily deals with ε, ε-differential privacy, and the Laplace mechanism, it is similarly applicable to the other two variants of differential privacy: approximate Differential privacy and concentrated Differential privacy, both of these variants can use Gaussian mechanisms. These variants are well-known in the field of differential privacy research. This same point regarding cross-applicability also applies to other sections.

[0148] 1.1 Background on the privacy risks of publishing aggregated statistical data In some cases, publishing aggregated statistics (e.g., contingency tables) about private datasets can lead to the disclosure of private information about individuals. Often, it is not obvious how aggregated statistics about a group can leak information about individuals, and manual output checks cannot detect all these unintended disclosures. Researchers have devised techniques to mitigate the risk of private information leaks. Two such techniques are suppressing statistics about small groups and adding random noise to the statistics.

[0149] Few techniques exist for measuring the risks associated with publishing aggregated statistics. One approach to assessing risk is to use theoretical privacy models, such as differential privacy. These models provide some measure of the privacy security of statistics, but they suffer from two problems. First, the theoretical measure is difficult to map to an intuitive understanding of privacy: what does a value of 0.5 (the primary parameter of differential privacy) actually mean? Second, theoretical models account for worst-case scenarios and therefore tend to be unrealistically pessimistic about the magnitude of risk in data publishing.

[0150] Other methods are needed to measure the privacy risks of aggregated statistics.

[0151] Furthermore, privacy-preserving technologies that prevent the disclosure of private information involve a trade-off between the privacy protection achieved and the loss of data utility. For example, suppressing statistics about small groups prevents the direct disclosure of private attributes, but at the same time reduces the amount of information that can be published. Therefore, it is important to assess the utility of data published under privacy-preserving technologies. However, it is not always clear how best to measure the loss of utility or data distortion. In the absence of a clearly defined utility cost of distortion and data loss beforehand, alternative methods are needed to measure the data utility of privately aggregated statistics.

[0152] Using adversarial testing to test defenses is a method that may be easy to understand. However, testing a large number of attacks remains difficult, and there is a risk of overfitting one's defenses to attacks that are only attempted during the testing process.

[0153] In contrast, differential privacy does not require knowledge of the attack type. However, as mentioned above, understanding how to set ε is a difficult task.

[0154] Lens combines the advantages of adversarial testing methods with privacy-preserving technologies such as differential privacy.

[0155] 1.2 Overall Objectives of the Adversarial Testing and Analysis System Figure 13The three components of this system are shown: Abe 130, Canary 132, and Eagle 134, each with a different but related purpose.

[0156] Eagle 134 focuses on the utility of measured statistical releases. Eagle extracts advanced conclusions from a set of aggregated statistical data. These conclusions are those that human analysts might draw from observing statistical data. For example, they might take the form of "People with variable X = x are most likely to have variable Y = y" or "There is a correlation between variable X and variable Y".

[0157] Canary 132 focuses on detecting the risk of disclosure of an individual's private information. Canary models different types of adversaries and performs a series of privacy attacks on a given statistical release. A Canary attack is a method of combining information from a set of statistical data to determine a person's private attributes. For example, an attack on a SUM table might be subtracting the value of one cell from the value of another. If the groups associated with the two cells differ by one person, this attack will reveal that person's private value. Canary attack outputs a measure of the risk of disclosure of private attributes against a set of aggregated statistical data. For example, a SUM attack outputs a list of individuals from whom one's private value can be known from aggregated data.

[0158] Canary and Eagle each have their own uses, and they are also useful for Abe 130.

[0159] Abe Evaluating the privacy-utility trade-offs of various privacy-preserving techniques136. Most privacy-preserving techniques are parameterized; for example, small-count suppression is parameterized by a threshold, below which counts are suppressed. For any given privacy-preserving technique, such as differential privacy, Abe chooses a parameter, if possible: ● Retain the high-level conclusions of the original table. This step uses Eagle's output.

[0160] ● Defend against all known privacy attacks. This step uses the output of Canary.

[0161] It's possible that no parameters can simultaneously provide good privacy and usability. In this case, Abe detects this fact and can report it to the user.

[0162] Abe, Canary, and Eagle possess some key characteristics that make them valuable technologies.

[0163] ● Measuring utility loss without a defined cost function: Privacy mechanisms often lead to data distortion or suppression. Measuring the impact on data utility is always a challenge. Distortion measures such as root mean square error (RMSE) can be used, but this implies that the user knows how to interpret the distortion. In addition to performing standard distortion measures such as RMSE, Abe uses Eagle to perform a more advanced testing method that preserves critical... Insights derived from data In some cases, if the same insights are derived from distorted data as from the original data, then the distortion of the data is irrelevant. Eagle can be configured to capture many different types of insights.

[0164] ● Real-world risk measurement: Even when using models such as k-anonymity or differential privacy, it is difficult to determine the extent of potential privacy risks in the release of statistical data. The Abe-Canary attack combination uses a method similar to penetration testing in cybersecurity. It attacks statistical data to the best of its ability and logs its execution. This is an interpretable and useful way to measure privacy risks.

[0165] 1.3 Input Data All components analyze aggregated statistics and / or generate row-level data for said aggregated statistics. Ideally, the aggregated statistics should be described as the result of a SQL-like statistical query in the following form. AGGREGATE (Private variables) GROUPBY (property1&property2&...) AGGREGATE This can include SUM, COUNT, AVERAGE, or MEDIAN. For example, this could be a COUNT query performed on a statistical database for all people in a dataset who have a specific set of attributes, such as: COUNT(*) GROUPBY(Gender & Salary Level) Or a SUM query targeting private values, such as: SUM (Monthly Income) GROUPBY (Gender & Department) The results of these queries against the database generate numerous aggregate statistics, the structure of which is as follows: Figure 14 As shown.

[0166] This is an example of Lens output with Eagle and Canary data publishing types applied.

[0167] 1.4 Encoding aggregated information into equations A procedural method is needed to express information about each person. Statistical data, such as sums and counts, are linear functions of individual values ​​and can be expressed by a system of linear equations. Many Canary attacks require summarizing aggregated information into a set of linear equations of some form. The following sections describe how to represent different types of aggregated statistics.

[0168] 1.4.1 Encoding the SUM and AVG tables Consider a summation table displaying the sums of private attributes for various population groups. For example, a table might show the total salaries for each department in a company. In this case, each person's private attribute is a continuous value, and the system encodes these continuous values ​​as variables. For example, if there are 10 people in the sample population, their private attributes are represented by variables v1, ..., v10. The attack aims to recover the exact value of each variable in the population (e.g., v1 = 35000, v2 = 75000, etc.). Now, each cell in the SUM table corresponds to a group of people and can be transformed into a linear equation. For example, if the cell corresponds to people 2, 5, and 7, and say the sum of the private attributes is 99, then we derive the following equation: v2 + v5 + v7 = 99 We refer to each statistical data point in the table as a "cell", "aggregate query", "aggregate", or "statistical data".

[0169] For the sum table, all information from the aggregate is summarized in a system of linear equations:

[0170] For example, if we publish about n Personal m Then A It has 0 and 1 m×n A matrix, where each row represents a sum and an individual included in the sum is labeled 1, while other individuals are labeled 0. The vector υ is... n A column vector representing the private attribute values ​​of each individual. d The length is m And the values ​​of and are used as entries in the vector.

[0171] The AVERAGE table can be reformulated as a SUM table. In the case of AVERAGE queries, sometimes all dimensions of the table are known background variables, while the unknown private attributes are the variables to be used in the AVERAGE operation. With this background knowledge, the count of each cell can be known, and therefore the count can be multiplied by the average to obtain the sum. In this way, the AVERAGE table can be reduced to the case of a SUM table, and the methods used for SUM tables can be exploited.

[0172] By knowing the size of each query set from background knowledge such as all people and all groups partitioned by variables, it is possible to perform round-trip calculations between AVERAGE and SUM.

[0173] 1.4.2 Encoding the COUNT table The coding process for COUNT tables (also known as contingency tables) is as follows.

[0174] One-hot encoding is used to split the categorical variable into several binary variables, and a set of equations is used to express each statistic. Then another set of equations is used to express that each person is associated with only one category.

[0175] Assume the COUNT table has N Dimensions, and in the dimensions N-1 This is a well-known property. For example, for N= 2 There may exist a 2D contingency table based on counts of age and drug use, where the contingency table can represent age ranges on one axis. On the other axis, drug use is represented. Assume age is a known attribute, but assume drug use is an unknown and private attribute.

[0176] Canary uses one-hot encoding for private categorical variables, so for a private categorical variable with 3 classes, each person has 3 associated variables that can take values ​​of 0 or 1—we call these variables υ. 1:x υ 1:y and υ 1:z —These variables correspond to whether the person marked 1 belongs to the category. x y or z And make , Intuitively, each person can only belong to a part of one category. In the use case of drug use, this would be: .

[0177] Canary then encodes the information from the COUNT contingency table. Suppose a row of cells (e.g., a row of cells representing ages 20-30) is known to consist of three people (people 4, 9, and 19), but the private attribute categories to which these people belong are unknown. If the row looks like... Figure 15 As shown in the table.

[0178] Canary encodes this into three equations using the same variables as before, one for each cell.

[0179]

[0180] For the COUNT table, all information is summarized in these equations, with the constraint that all variables must be either 0 or 1. Solving these equations restores all the variables. The value of is the subject of a famous computer science problem known as zero-one-integer linear programming (Crowder, Harlan, Ellis L. Johnson, and Manfred Padberg's "Solving large-scale zero-onelinear programming problems"). Operations Research 31.5 (1983): 803-834), and a suitable solver can be used to find vulnerable variables in the dataset based on the set of linear equations.

[0181] Other COUNT attacks using this equation structure will be discussed below.

[0182] 1.4.3 Encoding of tables where the sensitive value is part of GROUP BY Consider the following situation: one of the variables on which the grouping is based and the variable being counted or summed are both private. For example, in the example above, if age and Drug use These are all private values ​​that must be protected. Therefore, the age will be unknown, and we will be unable to write the above equation.

[0183] We use multiple private variables Flat This problem is solved with a single private variable, thus reverting to the more standard case where only one variable is secret. The flattening approach we use involves one-hot encoding for every possible combination of secrets: for example, the first secret takes the value a or b, and the second secret takes the value x or y, so the flattened private variable would take the values ​​(a, x), (a, y), (b, x), (b, y); in the example above, if age is also private, then the private value would consist of the pair (age, drug use), and therefore could be (20-30, never).

[0184] After flattening the secrets, we return to the standard case for categorical variables, which can be handled as described in the paragraphs above. It's important to note that if one of the secrets is a continuous variable, such as salary, flattening must be performed with caution. Indeed, if flattening is applied directly, the resulting categorical variable might take on so many distinct values ​​that each private value might be observed only by one person (no two people in a database have exactly the same salary down to the last digit). Such private columns would be unprotected. Therefore, we advocate reducing the precision of continuous variables or binning them before flattening them.

[0185] 1.5 Eagle Eagle is a program that processes a set of published statistics (such as contingency tables) and outputs a set of high-level conclusions or insights. These insights are findings that human analysts might extract from the data; for example, in the table above, companies spend the most on paying male sales staff. Insights can be encoded as sentences or structured data (e.g., {"finding_type": "max_val", "value": {"gender": "female", "eye": "brown"}}).

[0186] Testing whether the high-level conclusions or key insights of the original sensitive dataset are preserved allows us to determine how distortion of statistics affects its usefulness or utility. This is accomplished by assessing whether the same high-level conclusions can be drawn from the perturbed statistics compared to the original sensitive dataset. The conclusions drawn indicate that the phrasing and utility more closely approximate the reality of the business value of the data product.

[0187] All high-level conclusions are encoded into a single program so that utility tests can be performed automatically. A representative set of 'conclusions' can be run on any table.

[0188] Some of the advanced conclusions that Eagle discovered are: ● Maximum value ● Relevant variables ● Differences in group means ● Time Mode Maximum value. Eagle iterates through each contingency table and finds the maximum value in the contingency table. It has a threshold. t (between 0 and 1), and only when the second highest value is less than the maximum value. t The maximum value is recorded only when the value is doubled. For example, if the cell with the highest value is cell X with a value of 10, and the cell with the second highest value has a value of 8, and... t If the value is 0.9, Eagle will record the following conclusion: the largest cell is cell X. However, if t If the value is 0.7, then it will not record this finding.

[0189] Eagle can also calculate the maximum value in a contingency table when one of the variables is fixed. For example, if the contingency table is a count of medical conditions by gender, it might record the maximum medical condition / gender pair, the most frequent medical condition for each gender, and the most frequent gender for each medical condition.

[0190] Related variablesIf one of the factors grouping the data is numerical, such as age, Eagle will test whether there is a strong positive or negative correlation between this attribute and the private value. This test is performed only on SUM or AVG tables. Eagle calculates a Pearson correlation coefficient, which measures the linear dependence between two variables. Findings are only recorded if the correlation coefficient is above a certain threshold.

[0191] Differences in group means. For a table containing the mean private value for each group, Eagle assesses whether there are any statistically significant differences between the group means. For a given table, it performs a one-way or two-way analysis of variance (ANOVA) hypothesis test and calculates the p-value as a measure of statistical significance and the eta-square as a measure of effect size. Two different insights can be recorded as the results of this test: ● If p is less than a given alpha level and the effect size is greater than a given level, there are significant differences between the mean private values ​​of the groups. For example, Cohen's "Statistical Power Analysis," first published by Jacob Cohen on June 1, 1992, in Current Directions in Psychological Science, Volume 1, Issue 3, pp. 98-101. https: / / doi.org / 10.1111 / 1467-8721.ep10768783 The proposal is to use 0.25 as the threshold for medium or large effect size.

[0192] ● If these conditions (high statistical significance and large effect size) are not met simultaneously, there is no significant difference between the mean private values ​​of the groups.

[0193] Time mode. When provided with a table representing the same statistics across time periods, Eagle can detect time patterns in the data. For a given statistic, these include whether there is a specific upward or downward trend, whether the distribution across multiple groups changes consistently over time, and whether there are any outliers in the given time series. For example, one finding is that total spending statistics increased every year for eight consecutive years. Another finding is that the spending ratio between men and women remained approximately the same for ten consecutive years.

[0194] Eagle can extract any type of insight, which can be formulated using the same structure as the example given above. Additional insights can be derived from the results of other statistical tests, such as chi-square tests of independence, or statements about ranking lists.

[0195] Different users may focus on different conclusions. Therefore, end users should be allowed to specify their own customized conclusions relevant to their use cases.

[0196] Finally, users can submit their own conclusions for testing. For example, these conclusions can be entered as a piece of code (such as Python code). The system processes user-submitted conclusions, such as those provided by its built-in conclusions.

[0197] 1.6 Canary Canary is a system that automatically assesses the risk of privacy violations in data publication. Canary processes a set of published statistics (e.g., contingency tables) and outputs information about the risk of individual private values ​​being disclosed through a set of privacy attacks. A privacy attack is a function that takes a set of aggregated statistics as input and outputs guesses about the private values ​​of one, some, or all of the individuals in the dataset.

[0198] Canary contains a set of attack algorithms. Some privacy-preserving attack algorithms return additional information about the attack. Example attacks and outputs might be: ● Direct Cell Lookup: The most trivial attack. If you have a SUM table and a cell reflects a singleton (a group of size one), directly returning the value of that cell is an accurate guess of the person's private value. Furthermore, an attacker can learn this value with 100% confidence and mark the person as 'vulnerable'. The term 'vulnerable' means that the individual can be completely determined by an attack (note that in the case of raw statistics, this means unprotected against added noise).

[0199] ● Differential Attack: If you have some SUM tables and two cells (in different tables) reflecting groups X and Y respectively, and groups X and Y differ by only one person, then returning the value in Y minus the value in X is an accurate guess of that person's private value. Differential attacks with more than two cells are more complex.

[0200] A large number of attack functions are kept together in the suite and stored in the attack library. The attacks have also been standardized so that one or more attacks can be easily added to the suite at any point.

[0201] The attack function is run to automatically guess sensitive data based on aggregated statistics. By expressing the statistics as a set of linear equations about the variables to be aggregated, the solver can find efficient solutions (i.e., values ​​of the sensitive variables consistent with the statistics). The output of the attack function is then used to set ε.

[0202] When there is a combination of statistical data where the sensitive variable is completely determined, the solver can find the exact value of the sensitive variable. By comparing the guess with the actual value, it is said that a person is easily attacked when there is a match. Constraints on the range of the sensitive variable can also be added directly to the solver.

[0203] The following sections describe many different attacks.

[0204] 1.6.1 Differential attack scanner for sums, averages, counts, and medians Differential attacks are a common type of privacy attack targeting aggregated statistics. A differential attack works by sorting the statistics by queryset size and only checking for differential attacks targeting pairs of statistics whose queryset sizes differ by one. This is more efficient for differential attacks than naively checking every pair of statistics. After finding a differential attack, we can update the queryset to remove vulnerable individuals. This removal may potentially expose further differential attacks against others.

[0205] As described below, the process of finding differential attacks has been automated.

[0206] Differential attack scanners search a given statistical publication for groups that differ by a single individual. This allows for the formation of "difference of one" attacks, which can then expose an individual's private values.

[0207] The SUM example nicely illustrates the one-difference attack. If the linear equations associated with two separate cells (as described in Section 1.3) are... v1 + v2 + v3 + v4 = x v1 + v2 + v3 = y Then we can clearly deduce that v4 = xy For the original statistical release without applying any differential privacy mechanisms (such as adding Laplace noise), this method is recursive, since v4 has now been discovered, so the other two equations can now be solved by subtracting v4. Consider more than two linear equations from the same statistical release. v4 + v5 +v6 + v7 + v8 + v9 = a v5 + v6 + v7 + v8 = b Knowing that v4 allows us to change the first equation v5 + v6 + v7 + v8 + v9 = a - v4 This, in turn, allows us to construct another one-out-of-charge attack. v9 = ab-v4 The differential attack scanner searches for systems of equations associated with a given statistical release for linear equations that differ by a single individual. When operating on raw statistics, it then removes the individual and its value from the system of equations, and rescans for the differencing attack. This method also applies to equations derived from AVERAGE contingency tables, as these equations can be re-expressed as sums (as outlined in Section 1.4.1).

[0208] The differencing scanner can also operate on COUNT tables, because COUNT statistics are also represented as linear equations, where the right-hand side of the equation represents the number of individuals in a given category. Expressing a COUNT table as a system of equations is outlined in more detail in Section 1.4.2.

[0209] MEDIAN statistics are also vulnerable to differencing attacks, although the information produced by such attacks is a constraint on the value of a private variable, rather than the exact value itself. Instead of a linear equation, a given median equation can simply be treated as a set of variables. Consider the medians: MEDIAN{v1, v2, v3, v4} = x MEDIAN{v1, v2, v3} = y In this case, if x>y, we can state that the difference of the sets gives v4>y. Similarly, if x<y, we can state that v4<y.

[0210] It is critical to note that, in the above sense, differencing attacks on MEDIAN statistics are not recursive even with the original statistical releases. This is because, continuing the example above, v4 can no longer be removed from other existing sets that contain v4 (i.e., median statistics), and no new differencing pairs can be found.

[0211] Within Canary, the differencing scanner is efficiently implemented by sorting all given statistics by query set size (i.e., the number of variables contributing to a given statistic), also referred to as QSS. For a given reference statistic, set difference is taken under all other statistics whose QSS differs by 1 relative to this reference. If this set difference contains a single variable, a differencing pair is found. The above rules for differencing are applied depending on the type of the released statistic.

[0212] For AVERAGE, SUM, and COUNT statistics that are derived from the original statistical output, the scanner removes all found variables from the equation set and rescans. This recursive process terminates once no new difference is found. For the original MEDIAN statistics, or any noisy statistics, the scanner terminates after a first scan of all statistics. The scanner then returns all derived variables (for AVERAGE, SUM, and COUNT statistics) or the found variable limits (for MEDIAN statistics). The scanner can also return the attack for each derived variable as a set difference or a set difference chain.

[0213] This one-difference scan procedure can be used in a variety of ways, either as a quick method to illustrate easily interpretable attacks on statistical releases, or as the initial phase of an iterative attack method.

[0214] Risk metric output by the difference scan algorithm.

[0215] The algorithm is: 1. Convert the sum table into a system of equations. 2. For scans with a difference of one.

[0216] 3. If applicable, remove the difference of one and then rescan.

[0217] This algorithm returns a set of variables that are susceptible to one-difference attacks or a series of variables that are one-differenced (if applicable). For each variable found to be vulnerable, the algorithm also returns the resulting estimate. Or the range of estimated values.

[0218] 1.6.2 Least Squares-Based Iterative Attacks on Summation Tables In order to find at-risk individuals through a more sophisticated difference attack using a given set of sums and tables, Canary needs to solve a system of linear equations.

[0219] Finding individuals at risk of having their secrets revealed through published summary statistics is equivalent to finding all variables whose values ​​are entirely determined by a set of equations. υ i (Referred to as 'variables vulnerable to attack'). A fully determined variable is equivalent to a private property that can be attacked by looking at the SUM table alone; the information in the aggregated statistics is sufficient to fully determine the private property expressed by these variables.

[0220] Canary's least-squares SUM attack algorithm searches for the least-squares solution of a system of linear equations.

[0221] Use an iterative linear solver and return this best guess solution for all variables in the dataset.

[0222] The iterative solver does not directly solve the system of equations. Instead, it starts with a first approximation of the solution and iteratively calculates a sequence of approximations (hopefully improving). Several parameters define the termination condition of the iteration and how close the obtained solution is to the true solution. Typically, systems of equations collected from all sum tables are indeterminate because the amount of statistical data is likely less than the amount of variables in the dataset. If such a linear solver is given an indeterminate system of equations, it will output the equations... one Solution, this is to reduce the distance A·υ - d The solution that minimizes the L2 norm.

[0223] Using this type of solver, it is possible to find variables in a dataset whose values ​​are fully constrained in the following ways: 1. Use a solver to generate solutions to the system of equations.

[0224] 2. Iterate through the variables and compare the solution value with the actual value (found in the original data).

[0225] 3. If the solution value is the same as the actual value, we say that the value is completely determined by the system of equations. Note that we may not want to use strict equality – because the solver is not always accurate, we may want to treat the values ​​as the same if the difference is less than a threshold (e.g., 0.000001).

[0226] It is worth noting that this method can return false positives. If the variables in the system of equations are not fully determined, the solver may arbitrarily choose a value that happens to match the actual value of the variable. Therefore, Canary provides methods to handle false positives, which are discussed below.

[0227] Alternatively, Canary could perform this attack while skipping the step of identifying which variables are fully constrained. Instead, it could simply provide a guess for each variable. If used in this way, Lens could add range constraints to the solver. For example, if the range of the sensitive variable is 0 to 10, Lens would put 0 <= v_i <= 10 for all v_i into the solver.

[0228] An alternative method using orthogonal equations. If many statistics about the same dataset are published ( If the problem involves an overdetermined system of equations used to attack statistical data, then Canary needs to solve it. In these cases, the least squares solution can be calculated by solving the orthogonality equations. .

[0229] In this method, the system of equations is transformed into dimensionality. The system of symmetric equations can then be solved using a fast numerical solver. This method is only applicable to... The case where it is a non-singular matrix and invertible is the result of m being appropriately large relative to n.

[0230] Risk metric output by the iterative least squares attack algorithm.

[0231] The attack algorithm is: 1. Convert the sum table into a system of equations. 2. Solve the system of equations by running an iterative solver or by solving the orthogonal equations to obtain the potential solutions for each private attribute.

[0232] For all the vulnerable variables found, the algorithm returns a guess. .

[0233] 1.6.3 Attacks on Summation Tables Based on Pseudo-Inverses Another Canary attack algorithm also finds the least-squares solution to the observed system of equations, but the attack works differently. It uses the system of equations matrix. A The false rebellion.

[0234] Pseudo-inverse attacks use linear algebra to compute combinations (i.e., formulas) of statistical data, thereby guessing a person's sensitivities most accurately (even when noise is added to the statistical data). This not only allows finding all individuals vulnerable to differential attacks but also allows identifying specific differential attacks, which can be shown as examples of privacy attacks.

[0235] The solution is obtained by calculating the pseudo-inverse. A method to find the least squares solution that minimizes the error norm. The method is to calculate the matrix. A The Moore-Penrose pseudoinverse is usually represented as A + This method applies to both underdetermined and overdetermined systems of equations.

[0236] A + Through matrix The singular value decomposition (SVD) is approximately as follows: After calculating A + Then, vulnerable variables can be identified as matrices. The diagonal entries, wherein the diagonal entries are 1 or close to 1 within a certain numerical error tolerance.

[0237] matrix A + Provides access to statistical data sets d Description of privacy attacks.A + Each line of description A The linear combination of each row (i.e., the published sum) restores the private value of a variable.

[0238] Using this type of solver, it is possible to find variables in a dataset whose values ​​are fully constrained in the following ways: 1. Calculate the matrix A The approximation of the pseudo-inverse.

[0239] 2. Calculate matrix multiplication And in B Find the diagonal entries that are equal to 1. These are the indices of the variables that are uniquely determined by the system of equations.

[0240] Specific privacy attacks on vulnerable variables are encoded using pseudo-inverses, and therefore, this method provides a way not only to detect at-risk individuals but also to recover from attacks themselves—by calculating a formula for sensitive values ​​based on published statistics. Furthermore, the attack function can be directly applied to any new statistical release based on the same query, i.e., any... m Dimensional result vector d And without any further computational work.

[0241] Because the pseudo-inverse is approximated by its SVD, numerical inaccuracies can lead to problems even if the corresponding variables are not completely determined by the system of equations. V Some diagonal entries are close to 1. Therefore, the results can be optionally double-checked to ensure there are no false positives.

[0242] Risk metric output by pseudo-reverse attack algorithm.

[0243] The attack algorithm is: 1. Convert the sum table into a system of equations.

[0244] 2. Attack matrix A + Multiply by a vector of statistical data described by a set of contingency tables. In order to obtain the potential solutions for all variables.

[0245] This algorithm returns a guess for all the vulnerable variables found. υ i , and a list of vulnerable variables.

[0246] 1.6.3.1 Using SVD to reduce the computational complexity of pseudo-inverse attacks If the matrix under consideration A If the value is very large, it may be impossible to compute the pseudo-inverse of the matrix within a reasonable amount of time. A+ Therefore, it is important to try and reduce the computational burden of the operation. We do this by calculating... A This is implemented using SVD. Specifically, we first calculate... A First, we use SVD (a simpler and faster way to compute pseudo-inverses), and second, we use SVD only to compute those that can execute the attack. A + The lines. Now, we will describe each step in turn: 1. We calculate A SVD; that is U, S and V , making .

[0247] 2. We noticed (in (Represents the product of matrix entries) recovery B The diagonal of the line allows us to immediately locate vulnerable variables. Let... Z A vector of indices for vulnerable variables.

[0248] 3. Recall that the attack was A + The row, the index of which is in Z Therefore, we only need to calculate these rows. Using ,exist Z Middle Mark V The line we get This greatly reduces the amount of computation required.

[0249] 4. Then, the output of the method is the same for the previously introduced pseudo-reverse attack, and therefore can be used in the same way.

[0250] 1.6.3.2 Using the GROUPBY structure for efficient SVD calculation The unique structure of the studied linear equation system can be used to achieve parallel computation on extremely large databases. It can also improve the computation of attacks by using a basic query structure. By using the query-based structure, large equation systems can be decomposed into multiple sub-equation systems, which can be solved separately and then combined.

[0251] In cases involving massive datasets and distributions, no standard library can perform SVD. In such situations, we use... A The GROUP BY structure. Specifically, the structure corresponding to a given GROUP BY. A All rows are orthogonal (the inner product is zero), therefore A The SVD for this block is very simple to execute.

[0252] Therefore, we first perform SVD for each GROUPBY, and then sequentially merge the SVDs. To merge the SVDs, we proceed in two steps. First, we perform QR decomposition on the stacked right singular vectors. Since QR requires no optimization, we can obtain orthogonal matrices with minimal computational cost. Q Right triangle matrix R rank of the system of equations r Then, by maintaining R The former r Given a set of singular values ​​and vectors, we can reconstruct the SVD of stacked singular vectors, and finally reconstruct... A SVD.

[0253] Stacking can be done in parallel (by merging GROUPBYs 2x2, then merging again until complete), recursively (by adding GROUPBYs one by one to an ever-growing stack), or globally (merging all at once). The most efficient strategy depends on the system's capacity: the global approach is optimal but requires a large amount of memory; the parallel approach is most useful when parallel sessions are needed but has high communication overhead; the recursive approach is suboptimal but requires only one session, which limits memory consumption.

[0254] 1.6.3.3 Using QR decomposition to reduce the computational complexity of pseudo-inverse attacks All the previously presented schemes assumed the role of the attacker and only used knowledge available to the attacker. However, to make the attack system more effective, we can use our knowledge of the secrets... υ Understanding this can help reduce computational costs.

[0255] Therefore, it will be carried out as follows: 1. Obtain the QR decomposition of the equation matrix.

[0256] 2. Obtain the triangular components from QR decomposition using backward substitution. ,equation The least squares solution.

[0257] 3. Match the entry to the true vector of the secret value. Matched entries are considered vulnerable. This is a step a real attacker cannot perform.

[0258] 4. For each vulnerable row Use backward substitution as in step 2 to modify the equation. Solve for, where Is it in the index? A vector whose values ​​are all zero except for the value 1. This is called the obtained solution. Then, It is the attack vector. A + The i OK.

[0259] Note that the method described can also be parallelized as in Section 1.6.3.2.

[0260] 1.6.3.4 Using a solver to generate the optimal pseudo-inverse attack Given a data product and the existence of a differential attack, it is possible to generate a guess about the secret. This guess is also random due to the addition of noise. This section describes a method for finding differential attacks that generates guesses with minimal variability.

[0261] The method described below finds the most accurate minimum variance difference attack and seeks the optimal attack on the data product, not just the data product itself. The method optimally utilizes the varying levels of variability present in each published noisy statistical statistic.

[0262] Through attack vectors We came to a guess. .because d It is random, therefore It's also random. The accuracy of the attack can be determined by... variance var( ) To measure. Now, for any z This allows us to obtain , making This is another attack vector. To make the attack as accurate as possible, we are looking for... z to make and Minimize as much as possible. Relying on a linear solver, the method is then expanded as follows (we use the same notation as in the previous section): 1. Use any of the methods in 1.6.3 to identify vulnerable lines. i .

[0263] 2. Using a linear problem solver in Minimize under constraints .

[0264] 3. Return to optimal attack .

[0265] 1.6.3.5 Using Rank Revealed QR Decomposition to Generate Optimal Pseudo-Inverse Attacks Finding the minimum variance attack is a computationally intensive task, impossible to scale to large data products, and too time-consuming to easily achieve privacy risk assessment when building data products. A faster, scalable solution is needed to achieve reasonable usability.

[0266] The method described in this section overcomes technical hurdles by revealing a QR factorization technique that makes solving any system of equations much faster and more scalable.

[0267] There is an incentive to make finding the optimal attack as efficient as possible, especially since we will need to repeat this process many times: for each vulnerable line i However, for each presupposition, how should we find out? d The noise-adding mechanism makes the resulting minimum variance attack less accurate.

[0268] It is possible to improve efficiency by relying on the rank-revealed QR decomposition of the equation matrix. Rank-revealed QR decomposition (or factorization) is a standard procedure available in most linear algebra software. This decomposition reorganizes the columns of the R components of the QR such that all z , making Set the first entry to 0 (when the rank of the equation matrix is ​​0). r hour, z The former r Each entry must be 0). This makes it easier to satisfy the constraint. This greatly reduces the amount of computation. The process is as follows: 4. Generate the equation matrix A The rank of QR is revealed.

[0269] 5. Use the QR method described in section 1.6.3.3 above to find vulnerable rows. i .

[0270] 6. Generate a basic attack using the QR method described in section 1.6.3.3 above. a .

[0271] 7. Call V , d The variance-covariance matrix. Then, our problem might be restated as finding the variance-covariance matrix that makes... Become the smallest z This will be achieved by solving... The first derivative is zero, which is achieved by solving a system of linear equations, and therefore can be achieved using the QR decomposition as described above in 1.6.3.3.

[0272] 1.6.4 Symbolic Solver Attacks on SUM Tables One of Canary's privacy attackers uses a symbolic equation solver method. The symbolic solver takes a system of linear equations and generates an expression for each variable. Therefore, the symbolic server is able to determine when a variable and its value are fully determined. For example, it can be said that v2 equals "99 – v5 – v7". Canary processes these expressions to identify linearly dependent groups of variables (variables whose expressions depend on the values ​​of other variables in the group) and fully determined variables (variables marked as vulnerable via difference attacks). The symbolic solver also provides multiple sets of interrelated variables and equations associated with those variables (e.g., v1 = 100 - v2).

[0273] This method of solving systems of equations is known in scientific literature as Gauss-Jordan elimination, but it cannot be well extended to large systems of equations.

[0274] Canary's symbolic solver attack can take additional steps to pinpoint variables that are not precisely identified but are determined to fall within a sufficiently small interval to still pose a privacy risk. For example, if someone could determine your salary is between 62,000 and 62,500 based on published statistics, it would be as if they knew exactly your salary, potentially feeling like a privacy violation. To detect these variables, Canary uses the Monte Carlo method to explore the possibilities each variable can take. As a step function of the Monte Carlo process, one variable is modified, and equations are used to calculate how said variable affects other variables. At the end of the Monte Carlo process, information about the distribution of each individual variable is obtained. Variables that fall only within a very narrow range may pose a privacy risk.

[0275] In each relevant group of variables (discussed above), Canary performs the following Monte Carlo procedure: 1. Initialization steps: Assign variables their actual values. 2. Select a variable and increase or decrease it (the rules for this process can be customized; for example, the rules could be to add a random selection {+5, -5}, or a random selection from an interval [-10, 10] or from an interval [-x, x], where x is a value or a fixed percentage of a variable range). 3. Use a symbolic equation to adjust another variable in the related group in the opposite direction (thus maintaining a linear relationship). 4. Test for any violations of constraints. A constraint might be that the private variable must be greater than 0 and less than 1,000,000. If a constraint is violated, return to step 2 and retry. If no constraints are violated, implement the changes and repeat from step 2.

[0276] This process can continue (steps 2-4) to create a series of states. These states can be sampled to approximate the distribution of all variables. Variables whose distribution is constrained to small intervals are then considered vulnerable.

[0277] Attack the risk metric output by the symbolic solver algorithm.

[0278] The attack algorithm is: 1. Convert the sum table into a system of symbolic equations.

[0279] 2. Solve the system of equations using the Gauss-Jordan elimination method.

[0280] 3. Perform (optional) checks on variables that are determined to be within small intervals.

[0281] For each vulnerable variable identified, the algorithm returns an estimate (or, if from step 3, a value interval) and a combination of statistics determining the estimate. The algorithm may also optionally return the variable within a small interval and what that interval is.

[0282] 1.6.5 Attacks on the COUNT table as a constrained optimization problem Since counting tables can also be expressed as linear equations, solvers can be used to attack counting tables.

[0283] In the case of COUNT, the value of a private variable is one of several possible categories. For example, a sensitive attribute could be whether an individual takes a certain medication, and the private value could be one of {never, rarely, often}, and an attacker is trying to find out which of these categories the variable belongs to.

[0284] Canary's COUNT attack, like its SUM attack algorithm, summarizes all the information from the COUNT table into the system of linear equations (see Section 1.4.2). However, unlike the SUM attack, the COUNT attack constrains the solution space in which they search for the values ​​of the variables to a specific range. To see this, let's use... v This is a matrix representing private values. In our example, we have: for all i As v The i OK v i use In the form of. Then, using v The list Query: , and , They are the same. Therefore, for the equation matrix associated with the latter query... A and the contingency table of counts to be published d We obtain: .

[0285] Therefore, attack counting can be viewed as solving the following system of constrained equations:

[0286] in c This is the number of possible categories (e.g., in our drug use example, c=3). Canary COUNT attackers use a range of techniques to obtain solutions to multiple variations of this problem within a reasonable timeframe. Some attacks simply recover the private values ​​of fully determined variables, while others attempt to guess as many values ​​as possible correctly.

[0287] 1.6.5.1 Notes on the norms used Please note that we did not specify the norm used in the above equations, but rather used a series of possible norms; that is, It represents any norm or pseudo-norm, but specifically, it represents... norm, where p =0, 1, and 2. In the noise addition settings, it's important to note that if the added noise is Laplace or Gaussian, then use... L 1 and L The 2-norm corresponds to using the correctly specified maximum likelihood, thus making the proposed optimization scheme below the lower bound of Cramer-Rao efficiency (no unbiased estimator would be more accurate). 1.6.6 Attacks on COUNT Tables Based on Discrete Solvers The first and simplest way to attack a COUNT table is to solve the problem directly using a suitable integer linear programming solver. Several algorithm libraries offer this possibility.

[0288] The risk metric returned by the discrete solver attack method.

[0289] The attack algorithm is: 1. Encode the set of the COUNT table into a system of equations.

[0290] 2. Run using the discrete solver.

[0291] The attack algorithm returns the following guess for each variable: 1. It has the appropriate format; that is, a vector such that each entry is in... In the vector, the sum of the entries is equal to 1.

[0292] 2. Make It's very small.

[0293] While general and very useful for small systems of equations, this attack has the drawback that it cannot be extended to large problems, and we don't know which of these guesses are accurate. The alternative Canary COUNT attacker addresses both of these issues.

[0294] 1.6.7 Pseudo-inverse-based attacks on COUNT tables Another type of Canary attack on the COUNT table is performed in the same way as the Canary SUM attack based on pseudo-inverse. This attack algorithm ignores the fact that the private values ​​of variables can only be located in... Constraints in the process.

[0295] The risk metric returned by this COUNT pseudo-reverse attack algorithm.

[0296] The attack algorithm is: 1. Encode the set of the COUNT table into a system of equations.

[0297] 2. Attack matrix A + Multiply by a vector of statistical data described by a set of contingency tables. d In order to obtain the potential solutions for all variables.

[0298] 3. Most of these potential solutions will not be in the state of... In the middle, or even not very close, however, by building A + The vulnerable variables will be (or very close to, until the precision of matrix inversion is achieved).

[0299] 4. For all variables found to be vulnerable (determined using the same method as described above for pseudo-inverse attacks on the SUM table), the guess is approximated as... The closest value in the range.

[0300] This algorithm returns a list of all variables found to be vulnerable, and a guess of the private value of each of these vulnerable variables.

[0301] 1.6.8 Saturation Row Attack on a Counting Table The attacker makes the following two observations. First, the attacker needs to know how many secret values ​​need to be accumulated to calculate the statistics. Second, the attacker needs to know the maximum and minimum values ​​that the secrets might use. With these two pieces of information, the attacker can deduce the maximum and minimum values ​​that the statistics might use. If the published statistics are close to the maximum value, then each secret value used to calculate the statistics is likely also close to the maximum value, or vice versa for the minimum value.

[0302] Discrete solver attacks output correct guesses about most of the dataset. This relies heavily on the fact that private values ​​can only be 0 or 1 to make a good guess. The main drawback of this attack is its inability to handle large systems of equations or to provide a measure of confidence in the guesses about the values ​​of the variables it returns. In contrast, pseudo-inverse-based methods only output guesses about fully determined variables that are known to be vulnerable. Pseudo-inverse-based methods ignore constraints on the possible private values ​​that variables can take, and therefore risk missing vulnerabilities. These constraints reduce the number of possible solutions, and thus allow attackers to make much more accurate guesses.

[0303] Therefore, another Canary COUNT attack algorithm, namely the saturation row attack algorithm, aims to combine the capabilities of a discrete attacker (utilizing solution space constraints) with the capabilities of pseudo-inverse-based attacks to handle larger systems of equations. The saturation row attack algorithm proceeds as follows: First, it finds the saturated cells: - A cell is said to be positively saturated if the count it contains is equal to the query set size; that is, if the sum of the entries in the equation matrix equals the published count. Then, all private values ​​in the query must be equal to 1.

[0304] - A cell is said to be negatively saturated if it contains a count of 0 and the query set size is not 0. Then, all variables considered in the query must have a private value of 0.

[0305] The algorithm then removes all variables from the observed set of equations whose private values ​​can be determined using the saturation method, and applies a pseudo-inverse attack to the remaining set of equations to recover the unknown variables.

[0306] The risk metric returned by the saturated row COUNT attack algorithm.

[0307] The attack algorithm is: 1. Encode the set of the COUNT table into a system of equations.

[0308] 2. Analyze the cells and detect cells with positive and negative saturation.

[0309] 3. If a saturation entry is found, a pseudo-inverse attack can be applied as described below: a. From d Subtract the contribution of private values ​​inferred from saturated cells.

[0310] b. From A Remove rows and columns corresponding to cells that have been found to be saturated and private values, resulting in... .

[0311] c. Use The pseudo-inverse is used to find vulnerable variables.

[0312] d. If a new vulnerable attacker is found, return to step 1; otherwise, terminate.

[0313] The algorithm returns a list of all vulnerable variables identified through saturated cells, along with guesses of the private values ​​for these variables. The algorithm also returns a list of vulnerable variables and corresponding private value guesses generated from the pseudo-inverse of the attack.

[0314] 1.6.9 Consistency Check-Based Attacks on COUNT Tables Another COUNT attack algorithm further improves the quality of guessing the private values ​​of variables by determining impossible solutions. To do this, the algorithm fixes one of the private values, which is equivalent to adding an additional constraint to the system of equations. Instead of solving the original system of equations, for a given set of variables... i and variables i Presumed private value s The algorithm then tests whether it exists. v , so that: , , and That is, the solver must test whether the system of equations remains consistent when fixing a given private value to a specific solution.

[0315] Checking for the existence of such a solution is a feature provided by most convex optimization software and is much faster than actually solving the system of equations. Therefore, this feature can be implemented iteratively to cover the entire set of possible solutions for a reasonably sized system of equations.

[0316] The main advantage of this attack method is that... d In a real-world scenario (i.e., when accurate statistics are published and no noise is added), the method will only produce accurate guesses. Additionally, note that to make this test faster, the conditions can be (as described in the following paragraphs) changed from... Relaxed to That is, instead of constraining the system of equations to solutions equal to 0 or 1, we constrain the system of equations using any actual value greater than 0 and less than 1.

[0317] The risk metric returned by the consistency check attack algorithm.

[0318] The attack algorithm is: 1. Perform a "saturation row attack on the counting table". 2. For each variable Given a hypothetical solution *s*, we test whether such a solution is possible. If only one solution *s* is possible for any variable *i*, then we infer that the private value of variable *i* must be *s*, and therefore we must update the system of equations accordingly: a. Subtract the contribution of the inferred private value from d.

[0319] b. Remove the rows and columns corresponding to saturated cells and private values ​​from cell A, respectively, to generate... .

[0320] c. Return to step 1. Use Replace A.

[0321] 3. If a solution cannot be determined for any variable, then terminate.

[0322] The algorithm returns a list of all vulnerable variables that can be accurately guessed, along with their corresponding private values.

[0323] 1.6.10 Attacks on COUNT Tables Based on Linear Constraint Solvers Another possibility is to impose constraints on the problem. eased to That is, instead of constraining the system of equations to solutions equal to 0 or 1, we instead constrain the system of equations using any actual value greater than 0 and less than 1. Then, each resulting guess is rounded to the nearest integer.

[0324] The key computational advantage of this approach is that the system of equations then falls into the category of convex optimization. Most scientific computing software provides very efficient solvers for such problems. However, to handle very large systems of equations, we present them in two forms, either simultaneously or sequentially. The constraint relaxation is solved for all columns.

[0325] The risk metric returned by the attack algorithm is obtained by attacking the linear constraint solver. The attack algorithm is: 1. Encode the set of the COUNT table into a system of equations.

[0326] 2. If the system of equations is small, then solve the entire system of equations; , Under the constraints Minimum.

[0327] 3. If the system of equations is too large to be handled using the first method, then solve each column separately; that is, for each... j=1,2,…,c Use subscripts to independently represent columns, in constraints lower envoy Minimum.

[0328] 4. In both cases, we obtain an estimate. It is difficult for us to set a threshold for the estimator to obtain... That is, for each variable i Hexing j , ,if Otherwise, it is 0.

[0329] The algorithm returns a guess of the private value for each variable.

[0330] 1.6.11 Measuring the accuracy of a COUNT attacker's guesses The system of equations measures or estimates the accuracy of a COUNT attack in guessing the correct value of a single record.

[0331] Heuristics suggest that stable guesses consistent with published data are more likely to be correct than others. We first consider the stability of adding or removing accessible information. Because information is delivered through published statistics, the likelihood and extent of guess variation should be considered if an attack is carried out using only a subset of published statistics. By doing this multiple times, using a different but random subset each time, we can see the stability of the guess. This thus accounts for the attacker's uncertainty.

[0332] Despite their power, none of the solver-based attacks listed above can easily derive a measure of the accuracy or likelihood of correctness of the proposed guesses after adding noise. Note that solver-based attacks do not include methods using pseudo-inverses, which instead provide a direct measure of the quality of the guesses. We offer three solutions: 1. As mentioned above, this method uses pseudo-inverses to identify which guesses are accurate. This approach identifies guesses that can be obtained from statistical releases. d It is a conservative view to infer precisely which variables are involved. This is because the fact that counting is discrete makes it much easier to guess, and therefore much more accurate than guessing from a list that is completely deterministic of pseudo-inverses.

[0333] 2. Measuring the stability of changes in available information. That is to say, if only the publication is observed... d A small fraction of the probability will be different if the measurement guesses are made.

[0334] 3. Another way to measure stability is to quantify how changes in the guess will affect the goodness of fit. Consider the gradient of the objective function; that is, the objective function relative to the unknown variables. v The first derivative of the gradient (this gradient varies depending on the norm used for optimization) is used. If the proposed solution is 1 and the gradient is negative, this solution is considered stable because we can only reduce the error by increasing the guess. Conversely, if the guess is 0 and the gradient is positive, the solution is considered stable. The gradient is used to determine how much the overall ability of the guess to replicate the observed publication changes when a given entry of the guess is perturbed. Additionally, the gradient informs the stability of the guess by estimating the worst possible fit of changing the guess value to the overall population.

[0335] 1.6.12 False Alarm Check Detecting false positives allows for avoiding overestimating the level of privacy risks and flags potential attacks that could actually lead to incorrect guesses.

[0336] Some attacks, such as the SUM iterative least squares attack, risk false positives; that is, they can claim that a variable is vulnerable when it is not. In response to this risk, a double-checking process is included in the system of equations.

[0337] To check whether the proposed privacy attack can accurately recover the secret, an additional equation is simulated and inconsistency checks are performed. Inconsistency checks can also be performed on large sets of equations.

[0338] To verify the existence of an attack, one of the following methods can be used: 1. Add a new equation to the system of equations that constrains a supposedly vulnerable variable to a value different from the solution returned for that row in step 2. For example, if the solution is v17 = 88, add the new equation "v17 = 89" to the system of equations. This correspondingly enhances the vector of statistical data. d .

[0339] 2. Perform any of the following operations: a. Solve the enhanced system of equations using an iterative solver. The solver returns whether the system of equations is considered inconsistent. If the system of equations... Still consistent Then we know that the value is not actually vulnerable; it is a false alarm.

[0340] b. Calculate the left-hand side (matrix) of the system of equations. A rank and augmented matrix The rank of the augmented matrix is ​​and the size of the augmented matrix is ​​. The matrix, through A Add a vector of statistical data to the right side d To establish. If A The rank is less than If the rank is , then according to the Rouche-Capelli theorem, the variables in the last equation cannot be denoted by . A Completely certain.

[0341] If the value of a row was previously fully constrained by the rest of the equations, adding this new linear constraint will make the system of equations inconsistent because the new constraint contradicts the rest. Therefore, there is no solution for this new system of equations. If adding such a constraint does not make the system of equations inconsistent, it means that the value of the row is not fully constrained by the rest of the equations, and therefore an attack on it would be a false alarm. If necessary, Canary performs this consistency check on each row considered vulnerable in the second step, and can thus verify which row is truly at risk.

[0342] 1.6.13 Multi-Target Optimization (MOO) Attack Another approach to adversarial testing within the Canary system is based on multi-objective optimization (MOO) gradient descent, and is known as Canary-MOO. As described below, Canary-MOO constructs a set of estimated variables and updates these estimates iteratively based on the error between published statistics and the same statistics calculated from these estimates. The error of each pair of published / estimated statistics is considered the objective to be minimized (i.e., the objective is to reduce the error in each pair).

[0343] The algorithm iteratively updates the estimated set of private values ​​in a manner that minimizes the error between the published aggregate query and the same query executed on the estimated private values. Unlike, for example, Canary-PINV, Canary-MOO makes a "best guess" of the values ​​of private variables for which the system of equations cannot be fully determined, and is able to handle a wider range of aggregation types; both can be used individually and in combination.

[0344] Canary-MOO estimates a vector of private values. Initialize to real private values The average value is uniformly distributed. It is assumed that this average value is known to the opponent, or that she can make an educated guess about it. At this stage, general background knowledge can optionally be incorporated by adjusting the uniform initialization to account for the known distribution of private values ​​related to the quasi-identifier. For example, if Let be a vector of wages, and given that the manager's income is above average while the doorman's income is below average, then the wages belonging to the manager are... All will increase slightly, while the wages of all individuals belonging to the gatekeepers will decrease slightly. This will be achieved through specific... Setting values ​​to known values ​​also allows for the incorporation of specific background knowledge during the initialization phase. General background knowledge regarding the constraints on specific variable values ​​can be incorporated into the gradient descent process itself.

[0345] Additionally, a small amount of random Gaussian noise can be used to... Initialization allows for multiple Canary-MOO runs from different initialization states, thus providing the following confidence metrics in the results.

[0346] Where G represents the iid random variable derived from the Gaussian distribution, and G represents the... and The iid random variable is derived from a Gaussian distribution. Values ​​other than 100 can also be used.

[0347] After initialization, the MOO algorithm executes the following process iteratively: 1. Regarding Execute queries on the data to obtain estimated aggregate statistics. .

[0348] 2. Calculation The error between the published aggregate d and the aggregate d.

[0349] 3. Based on error pairs Update.

[0350] 4. Normalization makes the mean equal to the mean of the original private values.

[0351] 5. Any value lower than the minimum value of the original private value or higher than the maximum value of the original private value. Set a threshold.

[0352] 6. (Optional) Based on background knowledge regarding the constraints of specific variables, for any particular Set a threshold.

[0353] The algorithm can be configured to... Terminate immediately after all private variables have ceased to change significantly, after all private variables have been stably determined to be a set threshold percentage of the actual values ​​of all private variables, or after the maximum number of iterations has been completed (e.g., the number of times a reasonable adversary might use).

[0354] Risk metrics returned by Canary-MOO: Figure 16 A diagram of the risk measurement algorithm is shown. The algorithm (including all variants described below) returns a guess corresponding to a private value for each variable.

[0355] The specific implementation of multi-objective optimization is highly customizable and flexible, and it is possible to combine gradient descent based on different types of statistics, more heuristic update rules, and initialization strategies (e.g., initializing certain values ​​to the output of other attacks, as in 1.6.13.7).

[0356] 1.6.13.1 Batch Updates under SUM Statistics Batch updates for multi-objective optimization are used to infer sensitive variables from a set of published statistics.

[0357] This improves the efficiency of multi-objective optimization when processing SUM aggregated statistics by using multiple error terms simultaneously to update variable estimates. Instead of updating based on a single objective (i.e., an error based on a single published and estimated statistical pair), it immediately considers the errors of any number of pairs. The errors are scaled relative to their objective to avoid a single error dominating batch updates. For each variable, the scaled errors are averaged and used to update each variable at a time.

[0358] Implement error-pair-based updates through batch updates. Update, where batch size B It can be from 1 to m any value (where m (This refers to the number of aggregated statistics published). B=1 In this case, the algorithm selects the largest error statistic and updates based on it. Batch updates are used to achieve error-based updates. Update, including batch size B It can be from 1 to m any value (where m (This refers to the number of aggregated statistics published). B=1 In the case of [missing information], the algorithm selects the largest error statistic and updates it based on that.

[0359] exist B <m In this case, the algorithm selects the preceding one. B The maximum error statistics and based on B Update according to each error. Due to batch size... B <m The reason for the computational efficiency in this case is that the algorithm only considers... The elements involved in the aggregate statistics of the batch are those that are present in the data. B=m In this case, statistical data is not selected based on error, and the update is changed to consider all statistical data at once.

[0360] A crucial concept for batch updates is that all errors must be scaled according to the target statistics of the errors. This prevents numerically large but proportionally less significant errors from dominating the market. renew.

[0361] For SUM statistics, the following will be used: B=m The batch update rule is implemented as follows B=m Implement as

[0362] in j index m A collection of statistical data, index i n A private variable, and A i This indicates a vector slice of the equation matrix for the private variable i. This update rule can be intuitively considered as an average scaling error across all statistics. To update, this will be done by first scaling the errors to their target statistics, and then multiplying each of these scaled errors by 1 or 0, depending on... Does it exist in such A i The indicated statistical data. Divide the summed scaling error by . Total number of statistics involved This allows for averaging of updates. For smaller batches, this can be done by targeting the first few batches where the scaling error is not the largest. B One of the statistical data is used to temporarily modify the vector of statistical membership. A i This sets its entries to 0.

[0363] 1.6.13.2 Perform batch updates for AVG statistics Canary-MOO can refactor AVG statistics into SUM statistics and include them in batch updates of SUM statistics. This is accomplished simply by converting AVG to SUM by multiplying the AVG statistics by their queryset size.

[0364] in It has 1 and 0 n A dimensional vector indicating which elements contribute to AVG statistics. This vector can be appended to... A The new SUM statistics can be appended to d Based on this approach, AVG is considered equivalent to SUM. It has 1 and 0 nA dimensional vector that indicates which elements contribute to AVG statistics.

[0365] Version 1.6.13.3 performs batch updates for MEDIAN statistics. This improves the efficiency of multi-objective optimization when processing MEDIAN aggregated statistics by using multiple error terms simultaneously to update the estimates of variables. This is accomplished by linearizing the updates from nonlinear median statistics, which is achieved by considering only those variables that directly affect the median. MEDIAN statistics only carry information about the central values ​​of a set of variables. Therefore, the same batch update rules as SUM and AVG statistics are used, but only the central values ​​(the median of the odd-numbered set of variables and the two central values ​​of the even-numbered set) are updated.

[0366] Many specific update rules have been developed for median statistics, representing non-linear statistics for specific categories. MEDIAN statistics present more complex problems than AVG and SUM statistics because the error in the median value does not provide specific information for the same category: the MEDIAN error does not convey information about all members of the queryset, but only where the partition should be to divide the queryset in two. Therefore, the default option for MEDIAN statistics is the same as the batch update rule used for SUM statistics, but with minor modifications: only update the median value (for odd-numbered QSS querysets) or the values ​​on either side of the median (for even-numbered QSS querysets). This can be achieved by applying the median error to a given... To achieve this, all non-median entries are temporarily set to 0 to access the query matrix. A The operation, in which This indicates the current median query. In this way, only the median entry is updated, as it is temporarily the only variable that helps with the statistics. This aligns with the intuition that the median incorrectly conveys limited information about the members in the query set whose values ​​are not directly related to determining the median itself.

[0367] 1.6.13.4 Noisy Gradient Descent By adding a cooling factor based on the noise distribution during gradient descent, the convergence of multi-objective optimization is improved when dealing with noisy statistics. The cooling factor, proportional to the noise added to the published statistics, is incorporated into the gradient descent to help prevent noise from dominating the process.

[0368] Assuming that Canary-MOO is frequently used to estimate privacy risks with noisy data, the algorithm can be modified to scale its iterative updates. times, of which λ Defined as , where GS is the global sensitivity of the statistical data (this term comes from the differential privacy literature). This 'cooling factor' allows gradient descent to take into account noisy statistics, thus converging to a stable solution that is not dominated by noise.

[0369] 1.6.13.5 Specific Uses of the Median: Median Snapshot Procedure Median statistics are difficult to use in optimization strategies because they are non-linear functions of variables. However, median statistics convey a wealth of information about the variables that can be used in other ways. The median of an odd number of variables corresponds to the value of one of the variables themselves. Therefore, given an estimate of the value of each variable in an odd number of variables, the variable closest to the known median is "captured" at that median value. This technique can be used to aid optimization during gradient descent or as a post-processing step. This snapshot procedure can be used, for example, in combination with any of 1.6.13.1, 1.6.13.2, 1.6.13.3, or 1.6.13.6.

[0370] When feeding median statistics to Canary-MOO, a specific method can be used for the statistics where the number of variables contributing to each statistic (referred to as the query set size (QSS)) is odd. For these statistics, the published true median directly corresponds to one of the values ​​in the query set. Canary-MOO uses this by iterating through the median statistics for each odd-numbered QSS to find the corresponding... median value, and put this The value is "captured" to the published median. This process can be performed after the iteration terminates, or it can be performed repeatedly at regular intervals as part of the iteration process.

[0371] 1.6.13.6 Canary-MOO "bag grabbing" method with multiple query types It can effectively attack statistics of multiple aggregate types with the same sensitivity value.

[0372] Canary-MOO's flexibility allows for efficient updates from various query types, provided appropriate update rules are provided. If necessary, the algorithm can also provide the option to input custom update rules in addition to the update rules already provided for SUM, AVG, and MEDIAN. Using the methods described above ( (Batch update of average statistics) Statistical data can be used d j and n dimensional vector To represent a non-SUM query, the vectors can be appended to the statistical data respectively. d The existingm dimensional vectors and equation matrices A Assuming A of m Each column in the query is associated with a query type and a corresponding update rule (user-specified or hard-coded), allowing Canary-MOO to provide a set of aggregated statistics and generate... It iteratively approaches the true private value by taking into account the most erroneous statistics individually or as part of a batch update and using the provided update rules corresponding to the type of statistics.

[0373] This allows for the simultaneous use of information from multiple types of aggregated statistics, thereby collectively improving estimates of sensitive variables. Any combination of any type of statistics can be considered, provided an update rule is provided for each statistic.

[0374] 1.6.13.7 Combinations of attacks using Canary-MOO Combining different attackers can increase the intensity of a collective attack.

[0375] Some attacks only guess the values ​​of a subset of variables that can be derived with high determinism. The results of such attacks (such as discovered variables from 1.6.1 or fully deterministic variables from 1.6.3) can be used to improve the optimization for attack guesses on variables that are still unknown. This is accomplished by initializing the optimizer's startup state to include known variables from other attacks.

[0376] Canary-MOO can be integrated with other parts of Canary. In particular, because... With its flexible initialization, Canary-MOO can be initialized using any other attack (such as Canary-PINV (Section 1.5.2) or a simple difference scan procedure). quick Rapid Heuristics The output estimates are initialized using private variables. Known variables can be removed from the SUM and AVG equations to which they contribute, if this has not already been achieved via a one-difference scan procedure. If the variables are only known within certain constraints (e.g., using median statistics from a one-difference attack), these constraints can be incorporated into the gradient descent process.

[0377] 1.6.14 Modeling Background Information Canary can also directly encode the opponent's background knowledge into the set of linear equations.

[0378] The opponent may possess different types of auxiliary information, which Canary can encode: ● Percentage of known private attributes:An adversary may access the private values of a subset of all individuals. For example, this may occur if data is collected across departments, and the attacker can access the data of their own department but wants to learn the private attributes of all persons in other departments. For SUM tables, this type of background knowledge is encoded as additional linear equations in the system of equations. The additional equations fix the values of variables to their actual values, for example, v1 = 18200.

[0379] ● Common sense about a group of people: An adversary may have specific knowledge about the population, either because she is part of the group, or because of "common facts". For example, she may know that the monthly salary of a manager will always be in the range of 5k-10k. For sum tables, this type of background knowledge is encoded as inequality constraints, for example, 5000<v2<10000.

[0380] ● Lowest and highest rankings: An adversary may know the ranking of private values, such as which persons have higher income than others, or she may know that the private value of a target is the maximum or minimum value among all values. This additional information makes it easier to extract the values of individuals. This type of background knowledge is encoded as additional linear or inequality constraints, for example, v10<v1<v7, or v1>vX for all X in the dataset 1.7 Abe Abe is a system that can be used to explore the privacy-utility tradeoff of privacy-preserving techniques (such as noise addition) for aggregated statistical data. It can be used to compare different techniques or different sets of privacy parameters for a given data privacy mechanism.

[0381] Abe is integrated with Eagle and Canary. For a specific privacy technique and the parameterization of said technique, Abe tests whether all interesting insights that Eagle can extract from a set of statistical data still hold. At the same time, Abe tests whether all individuals who are at risk in the original release are protected. Therefore, Abe evaluates both privacy and utility simultaneously.

[0382] Abe takes the following as inputs: a set of aggregated statistics or statistical queries; a privacy-preserving technique (for example, a noise addition function) and a list of different sets of privacy parameters for this privacy function (for example, a list of noise scale values).

[0383] For each privacy function and set of privacy parameters, Abe evaluates how well the aggregated statistics generated by the data privacy mechanism preserve data insights using the given parameter settings (utility test) and the possibility that the aggregates still expose the private data of individuals (attack test).

[0384] Alternatively, Abe can output a privacy parameter (e.g., ε in the case of a differential privacy mechanism) that satisfies a certain criterion (e.g., maximum ε) to defend against all attacks.

[0385] The Discovery Tester module in Abe tests whether all insights, such as "the largest number of people in group X have attribute Y," are also found in private statistics. As an example, if the privacy protection function being tested is noise, and in the original statistics, all employees in the sales department have the highest SUM (salary), then Abe's Discovery Tester module tests whether this still holds true when looking at the noisy SUM with a certain amount of noise added.

[0386] Abe can also employ a simpler method to measure utility and various settings for privacy parameters to easily calculate distortion statistics (e.g., root mean square error, mean absolute error).

[0387] It also displays distortion measures related to noise to the end user. Measures such as root mean square error and mean absolute error are used to capture the amount of data that has been disturbed.

[0388] The attack system module in Abe tests whether all privacy attacks have been defended against. This step uses Canary's privacy attacks. Abe tests how accurately this set of privacy attacks can recover an individual's private data from the private statistics compared to the original statistics. For example, if one of the attackers using Canary's SUM function can 100% accurately and reliably learn a person's salary from a set of original SUM tables, then using Canary, Abe tests how accurate the attacker's guesses about that person's secrets are from a noisy SUM table.

[0389] Lens measures both the privacy and utility effects of various ε settings and can be used to present detailed, realistic, and understandable information about the impact of various ε settings on both privacy and utility. The system automatically captures and displays all of this information.

[0390] ε can be set using many user-configurable rules.

[0391] As an example, the system can be configured to determine the highest ε consistent with defeating all attacks. Therefore, if the set of various attacks applied to the release of data products constitutes a representative set, sufficient security protection for sensitive datasets can be provided while maximizing the utility of the data product release.

[0392] As another example, the system can also be configured to determine essentially the minimum ε, thereby preserving the utility of the data product release. Thus, all findings from the data product release will be retained while maximizing sensitive privacy.

[0393] 1.7.1 Determine if the attack was successful How Abe determines whether a privacy protection function successfully defends against an attack depends on the type of privacy attack. Abe relies on certain definitions of attack success and what constitutes a data breach. For example, for continuous private variables, such as salary, the rule defining a "correct guess" could be whether the guessed value is within a configurable range of the actual value (e.g., within 10%). The rule could also be whether the difference between the actual and guessed values ​​is less than a certain amount, or whether the actual and guessed values ​​are within a certain proximity to each other in the cumulative distribution function (taken over the dataset). For categorical variables, the rule tests whether the correct category was guessed.

[0394] The following sections describe in more detail the Abe attack testing process used for different types of privacy attacks on different aggregates.

[0395] 1.7.1.1 When to stop an attack Figure 17 A diagram illustrating the rules for testing attacks and determining their success is shown. Abe contains rules regarding when to block attacks and when not to block them (e.g., at what noise levels).

[0396] There are two methods for finding privacy parameter thresholds that prevent attacks, but both methods rely on the same definition of a successful attack.

[0397] If the probability of an attack correctly guessing a private value from noisy statistical data is greater than an absolute threshold Therefore, an attack can be considered successful if the attacker is likely to make a good guess, and if the attacker's chance of making a good guess is significantly higher than the baseline before observing the statistics.

[0398] Another definition of a successful attack is... replace condition.

[0399] Variable focusing method. In this method, there is a list of variables to be targeted. This list can be output by the attack itself (see, for example, 1.6.3), or it can simply be a list of all variables.

[0400] In the variable-focused approach, we independently test whether an attack could lead to a privacy breach for each variable. The method considers both absolute confidence and confidence changes. Each individual entity (i.e., each sensitive variable) is examined, and an attack on that entity is considered successful if both relative and absolute conditions are met.

[0401] To test whether the attack was successful, Abe's attack module performed the following: 1. We perform a baseline attack on private variables. This is a simple method for guessing the configurability of private variables (see Section 1.7.1.2). A baseline attack gives the probability of success of the attack without publishing statistics and is called... .

[0402] 2. We measure the probability that an actual attack on private statistics will produce a guess that closely approximates the actual value. We call this probability... .

[0403] 3. We compare these metrics with our thresholds. a. ? b. ? Furthermore, if both of these conditions are met, then under this parameter setting, we will mark this variable as still vulnerable.

[0404] As an example, suppose we sample from the distribution of private variables in the dataset, and this baseline attack... =20% of the time, correctly guessed a person's private value. Then, we discovered that a Canary SUM-PINV attack could be performed on noisy versions of certain SUM tables. = 85% of the time, guessing correctly. We say that if, after we release the statistics, attackers gain at least a better chance of guessing private values... = 20%, then the attack constitutes an invasion of privacy, and if this leads to = It's only risky to guess correctly 80% of the time. Therefore, in this case, we find that attacks on noisy statistics about private values ​​are risky, and the noise is insufficient to prevent such attacks.

[0405] Holistic Approach. In this approach, we don't consider each row in isolation. Instead, we consider how many correct variables the attack as a whole has obtained. Therefore, all vulnerable variables are considered together, and the method determines the proportion of variable groups that will be correctly guessed.

[0406] Similarly, we can use the baseline method described above and see the percentage of variables that were correctly identified by that method. .

[0407] Then, we can look at the percentage of variables that were actually obtained correctly during the attack (as a function of the privacy parameter), and call this... .

[0408] Now, we again compare the baseline percentage and attack percentage with relative and absolute thresholds to determine whether the attack was successful. These thresholds can be set to the same or different values ​​as the thresholds in the variable focusing method.

[0409] Consider the following scenario: We want to test whether the noise from the DP mechanism is high enough to protect the publication of the COUNT table. The COUNT table is a breakdown of patients' medication use based on other publicly known demographic attributes, while the private category (individual's medication use) has three different subcategories {never, rarely, often}. We might first set the baseline to... = 33%, because if an attacker needs to guess someone's category without any other information besides the existence of these three categories, the attacker has a one-third chance of guessing correctly. Then, we perform a Canary discrete solver COUNT attack on a noisy version of the COUNT table we want to publish. The COUNT attack results in the variables being correctly guessed. = 60%. For the row-based approach, we then compare these percentages with relative and absolute thresholds to determine whether the overall attack was successful.

[0410] Notes on relative thresholds and absolute thresholds Relative threshold and absolute threshold These are user-configurable system parameters. Note that for both methods, sometimes absolute thresholds are used. Setting it to 0 might be appropriate. In this case, any meaningful improvement in the guessing aspect compared to the baseline method seems problematic. Therefore, in this situation, it could be suggested to set the absolute threshold to 0 and use only the relative threshold.

[0411] 1.7.1.2 Baseline Method for Guessing Private Values For relative thresholds, a baseline is needed for comparison. This baseline represents how confident an attacker would be in guessing a person's value if that person's data were not included in the dataset.

[0412] A baseline guessing component can be constructed, and several baseline guessing strategies can be implemented, such as random sampling from the distribution or guessing only the most likely value each time.

[0413] Baseline This measures an attacker's confidence in determining an individual's private value without the release of statistical data. There are different ways to define this prior guess.

[0414] One approach is to uniformly sample the private values ​​of individuals from the original dataset. i The times, and the measurements were taken at the time. i The frequency of correctly guessed in the next sampling.

[0415] Alternatively, one can formalize Bayesian priors on private attributes based on general background knowledge. For example, wage distributions across different European countries can be inferred from official statistics (Eurostat income distribution statistics: http: / / ec.europa.eu / eurostat / web / income-and-living-conditions / data / database), and an attacker attempting to guess a person's wage without any specific information about that person is likely to make a reasonable guess using this external information.

[0416] People can also provide Abe with a hard-coded list of prior confidence values, or a list of prior guesses, for each entity in the dataset. This list can be based on the attacker's profile. For example, an employee working in a company's human resources department trying to understand other people's salaries from aggregated statistics might have high confidence in the income of their direct colleagues but less confidence in the income of other members of the company. This feature can be useful when people want to protect against very specific risks or only publish statistics to a limited user group.

[0417] 1.7.1.3 Sampling-based methods for determining the probability of attack success Abe used a set of attacks from Canary to test whether the parameter settings of the data privacy mechanism adequately reduced the risk of breach. Different attacks used different methods to test their success. For all privacy attacks, there was a common mechanism to test their success. This method independently sampled statistics several times and evaluated the frequency of attack success across all trials. The percentage of times the attack guessed correctly determined the confidence level of the attack.

[0418] For example, to test whether noise added at a specific ε via a differential private release mechanism is sufficient to defend against symbolic solver attacks on the SUM table, Abe tested noise with this ε. i These different noisy release samples attack the noisy tables. i There are several different versions, and for each of them, the guesses about the rows are tested to see if they are correct (as defined above in 1.7.1). Then, for the tested ε value, the correct guesses are divided by the total number of guesses to obtain the estimated success rate of the attack on each vulnerable row. .

[0419] 1.7.1.4 Calculate the relationship between noise and attack success By modeling an attack as a linear combination of random variables, the probability of an attack's success can be calculated (where, for continuous variables, success is defined as falling within a certain range around the actual value). In contrast, determining attack success by regenerating noise and repeatedly attacking is neither fast nor accurate.

[0420] Abe's attack testing module can be used to test the effectiveness of noise addition in preventing attacks. However, for certain Canary attacks, there are alternative methods for evaluating attack success. These will be explained in the following sections.

[0421] To identify privacy risks in the SUM or AVG tables, Canary summarizes all information that could be used by attackers into a system of linear equations.

[0422] Having statistical data vectors ,in q It is in all q The table generates a total m The number of times a statistical statistic is queried.

[0423] Canary's PINV version calculates the query matrix. A pseudo-reversal A + And return a matrix A + row index i ,in

[0424] Except for the entries i=1 A vector whose outermost element is all zero. If the above relation applies to the first... i If the statement is valid, it means that the private value is valid. It is completely determined by the set of statistical data.

[0425] Lens generates differentially private noisy statistics to protect these vulnerable variables. If a Laplace mechanism is used to generate the differentially private distribution, the vector of noisy statistics can be described as...

[0426] From having a mean of 0 and a scale The noise value obtained independently from the Laplace distribution

[0427] Added to each statistic via the Laplace mechanism The noise will be determined based on the query. Overall sensitivity and privacy parameters Scaling is performed. In the most common case... All statistics in the dataset come from the same aggregation and have constant sensitivity, while in the simplest case, they are derived from the same aggregation. The measured privacy budget will be evenly distributed across queries, making ε and GS It is a constant. To simplify the notation, the query index can be omitted. k And use j for Indexing statistical data and noisy values .

[0428] The goal of Abe is to find ε, which adds enough noise to the statistical data to defend against all attacks identified by Canary. Based on the analysis of attacks on the SUM and AVG tables above, the following methods can be used to find a suitable ε.

[0429] Gaussian approximation of attack probability The PINV attack returned by Canary is achieved by applying the attack vector. From a set of noisy statistical data Generate private values ​​for individuals speculation

[0430]

[0431] Therefore, attacks on noisy statistical data produce the following noisy guesses: actual value Plus RV η . η yes j 1 independent Laplace variable The weighted sum, where

[0432] η The distribution of is not irrelevant to the analysis and calculation. However, η The moment generating function is known, and therefore can be computed. η First and second moments

[0433] as well as

[0434] It's okay i The L2 norm of the attack vector on, and Given that all statistics are derived from queries with constant query sensitivity GS and the same ε, the variance of the attacker's guess becomes:

[0435] In this particular case, one way to measure the success of an attack is to calculate the attacker's value. v i The cumulative probability of making an accurate guess, i.e., noise. η The probability of the attack being less than a certain error tolerance. In this case, Abe calculates the percentage of actual attack success as:

[0436] Although it is difficult to derive the probability density, and therefore also difficult to derive the cumulative distribution, η The function is , but there still exists a distribution that can be well approximated by the sum of multiple Laplace RVs.

[0437] For a large number of summated Laplace RVs, these sums approximately follow a Gaussian distribution.

[0438] The better the approximation of the normal distribution, the better the statistical data. m And therefore the larger the sum of the Laplace RVs, the greater the number of RVs.

[0439] Under this Gaussian distribution approximation, the probability of a successful attack, i.e., the attacker's noisy guess, is in the actual value... v i One around α Within the required accuracy range, the following analysis and calculations can be performed:

[0440] in erf It is an error function, and . It follows a semi-normal distribution, and for detected attacks... For each of these, Abe uses its cumulative distribution function. To approximate Abe uses the same baseline comparison and absolute confidence threshold as described above to determine whether an attack is likely to succeed for a given value of ε.

[0441] Mean absolute error of the attacker's noisy guess Based on the attacker's guess about noise η Instead of suggesting a list of different ε values ​​to test, Abe can directly suggest possible defenses with attack vectors, using the same Gaussian approximation of the distribution. Given the attack ε. If we assume Then the attacker's guess of the relative mean absolute error is

[0442] Abe can now calculate the maximum ε, at which the average error of the attacker's guess is expected to deviate from the actual value by more than α%.

[0443] This ε serves as an upper limit on how high ε can be set before an attack is likely to succeed.

[0444] Root mean square error of the attacker's noisy guess If one does not wish to rely on the Gaussian assumption, Abe can still analyze and derive the expected defense against a given attack. The solution is based on calculating the relative root mean square error (rRMSE) of the attacker's noisy guesses.

[0445] Similar to the relative mean absolute error, Abe will use this metric of the expected error of the attacker's guess given ε to derive a privacy-preserving upper bound for ε.

[0446] Converting one type of risk measurement to another Under the assumption that the attack guesses follow a normal distribution (i.e., Gaussian), these three metrics can be transformed into each other. This is because all existing parameters depend only on the norm of the attack vector, the secret value, and the sensitivity. Therefore, algebraic operations allow one to be expressed as a function of the others.

[0447] From a user's perspective, this means that if a user prefers to understand their risk through a root mean square error (RMSE) threshold, then we can calculate a threshold corresponding to the provided probability of a successful attack. Conversely, given an RMSE, we can suggest a probability of a successful attack that would produce that threshold.

[0448] This ability to move between metrics is key to enabling a wider range of users to accurately assess risk. Depending on the technicality of the user's background or the nature of the proprietary value, different metrics will become more relevant.

[0449] COUNT query results When attacking COUNT queries, we have two main types of attackers. One type uses a pseudo-inverse as an attack against SUM queries. In this case, the same method described above can be used to generate an upper bound on ε; that is, if the value ε exceeds the upper bound, the attack successfully generates a good guess of the entity's private value. The second type of attack against COUNTS uses a high-level constraint solver. In this latter case, the analytical method described above cannot generate an upper bound on ε. However, the iterative method still performs well and is a valid option in this case. In the following, we provide an analytical version that does not require executing the attack multiple times as the iterative method, making it possible to generate a scalable method for determining the appropriate value of ε.

[0450] To generate an upper bound on ε in the case of an attacker using a solver, we proceed in two steps. First, we define the attacker's success as the fraction of an accurate guess. Let this quantity be called... p This is because it can be interpreted as the marginal probability of guessing correctly. Note here that... p It's not what the attacker observes, but rather a measure of the damage such an attacker might cause. Unfortunately, there's usually no way to calculate this. p The formula in closed form. Therefore, as the second step, we generate p An approximation of this value is called... To generate this approximation, we use an attacker to implicitly perform maximum likelihood estimation of the private value. Then, before thresholding, each estimate of the private value approximates a normal distribution with known mean and variance. This allows us to generate values ​​using average guesses and variance. p The approximate value of the mean field is obtained, thus yielding:

[0451] in If one category is dominant, adjustments can be made. α This ensures that, within a large ε-limit, the recovered guess score is the same as the guess score obtained when attacking the statistics without adding noise. β It is the variance and equal to , in g It is the number of GROUPBYs that have been released. yes A The average of the squares of the singular values, n It is the number of records, and d It is the number of possible values ​​for the discrete private value. Then, using... This allows us to approximate how good the attacker is.

[0452] All different attack testing mechanisms can measure whether an attack is likely to succeed or is defensible given ε. Which method is appropriate depends on the specific privacy attack and the user's risk concerns.

[0453] 1.7.1.5 A method for defining attack success based on distinguishing between minimum and maximum values Differential privacy relies on the fundamental idea of ​​making it difficult to distinguish whether someone is in a dataset, which is equivalent to making it difficult to distinguish between minimum and maximum values. However, it has not yet been implemented to use this concept to measure the success of a particular attacker.

[0454] For consecutive sensitivity values, another way to define a successful attack is to determine whether someone's value is within the minimum or maximum of the allowed range. This definition of a successful attack also does not depend on the sensitivity value of any particular individual (unlike other definitions of successful attacks mentioned above, such as "within 10% of the actual value").

[0455] The system assumes that to determine this, an attacker would take a range of variables, and if an estimated value of someone's value is reportedly in the upper half of that range, the attacker would guess that value is the largest, and if the estimated value is reportedly in the lower half of that range, the attacker would guess that value is the smallest. The system can then measure the probability that the attack correctly guessed the value as the minimum for the actual minimum value (or, similarly, the probability that the attack correctly guessed the value as the maximum for the actual maximum value). The system can calculate this probability by analyzing the probability distribution of the guesses (determined by the level of noise added) and looking at the probability that the guess would fall in either half of the range. The best-case scenario for privacy is that the attack will succeed 50% of the time (equivalent to random guessing). The worst-case scenario for privacy is that the attack will succeed 100% of the time.

[0456] Users can configure the percentage of time during which such an attack is allowed to succeed. Abe can then use this percentage to determine how much noise must be added.

[0457] 1.7.2 Reports generated by Abe Abe generates various summary reports that help users understand the privacy-utility trade-offs of privacy protection mechanisms such as differential privacy.

[0458] Results of variable focusing attack test Some privacy attacks generate guesses about every row in the dataset, and Abe tests each of these attacks individually. Abe generates the following report for these attacks. Figure 18 A horizontal bar graph showing findings generated by Eagle illustrates the case where information is retained, and the findings are a function of the value of ε. Figure 19 A horizontal bar chart is shown showing at-risk individuals identified by Canary for different attacks, where at-risk individuals are a function of the ε value. Sliding the vertical lines on the chart helps to immediately understand which attacks will be stopped and which findings will no longer be retained.

[0459] Differential private noise addition has been used as a privacy mechanism and ε (the parameter for DP noise addition) has been varied. For each ε, it has been tested which findings were preserved and which individuals were protected. The bars represent the best-fit thresholds for the range of ε that allow findings to be preserved, or the individuals to be protected. Larger ε (further to the right) means less noise and less privacy.

[0460] This image illustrates how ABE can be used to assist in determining the parameters for privacy mechanisms. Good parameter selection results in no successful attacks, but most findings are preserved.

[0461] 1.7.3 Abe's regular statistical releases of constantly changing datasets When planning numerous data releases over time, privacy protection parameters need to take into account not only the parameters of the current release, but also all subsequent releases and any updates to the specifications of sensitive datasets or data products.

[0462] Several techniques are proposed that first infer the strength of the attack as the number of releases increases, and then adjust the required privacy-enhancing noise addition accordingly.

[0463] So far, Canary and Abe have been used to run on a given dataset and a list of statistical queries. However, in many cases, the data used to generate aggregates changes over time, and new statistics about the same individuals will be published periodically. The more statistics published, the higher the risk of privacy breaches. This must be taken into account when using the output from Abe to select the appropriate level of privacy protection (such as, for example, the ε value for adding noise for the first private data release).

[0464] To understand why changing data is important, consider the following example scenario: A company decides to publish average salaries each quarter. In Q1, the average salary is $56k. In Q2, only one new person joins the company: a new salesperson. The average salary in Q2 is $58.27k. Knowing the number of people in the company, it's possible to calculate the exact salary of this new salesperson, which is a privacy violation.

[0465] Abe can be used to infer the risks of future data releases. Users need to tell Abe: 1. Which queries will be run repeatedly on the data? 2. At what frequency will the results be published? This frequency will be referred to as...F .

[0466] 3. How long will any given user remain in the analyzed dataset (e.g., approximately 12 years for a school admissions dataset). This duration will be referred to as... D .

[0467] exist D With historical data available for years, Abe uses the following process to infer risk: 1. During the duration D According to frequency F Historical data is split into snapshots.

[0468] 2. Generate all the statistics that should be published on each of those snapshots. 3. Run Canary and Eagle on the statistical data set to extract vulnerabilities and insights. 4. It also generates a comprehensive risk analysis based on historical data.

[0469] If we assume that changes in historical data roughly represent future data, then in the past... D The privacy parameters effective in the middle of the year will have an impact on the future. D Valid for one year. As an example, consider a database of student performance where students will be in the dataset for 12 consecutive years, and four different reports will be published each year, each containing a set of summary statistics on student performance. Historical data from students who have already left school can be used to set appropriate levels of privacy parameters for current students.

[0470] Abe simulation in the absence of historical data or with insufficient historical data available. D Database changes within the year, frequency of change: F Some key dataset characteristics are needed (such as, for example, the average rate at which users enter and leave the database, typical changes in an individual's private attributes, or patterns of user change across segment groups) to simulate database changes.

[0471] Another approach, independent of relying on real or simulated historical data, is to use theorems about data privacy techniques, such as the differential privacy theorem, to infer future risks. For example, for continuous data releases, one can predict how existing linear correlations in a person's data will reduce privacy protection by adding noise. Combinatorial theorems allow one to calculate the privacy level at each iteration. Make p The overall privacy level (ε) following the initial release. These theorems can then be used to infer an individual's risk based on future statistical data.

[0472] Furthermore, following Section 1.7.1.4, we can understand the attack vectors. a To assess the required privacy level ε. We observed there that if the data product query and GROUPBY variables remain constant, an attack on the first release of the data product will also be a valid attack on the second release. Furthermore, by simply averaging the results of the two attacks, it is possible to combine the two attacks into a single, more powerful attack. Using the same theory, it may be found that... p After the release, people can use the original attack vector. a To attack each release, then combine the attacks to create a more powerful attack. There we see: [This is achieved through merging...] p The attack vector obtained from this attack has an L2 norm, which is equal to the original vector. a Divide by Therefore, if Sufficient to protect the first release from attack vectors a Then it is necessary Let's protect it together p One release.

[0473] In some cases, in addition to theorems, the characteristics of data privacy mechanisms observed empirically can also be used to infer future risks.

[0474] In some cases, this may help reduce the privacy-utility trade-off. D This can be accomplished in the following ways: ● When the user exists D The user will be removed from the analytics database after the New Year.

[0475] ● Users are resampled for each release so that not every user is always included in the release, thus users will end up being included in (discontinuously) D Among the valuable releases of the year.

[0476] 1.7.4 Setting ε based on Canary and Eagle Canary can include multiple attacks. It runs all attacks on the expected release of statistics and recommends setting ε low enough (i.e., high enough noise) to block all attacks. For variable-focused attacks, it suggests the minimum ε required to defend each variable. The behavior of a global attack differs; there is no different ε for different variables. As the global ε decreases (i.e., as the noise increases), the global attack becomes less effective (i.e., makes less accurate guesses). Note that this number may depend on the specific noise added, so we might want a practical attack in a lot of noise attraction to get the correct variables. average percentage.

[0477] Abe uses this feature to recommend the ε used in Lens. It combines the output of row-based and global method attack tests. Abe may recommend setting the highest ε low enough to block all attacks, or it may leave an additional safety buffer (e.g., reducing ε further by 20%) for a more conservative configuration.

[0478] To find a sufficiently low ε to prevent all attacks, Abe can iterate through a list of candidate εs (e.g., "[1, 5, 10, 20, 50, 100]"), adding noise to the statistics based on the ε, and then use a Canary attack to attack the noisy statistics and see if the attack succeeds. This may require averaging many noisy trials. Abe will then select the highest ε so that the attack fails. Alternatively, Abe can directly calculate the desired ε using the formula from Section 1.7.1.4 above. However, by testing a series of different εs, simulating the addition of noise based on each ε, and attempting to attack the noisy statistics associated with each ε, the highest ε (i.e., the lowest noise level) can be selected to make all attacks fail.

[0479] Abe can also include utility in the decision to set ε. For example, it can set ε as low as possible under constraints that preserve all important findings (determined by Eagle) or that some distortion measure (e.g., root mean square error) is low enough.

[0480] 1.7.4.1 Setting ε when there is no differential attack in a single release As described in Section 1.7.3, Abe can be used to periodically publish a set of statistics about a dataset that changes over time. The goal of Abe is to segment the total amount of noise required to defend against attacks on all statistics published uniformly across the releases. To this end, in the absence of historical data, an attack on the first periodic release must well represent the risk of future attacks.

[0481] As an example, imagine a user wants to publish annual statistics on student characteristics, such as special education needs broken down by local authority and school type, and a student will be retained in the database for 12 years. For the initial release, Abe uses the ε suggested by Canary and assumes that the attack will become more robust over time as more information about the same student is released. Abe recommends using a time-adjusted ε, rather than simply adding the minimum amount of noise needed to defend against the current attack. This helps avoid the need to add a much larger, unequal amount of noise later to compensate for the fact that the attack has become more accurate.

[0482] This means that, given that no row-based attacks were detected in the initial release and that the overall attack was blocked by the highest ε tested, there is a risk that Abe underestimated future risks. It is highly likely that new attacks will emerge over time as people change their quasi-identifiers or exit the dataset, making them vulnerable to differential attacks.

[0483] To avoid publishing highly accurate information about people initially only to later add a lot of noise, Canary can generate a synthetic attack on the first release. Abe takes the obtained ε and applies its budget splitting rules to obtain the ε of the first release, thus avoiding the need for major adjustments later.

[0484] In the Canary system, a diff-of-two synthetic attack can be performed by adding rows to the query matrix that differ from existing rows by one entry. An effective method to do this is to add one or more columns to the query matrix that are all 0s except for the added query row, ensuring that the added information does not cause any inconsistencies in the query matrix. The added query row will be a copy of the last row in the query matrix, with the only modification being setting the entries in the artificial columns to 1. This corresponds to an extra record in the dataset that has only one quasi-attribute and one secret.

[0485] There are different strategies for developing synthetic differential attacks that are useful against calibration risks: ● Attacks with the minimum possible L2 norm ● Attacks on sensitive values ​​originating from the very end of the sensitive range. ● Attacks on sensitive values ​​with the lowest baseline guessing rate Canary uses one of these strategies to perform a synthetic attack on the first release in a series of releases, while Abe assumes the attack is real and thus finds the appropriate amount of noise to add to the release.

[0486] Performing synthetic differential attacks when there are no vulnerabilities in the initial release helps avoid the need to add a larger amount of unequal noise to subsequent releases, because ABE needs to compensate for the fact that the information in the initial release was already very accurate, but now the attack has occurred.

[0487] 1.7.5 Factorization of the computational power available to the attacker Some of the attacks described in the Canary section require considerable computational power to run within a feasible timeframe. Since computational power comes at a cost, some attacks may be too expensive for certain attackers to execute.

[0488] Abe can take this limitation into account. The user provides information about how much computing power an attacker has available. Abe then only runs Canary attacks that can be carried out using that computing power.

[0489] Users can provide information about an attacker's available computing power in several ways: ● Lens may have pre-loaded profiles of various types of attackers and encoded estimates of the available computing power for each of these attackers.

[0490] ● Lens may directly query the attacker's available computing power (e.g., in core hours).

[0491] ● Lens might ask an attacker how much they are willing to spend and then convert that into computing power at market prices on cloud service providers (e.g., by querying the price of Amazon Web Services).

[0492] After obtaining the computational power limit, Abe then only performs attacks that can be executed with computational power equal to or less than the limit. This can be done, for example, by attempting to run each attack and cutting it off when it exceeds the computational power limit. It can also include pre-configured models that determine how much computational power is required to run each attack task based on factors such as data size and data shape, and using these models, only attacks that the models indicate will be completed with the allowed computational power are run.

[0493] The model can also include functions that express expected runtime as computational cores, dataset size, and data release size, for example. Computational capabilities can be expressed as preloaded configuration files or as user input (expressed as time or money). Attacks exceeding computational capability constraints will not run. Additionally, if ABE runs in an environment with computational resource constraints, it may not be able to execute all attacks. A further improvement is that ABE can run attacks in order from fastest to slowest. In this way, if it detects that one of the earlier attackers has successfully attacked a release with a certain amount of noise, it can stop the attack and not run subsequent, slower attackers, thus saving overall computational time.

[0494] 1.7.6 Attacking a subset of the dataset If the computational cost of performing the attack is too high (see the previous section), Abe can instead attack a subset of the dataset. Attacking a subset instead of the entire dataset reduces processing time. Choose a subset so that the attack will produce results similar to those obtained by attacking the entire dataset.

[0495] If Abe finds that an attack succeeds on a subset of the dataset, it can infer that the attack will succeed on the entire dataset. (The reverse reasoning is incorrect.) Methods for selecting subsets include, but are not limited to: ● Select a random subsample of personnel, regenerate the statistical data of the subsample, and then attack those statistical data.

[0496] ● Select individuals with specific attributes (e.g., married individuals), and then attack only the statistics applicable to that subgroup.

[0497] ● Suppose you already know the sensitive attributes of a random subsample of people, and use this information to calculate statistics only for unknown people (e.g., if the sum of the values ​​of people A, B, and C is 37, and you know that the value of C is 6, then the sum of the values ​​of A and B is 31), and then attack those statistics.

[0498] ● Use singular value decomposition of the equation matrix to determine which queries are most useful in the attack (i.e., keep the queries with large weights in the singular vector with the smallest singular value).

[0499] 1.8 Independent Use Cases for Abe and Canary Abe, which is equipped with Canary attack capabilities, is also used as a standalone system. The following use cases are examples of how it can be used.

[0500] 1.8.1 Generating the "Risk Function" for the Dataset Before a dataset becomes easily reconstructable, users can use Abe to understand the amount of aggregate statistics they can publish about the dataset. Reconstruction carries a significant risk: when too much aggregate statistics have been published, it becomes possible to accurately identify all or most of the individual private variables.

[0501] Abe allows people to simulate the risk of different numbers of statistical tables and measure the number of variables that are vulnerable to attack. These experiments can be performed on a specific private dataset in question to obtain dataset-specific results, thereby deriving an approximate function that outputs the amount of risk given a number of published tables.

[0502] 1.8.2 Replace manual output checks with automated risk detection (risk monitoring). A user might be considering publishing a set of summary statistics about their private data in the form of a contingency table. Abe can determine whether the statistics would make any individual vulnerable to a privacy attack. If any of the variables identified in a Canary attack are found to be vulnerable, the user knows not to publish those statistics.

[0503] 2. Processing datasets with multiple private attributes Lens typically aims to protect the privacy of individuals, but it can also protect the privacy of any otherwise defined private data entity (e.g., families, companies, etc.). In many cases, a database contains several records about an entity, and often more than one column in the dataset is considered private information. This presents a challenge when using Lens to publish differentially private statistics about this data: the differential privacy protection provided for a secret and an entity can be compromised by publishing statistics about other related private attributes belonging to the same entity. Protecting datasets with relevant sensitive variables can be tricky because it's necessary to consider how much knowledge about a secret might reveal all the relevant sensitive variables.

[0504] Three different scenarios need to be considered: 1. Publish statistics on two (or more) different private properties that are unrelated or whose relationship to private values ​​is unknown. 2. Publish statistics on two (or more) different private properties that are highly relevant and where knowing one is sufficient to infer all related secrets.

[0505] 3. Publish statistics on two (or more) different private attributes related to a certain part.

[0506] An example of the first scenario is a database containing various demographic data, including private attributes such as a person's blood type plus that person's salary. Since these secrets are unrelated, Lens can run Abe on each of these attributes separately to determine how much noise needs to be added (and, in cases where the suggested noise for the same table conflicts with each individual run, the maximum noise is used). When determining ε for one of the private attributes, Lens can conservatively assume that other private attributes are available to the attacker as background knowledge.

[0507] An example of the second scenario is a healthcare database containing medical data, such as diagnoses of a particular type of cancer, as well as data on the use of drugs used to treat cancer. Calculating the combined privacy risks of publishing statistics on both cancer diagnoses and drug use is very tricky because information published about one needs to be considered useful for inferring the other. If the relationship between these two secrets is ignored, one might underestimate the privacy risks of publishing such statistics.

[0508] Imagine two different tables are published regarding this dataset: one table contains counts of patients with a specific type of cancer, and the other table contains counts of patients taking a certain anti-cancer drug to treat their condition. The statistics in these two tables are highly correlated, and information about an individual learned from one of the tables can help derive a second private value. Say an adversary has identified person X as having type A cancer from the first table. When trying to find out which patients in the second table are taking which anti-cancer drug, she can already guess with high probability that person X is taking that drug to treat type A cancer. This not only puts person X at risk of having two secrets exposed, but could also create a snowball effect, making other patients in the second table vulnerable.

[0509] To correctly model the risk in all the above scenarios, Lens derives and detects relationships between groups of private attributes based on user input and automated processing. The inferred relationships can be of different types: ● Parent-child relationship: A private column contains a subcategory of another private column. Example: The "Cancer Type" column with categories {"Acute Lymphoblastic Leukemia", "Acute Myeloid Leukemia", "Gastrointestinal Carcinoid", "Gastrointestinal Stromal Tumor"} is a subcolumn of the "Cancer Category" column with categories {"Leukemia", "Gastrointestinal Tumor"}. These relationships are automatically detected by scanning multiple pairs of categorical columns to find the simultaneous occurrence of words, and hierarchical sorting is suggested using the cardinality of the column with the highest matching score.

[0510] ● Linear Relationship: A simple linear model exists that predicts the value of a private column based on the values ​​of a second private column or a set of related private columns. Example: An individual's "net worth" y can be predicted as y = x2 - x1 based on their "liabilities" x1 and "assets" x2. These relationships will be automatically detected through statistical tests for linear correlation, such as the chi-square test.

[0511] ● Nonlinear Relationship: A nonlinear model exists that predicts the value of a private column based on the values ​​of a second private column or a set of related private columns. Example: An individual's "CD4+ cell count" can be calculated using a known nonlinear equation based on the gene expression levels of different HIV genes (such as "CD4+ cell count"). gag "Level of expression" pol "Level of expression" or " env The expression level is used for prediction. All of these attributes are themselves considered private.

[0512] ● Semantic Relationship: It is possible to know that two private columns are semantically related without needing to know the explicit relationship between them. Example: A medical diagnosis may be known to be associated with symptoms such as migraine attacks or high blood pressure, but it is unclear how one symptom can predict the other.

[0513] In Lens, users can define relationships between private columns and provide explanations for various types of relationships. Lens can also automatically detect certain relationships.

[0514] Lens's attack-based assessment system uses the output of this process to inform its risk estimation process. First, "secret groups" are formed. Then, depending on the type of relationships between private columns in the "secret groups," and how those private columns fit into Abe's attack modeling section, the process is further refined. For example: ● Parent-child relationships: If a parent-child relationship exists between columns in a set of secrets, the Canary equation for the parent class in Abe can include additional equations or inequalities expressing this relationship. Let the variable expressing the first attribute be P_i for individual i, and the variable expressing the second attribute be O_i for individual i. A constraint can be added for each i: O_i <= P_i.

[0515] ● Linear relationships: Linear relationships between variables can be directly incorporated into linear Canary equations as additional equations.

[0516] Therefore, by encoding information about the relationships between the sensitive variables into the set of linear equations, ABE is able to model multiple sensitive variables together. When there is no relationship between the sensitive variables, ABE runs the independent sensitive variables separately and applies the recommended maximum noise to each statistic.

[0517] 3. Processing time series or longitudinal datasets Databases typically have more than one table. For example, a common way to represent data about payments is to have one table for people and another table for payments. In the first table, each row represents a person. In the second table, each row represents a payment (it may contain identifiers for the payer and payee, which can then be looked up in the people table). There can be many payments associated with a single person.

[0518] We call this type of data transaction data. Transaction data contrasts with rectangular data, which consists of a table where each row represents a person. Figure 20 An example of a transaction data pattern is shown.

[0519] Lens publishes differentially private aggregate queries. In order to use, for example, a Laplace mechanism to calculate how much noise to add to each aggregate query result, Lens must know: a) the sensitivity of the query (“sensitivity” found in the differential privacy literature) and b) what the appropriate ε is. For transactional data, achieving both becomes even more difficult.

[0520] Both can be made easier by applying a "re-rectangularization" process to each query.

[0521] 3.1 Rectify the transaction data query Rectifying transaction data queries means transforming queries about transaction datasets into queries about rectangular datasets. The rectangular datasets we care about contain only one row per person, and our goal is to protect everyone's privacy.

[0522] The system uses a rectangularization process to express queries on transaction data (one row per event, potentially multiple rows per person) as queries on an intermediate rectangular table. SQL rules have been developed that transform SQL-like queries on transaction data into SQL-like queries on the rectangular data.

[0523] The starting point of the rectangular dataset is the table in the dataset, which has one row per person. Suppose we want to protect the customers in the example transaction database above; the "CUSTOMER" table is the starting point of the rectangular dataset.

[0524] Now, suppose a user wants to publish the results of the query "SUM(TOTALPRICE)FROM ORDERS". This involves the ORDERS table. However, we can create a new column in the CUSTOMER table to allow answering this query: the total price per customer.

[0525] We call this process a GROUP BY rule because it's done by grouping the query by type. Below is a complete example of a GROUP BY rule in the action of the query "SUM(TOTALPRICE) FROM ORDERS": 1. Execute SUM (TOTALPRICE) FROM ORDERS GROUP BY CUSTKEY.

[0526] 2. Make the result of this query a new column in the rectangular dataset (i.e., CUSTOMER). Name this new column INTERMEDIATE_SUM.

[0527] 3. Execute SUM(INTERMEDIATE_SUM) FROM CUSTOMER.

[0528] The dataset we created in step 2 is a rectangular dataset, and the query we uttered in step 3 produces the exact same answer as the original query. We created an intermediate rectangular table to answer queries about the transaction dataset.

[0529] Summation can be calculated as the sum of intermediate sums; in other words, we sum by person to obtain intermediate features, and then sum over those features. Regarding counting, we count by person and then sum over the features.

[0530] Please note that in step 1, we grouped by CUSTKEY because it happens to represent a single person and is included in the ORDERS table. However, if we want to query LINEITEM, for example, "SUM (QUANTITY) FROM LINEITEM"? No reference to the customer can be found in this table.

[0531] In this scenario, we must join with another table to obtain a reference to the customer. This process is called a JOIN rule. For example, we can join LINEITEM with ORDERS on ORDERKEY to be able to reference CUSTKEY. Below is a complete example of the JOIN and GROUP BY rules for the query "SUM (QUANTITY) FROM LINEITEM": 1. Create a new table: LINEITEM JOIN ORDERS ON (L_ORDERKEY = O_ORDERKEY) 2. Execute SUM(QUANTITY) FROM LINEITEM JOIN ORDERS ON (L_ORDERKEY = O_ORDERKEY) GROUP BY CUSTKEY.

[0532] 3. Make the result of this query a new column in the rectangular dataset (i.e., CUSTOMER). Name this new column INTERMEDIATE_SUM.

[0533] 4. Execute SUM(INTERMEDIATE_SUM) FROM CUSTOMER.

[0534] Step 1 enables referencing the CUSTKEY. Then, the GROUP BY rule can work as before in steps 2-4.

[0535] Using these two rules, Lens can transform many queries on transaction data into queries about intermediate rectangular datasets. The sensitivity of the transformed versions of the queries can be evaluated, and ε can be set for them as rectangular queries. In this way, Lens can support the publication of statistics about transaction datasets.

[0536] To perform this rectangularization, Lens needs to know the database schema and the tables within the schema that contain rectangles (i.e., tables with one row per person). It also needs to know which column in this rectangular table is the identifier.

[0537] 4. Defining "Sensitivity": A Key Concept in Differential Privacy Knowing the range of sensitive variables in the data is necessary to ensure differential privacy.

[0538] Lens uses a Laplace mechanism to publish a differential, private version of aggregated statistics (it could similarly use a Gaussian mechanism, but the Laplace mechanism is discussed here). The Laplace mechanism adds Laplace noise to the query results. It calculates how much noise to add as sensitivity / ε, so understanding the query's sensitivity is important.

[0539] Extracting ranges directly from the raw dataset poses a potential privacy risk because it reveals the minimum or maximum values ​​within the data. Therefore, instead, the ranges are extracted and displayed to the data holder. The system then queries the data. Theoretically the greatest possibility The scope could be whatever, and it warns data holders that what they type will be made public. This prevents data holders from simply reporting the actual scope of the current data in the original dataset.

[0540] The sensitivity of a COUNT query is 1. The sensitivity of a SUM query is equal to the size of the variable's range. Importantly, this does not mean the range of the variable at any given point in time, but rather the maximum conceivable range of the variable. For example, a variable representing human age might have a range of approximately 0-135.

[0541] Lens prompts users to enter a range for any column to be summed. Leaving this to the user's own device could tempt them to simply look up the range of variables in their available data and use that range. This poses a privacy risk, and the variables could potentially exceed limits in future releases. Therefore, to prevent users from doing this, Lens calculates the current range of the user's data and displays this range using a dialog box that prompts the user to change the number to the maximum possible range. The dialog box also informs the user that whatever they enter as the variable range should be considered public.

[0542] As an example, suppose a user has a database of employee get off work hours and wants to publish statistics about this database. One feature they are interested in is the average workdays. They calculate this as the average of the average workdays per employee (“average workdays per employee”) (“final average workdays”). Lens needs to know the sensitivity of this feature: average workdays per employee. Therefore, the user must enter a range. Lens queries the data and finds that the current minimum is 3.5 hours and the maximum is 11.5 hours. Lens provides this information to the user, along with the aforementioned warning about the transparency of the input. Considering possible future scenarios, the user decides to enter 2 and 12 as the limits of the range. Lens can then calculate a sensitivity of 10 (12 minus 2) and use that sensitivity to calibrate the noise it adds to the average statistics.

[0543] Lens can then clamp or suppress future data points that fall outside this configuration range. For example, if an unexpected sensitive value of 13 is collected, while the range is 2-12, the data point can be discarded or converted to 12.

[0544] 5. Output synthetic microdata instead of aggregated statistics. In some situations, outputting aggregated statistics may not be appropriate. For example, if an existing data mining pipeline exists, outputting synthetic microdata copies of the actual data would enable the use of that pipeline while protecting privacy with minimal pipeline changes.

[0545] By viewing synthetic microdata as an alternative way to convey aggregate statistics, Lens can easily output either synthetic microdata or aggregate statistics with the same settings. This is done by embedding the pattern of the aggregate statistics into the synthetic microdata.

[0546] For this reason, Lens includes an option to output a dataset of privacy-protected synthetic microdata in response to a user-defined query, rather than a set of perturbative aggregate statistics. Lens allows data holders to publish DP aggregates and / or DP synthetic data, in either case ε is centrally managed and configured by the same automated analytics.

[0547] The way synthetic microdata is constructed allows for close, but not precise, matching between the answers to user-defined queries on the original dataset and the answers to the same queries on the synthetic dataset. The tightness of this matching is parameterized. This allows for the simultaneous capture of relevant insights of interest from protected datasets, while the tightness of these answers provides a formal constraint on the amount of personal information disclosed from the original data.

[0548] Lens offers several options for outputting synthetic microdata. One option within Lens is to employ a method based on the Multiplicative Weighted Exponent (MWEM) algorithm (Hardt, Ligett, and McSherry (2012), A Simple and Practical Algorithm for Differentially Private Data Release, NIPS Proceedings). This method uses differential privacy to release synthetic microdata.

[0549] The algorithm consists of several steps: An initial synthetic dataset is constructed and uniformly drawn within the domain of the original dataset.

[0550] User-defined queries are computed on the original dataset using a Laplace mechanism in a differentially private manner (Dwork (2006) Differential Privacy. In the proceedings of the International Colloquiumon Automata, Languages ​​and Programming (ICALP) (2), pp. 1-12). This keeps the original statistics and their differentially private counterparts secret.

[0551] User-defined queries are calculated from the initial synthesized data.

[0552] Then, the initial synthetic dataset is iteratively optimized by minimizing the error between the perturbation statistics generated on the original dataset and the perturbation statistics generated on the synthetic dataset. Specifically, the algorithm uses another differential privacy mechanism: the Exponential Mechanism (McSherry and Talwar. (2007) Mechanism Design via Differential Privacy, Proceedings of the 48th Annual IEEE Symposium on Foundations of Computer Science, pp. 94-103) to select the maximum error statistic, and then modifies the synthetic data to reduce this error.

[0553] The combined use of these two differential privacy mechanisms allows for the construction of synthetic datasets with mathematically quantifiable disclosures of a given single variable within the original dataset.

[0554] 6. Privacy protection for multiple entities Typically, data privacy mechanisms are designed to protect the privacy of the people in a dataset; in other words, to ensure that no secrets about individuals are disclosed. However, this does not address the possibility that the privacy of another entity in the real world needs to be protected. For example, consider a dataset of store purchases. It would be desirable to protect everyone's purchase history. However, it might also be necessary to protect the sales history of each store.

[0555] This is called “protection against multiple entities” because there are more than one entity that needs privacy protection (in this case, a person is one entity and a store is another).

[0556] These two entities may be related or unrelated. We consider two cases: one where an entity is 'nested' within another, and the other where there is no nesting. For example, in a census, people and households are nested entities, where each person is exactly in one household, and each household has at least one person. In the purchase dataset example above, people and stores are not nested entities; each person can shop at more than one store, and each store has more than one customer.

[0557] 6.1 Differential privacy protection for two (or more) nested entities If entity A is nested within entity B, then protecting A with a specific differential privacy level requires less noise than protecting B. For example, since a person is nested within a family, protecting the person requires less noise than protecting the family. Therefore, if we provide ε-differential privacy for B, we also provide ε-differential privacy for A.

[0558] To protect nested entities, the system needs to determine which entities are nested by examining the many-to-one relationships between columns. This information can be provided by the user or learned automatically. To learn this information automatically, the system can use metadata describing the data or analyze the data itself. Suppose a dataset has one column representing identifiers of A and another column representing identifiers of B; the system checks if a one-to-many relationship exists from A to B (if so, B is nested within A).

[0559] To set ε, ABE sets ε based on entities that are difficult to protect (external entities). External entities are more difficult to protect because they have a greater imprint on statistics; for example, a family of six has a greater impact than a single person. Lens can then report the level of differential privacy ε provided to each entity.

[0560] After setting ε, you can also run Canary on the internal entity to double-check that this ε is sufficient to protect the entity.

[0561] Please note that this method extends to more than two entities as long as there is a nesting relationship between each pair of entities.

[0562] 6.2 Differential Privacy-Maximum Noise Method for Two Non-Nested Entities If the entities are not nested, ABE can set ε by calculating how much noise each entity needs independently and then selecting the maximum value of the resulting noise level. Lens can then report the level of differential privacy provided to each entity.

[0563] After setting ε, you can run Canary on other entities to double-check that it is sufficient to protect those entities.

[0564] Please note that this method extends to more than two entities.

[0565] 7. Heuristics for rapidly assessing the security of data products Lens contains numerous heuristics to help identify the privacy risks associated with publishing statistics. All of these can be evaluated within Lens before conducting any adversarial testing itself, and provide a rapid method for estimating the privacy risks of publishing aggregated statistics.

[0566] The combination of a dataset and a set of user-defined queries clearly presents privacy risks, which can be detected using these heuristics without requiring comprehensive adversarial testing. Lens can leverage these quick heuristics to provide feedback after query setup and before adversarial testing, informing the user whether any of these methods indicates a data product configuration that poses a significant privacy risk. In this way, users can choose to reconfigure their data products before adversarial testing reveals a level likely to result in poor utility.

[0567] The number of published aggregate statistics and the number of variables within the dataset. Consistent with existing privacy research, the number of published aggregate statistics relative to the number of people (or other entities) in a dataset is a good indicator of risk. The ratio between the number of published statistics and the number of people in the dataset is related to the likelihood of a reconstruction attack occurring (e.g., if the ratio is too high, such as greater than 1.5, there is risk). Therefore, this ratio can be used as a quick indicator of the privacy risks of publishing aggregate statistics.

[0568] For example, Lens can calculate the ratio of the number of statistics to the number of people and issue a warning to the user when the ratio is too high.

[0569] This heuristic can be further optimized by considering the number of statistics for a given individual at each variable level and issuing a warning if any variable appears in too many statistics.

[0570] The number of uniquely identified individuals in the statistics release. Another heuristic for privacy risk is the number of individuals with uniquely known attributes (considering only attributes relevant in statistics).

[0571] For example, when more than one person shares the same quasi-identifier (within the attributes used in the data publication), these individuals are not vulnerable to differential attacks in aggregated statistics. These individuals have inherent protection against attacks. Therefore, the number of people who are uniquely identified (i.e., do not share the quasi-identifier with anyone) is a good representation of the number of people who could potentially be attacked. For example, if no one is available to attack, we know there is no risk.

[0572] For example, if a table is generated (average income by gender and age), the heuristic method will calculate how many individuals in the dataset have a unique gender-age combination.

[0573] One attack away As mentioned earlier (Section 1.5.2), the difference attack returned by the difference attack scanner can serve as a quick heuristic indicator of whether a specific statistical release has leaked personal private values.

[0574] Small queryset size The distribution of the number of variables contributing to each statistical statistic (known as the query set size (QSS)) is another heuristic of risk. If several statistical statistics have low query set sizes, the likelihood of an attack is lower. The risk of publishing aggregate statistics with QSS=1 stems from the self-evident fact that this statistic is not an aggregate but rather discloses a single variable. However, aggregate statistics with QSS=2 also pose a significant disclosure risk because, intuitively, for each aggregate statistic with QSS=2, discovering just one variable is enough to reveal the values ​​of two variables. For this reason, the number of smaller QSS statistics can serve as a valuable measure of the inherent disclosure risk within a set of aggregate statistics.

[0575] COUNT query saturation For a set of aggregate statistics that take into account a private category variable (e.g., the COUNT for individuals who are HIV positive), a saturation query serves as a quick heuristic assessment of risk.

[0576] A saturation query is one in which the number of variables that contribute to a given COUNT statistic matches the count itself. For example, if the COUNT of HIV-positive individuals in a specific subset of data equals the size of that subset, it is clear that all members of that subset are HIV-positive. Similarly, if the COUNT of this subset is 0, we know that all members of that subset are HIV-negative. This approach extends to non-binary categorical variables.

[0577] 8. Lens Use Cases This section describes how to use the Lens system.

[0578] 8.1 Setting up Differential Private Data Products Without Data Privacy Expertise 8.1.1 Payment Data Products One use case for the Lens system is creating data products related to payments. Payment processor companies or credit card companies possess datasets of millions of transactions and customers. This data contains rich patterns that can be useful to companies, consumers, and third parties. However, the data is sensitive because it consists of people's purchase history, which is private.

[0579] Credit card companies can use Lens to create data products consisting of useful payment details, such as how much people spend on average at grocery stores, restaurants, and delivery orders. They can capture these statistics quarterly and provide them to customers, for example, to let them know how they are performing relative to the average.

[0580] Lens will ensure that the company publishes all statistics with properly calibrated differential privacy guarantees. Therefore, the workflow will continue: 1. The companies configure the Lens with the statistical data they are interested in publishing. 2. Abe runs on these statistics to determine how much noise is needed to stop a Canary attack.

[0581] 3. Lens asks users if they want to apply this noise to their releases, and the users approve or adjust the noise.

[0582] 4. Noisy broadcasts are generated.

[0583] This use case relies on some of the innovative elements discussed above. For example: ● Published regularly over time; ● The data is vertical (one line per transaction, even though what we want to protect is people).

[0584] 8.1.2 Government Statistical Data Products Another use case for Lens is publishing socioeconomic and demographic data in agencies such as censuses. Governments responsible for conducting censuses want to publish these statistics for the public interest, but they don't want to disclose sensitive information about any individuals or households.

[0585] The Census Bureau uses Lens to configure the data they want to publish. Lens parameterizes the noise-adding mechanism using the same process described in previous use cases to ensure the publication receives good differential privacy protection. The Census Bureau then publishes the noisy release generated by Lens.

[0586] This use case relies on protecting the privacy of multiple entities (people and families).

[0587] Now, suppose a census uses older aggregation software (software that calculates aggregated statistics from raw data) that takes raw data files (i.e., not yet aggregated) as input. They don't want to change the older software. However, they want to anonymize the data before feeding it into this older software. In this case, Lens can output synthetic data instead of noisy statistics, and this synthetic data can be fed into the older software. Because the synthetic data contains roughly the same patterns as the noisy statistics, the older software will calculate approximately accurate aggregated statistics.

[0588] 8.2 Quickly assess whether data release is likely to have good privacy and utility. Lens allows users to quickly understand whether the statistics they want to publish can be published with a good privacy-utility tradeoff. For example, publishing 500 statistics per day about the income of the same 10 people is unlikely to be feasible under any meaningful privacy and utility conditions. If a user tests this publishing in Lens, Lens's quick heuristics can quickly signal to the user that this attempt involves too many statistics per person and will not be successful. The user can then reduce the number of statistics accordingly and try again.

[0589] If the heuristics indicate that the release is likely to be successful, the user can proceed to release data products as discussed in previous use cases.

[0590] Appendix 1: Summary of key concepts and features This appendix is ​​a summary of the key concepts or features (C1 to C82) implemented in the Lens platform. Note that each feature can be combined with any other feature; any sub-feature described as 'optional' can be combined with any other feature or sub-feature.

[0591] C1. A data product platform with multiple features for calibrating the appropriate amount of noise addition needed to prevent privacy leaks. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein privacy-preserving parameters are configurable as part of the data product publication method or system to alter the balance between maintaining the privacy of the sensitive dataset and making the data product publication useful.

[0592] Optional features: ● Privacy parameters include one or more of the following: noise value distribution, noise addition amount, ε, increment, or the score of the resampled rows of the sensitive dataset.

[0593] ● The usefulness of the data product is evaluated by determining whether conclusions that can be drawn from the sensitive dataset or from the release of a data product that is not protected by privacy can still be drawn from the release of the data product.

[0594] ● Conclusions include any information or insights that can be extracted from the sensitive dataset or from data products that are not protected by privacy, such as: maximum values, relevant variables, differences in group means, and time patterns.

[0595] ● The privacy of the sensitive dataset was assessed by applying a variety of different attacks to the release of the data product.

[0596] ● Add the noise value distribution to the statistics in the data product release.

[0597] ● The noise distribution is a Gaussian noise distribution or a Laplace noise distribution.

[0598] C2. Workflow for collecting product specifications data A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein one or more privacy-preserving parameters are automatically selected, generated, determined, or set, and wherein the privacy-preserving parameters define a balance between maintaining the privacy of the sensitive dataset and making the data product publication useful.

[0599] Optional features: ● The release of data products is configured by the data holder.

[0600] ● User-configurable data product-related parameters are input by the data holder.

[0601] ● Sensitive datasets are entered by the data holder.

[0602] ● The graphical user interface of the data holder is implemented as a software application.

[0603] ● Data product related parameters include: ○ The scope of sensitive data attributes; ○ Query parameters, such as: query, query sensitivity, query type, query set size limit; ○ Outlier range, values ​​outside the outlier range are suppressed or truncated; ○ Preprocessing transformations to be performed, such as rectangularization or generalization parameters; ○ Sensitive dataset pattern; ○ A description of the aggregated statistical data required for the release of the data product; ○ Prioritization of statistical data in the release of the aforementioned data products; ○ Data product description.

[0604] ● Data products are released in the form of APIs or synthetic microdata files.

[0605] ● Data product releases include one or more of the following: aggregated statistical data reports, infographics or dashboards, and machine learning models.

[0606] C3. Automatic PUT Evaluation A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein a privacy-utility trade-off (PUT) is automatically evaluated.

[0607] C4. Detailed Report A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; wherein a privacy-utility trade-off (PUT) is automatically evaluated, and wherein the data product publication method and system generate reports or other information describing the characteristics of the intended data product publication, the characteristics relating to a balance or trade-off between: (i) maintaining the privacy of the sensitive dataset, including whether an attack is successful and / or unsuccessful; and (ii) making the data product publication useful.

[0608] C5. Guidance on how to modify data products to achieve better PUT. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein a privacy-utility trade-off (PUT) is automatically evaluated, and recommendations for improving the PUT are subsequently automatically generated.

[0609] Optional features: ● Recommendations include modifying one or more of the following: the dimensions of one or more tables in the data product; the release frequency of the data product; the statistical generalization to be performed; outlier suppression; noise distribution values; or any data product-related parameters.

[0610] C6. Duplicate Release A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the method or system is configured to generate multiple refreshed or updated versions of the data product publication, and is configured to display how the privacy-utility trade-off changes in each refreshed or updated version of the data product publication.

[0611] C7. Republishing any updated version of the sensitive dataset takes into account. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the method or system is configured to generate multiple refreshed or updated versions of the data product publication, and is configured to display how the privacy-utility trade-off changes in each refreshed or updated version of the data product publication; Furthermore, each generated data product release takes into account any updated versions of the sensitive dataset.

[0612] C8. Repeated releases under a reassessment of privacy parameters A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the method or system is configured to generate multiple refreshed or updated versions of the data product publication, and is configured to display how the privacy-utility trade-off changes in each refreshed or updated version of the data product publication; Furthermore, for each generated data product release, the protection parameters are automatically updated by taking into account any updated versions of the sensitive dataset, any updated versions of the data product release, or any user-configurable parameters.

[0613] C9. Compare distortion with sampling error A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein one or more privacy-preserving parameters are automatically generated, and the method or system is configured to automatically generate a comparison between (i) the utility of the privacy-preserving parameters and (ii) a sampling error.

[0614] C10. A system that automatically performs adversarial testing on data releases. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein one or more privacy-preserving parameters are applied, and the method or system is configured to automatically apply a variety of different attacks to the data product publication and automatically determine whether the privacy of the sensitive dataset is compromised by any attack.

[0615] Optional features: ● The attacks are stored in the attack database.

[0616] ● The privacy protection system assesses whether the various attacks are likely to succeed.

[0617] ● Each attack estimates whether any sensitive variables from the sensitive dataset pose a risk determined from the release of the data product.

[0618] ● Each attack output is identified as a sensitive variable that is vulnerable to the attack in relation to that attack.

[0619] ● Each attack output is determined as a guessed value for each vulnerable sensitive variable.

[0620] C11. A system that automatically performs adversarial testing on a set of aggregated statistics. A computer-implemented method for managing the privacy of a set of aggregated statistics derived from a sensitive dataset, wherein the method uses a penetration testing system configured to automatically apply multiple different attacks to the set of aggregated statistics to automatically determine whether the privacy of the sensitive dataset has been compromised by any attack.

[0621] Optional features: ● Aggregated statistical data includes machine learning models.

[0622] ● The penetration testing system implements any of the methods implemented by the privacy protection system.

[0623] C12. Calculate ε directly using adversarial testing. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a differential private system; and the method or system is configured to apply multiple different attacks to the data product publication and determine a substantially highest ε consistent with defeating all attacks.

[0624] C13. Calculate ε directly based on the attack. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, said set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein ε is directly calculated based on attack characteristics to obtain the desired attack success.

[0625] Optional features: ● Attack characteristics include probability density functions.

[0626] C14. Use adversarial testing to measure whether a certain ε will defeat a privacy attack; then use that adversarial test to set ε low enough that the attack will not succeed. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a differential private system; and wherein a value of a privacy protection parameter ε is applied, and the method or system is configured to apply a variety of different attacks against the value of ε to the data product publication and determine whether the privacy of the sensitive dataset is compromised by any attack; and then determine a substantially maximum ε consistent with maintaining the privacy of the sensitive dataset.

[0627] Optional features: ● Maintain the privacy of the sensitive dataset when all the various attacks applied to the release of the data product are likely to fail.

[0628] C15. ε scan A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a differential private system; and wherein a privacy protection parameter ε is iteratively applied, and the method or system is configured to automatically apply multiple different attacks to the data product publication for each ε value and automatically determine whether the privacy of the sensitive dataset is compromised by any attack and to determine a substantially highest ε consistent with maintaining the privacy of the sensitive dataset.

[0629] C16. Use automated adversarial testing to set ε A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a differential private system; and wherein a value of a privacy protection parameter ε is applied, and the method or system is configured to automatically apply a variety of different attacks against the value of ε to the data product publication and automatically determine whether the privacy of the sensitive dataset is compromised by any attack, and then automatically determine a substantially maximum ε consistent with maintaining the privacy of the sensitive dataset.

[0630] Optional features: ● Subtract a user-configurable security buffer value from the determined highest ε to enhance the privacy of the sensitive dataset.

[0631] ● Add a user-configurable security buffer value to the determined maximum ε to improve the utility of the data product release.

[0632] C17. Encoding statistical data into linear equations A computer-implemented method for querying datasets containing sensitive data, wherein the method includes the step of encoding statistical data such as sums and counts using a system of linear equations, the statistical data being a linear function of values ​​in the dataset.

[0633] Optional features: ● The method includes the following steps: (i) receiving a linear query specification; (ii) aggregating data in the sensitive dataset based on the query specification; and (iii) encoding the aggregated data using a set of linear equations.

[0634] ● When the received query is a SUM, relating to the sums of m variables in a dataset, the set of linear equations is defined by the following terms:

[0635] in A has 0 and 1. A matrix, where each row represents a sum and the variables included in the sum are marked as 1, and the other variables are marked as 0; v yes n A column vector representing the sensitivity value of each variable in the sensitive dataset; And d is a length of m A vector, with the values ​​of the statistics as entries in the vector.

[0636] C18. Encode the AVERAGE table as a SUM table. A computer-implemented method for querying a dataset containing sensitive data, wherein the method includes the step of: encoding an AVERAGE table into a SUM table of the query set using the size of the query set.

[0637] C19. Encode the COUNT table. A computer-implemented method for querying datasets containing sensitive data, wherein the method includes the step of encoding a COUNT table into a system of linear equations.

[0638] Optional features: ● Use one-hot encoding to split the sensitive variable into several binary variables.

[0639] C20. Handling a hybrid of sensitive and public packets. A computer-implemented method for querying a dataset containing multiple sensitive data columns, wherein the method includes the step of encoding multiple sensitive data attributes into a single sensitive data attribute.

[0640] Optional features: ● Use one-hot encoding to encode every possible combination of variables in the sensitive data column.

[0641] ● Generalize the continuous variables before performing the one-hot encoding step.

[0642] C21. Display distortion measurements related to noise. A computer-implemented method for querying datasets containing sensitive data, wherein the method includes the steps of: using a privacy-preserving system such as a differential private system; and wherein one or more privacy-preserving parameters are automatically generated together with a distortion metric describing noise addition associated with the privacy-preserving parameters.

[0643] Optional features: ● Distortion measures include the root mean square error, mean absolute error, or percentile of the noise value distribution. C22. Determine whether utility is preserved in perturbed statistics by assessing whether the same advanced conclusions are drawn from perturbed statistics. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein one or more privacy-preserving parameters are applied, and the method or system is configured to automatically determine whether conclusions that can be drawn from a data product publication that is not privacy-preserving can still be drawn from a privacy-preserving data product publication.

[0644] Optional features: ● The method includes the step of encoding the conclusion into a program.

[0645] ● The method includes the step of encoding the maximum value conclusion.

[0646] ● The method includes the step of encoding the conclusions of the relevant variables.

[0647] ● The method includes the step of encoding the conclusion of the difference in group means.

[0648] ● The method includes the step of encoding the time pattern conclusions.

[0649] C23. Allow users to specify their own custom conclusions. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein user-defined conclusions are input, and the method and system automatically determine whether the data product publication retains the user-defined conclusions.

[0650] C24. An attack scheme that processes aggregated statistics and outputs guesses about the individual values. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein a suite or set of different attacks is automatically accessed and deployed to attempt to recover information about an individual from the data product publication.

[0651] C25. Differential Attack Scanner A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein differential attacks are automatically searched for.

[0652] Optional features: ● Automatically apply differential attacks to data product releases; ● The search method includes the following steps: (a) Sort the statistical data in the data product release by query set size; (b) For a one-difference attack, check every pair of statistics whose query set sizes differ by one; (c) For each attack that is one step behind that is discovered: The query set is updated by removing vulnerable variables that differ by one. Repeat steps (a) to (c); and (d) Output privacy risks relative to data products published by differential attacks.

[0653] ● A one-difference attack is detected when a pair of query sets with the same variables (excluding one) that differ in size by one are found.

[0654] C26. Iterative Least Squares Attack on SUM Tables A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein an iterative least squares attack on the aggregated statistical data is performed.

[0655] Optional features: ● Least squares-based attacks include the following steps: a) The solution to the set of linear equations that generate the equations: ,in It is a one-dimensional vector containing the calculated variable values ​​for each variable in the sensitive dataset.

[0656] b) Compare the calculated variable values ​​with the original variable values ​​for each calculated variable; c) Output the privacy risks of publishing data products relative to least squares-based attacks.

[0657] ● If the comparison in step (b) is less than a predefined threshold, the original variables in the dataset are considered vulnerable to attack.

[0658] C27. Use an alternative method using orthogonal equations.

[0659] A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein orthogonality equations are used to perform attacks on aggregated statistical data.

[0660] Optional features: ● Least squares-based attacks involve solving the following equations: ;in It is the transpose of A.

[0661] ● The data product release includes m statistical data points about n individual variables, where m > n.

[0662] C28. Pseudo-inverse-based attack on the SUM table A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein an attack on the aggregated statistical data is performed using a pseudo-inverse-based method.

[0663] Optional features: ● Attacks based on pseudo-inverses include the following steps: a) Calculate the matrix A The Moore-Penrose pseudoinverse is represented as A + ; b) Calculate matrix product And in B Find the diagonal entry that is 1, the diagonal entry corresponding to the index of the variable that can be determined by the set of linear equations.

[0664] c) Output the privacy risks associated with publishing data products relative to attacks based on pseudo-inverses.

[0665] ● Attack Matrix A + Multiply by statistical data vector d In order to obtain the potential solutions for all variables.

[0666] C29. Attacks based on pseudo-inverses using SVD A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein an attack on the aggregated statistical data is performed using a pseudo-inverse-based method employing singular value decomposition.

[0667] Optional features: ● The pseudo-inverse-based attack includes the following steps: calculating the singular value decomposition (SVD) of A and obtaining the matrix. U , S and V , making Therefore, only calculate A + The rows that uniquely identify the variables in v; ● Attacks based on pseudo-inverses using SVD include the following additional steps: a) Observation recover B Locate the diagonal of the vulnerable variable and generate Z, a vector of the indices of the vulnerable variable; b) Memories A + The row that uniquely determines the variable in v is in z Indexed and computed To output vulnerable variables; c) Output the privacy risks of publishing data products relative to SVD-based pseudo-inverse attacks.

[0668] C30. Pseudo-inverse-based attacks using grouping structures and SVD A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein an attack on aggregated statistical data is performed by decomposing a large set of statistical data equations into sub-equations that can be solved individually using a query infrastructure, and then merging the solutions.

[0669] Optional features: ● A pseudo-inverse-based attack algorithm using SVD leverages the GROUP BY structure of A and includes the following steps: a) Perform SVD on each GROUPBY query result, and b) Merge SVDs sequentially.

[0670] ● Merging SVDs involves performing QR decomposition on the stacked right singular vectors to generate orthogonal matrices. QRight triangle matrix R rank of the system of equations r .

[0671] ● Wherein by maintaining R The former r Given a set of singular values ​​and vectors, we will reconstruct the SVD of stacked singular vectors and... A SVD.

[0672] ● Stacking will execute in parallel, recursively, or in batches.

[0673] ● Output the privacy risks of publishing data products relative to SVD-based pseudo-inverse attack methods.

[0674] C31. Attacks based on pseudo-inverses using QR decomposition A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein an attack on the aggregated statistical data is performed using a pseudo-inverse-based attack employing QR decomposition.

[0675] Optional features: ● Attacks based on pseudo-inverses using QR decomposition utilize information about secrets. v The knowledge, among which v yes n A column vector representing the value of each variable in the sensitive dataset.

[0676] ● The algorithm includes the following steps: (a) Execution equation matrix A QR decomposition; (b) Obtain the triangular components from the QR decomposition using backward substitution. ,equation The least squares solution; (c) will With secrets v The comparison is performed, where any matching variable is determined to be vulnerable; (d) For rows i For each corresponding vulnerable variable, backward substitution is used to modify the equation. Solve for, where Is it in the index? i A vector whose values ​​are all zero except for the element equal to 1. It is the attack vector.

[0677] (e) Output the privacy risks of publishing data products relative to the pseudo-inverse attack method using QR.

[0678] C32. Find the most accurate minimum variance difference attack. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein differential attacks with minimum variance are automatically identified.

[0679] Optional features: ● Identify attacks with minimum variance using the following steps: a) Use a pseudo-inverse-based method to find vulnerable rows. i Call The associated one-hot vector (where except at the index) i (Except for the entry at position 1, all other entries are equal to zero.) b) In Under the constraints Minimize, and where d It is a noisy vector of statistical data.

[0680] c) Return to optimal attack .

[0681] C33. Using rank-revealed QR factorization to efficiently find the minimum variance attack. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein rank-revealed QR factorization is used to automatically identify differential attacks with minimum variance.

[0682] Optional features: ● Identify attacks with minimum variance using the following steps: a) Generating equation matrix A The rank reveals the QR decomposition.

[0683] b) Use a pseudo-inverse-based method to find vulnerable rows i c) Use a pseudo-inverse-based method to generate basic attacks. a .

[0684] d) Use rank-revealed QR decomposition to generate a projection operator onto the core of A. This is called p.

[0685] e) Call V , d The variance-covariance matrix. Then, our problem might be restated as finding the variance-covariance matrix that makes... The minimum value of z is obtained by solving... This is achieved by setting the first derivative to 0, which is useful for solving linear equation systems and can be done using... PVP This is achieved through QR decomposition.

[0686] C34. Symbolic solver attack on SUM tables A computer-implemented method and system for publishing data products, wherein the data product publishing is derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein a symbolic solver is used to automatically perform attacks on aggregated statistical data.

[0687] Optional features: ● Symbolic solver attack algorithms include the following steps: a) Convert the summation table into a system of symbolic equations; b) Solve the system of symbolic equations using the Gauss-Jordan elimination method; c) Check whether the variable is within a small predefined interval.

[0688] ● If the algorithm correctly guesses the variable that is determined to be vulnerable within a predefined interval, it returns.

[0689] C35. Attacks on COUNT tables as a constrained optimization problem A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein a counting table is expressed as a linear equation, and an attack on the counting table is achieved by automatically solving a constrained optimization problem.

[0690] Optional features: ● An attack algorithm on the COUNT table includes the steps of solving the following set of equations: , in c It is the number of possible categories for a categorical variable.

[0691] C36. Pseudo-inverse-based attacks on COUNT tables A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein a counting table is expressed as a linear equation, and an attack on the counting table is achieved by using a pseudo-inverse-based attack.

[0692] Optional features: ● A pseudo-inverse-based attack is any attack based on a pseudo-inverse as defined above.

[0693] ● A pseudo-inverse-based attack on the COUNT table includes the following steps: (a) Attack matrix A + Multiply by a vector of statistical data described by a set of contingency tables. d To obtain the potential solutions for all variables; (b) For all identified vulnerable variables, round the guessed values ​​to the nearest integer. The closest value in the range.

[0694] C37. Saturation Row Attack on COUNT Tables A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein a counting table is expressed as a linear equation, and an attack on the counting table is achieved by using a saturated row method.

[0695] Optional features: ● The saturation row attack algorithm includes the following steps: (a) Analyze cell A and detect cells with positive and negative saturation; (b) If a saturation entry is found, then: a. From d Subtract the contribution of private values ​​inferred from saturated cells; b. From A Remove rows and columns corresponding to cells that have been found to be saturated and private values, resulting in... .

[0696] c. Use The pseudo-inverse is used to find vulnerable variables.

[0697] d. If a new vulnerable attacker is found, return to step (a); otherwise, terminate.

[0698] ● A cell is positively saturated when its count equals the size of the lookup set, and negatively saturated when its count equals 0.

[0699] C38. Attacks on COUNT tables based on consistency checks A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein an attack on the count table is carried out through a consistency check-based attack.

[0700] Optional features: ● Attack algorithms based on consistency checks include the following steps: For each variable i and the presumed solution s Test whether other solutions may exist; and if for any variable i If there can only be one solution s, then the inferred variable... i The private value must be s, and the system of equations must be updated accordingly: ○ From d Subtract the contribution of the inferred private value.

[0701] ○ From A Remove the rows and columns corresponding to saturated cells and private values, respectively, to generate... .

[0702] ● Combine saturation-based attacks and consistency-checking attacks as described below: (a) An algorithm for performing a saturation row attack on the counting table A; (b) Execute a consistency check-based algorithm to generate A' (c) Return to step (a) and replace A with A'.

[0703] (d) If a solution cannot be determined for any variable, then terminate.

[0704] ● A consistency-based attack algorithm returns a list of all vulnerable variables that can be accurately guessed, along with their corresponding private values.

[0705] C39. Attacks on COUNT tables based on linear constraint solvers A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein a counting table is expressed as a linear equation, and an attack on the counting table is implemented using a linear constraint solver.

[0706] Optional features: Attacks on COUNT tables based on linear constraint solvers include the following steps: a) Encode the set of COUNT tables into a system of equations.

[0707] b) If the system of equations is small, then solve the entire system of equations; , Under the constraints Minimum.

[0708] c) If the system of equations is too large to be handled in the first case, then solve each column separately; that is, for each j=1,2,…,c Use subscripts to independently represent columns, in constraints lower envoy Minimum.

[0709] d) In both cases, we obtain an estimate. Then, for each record (i.e., For each row in the table, guess the sensitive category of the row whose associated one-hot encoding is closest (by L1 norm).

[0710] C40. Measuring the accuracy of a COUNT attacker's guesses by changing available information. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein a measure of the accuracy of the attack on the count table is achieved by repeatedly attacking different subsets of the data product publication.

[0711] Optional features: ● The method also estimates the stability of COUNT attacks.

[0712] ● The method described takes into account the uncertainty of the attacker.

[0713] C41. Using gradients to measure the accuracy of a COUNT attacker's guesses. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the accuracy of an attack on a counting table is measured by analyzing gradients, the gradients defining how much the overall ability to guess and replicate the observed publication changes when given a perturbation of the guess.

[0714] Optional features: ● If the guess is 1 and the gradient is negative, the guess is considered stable; and if the guess is 0 and the gradient is positive, the guess is considered stable.

[0715] C42. False Alarm Check A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein false positive attacks are automatically checked.

[0716] Optional features: ● The method for detecting false alarms includes the following first step: adding equations to the linear equation system with incorrect variable values ​​and determining whether the equation system is consistent.

[0717] ● There are two different methods for determining whether a system of equations remains consistent after adding additional equations with incorrect variable values. a) Recalculate the solution to the system of linear equations that includes incorrect equations and check if a solution exists.

[0718] b) Calculate the rank of the system of equations that includes and excludes incorrect equations, and compare the ranks of the two matrices.

[0719] C43. Multi-target optimized attack A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein optimization attacks are employed.

[0720] Optional features: ● Optimization attacks are based on creating vectors from the estimated vectors. Exported synthetic statistics published ,in An estimate containing the value of each individual variable from the original dataset.

[0721] ● Optimizing attacks includes the following steps: ○ Using estimated values ​​of a single variable to initialization ○ Based on statistical releases and synthetic statistical releases calculated using estimated value vectors The error between them is iteratively applied to the estimated value vector. Update; Each statistical error of the statistical release-synthetic statistical release pair is considered as a set of objectives to be minimized.

[0722] ● To A threshold is applied to any estimate that is lower than the minimum value of the original private value or higher than the maximum value of the original private value.

[0723] ● The initial estimation vector takes into account the knowledge or background information that the attacker may know; ● The initial estimation vector has a uniform distribution with respect to the mean of the true private values; ● Add random Gaussian noise to the initial estimation vector.

[0724] ● Optimize the attack output to estimate the guess value of each individual variable.

[0725] ● Optimized attacks are flexible and include the possibility of incorporating: gradient descent based on different types of statistics; more heuristic update rules; and initialization strategies; C44. Batch update of SUM statistics A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein an optimization attack is used, in which batch updates under SUM statistical data are employed.

[0726] Optional features: ● This involves using batch updates for vectors. Update ● The vector is calculated using the average scaling error of all published statistical data. Update; ● For SUM statistics, the batch size will be used. B=m The batch update rule is implemented as follows

[0727] in j index m A collection of statistical data, i index n A private variable, and A i Indicate private variables i Vector slices of the equation matrix.

[0728] C45. Perform batch updates to AVG statistics. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein an optimization attack is used, in which batch updates under SUM statistical data are employed, and an AVG of a variable set of known size is recast into SUM by multiplying the AVG by the set size.

[0729] C46. Perform batch updates to median statistics. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein an optimization attack is used, in which batch updates under MEDIAN statistical data are employed.

[0730] Optional features: ● For the set of odd-numbered variables in the sensitive data column, update only the center value; or for the set of even-numbered variables in the sensitive data column, update both center values.

[0731] C47. Noisy gradient descent A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein an optimization attack is used in which a cooling factor proportional to the noise added to the published statistical data is incorporated into gradient descent to help prevent noise from dominating the gradient descent process.

[0732] C48. Median Snapshot Program A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; wherein an optimization attack is used, and wherein, given an estimate of the value of each variable in an odd-numbered query set, the variable representing the median of the estimated values ​​is changed to the median value already published in the data product publication.

[0733] C49. Multiple Query Types - 'Bag Grab' Method A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, said set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; wherein optimization attacks are employed, and wherein update rules are given for each statistical type in said publication, and Based on the statistical release and the synthetic statistical release calculated using the vector of the estimated values The error between them is updated iteratively.

[0734] C50. Combining attacks using Canary-MOO A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; wherein optimization attacks are used, and wherein combinations of attacks are used, and the startup state of the optimizer is initialized to include known variables from other attacks.

[0735] C51. Modeling Background Information A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; wherein examples of an attacker's assumed knowledge are directly encoded into the system of equations, and the statistical data of the data product publication is encoded into the system of equations.

[0736] Optional features: ● The attacker's assumed knowledge is the percentage of known sensitive variable values ​​in the sensitive dataset.

[0737] ● The attacker's assumed knowledge is a random selection of the percentage of known sensitive variable values ​​in the sensitive dataset.

[0738] ● The attacker's assumed knowledge is one or more of the following: ○ The variable values ​​in the aforementioned sensitive dataset; ○ The range of variable values ​​in the sensitive dataset; ○ Whether the variable values ​​in the sensitive dataset are less than or greater than predefined values.

[0739] ○ Whether the value of a variable in the sensitive dataset is less than or greater than the value of another variable.

[0740] ● The attacker's assumed knowledge is user-configurable.

[0741] ● The attacker's assumed knowledge is encoded as an additional set of linear equations.

[0742] ● The attacker's assumed knowledge is encoded as a set of linear and nonlinear constraints.

[0743] C52. Present privacy-utility trade-off information to notify ε settings. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein a privacy-utility trade-off (PUT) is automatically evaluated and displayed to an end user so that the end user can control the level of their privacy and utility.

[0744] Optional features: ● The method includes the following steps: displaying the highest ε that prevents all attacks to the data holder.

[0745] ● The method includes the following steps: displaying a minimum ε to a data controller, the minimum ε representing a percentage of user-configured statistics within a set of user-configured conclusions or user-configured thresholds.

[0746] ● Show the privacy implications as a function of ε.

[0747] ● Show the utility effect as a function of ε.

[0748] ● Display sensitive variables with reconstruction risk as functions of ε.

[0749] ● Display one or more potentially successful attacks as functions of ε.

[0750] C53. Set ε based on a rule for privacy / utility information, for example, the highest ε blocks all attacks, and the lowest ε preserves utility. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein a privacy-utility trade-off (PUT) is automatically evaluated, and rules are used to automatically recommend privacy-preserving system parameters, such as ε, based on the PUT.

[0751] C54. Determine if the attack was successful using the variable focusing method. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the probability of success of an attack is determined by analyzing the absolute confidence in the success of an attack against a particular individual and the relativity or change in the attacker's confidence.

[0752] C55. Determine whether the attack was successful in the overall approach. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the probability of success of an attack is determined by analyzing the absolute confidence in the success of an attack on a group of individuals and the relativity or change in the attacker's confidence.

[0753] C56. Baseline methods for guessing private values A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, said set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the likelihood of a successful attack is determined by analyzing the relativity or change of confidence by comparing a baseline.

[0754] Optional features: ● One method for establishing a baseline is to sample uniformly from the sensitive columns of the original dataset. The measurement was performed on the second time. The frequency of correctly guessed in the next sampling.

[0755] C57. Sampling-based methods for determining the probability of an attack's success. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein random noise is regenerated multiple times and then an attack is launched on the noisy statistical data each time, wherein the percentage of attacks that are correctly guessed represents the confidence in the attack.

[0756] C58. Computational relationship between noise and attack success A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein an attack is modeled as a linear combination of random variables, and the probability of the attack succeeding is then calculated.

[0757] C59. Case of counting query A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; wherein an attack solver is applied to the data product publication; and an approximation of the marginal probability that the attack solver will succeed is calculated.

[0758] Optional features: ● The approximation considers the average of the correct guesses and the variance of the scores of the correct guesses generated by the attack solver.

[0759] C60. Define a successful attack as distinguishing between minimum and maximum values. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, said set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein an attack is considered successful if it can distinguish whether a given individual has the lowest or highest value within a range of sensitive attributes held in said sensitive dataset.

[0760] C61. Lateral bar chart representation of the results of attack-based assessments. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the data holder can move an indicator on a GUI that indicates a level of privacy and utility as a function of a changing ε.

[0761] Optional features: ● Side bar charts are used to display results.

[0762] C62. Abe modifies the data. A computer-implemented method and system for publishing data products, wherein a data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein multiple planned publications exist, and the privacy-preserving system is configured to ensure that privacy is preserved to a sufficient level in all planned publications.

[0763] C63. Calculate how to account for additional risks when there are multiple releases over time. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by the data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein there are multiple planned publications, and the privacy-preserving system is configured to ensure that privacy is preserved to a sufficient level in all planned publications, taking into account the increased attack intensity in future publications.

[0764] Optional features: ● The method considers one or more of the following: (a) the number of queries that are likely to be received repeatedly; (b) the planned frequency of publication F; (c) the possible duration D for each individual in the sensitive dataset. ● To maintain their respective levels of privacy of p The release of each plan calculates the overall privacy level (ε).

[0765] ● Calculate the overall privacy level ε using the following equation: .

[0766] ● Remove individuals from the original dataset that have already been published in the sensitive dataset for at least a predefined duration or at least a predefined number of times.

[0767] ● Subsample individuals for each release so that each individual is not always included in the release.

[0768] C64. Perform synthetic differential attacks when there are no vulnerable targets in the initial release. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein there are multiple planned publications and the privacy-preserving system is configured to apply privacy parameters, such as noise, to the first data product publication even when there are no data privacy vulnerabilities in the first data product publication.

[0769] Optional features: ● The privacy parameters applied to the release of the first data product take into account multiple planned releases.

[0770] ● Generate a synthetic differential attack and insert the synthetic differential attack into the first data product release for recommendation ε.

[0771] ● A synthetic differential attack is one or more of the following: ● Attacks with the minimum possible L2 norm; ● Attacks on sensitive values ​​originating from the very end of the sensitive range; ● Attacks on sensitive values ​​with the lowest baseline guessing rate.

[0772] C65. Cheapest attack priority A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to apply multiple attacks, wherein the fastest or least computationally expensive attack is used first.

[0773] C66. Factorization of computational power A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to model the computational resources required for an attack programmed to run the privacy-preserving system.

[0774] Optional features: ● If the privacy protection system determines that the attack will not be completed within the specified time, it will automatically not attempt the attack. C67. Attack a subset of the dataset A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to launch attacks on a subset of the dataset in the data product publication.

[0775] Optional features: ● Launch attacks on subsets of the dataset in a way that reduces computational overhead without significantly underestimating privacy risks.

[0776] C68. Datasets with multiple sensitive attributes A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to search for relationships between sensitive variables.

[0777] Optional features: ● If a linear relationship is found, add new equations expressing these relationships to the system of equations. C69. Rectify longitudinal or time series datasets A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to rectangularize longitudinal or time-series datasets.

[0778] Optional features: ● Generate rectangular datasets from longitudinal or time-series datasets.

[0779] ● Use SQL rules to automatically transform SQL-like queries on transaction data into SQL-like queries on rectangular data, ensuring the same query results.

[0780] C70. Determine sensitivity A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, said set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein said privacy-preserving system is configured to query a user what the theoretical maximum possible range of values ​​for sensitive variables might be.

[0781] C71. Output synthesized microdata / line level A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to output synthetic data as an alternative to aggregated statistical data, or, in addition to aggregated statistical data, output synthetic data.

[0782] C72. Multiple Entities A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to automatically detect nested entities and protect the privacy of the outermost layer.

[0783] Optional features: ● Protect the outermost layer of privacy as well as the innermost layer of privacy.

[0784] C73. Protecting the privacy of multiple entities (non-nested entities) A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to protect the privacy of multiple non-nested entities.

[0785] Optional features: ● The privacy protection system determines the noise level required to protect each entity independently, and then obtains the maximum value of these noise levels.

[0786] C74. Heuristics for rapidly assessing the security of data products A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to use heuristic computation to quickly approximate the risks or security of the data product publication.

[0787] C75. Compare the number of published statistics with the number of variables within the dataset. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to determine the ratio between the number of published statistical data and the number of individual variables or people in the dataset.

[0788] C76. The number of individuals uniquely identified. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to use a single variable or the number of people that are uniquely identified (i.e., not shared with anyone as a quasi-identifier) ​​as a representation of the number of people who may be vulnerable to attack.

[0789] C77. The existence of a one-way attack A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to use a differential attack scanner to reveal variables in the sensitive dataset that may be vulnerable to differential attacks.

[0790] C78. By querying the set size A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to use the distribution of the query set size as a measure of the likelihood that an attack will occur.

[0791] C79. Saturation Query by Counting A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to calculate the number of query saturation attacks.

[0792] C80. Improve the effectiveness of truncating or clamping outlier variables. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to improve utility by truncating or clamping outlier variables.

[0793] C81. Improving utility by generalizing variables A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to generalize variables.

[0794] C82. Set the query set size limit (QSSR) threshold. A computer-implemented method and system for publishing data products, wherein the data product publication is a bounded or fixed set of statistical data, the set being predefined by a data holder and derived from a sensitive dataset using a privacy-preserving system such as a differential private system; and wherein the privacy-preserving system is configured to set a query set size limit threshold.

[0795] Notes It should be understood that the above arrangement is merely illustrative of the application of the principles of the invention. Many modifications and alternative arrangements can be devised without departing from the spirit and scope of the invention. Although the invention has been shown in the drawings and described in particular detail above in conjunction with aspects presently considered to be the most practical and preferred examples of the invention, those skilled in the art will understand that many modifications can be made without departing from the principles and concepts of the invention set forth herein.

Claims

1. A computer-implemented method for publishing data products, the method comprising the following steps: A differential privacy system is used to derive a data product release from a sensitive dataset, wherein the derived data product release is a bounded or fixed set of statistical data, the bounded or fixed set of statistical data being (a) predefined by the data holder and (b) derived from the sensitive dataset using the differential privacy system; wherein the data holder configures the prioritization of statistical data in the data product release, wherein the statistical data is sum, count, average, median, min, or max; wherein the privacy protection parameters of the differential privacy system can be configured as part of the data product release method to alter the balance between maintaining the privacy of the sensitive dataset and making the data product release useful; The method is configured to automatically apply multiple different attacks to the data product release, thereby determining attack characteristics, and the privacy protection parameter ε is calculated based on the determined attack characteristics to obtain the desired attack success. The method includes the following steps: determining the noise distribution of the statistics to be added to the data product release by automatically applying a variety of different attacks and taking into account the prioritization of statistics configured by the data holder.

2. The method of claim 1, wherein the privacy protection parameters include one or more of the following: noise value distribution, noise addition amount, ε, increment, bin width, generalization parameter, or the score of the second-sampled rows of the sensitive dataset.

3. The method of claim 1, wherein the usefulness of the data product is evaluated by determining whether conclusions that can be drawn from the sensitive dataset or from the release of a data product that is not protected by privacy can still be drawn from the release of the data product.

4. The method of claim 3, wherein the conclusions include any information or insights that can be extracted from the sensitive dataset or from the release of data products that are not protected by privacy, including: Maximum value, related variables, group mean difference, and time pattern.

5. The method of claim 1, wherein the privacy of the sensitive dataset is assessed by applying a variety of different attacks to the release of the data product.

6. The method of claim 1, wherein the noise value distribution is a Gaussian noise distribution or a Laplace noise distribution.

7. The method of claim 1, wherein one or more privacy protection parameters are automatically selected, generated, determined, or set, and wherein the privacy protection parameters define a balance between maintaining the privacy of the sensitive dataset and making the data product useful for publication.

8. The method of claim 1, wherein the data product publication is configured by the data holder configuring data product-related parameters, and wherein the data product-related parameters include one or more of the following: the scope of sensitive data attributes; query parameters, including: Query, query sensitivity, query type, query set size limit; outlier range, values ​​outside the outlier range are suppressed or truncated; The preprocessing transformations to be performed include rectangularization or generalization parameters; sensitive dataset patterns; a description of the aggregated statistics required for the release of the data product; prioritization of the statistics in the release of the data product; and a data product description.

9. The method of claim 1, wherein the sensitive dataset is input by the data holder.

10. The method of claim 1, wherein the graphical user interface of the data holder is implemented as a software application.

11. The method of claim 1, wherein the data product is published in the form of an API or a composite microdata file.

12. The method of claim 1, wherein the data product release includes one or more of the following: aggregated statistical data reports, infographics or dashboards, or machine learning models.

13. The method of claim 1, wherein the privacy-utility trade-off PUT of the data product release is automatically evaluated.

14. The method of claim 1, wherein a privacy-utility trade-off PUT for the data product release is automatically evaluated, and wherein the data product release method generates a report or other information describing the characteristics of the expected data product release, the characteristics relating to a balance or trade-off between: (i) maintaining the privacy of the sensitive dataset, including whether an attack is successful and / or unsuccessful; and (ii) making the data product release useful.

15. The method of claim 1, wherein the privacy-utility trade-off PUT of the data product release is automatically evaluated, and recommendations for improving the PUT are subsequently automatically generated.

16. The method of claim 15, wherein it is recommended to include modifying one or more of the following: the dimension of one or more tables in the data product; the release frequency of the data product; the statistical generalization to be performed; outlier suppression; noise distribution values; or any data product-related parameters.

17. The method of claim 13, wherein the method is configured to generate multiple refreshed or updated versions of the data product release, and is configured to display how the privacy-utility trade-off PUT changes for each refreshed or updated version of the data product release.

18. The method of claim 1, wherein the method is configured to automatically generate a comparison of sampling errors and noise added to the statistical data in relation to inherent uncertainties in the statistical data released prior to the perturbation, in order to provide privacy protection.

19. The method of claim 1, wherein one or more privacy protection parameters are applied, and the method is configured to automatically apply a variety of different attacks to the release of the data product and automatically determine whether sensitive information from the sensitive dataset has been leaked by any attack.

20. The method of claim 19, wherein the differential privacy system assesses whether the various different attacks are likely to succeed.

21. The method of claim 19, wherein each attack estimates whether any sensitive variables from the sensitive dataset are at risk of being determined from the release of the data product.

22. The method of claim 19, wherein each attack output is determined as a sensitive variable that is vulnerable to the attack.

23. The method of claim 19, wherein each attack output is determined as a guessed value for each vulnerable sensitive variable.

24. The method of claim 1, wherein the attack characteristics include a probability density function.

25. The method of claim 1, wherein the method comprises the following steps: The statistical data is encoded using a system of linear equations, which are linear functions of the values ​​in the dataset.

26. The method of claim 25, wherein the method comprises the following steps: (i) Receive linear query specifications; (ii) Aggregate the data in the sensitive dataset based on the query specifications; (iii) Encode the aggregated data using a set of linear equations.

27. The method of claim 1, wherein one or more privacy protection parameters are automatically generated together with a distortion metric, the distortion metric describing noise addition associated with the privacy protection parameters.

28. The method of claim 27, wherein the distortion metric includes the root mean square error, mean absolute error, or percentile of the noise value distribution.

29. The method of claim 1, wherein one or more privacy protection parameters are applied, and the method is configured to automatically determine whether conclusions that can be drawn from the release of a data product that is not privacy-protected can still be drawn from the release of a privacy-protected data product.

30. The method of claim 29, wherein the method includes the step of encoding the conclusion into a program, wherein the conclusion includes one or more of the following: a maximum value conclusion, a correlation variable conclusion, a group mean difference conclusion, or a time pattern conclusion.

31. The method of claim 1, wherein a user-defined conclusion is input, and the method automatically determines whether the data product release retains the user-defined conclusion.

32. The method of claim 1, wherein a suite or set of different attacks that attempt to recover information about an individual from the release of the data product are automatically accessed and deployed.

33. The method of claim 1, wherein a pseudo-inverse-based approach is used to perform the attack on aggregated statistics.

34. The method of claim 1, wherein rank-revealed QR factorization is used to automatically identify the difference attack with the minimum variance.

35. The method of claim 1, wherein the counting table is expressed as a linear equation, and the attack on the counting table is achieved by using a pseudo-inverse-based attack.

36. The method of claim 5, wherein an optimization attack is used, the optimization attack being based on creating a synthetic statistical release derived from an estimation vector, wherein the estimation vector contains estimates of each individual variable value from the original dataset, and wherein an update rule is given for each statistical type in the data product release, and the estimation vector is updated iteratively based on the error between the statistical release observed from the original dataset and the synthetic statistical release calculated using the estimation vector.

37. The method of claim 1, wherein an optimization attack is used, and wherein a combination of attacks is used, and the startup state of the optimizer is initialized to include known variables from other attacks.

38. The method of claim 1, wherein the privacy-utility trade-off PUT of the data product is automatically evaluated and displayed to the end user so that the end user can control the level of their privacy and utility.

39. The method of claim 38, wherein the method comprises the following steps: Show the data holder the highest ε that prevents all attacks.

40. The method of claim 38, wherein the method comprises the following steps: The minimum ε is displayed to the data controller, which retains a set of user-configured conclusions or a percentage of user-configured statistics within a user-configured threshold.

41. The method of claim 40, wherein one or more of the following parameters of the data product release are displayed as a function of ε: privacy impact, utility impact, sensitive variables with reconstruction risk, and one or more attacks that could potentially succeed.

42. The method of claim 1, wherein the privacy-utility trade-off PUT of the data product release is automatically evaluated and rules are used to automatically recommend privacy protection parameters based on the PUT.

43. The method of claim 1, wherein random noise is regenerated multiple times and then an attack is launched against noisy statistics each time, wherein the percentage of correctly guessed attacks represents the confidence of the attack.

44. The method of claim 1, wherein an attack on a sensitive variable is modeled as a linear combination of noisy random variables, wherein the distribution of the noisy random variables is determined by random noise added to the statistics, and then the probability that the attack will succeed is calculated, wherein the success of the attack is a configurable definition.

45. The method of claim 1, wherein there are multiple planned releases, and taking into account the increased attack intensity in future releases, the differential privacy system is configured to ensure that privacy is preserved to a sufficient level in all of the planned releases.

46. ​​The method of claim 45, wherein the method takes into account one or more of the following: (a) queries that are likely to be received repeatedly; (b) the frequency of the planned release; and (c) the possible duration of each individual within the sensitive dataset.

47. The method of claim 1, wherein there are one or more planned releases, and the differential privacy system is configured to apply privacy protection parameters to the first data product release even when there are no data privacy vulnerabilities in the first data product release.

48. The method of claim 47, wherein the privacy protection parameters applied to the release of the first data product take into account the plurality of planned releases.

49. The method of claim 25, wherein the differential privacy system is configured to search for relationships between sensitive variables.

50. The method of claim 49, wherein if a linear relationship is found, a new equation expressing that relationship is added to the system of linear equations.

51. The method of claim 1, wherein the differential privacy system is configured to rectangularize the longitudinal or time series dataset by converting the longitudinal or time series dataset into a single table, and wherein the single table contains one row for each sensitive variable requiring privacy protection.

52. The method of claim 51, wherein SQL rules are used to automatically transform SQL-like queries on transaction data into SQL-like queries on rectangular datasets, such that the query results are the same.

53. The method of claim 1, wherein the differential privacy system is configured to query what the theoretical maximum possible range of the values ​​of a user's sensitive variables might be.

54. The method of claim 1, wherein the differential privacy system is configured to (a) automatically detect nested entities, wherein one or more entities are nested on the outermost entity, and (b) protect the privacy of the outermost entity.

55. The method of claim 54, wherein protecting the privacy of the outermost layer also protects the privacy of the nested entities.

56. The method of claim 1, wherein the differential privacy system is configured to protect the privacy of a plurality of non-nested entities.

57. The method of claim 1, wherein the differential privacy system determines the noise level required to independently protect each entity, and then obtains the maximum value of these noise levels.

58. The method of claim 1, wherein the differential privacy system is configured to use a differential attack scanner to reveal variables from the sensitive dataset that may be vulnerable to differential attacks.

59. A computer-implemented system that implements the computer-implemented method as defined in any one of claims 1-58.

60. A data product generated using the computer-implemented method as defined in any one of claims 1-58.

Images