Bayesian Smoothing for Confidential Data Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a challenge in collecting and maintaining confidential data, such as salary information, while ensuring its confidentiality and utilizing it for specific purposes, as users are hesitant to share due to privacy concerns and the technical difficulties in ensuring data remains anonymous and is only used for aggregated statistical analysis.
Innovation Solution
A system is developed that securely collects and tracks confidential data by using semantic representations of organizations to infer peer organization groups, enabling the generation of insights even with limited data, through Bayesian smoothing and peer organization scores, ensuring data accuracy and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If confidential data is collected and aggregated for statistical analysis, then useful insights can be provided, but user privacy concerns increase and data confidentiality cannot be guaranteed
Solution Approach 1:
The patent segments confidential data into two separate encrypted components: encrypted confidential data values and encrypted organization identifiers. These segments are stored independently in separate tables, preventing direct correlation and maintaining privacy while enabling aggregated statistical analysis when needed.
Solution Approach 2:
The patent introduces random transaction numbers as intermediary keys that link encrypted data values to encrypted organization identifiers without revealing the actual organization identities. This intermediary mechanism enables statistical aggregation while preserving confidentiality by decoupling the direct relationship between data and source organization.
2Loss of information
If confidential data is aggregated at organization level, then meaningful insights can be provided, but there is not enough data for small or medium sized organizations
Solution Approach 1:
The patent merges data from multiple organizations by retrieving encrypted confidential data values associated with the same organization identifier across different cohorts or time periods. This combining approach increases the effective data quantity for small and medium-sized organizations, enabling meaningful statistical insights while maintaining individual organization privacy through encryption.
3Measurement precision
If confidential data is shared across peer organizations, then data accuracy improves, but data security and confidentiality may be compromised
Solution Approach 1:
The patent creates encrypted copies of confidential data values and stores them in multiple tables with different access contexts. These encrypted copies can be shared across peer organizations for statistical analysis while the actual plaintext data remains secure. The encryption ensures that even though data is copied and shared, the original confidentiality and security are maintained.
Data Source
AI summary
In an example embodiment, submitted confidential data of a certain cohort (e.g., title, region, organization) is augmented by modeling confidential data of a more generalized cohort based on peer organizations. The modeling may be performed using Bayesian modeling and the results used to infer confidential data values for the original cohort. The inferred confidential data values can then be used to generate statistical insights for display in a graphical user interface.