Organization Embeddings for Confidential Data Peer Grouping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a challenge in collecting and maintaining confidential data in computer systems, particularly in ensuring the privacy and security of sensitive information like salary compensation data, where users are hesitant to share due to concerns about data misuse and anonymity, and small organizations face difficulties in providing meaningful insights due to sparse data.
Innovation Solution
A system is developed that securely collects and tracks confidential data by generating semantic representations of organizations to compute peer organization groups, using encryption and anonymization techniques to ensure data security and inferring insights for cohorts with limited data through organization embeddings and peer scores.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If confidential data is collected and stored for analysis, then statistical insights can be provided to users, but user privacy and security concerns increase导致用户不愿分享数据
Solution Approach 1:
The patent segments confidential data into two separate encrypted components: organization identifiers and compensation values. These segmented components are stored independently in encrypted form, preventing direct linkage between specific organizations and their salary data, thus addressing privacy concerns while maintaining analytical utility
Solution Approach 2:
The patent introduces an intermediary mechanism using encrypted segment identifiers and threshold-based aggregation. Third-party data processors act as intermediaries who can perform statistical analysis on the segmented encrypted data without accessing the actual confidential values, enabling insights while protecting user privacy
2Object-affected harmful factors
If data is anonymized to protect privacy, then user security concerns are reduced, but the ability to provide organization-level insights deteriorates due to loss of granular information
Solution Approach 1:
The patent applies different levels of data processing to different analytical needs. Organization-level analytics use aggregated segmented data for high-level insights, while cohort-level analytics utilize the segmented structure with threshold-based reconstruction to provide meaningful statistics without exposing individual organization data, achieving local optimization for each analytical context
3Quantity of substance
If small organizations are included in the database, then data coverage is improved, but meaningful insights become difficult to provide due to sparse data at the organization level
Solution Approach 1:
The patent merges data from multiple small organizations within the same cohort group by combining their segmented encrypted data. This aggregation allows statistical analysis at the cohort level even when individual organizations have sparse data, providing meaningful insights through combined information while maintaining the segmented privacy protection structure
Solution Approach 2:
The patent implements threshold-based data reconstruction where only aggregated statistical properties are recovered when sufficient data points are available, rather than attempting to reconstruct individual organization data. This partial action approach enables insights when thresholds are met while avoiding spurious results when data is insufficient
Data Source
AI summary
In an example embodiment, a confidential data architecture is extended by providing components for reliably inferring confidential data insights (e.g., median salary) for cohorts with little or no actual submitted confidential data. This is performed by inferring confidential data values based on organizations that are considered to be peers to the organization of interest. This solution involves two parts: the generation of a novel, semantic representation (embedding) of organizations to be used to compute a similarity measure between any two organizations, and the use of the semantic representation to compute a peer organization group of a given company.