Matrix Factorization for Inferring Confidential Data Values

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

There is a challenge in collecting and maintaining confidential data, such as salary information, in computer systems due to user privacy concerns and the difficulty in ensuring that the data remains confidential and is used only for specific purposes, especially when certain combinations of cohorts have sparse data points, making it hard to provide meaningful statistical insights.

Innovation Solution

The system employs computerized matrix factorization and completion techniques to infer median/mean confidential values, ensuring data security through separate encryption of user identification and confidential data, and utilizing a submission table to track and anonymize data, while incentivizing users to contribute by providing insights based on their submissions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If confidential data is collected and stored in a computer system, then statistical analysis and insights can be provided to users, but user privacy concerns increase and data security risks arise

Engineering Contradiction:

Improvestatistical insightsVSAvoidprivacy concerns

Core Design Contradiction:

Loss of informationVSObject-affected harmful factors

Solution Approach 1:

The system segments confidential data into two separate encrypted columns: one for user identification and another for confidential data values. This segmentation ensures that even if the database is compromised, attackers cannot easily correlate user identities with their confidential data without both encryption keys, thereby reducing privacy concerns while maintaining the ability to perform statistical analysis on the segmented data

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary mechanism (encrypted submission table with separate columns) that mediates between data collection and privacy protection. The encryption scheme acts as an intermediary layer that allows statistical processing while protecting user identity, enabling the system to provide insights without directly exposing confidential information

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If confidential data is encrypted and stored separately, then data security is improved, but the complexity of data management and processing increases

Engineering Contradiction:

Improvedata securityVSAvoiddata management complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system changes the parameter of data storage from plain text to encrypted format with specific encryption parameters (separate columns, different encryption keys). This parameter change enhances security while the encryption framework is designed to maintain manageable complexity through systematic key management and structured table design

Inventive Principle:

Principle #35Parameter changes

3Object-affected harmful factors

If data is anonymized and aggregated, then privacy protection is improved, but the precision of statistical insights decreases

Engineering Contradiction:

Improveprivacy protectionVSAvoidstatistical accuracy

Core Design Contradiction:

Object-affected harmful factorsVSMeasurement precision

Solution Approach 1:

The patent applies local quality by providing different levels of data access and processing precision for different analytical needs. The encrypted submission table allows for precise statistical calculations on aggregated data while maintaining privacy, enabling the system to deliver accurate insights without exposing individual confidential information

Inventive Principle:

Principle #3Local quality

4Loss of information

If matrix factorization and completion are used to infer missing values, then data utility is improved for sparse cohorts, but computational complexity and processing time increase

Engineering Contradiction:

Improvedata completenessVSAvoidprocessing time

Core Design Contradiction:

Loss of informationVSLoss of time

Solution Approach 1:

The system performs preliminary actions by pre-processing confidential data through encryption and organization in a structured submission table format before statistical analysis. This preliminary structuring enables more efficient matrix factorization and completion operations when inferring missing values, reducing the computational burden during actual analysis by having data ready in the optimal format

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS10262154B1Computerized matrix factorization and completion to infer median/mean confidential values

Publication Date: 2019.04.16 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

In an example embodiment, an anonymized set of confidential data values is obtained for a plurality of combinations of cohorts having a first attribute type and a second attribute type. A matrix of the confidential data values having the first attribute type as a first axis and the second attribute type as the second axis is constructed. A set of candidate low rank approximations of the matrix is calculated using an objective function evaluated using a set of candidate data transformation functions, the objective function having one or more parameters and an error function. One or more parameters that minimize the error function of the objective function are minimized to select one of the candidate low rank approximations of the matrix. Then one or more cells that are missing data, of the selected one of the candidate low rank approximations of the matrix, are inferred.