Encrypted Vector Data Quality Assessment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for determining data quality often breach privacy and are inefficient, especially when computing multiple data quality metrics, as they require direct access to data or reveal sensitive information, which is not practical for commercial data sharing.
Innovation Solution
A system and method that uses encrypted vectors to assess data quality without disclosing actual data, allowing potential recipients to select and test data quality metrics like completeness, validity, uniqueness, and timeliness, while maintaining privacy through encrypted product calculations and condensed data vectors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional data quality assessment methods (audits, data snippets, or authorized requests) are used to determine data quality, then data quality metrics can be obtained, but privacy of the actual data is breached or sensitive information is revealed
Solution Approach 1:
The patent introduces encrypted vectors as an intermediary representation of data. Instead of directly accessing or sharing actual data, the system creates encrypted vector representations that preserve data quality metrics while preventing access to underlying sensitive information. This intermediary layer enables quality assessment without privacy breach.
Solution Approach 2:
The patent creates encrypted copies (vectors) of the actual data that retain statistical properties needed for quality assessment. These vector copies allow computation of data quality metrics without requiring access to the original sensitive data, effectively decoupling quality measurement from data exposure.
2Loss of information
If Private Set Intersection (PSI) or Private Set Intersection Cardinality (PSI-CA) is used to preserve privacy during data quality assessment, then data privacy is protected, but computational overhead becomes extremely high and practical for only single metric computation
Solution Approach 1:
The patent transforms the data representation from raw data to encrypted vectors, changing the parameter space in which computations occur. This transformation enables efficient computation of multiple data quality metrics simultaneously while maintaining privacy, avoiding the exponential overhead of traditional PSI methods. The vector representation allows linear-time operations instead of complex set intersection protocols.
3Measurement precision
If multiple data quality metrics are computed using conventional privacy-preserving methods, then comprehensive quality assessment is achieved, but computational overhead and time costs become prohibitively high
Solution Approach 1:
The encrypted vector representation serves as a universal foundation for computing multiple different data quality metrics simultaneously. A single vector structure enables computation of completeness, validity, uniqueness, consistency, and timeliness metrics without requiring separate privacy-preserving protocols for each metric, dramatically reducing total computation time.
Data Source
AI summary
A computer-implemented system and method for analyzing data quality is provided. Attributes each associated with one or more elements are maintained. A request from a user is received for determining data quality of at least one attribute based on an interest vector having a listing of the elements of that attribute and a selection of elements of interest. Each element is encrypted. A condensed vector having the same listing of elements as the interest vector is populated with occurrence frequencies for each of the listed elements. The elements of the condensed vector are encrypted by computing an encrypted product of each element in the condensed vector and the corresponding element of the interest vector. An aggregate is determined based on the encrypted products of each element of the interest vector and the corresponding element of the condensed vector. The aggregate is provided as results of the data quality.