Anonymized Data Similarity via Order-Preserving Substring Obfuscation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems cannot compare irreversibly encrypted values for similarity, limiting the ability to determine if they represent similar original values, especially in anonymized or hashed data spaces, which is crucial for privacy-preserving data analysis and matching.
Innovation Solution
A system that divides values into substrings, processes them through an obfuscation function, and compares contexts and obfuscated substrings to calculate a similarity measure, determining a closeness degree between values while preserving order and context similarity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If irreversible cryptographic algorithms are used to obfuscate sensitive data, then data confidentiality and security are improved, but the ability to perform similarity measures on the data is lost
Solution Approach 1:
The patent divides values into multiple substrings before applying obfuscation. Instead of hashing the entire value as one unit, the system segments the data into smaller parts (e.g., prefixes, suffixes, middle portions) and applies different obfuscation techniques to each segment. This segmentation allows similarity measures to be performed on individual substrings while maintaining overall data confidentiality, resolving the contradiction between security and similarity measurement capability.
Solution Approach 2:
The patent transforms the obfuscation approach by changing parameters of the hashing process. It applies different salt values, hash functions, or obfuscation强度 to different substrings or contexts. By varying these parameters strategically, the system enables similarity detection (when appropriate substrings match) while maintaining strong security (when full values are compared). This parameter variation resolves the contradiction by making the obfuscation selective rather than uniform.
2Reliability
If exact match comparison is used on obfuscated data, then data security is maintained, but the ability to identify similar or related data is eliminated
Solution Approach 1:
The patent applies partial obfuscation rather than complete obfuscation to all data. It selectively obfuscates certain portions of values (e.g., middle digits of social security numbers, middle portions of names) while leaving other portions (prefixes, suffixes) less obfuscated or unobfuscated. This partial action enables similarity matching on the less obfuscated portions while maintaining security through obfuscation of critical portions, thus resolving the contradiction between security and matching capability.
Solution Approach 2:
The patent introduces context information as an intermediary element in the comparison process. Context (such as metadata, associated data, or environmental information) serves as a mediator that enables similarity detection without requiring direct comparison of the obfuscated values themselves. The context provides additional dimensions for comparison that can reveal similarities while the obfuscated values maintain their security, resolving the contradiction between security and adaptability.
3Adaptability or versatility
If reversible encryption is used to protect data, then data can be decrypted for analysis, but the decrypted data is vulnerable to unauthorized use or disclosure
Solution Approach 1:
The patent segments data into multiple portions with different security requirements. Critical substrings that require strong protection are heavily obfuscated, while non-critical substrings can be analyzed in plain text or lightly obfuscated form. This segmentation allows analysis of non-sensitive portions without exposing sensitive information, resolving the contradiction between analyzability and security by enabling partial analysis without full decryption.
Solution Approach 2:
The patent applies different levels of obfuscation to different parts of the data based on their sensitivity. Rather than applying uniform encryption or obfuscation, the system tailors the protection level to each substring or data element. Highly sensitive portions receive strong obfuscation while less sensitive portions allow for analysis, creating local variations in security quality that enable both analysis and protection simultaneously.
Data Source
AI summary
A similarity measure system selects a first value and a first context related to the first value, divides the first value into a first set of substrings in an order preserving way, and processes each of these substrings through an obfuscation function to produce a first set of obfuscated substrings. The system selects a second value and a second context related to the second value, and processes the second value to produce a second set of obfuscated substrings. The system calculates a context similarity measure for the first context and the second context. The system determines a value similarity measure from the first and second set of order preserved obfuscated substrings. The system determines a closeness degree between the first value and the second value and a closeness degree based on the context similarity measure.