Format-Preserving Encryption Library for Big Data Compatibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data security solutions in big data ecosystems fail to adequately protect sensitive information, maintaining format integrity while ensuring data accessibility and security, leading to potential exposure and incompatibilities with software applications, and complicating data handling in complex data models.
Innovation Solution
A system and method utilizing a distributed processing library for format-preserving encryption (FPE) that encrypts sensitive data with a unique key, preserving the original format and length, and allows authorized access through a user interface for decryption, while integrating with machine learning models for secure data handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If format-preserving encryption is applied to encrypt sensitive data, then data security is improved, but compatibility with software applications deteriorates due to changed characteristics of encrypted text
Solution Approach 1:
The patent applies parameter changes by transforming the data representation through encryption while preserving format constraints. The FPE algorithm modifies the numerical or textual parameters of the data while maintaining the same data type, length, and format structure, allowing encrypted data to remain compatible with existing software applications that expect specific formats.
Solution Approach 2:
The patent segments the encryption process into format-preserving encryption operations that can be applied to specific data fields without affecting the entire dataset. This allows selective encryption of sensitive columns while leaving other data unchanged, maintaining system compatibility and enabling gradual migration to encrypted data storage.
2Reliability
If FPE algorithms are used to encrypt data, then sensitive information protection is improved, but data relationship integrity deteriorates as checksum values and monotonic relationships are no longer valid
Solution Approach 1:
The patent introduces an intermediary layer of encryption management that handles the transformation of data relationships. This intermediary system maintains mappings between encrypted and unencrypted values, allowing the system to preserve relationships such as checksums and monotonic ordering by transforming both sides of the relationship together rather than encrypting in isolation.
Solution Approach 2:
The patent applies preliminary action by pre-computing and storing encryption mappings and relationship constraints before encryption occurs. This allows the system to maintain data relationships by having advance knowledge of how encrypted values relate to original values, enabling integrity checks and validation operations to continue working on encrypted data.
3Reliability
If data is encrypted in big data ecosystems, then data security is improved, but ease of operation deteriorates due to complex data handling requirements
Solution Approach 1:
The patent implements universality by creating a unified FPE library that provides multiple functions through a single interface: encryption, decryption, format preservation, and relationship maintenance. This multi-functional approach consolidates what would otherwise require multiple separate operations and tools into one cohesive system, simplifying data handling in big data ecosystems.
Solution Approach 2:
The patent applies self-service by enabling the encryption system to automatically manage its own operations through integrated key management, automatic format detection, and self-validating relationship constraints. The system handles encryption operations transparently without requiring manual intervention for each data type or relationship, reducing operational complexity.
4Reliability
If existing security solutions like RBAC and encryption zones are implemented, then data access control is improved, but device complexity increases due to multiple layers of security mechanisms
Solution Approach 1:
The patent merges multiple security functions into a unified FPE library that combines encryption, format preservation, and access control in a single integrated system. Rather than implementing separate RBAC layers, encryption zones, and key management systems, the patent consolidates these functions into one library that can be applied uniformly across big data platforms, reducing overall system complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention provides a robust and effective solution to an organization by enabling them to implement a system (110) for facilitating format preserving encryption capability such that the encrypted data will not be available with its original value in a big data system and render sensitive field data as non-sensitive. Thus, sensitive data may be hidden from data-stores/warehouses without worrying about downstream access to the data. The system (110) proposed may also preserve the data type and format of datasets but not limited to the like. The system encrypts a dataset with a unique key (404) and then allows a privileged user (902) to decrypt the encrypted dataset with the unique key (404) and view the decrypted values without getting access to the sensitive original dataset.