Microservice Data Masking via Segmentation and Intermediary
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data masking approaches are not well-suited for implementation in microservice deduplication environments, introducing complexities that hinder efficient data protection and confidentiality in scalable architectures.
Innovation Solution
A microservice design with a masker controller and masker worker architecture that performs data masking in inline and offline modes, allowing for scalable data masking operations, avoiding recalculation of fingerprints, and enabling efficient masking at both object and byte range levels, with options for storing both original and masked data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional data masking approaches are used, then data masking functionality is provided, but implementation complexities are introduced in microservice deduplication environments
Solution Approach 1:
The data masking system is divided into separate microservices: a masker controller microservice that manages masking operations and a masker worker microservice that performs the actual masking. This segmentation allows each service to be developed, deployed, and scaled independently, reducing implementation complexity while maintaining functionality in microservice deduplication environments
Solution Approach 2:
The masker controller acts as an intermediary between the deduplication system and the masking operations. It receives data from the deduplication process, coordinates masking through the worker microservice, and returns masked data, thereby simplifying the integration of masking into existing microservice architectures
2Reliability
If data masking is performed on all data, then data protection is maximized, but computational efficiency decreases
Solution Approach 1:
The system applies data masking selectively based on data characteristics and sensitivity levels rather than uniformly to all data. The masker worker microservice identifies and masks only the portions of data that require protection, optimizing computational efficiency while maintaining adequate data protection for sensitive information
Solution Approach 2:
The system performs fingerprint calculation before masking operations. By pre-computing fingerprints of the original data, the system can determine whether masking is necessary and avoid redundant processing, thereby improving computational efficiency while ensuring protection where needed
3Measurement precision
If fingerprints are recalculated for masked data, then data integrity is verified, but processing time increases
Solution Approach 1:
Fingerprints are calculated and stored for the original data before masking operations begin. When masking is applied, the system uses these pre-computed fingerprints to verify data integrity rather than recalculating from scratch, significantly reducing processing time while maintaining verification accuracy
Solution Approach 2:
The system creates a copy of the original data's fingerprint information and uses this copy for integrity verification of the masked data. This copying approach allows verification without requiring time-consuming recalculation, as the fingerprint copy already contains the necessary integrity information
4Ease of operation
If both original and masked data are stored, then data accessibility is maintained, but storage requirements increase
Solution Approach 1:
The storage strategy is made dynamic based on access patterns and security requirements. The system can configure whether to store only masked data, only original data, or both, allowing optimization of storage requirements while maintaining necessary data accessibility for different user roles and operations
Solution Approach 2:
Different data storage strategies are applied to different data types and access scenarios. Sensitive data may be stored only in masked form for general access, while original data is retained only where specifically needed for verification or administrative operations, reducing overall storage requirements while maintaining accessibility where required
Data Source
AI summary
A method includes retrieving, with a masker controller job, an object and an associated object ID from a masking bucket that is defined in storage, making a copy of the object, with a masker worker microservice, masking the copy of the object to create a masked object, transmitting the masked object to an object access microservice, with the object access microservice, transmitting the masked object to a deduplication microservice, with the deduplication microservice, deduplicating the masked object, and storing the masked object in the storage.