Microservice Data Masking via Byte Range Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data masking approaches are not well-suited for implementation in microservice deduplication environments, introducing complexities that hinder efficient data protection and confidentiality in scalable architectures.
Innovation Solution
The implementation of a microservice design with a masker controller and masker worker architecture that performs data masking in inline and offline modes, allowing for scalable data masking operations while avoiding unnecessary recalculation of fingerprints, and enabling efficient masking at both object and byte range levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional data masking approaches are used in microservice environments, then data masking functionality is provided, but system complexity increases and scalability is hindered
Solution Approach 1:
The data masking system is divided into separate microservices: a masker controller microservice that manages masking operations and a masker worker microservice that performs actual masking. This segmentation allows each service to be independently scaled, deployed, and maintained, resolving the contradiction between providing comprehensive masking functionality and maintaining system simplicity.
Solution Approach 2:
The masker controller acts as an intermediary between the data processing pipeline and the masking logic. It receives data, determines masking requirements, and coordinates with worker microservices to perform masking only when necessary, reducing overall system complexity while maintaining adaptability.
2Reliability
If data masking is performed on all data, then confidentiality is maximized, but computational overhead increases
Solution Approach 1:
Instead of masking all data uniformly, the system applies partial masking only to specific byte ranges identified as containing sensitive information. The masker worker analyzes data structure and applies masking selectively to portions that require protection, reducing computational overhead while maintaining necessary confidentiality.
Solution Approach 2:
The system changes the masking approach from a blanket operation to a parameter-driven operation. It identifies specific byte ranges and applies different masking strategies based on the data type and sensitivity level, optimizing computational resources while ensuring adequate protection.
3Measurement precision
If fingerprints are recalculated for all data operations, then data integrity is verified, but processing time increases
Solution Approach 1:
Fingerprints are calculated and stored during the initial data ingestion phase rather than being recalculated for every subsequent operation. This preliminary action allows the system to verify data integrity quickly by comparing stored fingerprints with current data hashes, significantly reducing processing time while maintaining integrity verification.
Solution Approach 2:
The system uses feedback from fingerprint comparisons to determine whether full processing is needed. When fingerprints match, the system can skip redundant processing steps, optimizing the balance between verification precision and processing efficiency.
Data Source
AI summary
A method includes retrieving an object from storage and copying the object, generating a list that identifies one or more byte ranges, of the copy of the object, to be masked, providing the list to a masker controller microservice that examines a recipe corresponding to the copy of the object, where the recipe references a slice of the copy of the object, and the slice includes one or more data segments, masking, by the masker controller microservice, a segment of the slice that is in one of the byte ranges, to create a masked segment, and replacing, in the slice, the segment with the masked segment, to create a masked slice and creating a masked object recipe that contains a reference to the masked slice, creating a masked object that includes the masked slice, and that references any unmasked segments of the slice, and deduplicating the masked object.