Microservice Data Masking via Byte Range Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data masking approaches are not well-suited for implementation in microservice deduplication environments, introducing complexities that hinder efficient data protection and confidentiality in scalable architectures.
Innovation Solution
A microservice-based data masking architecture that utilizes masker controller and masker worker microservices to perform inline and offline masking, allowing for flexible data masking modes, efficient reuse of hashes, and granular masking at the object or byte range level, while avoiding recalculation of fingerprints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional data masking approaches are used in microservice environments, then data masking functionality is provided, but system complexity increases and scalability is hindered
Solution Approach 1:
The data masking system is divided into separate microservices: a masker controller microservice that manages masking operations and a masker worker microservice that performs the actual masking. This segmentation allows each service to be developed, deployed, and scaled independently, reducing overall system complexity while maintaining reliable data masking functionality.
Solution Approach 2:
An internal load balancer is introduced as an intermediary component between the object access microservice and the masker worker microservices. The load balancer distributes masking requests across multiple worker instances, simplifying the architecture by providing a centralized request routing mechanism rather than requiring direct point-to-point connections between services.
2Reliability
If data masking is performed on all data, then data protection is improved, but computational overhead increases
Solution Approach 1:
The system performs masking at the byte range level rather than on entire objects. The masker worker identifies specific byte ranges within objects that contain sensitive data and applies masking only to those regions. This local quality approach protects sensitive information while minimizing computational overhead by avoiding unnecessary processing of non-sensitive data.
Solution Approach 2:
The system calculates and stores fingerprints of objects before masking. When masking is required, the system uses these pre-calculated fingerprints to quickly identify which objects need masking and which byte ranges require processing. This preliminary action eliminates the need for full object scanning and fingerprint recalculation, significantly reducing computational overhead while maintaining comprehensive data protection.
3Measurement precision
If fingerprints are recalculated for masked objects, then data integrity is verified, but processing time increases
Solution Approach 1:
The system calculates and stores fingerprints of objects before masking operations. When masking is applied, the system reuses these pre-calculated fingerprints to verify data integrity rather than recalculating them from scratch. This preliminary action maintains accurate data integrity verification while significantly reducing processing time for masked objects.
4Ease of manufacture
If masking is applied at object level, then implementation is simplified, but data accessibility is reduced
Solution Approach 1:
The system implements masking at the byte range level within objects, allowing different parts of the same object to have different masking characteristics. This enables the system to maintain implementation simplicity by processing data in manageable segments while improving data accessibility by preserving unmasked portions of objects that do not contain sensitive information.
Data Source
AI summary
A method includes receiving an object from a client as part of a data ingestion process, by a masker worker, generating a list that identifies one or more byte ranges, of the object, to be masked, providing the list to an object access microservice, by the object access microservice, creating an object recipe of the object, and the object recipe includes one or more slices, and each of the slices includes one or more segments of data, masking only those segments of a slice that are in a byte range identified in the list, creating a masked slice that comprises the masked segments, and creating a masked object recipe that contains a reference to the masked slice, and the masking is performed before the object is stored in storage, creating a masked object that includes the masked slice, and deduplicating the masked object.