Sensitive Data Classification in Micro-service Applications
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In micro-service applications, it is difficult to correlate sensitive data with specific transactions or their originators, and determine which services access sensitive data, due to the lack of tracking mechanisms in existing systems.
Innovation Solution
A datastore layer service initiates queries and uses a sensitive data classifier to identify sensitive data items, sending transaction identifiers and classification information to a collector service, which stores records of sensitive data access and usage, enabling tracking of sensitive data transactions and their originators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If micro-service applications are used to improve scalability and maintenance ease, then application flexibility and productivity are improved, but the ability to track and correlate sensitive data with specific transactions deteriorates
Solution Approach 1:
The system performs preliminary actions by assigning transaction identifiers to transactions before they access sensitive data, and by pre-classifying data items as sensitive or non-sensitive. This enables tracking to be established in advance, allowing correlation of sensitive data access with specific transactions even in distributed micro-service environments.
Solution Approach 2:
The system implements feedback mechanisms by sending transaction identifier and classification information to a collector service that stores records of sensitive data access. This feedback loop enables continuous tracking and monitoring of sensitive data usage across multiple services, providing visibility into which transactions access which sensitive data items.
2Measurement precision
If data classification and tracking mechanisms are implemented to improve security monitoring, then measurement precision of sensitive data access is improved, but device complexity increases
Solution Approach 1:
The system segments the tracking functionality into distinct components: a sensitive data classifier that identifies sensitive data items, a collector service that stores access records, and integration points in the data access layer. This segmentation allows each component to perform its specific function independently, making the overall complex system manageable and maintainable through clear separation of concerns.
3Difficulty of detecting and measuring
If query information is analyzed to identify sensitive data items, then detection precision of sensitive data access is improved, but processing time increases
Solution Approach 1:
The system performs preliminary classification of data items in the datastore, marking them as sensitive or non-sensitive before queries are executed. This preliminary action allows the sensitive data classifier to quickly identify which data items are sensitive during query execution, rather than analyzing each data item in full during the query processing, thereby reducing the time overhead.
Data Source
AI summary
A datastore layer service of a plurality of services that compose an application receives, from an upstream service of the plurality of services, a request, the request being associated with a transaction submitted to the application, the request including a transaction identifier that uniquely identifies the transaction. The datastore layer service, in response to the request, initiates a query against a datastore to obtain a data item based on information included in the request. A sensitive data classifier analyzes query information associated with the query. The sensitive data classifier determines that the query requests a data item that has been classified as a sensitive data item. The sensitive data classifier causes the transaction identifier and classification information that indicates the query requested the data item that has been classified as a sensitive data item to be sent to a collector service.