Sidecar Proxy Label Enforcement for Microservice Data Governance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In decentralized systems, particularly edge computing systems, complying with diverse data governance requirements from various sources such as laws, regulations, and industry standards is challenging due to the diverse origins and processing of data through multiple microservices, which is not trivial in centralized systems.
Innovation Solution
Implementing a sidecar proxy that extracts label information from input data to determine compliance with data governance policies and tags output data with lineage information, ensuring compliance by filtering and transforming data as needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data processing is distributed across multiple microservices in decentralized systems, then system scalability and processing capability are improved, but data governance compliance becomes more difficult to enforce
Solution Approach 1:
The patent introduces sidecar proxies as intermediary components that accompany each microservice. These sidecar proxies intercept data flows, extract lineage information, and enforce data governance policies without requiring modifications to the microservices themselves. This mediator approach resolves the contradiction by maintaining system scalability while centralizing governance enforcement at the proxy level.
Solution Approach 2:
The patent segments data governance enforcement into independent sidecar proxy components that can be individually deployed with each microservice. This segmentation allows the system to maintain distributed processing capability while distributing governance enforcement responsibilities across multiple independent proxy instances, each handling its associated microservice's data compliance.
2Loss of information
If data lineage tracking is implemented across decentralized microservices, then data provenance visibility is improved, but system complexity and overhead increase
Solution Approach 1:
The sidecar proxies automatically extract lineage information from data payloads and append it to output data without requiring manual intervention or complex centralized tracking infrastructure. Each proxy independently performs lineage tracking for its associated microservice, reducing overall system complexity while maintaining comprehensive lineage visibility across the decentralized system.
3Reliability
If data governance policies are enforced at each microservice level, then compliance accuracy is improved, but processing speed and efficiency decrease
Solution Approach 1:
The sidecar proxies perform data governance compliance checks and lineage extraction as preliminary actions before data enters the microservice processing pipeline. By pre-validating data and preparing lineage information in advance, the system ensures compliance accuracy is maintained while minimizing the processing overhead during the main data transformation operations.
Data Source
AI summary
In one embodiment, a sidecar proxy executed by a device extracts label information from input data for input to a microservice associated with the sidecar proxy indicative of a lineage of the input data. The sidecar proxy makes, based on the label information, a determination as to whether the microservice processing the input data would violate a data governance policy. The sidecar proxy provides, based on the determination, the input data to the microservice. The sidecar proxy tags output data generated by the microservice with appended label information that includes the label information extracted from the input data and an indication of a data transformation performed by the microservice to the input data to form the output data.