Event Stream Constraint Enforcement Using Placeholder Events
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Event-driven architectures face challenges in complying with legal and data-security constraints that restrict data storage and processing based on geographical location or network security policies, leading to incomplete event streams and compromised continuity.
Innovation Solution
A constraint engine replaces sensitive data in events with placeholder events, storing the original events in compliant repositories while maintaining stream continuity and allowing authorized consumers to access the original events upon request.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If events containing sensitive data are transmitted through the event stream, then downstream consumers can access complete event data, but the system violates legal mandates and network security protocols
Solution Approach 1:
The system segments event data into two parts: the public event stream containing placeholder events for compliance, and a separate authorized access pathway to the event repository for complete data. This segmentation allows simultaneous compliance and data accessibility.
Solution Approach 2:
Placeholder events act as intermediaries in the event stream, representing sensitive events without exposing actual sensitive data. They maintain stream continuity while preventing unauthorized access, serving as a mediator between compliance requirements and data availability.
2Reliability
If sensitive events are removed from the event stream to ensure compliance, then security constraints are satisfied, but the continuity of the event stream is compromised
Solution Approach 1:
The system creates placeholder events that copy the structural and metadata properties of sensitive events without copying the sensitive payload data. These placeholder copies maintain event stream continuity and schema validity while eliminating security violations.
Solution Approach 2:
Instead of removing sensitive events entirely (which would break continuity), the system inverts the approach by keeping event positions and structures intact while replacing only the sensitive content with placeholders, thus maintaining continuity through inversion of the removal action.
3Loss of information
If the event repository stores complete sensitive event data, then authorized consumers can access full information, but the repository location may violate geographical data storage constraints
Solution Approach 1:
The system applies different quality requirements to different access pathways: the event stream uses placeholder events with reduced data quality for compliance, while the event repository maintains complete data quality for authorized access. This local quality differentiation resolves the contradiction between data completeness and storage compliance.
Data Source
AI summary
A computing system is configured to (i) receive an event stream produced by at least one event producer; (ii) evaluate each respective event in the event stream against a set of one or more constraints; (iii) based on the evaluation, determine that a given event in the event stream is governed by a given constraint in the set of one or more constraints; (iv) after determining that the given event is governed by the given constraint, cause the event to be replaced by a corresponding placeholder event within the event stream, wherein the placeholder event omits at least a portion of data included within the given event; and (v) after determining that the given event is governed by the given constraint, cause the given event to be stored in an event repository that complies with the given constraint.