Bidirectional Data Correlation Using Unidirectional Update Engines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in accurately detecting correlations between data objects due to unpredictable timing and order of update notifications, leading to incorrect correlations and system behavior, especially when scaling and handling multiple types of data objects.
Innovation Solution
A system comprising first-to-second and second-to-first correlation engines that detect and update unidirectional correlations, with storage engines ensuring consistency across repositories, mitigating race conditions and maintaining correct correlations despite varying notification orders.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single correlation engine tracks bidirectional correlations, then the system structure is simple, but incorrect correlations occur due to unpredictable message timing and order
Solution Approach 1:
The single correlation engine is divided into two separate unidirectional correlation engines: one tracking correlations from first objects to second objects, and another tracking correlations from second objects to first objects. This segmentation allows each engine to process messages in a controlled direction, eliminating race conditions and ensuring reliable correlation detection despite unpredictable message timing and order.
2Reliability
If manual intervention is used to correct incorrect correlations, then system behavior can be corrected, but operational efficiency decreases and human resources are consumed
Solution Approach 1:
The dual unidirectional correlation engines automatically detect and maintain correct correlations without requiring manual intervention. The system self-corrects by processing messages in defined directions and using storage engines to maintain consistent correlation states, eliminating the need for human operators to intervene and correct incorrect correlations, thus maintaining both high reliability and operational efficiency.
3Productivity
If correlation messages are processed in unpredictable orders, then the system handles concurrent updates, but detection of incorrect correlations occurs
Solution Approach 1:
The system establishes predefined processing directions for correlation messages before execution. The first-to-second correlation engine is configured to process messages from first objects to second objects, while the second-to-first engine processes messages in the opposite direction. This preliminary establishment of processing order ensures that concurrent updates are handled efficiently while maintaining precise correlation detection, as each engine processes messages in a controlled sequence regardless of their arrival order.
4Ease of operation
If bidirectional correlations are tracked in a single repository, then data access is simplified, but race conditions occur during updates
Solution Approach 1:
The single correlation repository is segmented into two separate repositories: one for first-to-second object correlations and another for second-to-first object correlations. Each repository is managed by its corresponding unidirectional correlation engine, which updates correlations in a controlled direction. This segmentation eliminates race conditions during updates while maintaining stable and consistent correlation data, and the separate repositories can still be accessed independently, preserving operational simplicity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A first-to-second correlation engine determines correlations between first objects from a first object feed, and second objects from a second object storage, and generates first correlation messages indicative of the correlations for a first-to-second object direction and a second-to-first object direction. A second-to-first correlation engine determines respective correlations between the second objects from a second object feed and the first objects from a first object storage, and generates second correlation messages indicative of the respective correlations for the second-to-first object direction and the first-to-second object direction. A first-to-second correlation storage engine receives the first and second correlation messages for the first-to-second object direction and updates first-to-second correlation storage based on the received messages. A second-to-first correlation storage engine receives the first and second correlation messages for the second-to-first object direction and updates a second-to-first object correlation storage based on the received correlation messages.