Data Matrix Layer for Scalable Multicast Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In replicated database systems, the master data source faces resource drainage and performance issues when updating multiple nodes, and maintaining accurate network addresses and schema information becomes cumbersome as nodes change or lose links with master databases.
Innovation Solution
A data distribution system with a producer layer, data matrix layer, and consumer layer, where producer nodes instruct data matrix nodes to listen to multicast addresses and send update messages with specifications for each data set, allowing data matrix nodes to determine and store relevant data sets, and consumer nodes register for data, enabling efficient data distribution without the master node needing to know consumer details.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the master data source sends update messages to each node individually, then data replication is achieved, but resource consumption increases and scalability deteriorates
Solution Approach 1:
The patent combines multiple individual update messages into a single multicast message that can be received by multiple slave nodes simultaneously. The master node sends one consolidated update message containing changes for multiple databases, and all subscribed slave nodes receive and process this single message, thereby reducing network traffic and master node resource consumption while maintaining reliable data replication across all nodes.
2Ease of operation
If the master data source maintains records of all node addresses and schemas, then data distribution is possible, but system complexity increases when nodes change
Solution Approach 1:
The patent implements a self-service mechanism where slave nodes automatically register with the master node by sending their database schema information and subscribed database lists. When nodes are added or removed, they automatically notify the master node of their presence or absence. This eliminates the need for the master node to actively track and maintain records of all node addresses and schemas, reducing management complexity while enabling seamless data distribution to dynamic node sets.
3Stability of the object's composition
If the master data source sends updates to all nodes, then data consistency is maintained, but network traffic increases
Solution Approach 1:
The patent applies local quality by allowing different slave nodes to subscribe to different subsets of databases based on their specific needs. Each slave node receives update messages only for the databases it has subscribed to, rather than receiving updates for all databases. This selective subscription approach maintains data consistency for each node's relevant databases while significantly reducing overall network traffic compared to broadcasting updates to all nodes for every change.
Data Source
AI summary
A method and system for distributing data from one or more main data sources to one or more consumer nodes. An abstract data matrix layer is provided between a producer layer and a consumer layer. A producer node multicasts data updates to selected data matrix nodes in the data matrix layer, and a consumer node registers with the data matrix layer to receive designated data updates. One or more data matrix nodes then serve the requested data updates on the requesting consumer node.