Database Node Transaction Log Transmission and Query Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face challenges in efficiently managing storage and memory resources when synchronizing data across multiple nodes, particularly in methods that require large storage and memory for data replication and processing queries, and in methods that replicate only a portion of data, which complicates data combination across nodes.
Innovation Solution
A database system with multiple nodes that includes a first node for holding transaction logs and a second node for receiving and processing these logs, where specific identification information is transmitted to indicate the transaction logs and data to be shared, allowing for efficient data transmission and processing without replicating the entire database, enabling query handling while minimizing storage and memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is synchronized between all nodes by applying log-based replication to all nodes, then query processing capability is improved, but storage and memory requirements increase
Solution Approach 1:
The patent segments the database into multiple partitions distributed across different nodes. Each node stores only its local partition data rather than complete database copies, reducing storage requirements while maintaining query processing capability for partition-specific operations.
Solution Approach 2:
The patent implements partial data replication where nodes replicate only necessary portions of data (specific partitions or frequently accessed data) rather than complete database copies. This allows query processing on replicated data while minimizing storage overhead through selective replication.
2Quantity of substance
If nodes only replicate a portion of data and process requests relating to copied data, then storage requirements are reduced, but data combination complexity increases
Solution Approach 1:
The patent introduces a coordinator node or metadata service that manages data location information and coordinates data retrieval across nodes. This intermediary handles the complexity of data combination by providing a unified interface for query processing, abstracting away the distributed data location complexity from end users.
Solution Approach 2:
The patent pre-computes and stores metadata about data partitions, their locations, and relationships across nodes. This preliminary organization of data location information enables efficient data combination during query processing without requiring complex real-time coordination, as the routing information is prepared in advance.
3Speed
If complete database replication is performed across all nodes, then query processing speed is improved, but network resources for synchronization increase
Solution Approach 1:
The patent implements partial data replication where only necessary database partitions or frequently accessed data subsets are replicated across nodes. This selective replication reduces network bandwidth consumption during synchronization while still enabling fast local query processing for the replicated portions.
Solution Approach 2:
The patent divides the database into segments or partitions that are distributed across nodes. Only relevant segments are replicated to each node based on query patterns and data access requirements, minimizing network synchronization overhead while maintaining query speed for local operations.
4Loss of energy
If selective data replication is implemented, then network resources are saved, but storage requirements at individual nodes increase for maintaining data consistency
Solution Approach 1:
The patent segments data into partitions distributed across nodes, with each node storing only its assigned partition. This segmentation reduces storage requirements at individual nodes compared to complete replication, while the distributed architecture maintains overall data availability and consistency through coordinated access protocols.
Data Source
AI summary
Provided is a database system wherein a first node, in accordance with a data transmission request from a second node, transmits, to the second node, specific identification information indicating a transaction log at a predetermined time point, and a transaction log and identification information from after the predetermined time point, and transmits the requested data to the second node from the predetermined time point onwards, and the second node transmits a transmission request for data to the first node in accordance with a search request, and when data received from the first node has been loaded into a memory, the second node executes, on the loaded data, a command relating to a transaction log that is newer than the transaction log indicated by the specific identification information, responds to the search request on the basis of data on which the command has been executed, and deletes the data on which the command has been executed, in accordance with a predetermined condition.