Data Migration Apparatus for Distributed Key-Value Store Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed Key-Value Store systems, stateful processing methods lose extensibility when trying to manage transaction consistency and prevent split-brain syndrome, leading to data mismatch issues during server scaling, which hinders the scale-out process and reduces the advantages of stateless extensibility.
Innovation Solution
An information processing apparatus that migrates data from a source to a destination server while maintaining data consistency by updating differential data and transaction logs, allowing the system to continue processing without stopping client access, and dynamically updating client management information to reflect changes in data arrangements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If stateful processing method is used to manage transaction consistency and prevent split-brain syndrome, then reliability is improved, but extensibility deteriorates due to loss of stateless property
Solution Approach 1:
The patent segments the data migration process into multiple phases: initial data copy, differential data tracking, and transaction log synchronization. This allows the system to maintain stateful processing for consistency while enabling incremental scale-out by migrating data in chunks rather than requiring complete system state transfer at once.
Solution Approach 2:
The patent performs preliminary data copying and consistency establishment before actual migration. By pre-synchronizing data and maintaining transaction logs during the migration process, the system ensures reliability is preserved while the migration itself enables extensibility through seamless server addition.
2Productivity
If data migration is performed during online operation, then productivity is improved by avoiding system downtime, but data consistency becomes more difficult to maintain
Solution Approach 1:
The patent implements feedback mechanisms through transaction logs that continuously track data changes during migration. The system monitors differential data between source and destination servers, and automatically synchronizes updates, ensuring consistency is maintained even while the system remains online and operational throughout the migration process.
3Reliability
If differential data and transaction logs are used to maintain consistency during migration, then data consistency is improved, but device complexity increases
Solution Approach 1:
The patent introduces transaction logs as an intermediary mechanism that simplifies consistency management during migration. Rather than directly comparing and synchronizing all data between servers, the system uses transaction logs to track changes and apply them incrementally, reducing the complexity of migration management while maintaining data consistency.
Data Source
AI summary
An information processing apparatus accepts migration of data from a migration source information processing apparatus. The information processing apparatus includes a processor to execute an operation for operation target data as a target of a processing request given from a client apparatus when the operation target data is migrated from the migration source information processing apparatus and the processing request is received from the migration source information processing apparatus, and to notify the client apparatus of a processing result of the operation, the result containing arrangement information of the operation target data.