Atomic Update Operations in Data Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database update techniques are resource intensive due to the need for multiple input-output operations, especially when incrementing counters, which can lead to significant computing resource consumption and performance reduction in data storage systems, and the use of synchronization locks further complicates the process.
Innovation Solution
Implementing atomic update operations in a data storage system that allows multiple updates to be performed without a read operation for every update, by storing operands and applying them in a single read request, reducing the number of read/write operations and minimizing resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional update techniques are used to increment a counter, then the counter value is updated correctly, but multiple input-output operations are performed consuming significant computing resources
Solution Approach 1:
The system performs preliminary actions by accumulating multiple update operations in a queue before executing them. Instead of immediately applying each update operation that requires reading the current value, computing the new value, and writing it back, the system queues these operations and applies them in batches, significantly reducing the number of read-write cycles to storage.
Solution Approach 2:
Multiple individual update operations are merged into a single batch operation. The system combines multiple counter increment requests into one atomic update operation that reads the current value once, applies all pending increments, and writes the final value back once, thereby consolidating multiple I/O operations into a single I/O operation.
2Reliability
If synchronization locks are used to manage concurrent counter updates, then data consistency is maintained, but computing resources are consumed and delays are introduced
Solution Approach 1:
The system introduces an intermediary mechanism in the form of a queue and batch processing logic that mediates between concurrent update requests and the actual storage operations. This intermediary layer manages the ordering and batching of updates without requiring traditional synchronization locks, allowing concurrent requests to be queued and processed efficiently in batches.
Solution Approach 2:
The system performs preliminary queuing of update operations before execution, allowing concurrent requests to be accumulated and ordered before being applied. This preliminary organization of operations eliminates the need for locks during the update execution phase, as the batch operation is designed to handle concurrent requests atomically without blocking other operations.
3Reliability
If synchronization locks are used in source code, then concurrent access is controlled, but the code becomes complicated and error-prone
Solution Approach 1:
The system implements self-service mechanisms where the batch update operation inherently handles concurrent access control without requiring explicit lock management in the application code. The atomic nature of the batch operation and the queue-based ordering mechanism automatically ensure data consistency, eliminating the need for developers to manually implement and manage synchronization locks.
Solution Approach 2:
The queue and batch processing logic serve as an intermediary layer that abstracts away the complexity of concurrent access control. Instead of requiring application code to manage locks, the system provides a higher-level abstraction where concurrent updates are automatically queued and processed in batches, simplifying the source code while maintaining reliability.
Data Source
AI summary
Technology is disclosed for performing atomic update operations in a storage system (“the technology”). The technology can receive an update command to update a value associated with a key stored in the storage system as a function of an input value; store the input value in a log stored at the storage system but not updating the value stored in the storage system; and update the value associated with the key with the received input values value based on the a function to generate an updated value, the updating occurring asynchronously with respect to receiving the update command.