Peer-to-Peer Snapshot Copying Between Memory and Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The input/output performance of disk storage devices has become a bottleneck in database systems, even with the use of solid-state disks, limiting the performance of database operations in real-time processing and high-performance scenarios.
Innovation Solution
A computer communication device with CXL-based connectivity facilitates peer-to-peer data transfer between memory and storage devices, utilizing a computing unit with a DMA engine to directly copy data between memory and storage without host intervention, enabling efficient snapshot operations for in-memory databases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional disk-based database operations are used, then device complexity is reduced, but input/output performance becomes a bottleneck limiting real-time processing capability
Solution Approach 1:
The system segments the database into two distinct parts: hot data residing in memory devices for high-speed access and cold data stored in storage devices for persistence. This segmentation allows the system to achieve high input/output performance for frequently accessed data while maintaining the benefits of disk-based storage for less frequently accessed data, thereby resolving the contradiction between performance and complexity.
Solution Approach 2:
The computing unit acts as an intermediary between the host and the memory/storage devices. It receives data from the host, determines appropriate destination devices based on data characteristics, and facilitates data transfer without requiring direct host intervention in the transfer process. This intermediary approach improves input/output performance while managing system complexity through automated data routing.
2Reliability
If host-managed device memory addresses are used for data storage, then ease of operation is improved, but data loss risk increases due to host dependency
Solution Approach 1:
The computing unit autonomously manages data transfer operations by receiving source and destination information from the host, determining target devices, and executing data copying without requiring continuous host intervention. This self-service capability improves reliability by reducing host dependency for data persistence while maintaining ease of operation through automated processes.
Solution Approach 2:
The patent extracts the data transfer and management functions from the host and relocates them to the computing unit. By taking out these critical functions, the system reduces host dependency and associated data loss risks while maintaining operational simplicity through the autonomous computing unit that handles data routing and transfer.
3Speed
If peer-to-peer data transfer is implemented, then speed of data copying is improved, but device complexity increases due to direct connectivity requirements
Solution Approach 1:
The computing unit serves as an intermediary that enables peer-to-peer data transfer between memory and storage devices. It receives data from the host, determines appropriate destinations, and facilitates direct data copying without requiring complex routing configurations. This intermediary approach achieves high data copying speeds while managing connectivity complexity through centralized coordination.
Solution Approach 2:
The computing unit performs multiple functions including receiving data, determining destination devices based on data characteristics, managing memory address mappings, and facilitating peer-to-peer transfer. This multi-functionality allows the system to achieve fast data copying through direct connectivity while reducing overall architectural complexity by consolidating multiple responsibilities into a single coordinated component.
4Productivity
If in-memory database operations are used, then productivity is improved, but loss of information occurs during data persistence transitions
Solution Approach 1:
The computing unit performs preliminary actions by receiving source information indicating data to be copied before the actual data transfer occurs. It determines destination information and prepares the data routing path in advance, ensuring that data persistence operations are completed accurately and completely. This preliminary preparation prevents information loss during the snapshot process while maintaining high real-time processing performance.
Solution Approach 2:
The system employs feedback mechanisms where the computing unit receives confirmation of data receipt and transfer completion from the host and storage devices. This feedback loop ensures that data persistence operations are verified and completed accurately, preventing information loss during transitions between in-memory and persistent storage while maintaining the high productivity benefits of in-memory operations.
Data Source
AI summary
A computer networking device includes a plurality of ports, a memory, and a computing unit. The computing unit receives source information and destination information for a snapshot from the host; obtains a read-value of a memory area corresponding to the source information in a memory device through a port identified based on mapping information and the source information; and transmit, to a storage device, the read-value based on peer-to-peer communication for a write operation of writing in a memory area corresponding to the destination information in the storage device through a port identified based on the mapping information and the destination information.