NVMe-oF Data Compression With Logical-to-Physical Block Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
NVMe-oF data transmission is limited by the requirement for manual compression at the target computing device, which complicates bandwidth and storage efficiency due to the fixed logical block size, and lacks automatic compression at the initiator device.
Innovation Solution
Implementing a compression engine at the initiator computing device to convert logical blocks into smaller physical blocks, accompanied by metadata for correspondence, allowing transparent data transmission and storage as physical blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data is transmitted using fixed logical block sizes in NVMe-oF, then protocol compatibility is maintained, but network bandwidth utilization is inefficient due to inability to compress data before transmission
Solution Approach 1:
The patent segments the data transmission process into distinct phases: compression phase (converting logical blocks to physical blocks), transmission phase (sending compressed data), and decompression phase (converting back to logical blocks). This segmentation allows compression to occur without disrupting the fixed logical block structure required by NVMe-oF protocol, thereby improving bandwidth utilization while maintaining protocol compatibility.
Solution Approach 2:
The patent introduces an intermediary compression layer that sits between the NVMe-oF protocol and the physical transmission medium. This intermediary converts logical blocks to compressed physical blocks for transmission, then converts them back to logical blocks at the receiving end. This intermediary mechanism enables compression without requiring changes to the NVMe-oF protocol itself, resolving the contradiction between bandwidth efficiency and protocol compatibility.
2Quantity of substance
If manual compression is performed at the target computing device, then data size is reduced for storage, but operational complexity and processing overhead are increased
Solution Approach 1:
The patent inverts the traditional compression approach by performing compression at the initiator (source) device rather than at the target (destination) device. This inversion moves the compression operation to where the data originates, reducing the storage size before data is transmitted and stored at the target device. This approach simplifies target device operations while achieving the same compression benefit.
Solution Approach 2:
The patent applies preliminary compression action at the initiator device before data transmission. By compressing the data payload into physical blocks before sending it over the network, the system reduces the amount of data that needs to be transmitted and stored at the target device. This preliminary action eliminates the need for manual compression operations at the target device, simplifying its operation.
3Quantity of substance
If logical blocks are converted to smaller physical blocks through compression, then storage capacity efficiency is improved, but data structure complexity increases due to mapping requirements
Solution Approach 1:
The patent creates a mapping structure that copies the relationship information between logical blocks and physical blocks. This mapping acts as a reference table that records which physical blocks correspond to which logical blocks. By using this copied relationship information, the system can efficiently manage the conversion between logical and physical blocks without requiring complex real-time calculations, thus improving storage capacity utilization while controlling structure complexity.
Data Source
AI summary
A method for data transmission is provided. The method includes, at an initiator computing device, receiving a data payload to be transmitted to a target computing device for storage using the nonvolatile memory express over fabric (NVMe-oF) protocol, wherein the data payload is represented as a plurality of logical blocks. Using a compression engine, the data payload is compressed into a plurality of physical blocks having a smaller total data size than the plurality of logical blocks. A write request is transmitted to the target computing device, the write request including the plurality of physical blocks, and metadata specifying a correspondence between each logical block of the plurality of logical blocks and one or more corresponding physical blocks of the plurality of physical blocks.