Smart NIC Data Segmentation With CRC Offload for Storage Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The smart network card and solid-state storage device in storage engines calculate CRC information differently, leading to processor load and performance degradation due to the need for the processor to perform CRC calculations on both data streams and logic blocks during data transmission and storage.
Innovation Solution
The Smart NIC performs segmenting of data streams into blocks and calculates CRC information for each segment, combining them with the data to form target logic blocks suitable for storage devices, thereby offloading CRC calculations from the processor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the smart network card and solid-state storage device both calculate CRC information independently, then data integrity is ensured, but processor computing power is consumed and working performance deteriorates
Solution Approach 1:
The patent segments the data processing workflow into distinct phases: the smart network card performs CRC calculation on the data stream before transmission, while the solid-state storage device performs CRC verification on received logic blocks. This segmentation of computational tasks between different devices eliminates the need for duplicate full CRC calculations, ensuring data integrity while reducing processor burden.
Solution Approach 2:
The smart network card performs CRC calculation on the data stream in advance before transmitting to the storage device. The storage device then uses this pre-calculated CRC information for verification purposes, avoiding the need to recalculate CRC from scratch and thereby preserving processor resources.
2Reliability
If the solid-state storage device calculates CRC information based on segmented logic blocks, then storage integrity is ensured, but processor computing power is consumed
Solution Approach 1:
The patent introduces CRC information as an intermediary element that carries verification data between the smart network card and storage device. Instead of the storage device recalculating CRC on each logic block, it uses the pre-computed CRC information transmitted with the data, significantly reducing computational energy consumption while maintaining storage integrity.
3Reliability
If the smart network card calculates CRC based on the entire data record, then transmission integrity is ensured, but the storage device cannot efficiently verify segmented logic blocks
Solution Approach 1:
The patent segments the CRC verification process to match the logic block structure. The smart network card calculates CRC for the entire data stream, then the storage device divides both the data and CRC information into corresponding logic blocks. This allows efficient verification of individual logic blocks during storage operations while maintaining overall transmission integrity.
Data Source
AI summary
A data processing method, system and device, and a medium are provided. The method comprises: receiving a data stream; dicing the data stream to obtain a plurality of segmented data blocks; respectively calculating verification information corresponding to the plurality of segmented data blocks; and sending the segmented data blocks and the corresponding check information as a target logic block to a storage device.