Conditional Transcoding for Bandwidth-Limited Encoded Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage devices, such as Solid State Drives (SSDs), face challenges in efficiently transferring data to host processors due to bandwidth limitations, especially when dealing with compressed or encoded data, which can result in increased data transmission and processing times, potentially negating the benefits of near-storage processing.
Innovation Solution
Implementing a transcoder within the storage device that filters and re-encodes data using embedded encoding information, allowing for the transmission of encoded data to the host processor without inflating the result, thereby reducing processing time and bandwidth usage, and utilizing a columnar format to leverage data compression and encoding schemes like Run Length Encoding (RLE) and Bit Packing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is transmitted from storage device to host processor using conventional bandwidth-limited interfaces, then data transfer can be performed, but the transfer time becomes excessively long and bandwidth is wasted
Solution Approach 1:
The patent applies preliminary action by performing data filtering and re-encoding operations at the storage device before data leaves the storage system. The transcoder component processes the encoded data, identifies and extracts only the relevant portions needed by the host processor, and re-encodes this filtered data. This preliminary processing eliminates the need to transfer unnecessary data over the bandwidth-limited interface, thereby significantly reducing transfer time while improving effective data transfer speed.
2Reliability
If encoded data is processed and transmitted without filtering, then data integrity is maintained, but the amount of data transmitted increases unnecessarily
Solution Approach 1:
The patent applies the extraction principle by implementing a filtering mechanism that selectively removes unnecessary data from the encoded data stream. The transcoder reads the encoded data, uses metadata to identify relevant data portions, and extracts only those portions that need to be transmitted to the host processor. This extraction process reduces the quantity of transmitted data while maintaining data integrity through selective rather than arbitrary data removal.
Solution Approach 2:
The patent applies parameter changes by modifying the encoding parameters of the data during the filtering process. The transcoder re-encodes the filtered data using updated encoding parameters that reflect the reduced data set. This parameter change allows the system to transmit smaller, more efficiently encoded data portions while maintaining the necessary data integrity for the host processor's operations.
3Adaptability or versatility
If all encoded data is transmitted to the host processor, then complete data availability is ensured, but processing time and bandwidth are wasted on unnecessary data
Solution Approach 1:
The patent applies local quality by making the data transmission adaptive to the specific needs of the host processor. The transcoder uses metadata and filtering criteria to identify which specific portions of the encoded data are relevant to the current host processor operation. This local quality approach ensures that data availability is maintained for the specific data portions needed, while eliminating unnecessary data transmission, thereby improving overall data processing efficiency.
Data Source
AI summary
A transcoder is disclosed. The transcoder may comprise a buffer to store input encoded data. An index mapper may map an input dictionary to an output dictionary. A current encode buffer may store a modified current encoded data, which may be responsive to the input encoded data, the input dictionary, and the map from the input dictionary to the output dictionary. A previous encode buffer may store a modified previous encoded data, which may be responsive to the input encoded data, the input dictionary, and the map from the input dictionary to the output dictionary. A rule evaluator may generate an output stream responsive to the modified current encoded data in the current encode buffer, the modified previous encoded data in the previous encode buffer, and transcoding rules.