In-Line Data Format Conversion in ML Hardware
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems require inefficient data format conversions and manipulations, often necessitating additional writes to memory, which wastes resources and increases latency.
Innovation Solution
The ML hardware performs data format conversions and manipulations in-line, eliminating the need for external memory writes and reducing resource usage and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data format conversion is performed by software component, then data can be converted from one format to another, but additional memory write operations are required which increases resource consumption and latency
Solution Approach 1:
The patent extracts the data format conversion functionality from the software component and relocates it to the hardware interface circuitry. This allows conversion to occur at the hardware level during data transmission, eliminating the need for software-based conversion and associated memory write operations.
Solution Approach 2:
The hardware interface performs data format conversion in advance, before the data reaches the processing unit. By converting formats during the data input phase rather than after retrieval from memory, the system eliminates subsequent memory write operations and reduces overall processing latency.
2Adaptability or versatility
If data manipulation is performed by software, then data layout and mapping can be changed, but additional memory write operations are required which waste resources
Solution Approach 1:
The patent extracts data manipulation functions (layout changes, mapping, shaping) from software and implements them in hardware interface circuitry. This enables real-time data transformation during transmission without requiring software intervention and associated memory operations.
Solution Approach 2:
The hardware interface performs data manipulation operations autonomously as data flows through the system. The interface circuitry automatically adjusts data layout, mapping, and shaping without external software control, eliminating unnecessary memory write operations and reducing resource consumption.
3Adaptability or versatility
If data is read from memory, converted by software, and stored back in memory, then data format conversion can be achieved, but the process is inefficient and resource intensive
Solution Approach 1:
The hardware interface performs data format conversion in advance during the data input phase, before data needs to be retrieved from memory for processing. This preliminary conversion eliminates the need for subsequent memory read-modify-write cycles, significantly reducing processing latency.
Solution Approach 2:
The patent removes the data conversion operation from the software processing chain and relocates it to hardware interface circuitry. This extraction eliminates the inefficient cycle of reading data from memory, converting it via software, and writing it back to memory.
Data Source
AI summary
A machine learning (ML) hardware includes a first data format conversion block configured to receive data generated by an application source in a first data format. The first data format conversion block is configured to convert the received data from the first data format into a second data format. The first data format is different from the second data format. The ML hardware includes a plurality of processing units configured to perform one or more ML operations on the data in the second data format to generate a processed data. The ML hardware includes a second data format conversion block configured to convert the processed data to a third data format. The ML hardware further includes a transmitting component configured to output the processed data in the third data format to a memory component for use by an application destination.