Neural Network Address Mapping for Reconstruction-Free Tensor Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Typical neural processors do not support operations for reconstructing data in generative adversarial networks and similar networks, leading to reduced utilization of multiply-accumulate (MAC) operations and lower performance.
Innovation Solution
A neural network operation apparatus with a memory and processors that validate data based on virtual tensor addresses and strides, obtaining real memory addresses through modulo operations and floor functions to perform operations like upsampling and transposed convolution without data reconstruction in physical memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data reconstruction operations are performed by transmitting data to host and receiving back, then data reconstruction is achieved, but MAC operation device utilization is reduced and total performance is lowered
Solution Approach 1:
The patent replaces the mechanical data transmission process (reading from host memory, writing to host memory) with a virtual memory addressing mechanism. The neural processor directly computes real memory addresses from virtual tensor addresses using mathematical operations, eliminating the need for physical data transfer and enabling in-place data reconstruction operations.
Solution Approach 2:
The patent introduces a virtual memory address space as an intermediary between the physical memory and the data reconstruction operations. By mapping virtual tensor addresses to real memory addresses through address translation, the system enables data reconstruction without actual data movement, resolving the contradiction between achieving data reconstruction and maintaining high MAC operation utilization.
2Use of energy by moving object
If typical neural processor scheme is used for data reconstruction, then data reconstruction is possible, but MAC operation device utilization is reduced
Solution Approach 1:
The patent changes the fundamental parameter of data address representation from physical memory addresses to virtual tensor addresses with stride information. This parameter transformation enables the system to perform data reconstruction operations directly within the neural processor using virtual address arithmetic, thereby maximizing MAC operation device utilization without sacrificing data reconstruction capability.
Data Source
AI summary
A neural network operation apparatus and method is provided. The neural network operation apparatus includes a memory configured to store data for a neural network operation, and a processor configured to validate the data based on a determination that the neural network operation should be performed on the data, obtain a real memory address to perform the neural network operation based on a result of the validating and a virtual tensor address of the data, and perform the neural network operation based on the real memory address.