AI Chip Operation Array Mapping for Neural Network Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial intelligence (AI) chips face inefficiencies due to low utilization of operation arrays when operating neural network models, as only a subset of rows are actively used while others remain idle, leading to poor compatibility and performance.
Innovation Solution
An operation method and apparatus that maps neuron data from D neural network layers to K rows of operation units, allowing for full-load operation by determining K pieces of neuron data to be operated per clock and inputting them to the operation array, thereby utilizing all rows and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neuron data is inputted row-by-row to the operation matrix, then the operation can be completed, but the utilization of the operation array is low and operation time is increased
Solution Approach 1:
The patent transforms the traditional row-by-row data input method into a multi-dimensional data mapping approach. By mapping D-dimensional neuron data to the K-row operation array using a mathematical transformation formula, the system enables parallel processing across multiple rows simultaneously, changing the operational dimension from sequential to parallel and resolving the contradiction between operation speed and compatibility.
Solution Approach 2:
The patent introduces a parameter transformation mechanism where the relationship between neuron data dimensions (D) and operation array rows (K) is established through a mathematical formula. This parameter change allows the system to adapt to different neural network model depths and operation array configurations, achieving full utilization of the operation array while maintaining compatibility with various model specifications.
2Productivity
If the depth of the neural network model matches the quantity of rows in the operation array, then full-load operation can be achieved, but compatibility with models of different depths is poor
Solution Approach 1:
The patent creates a universal data mapping formula that can accommodate any neural network model depth (D) with any operation array row count (K). The transformation formula serves multiple functions: it maps data appropriately for full utilization when D=K, handles cases where D<K by repeating patterns, and manages D>K by partitioning data, thereby achieving universal compatibility across different model configurations while maintaining full-load operation capability.
Data Source
AI summary
An operation method based on a chip having an operation array is provided, including: obtaining a neural network model including D neural network layers, each neural network layer being used for operating M*N neurons; determining, from M*N*D neurons of the D neural network layers, K neurons to be operated corresponding to each operation clock cycle, and inputting the K neurons to K rows of operation units of an operation array; and performing an operation on the inputted K neurons in the each operation clock cycle by using the operation array. The M*N*D neurons are mapped to K dimensions and then allocated to the K rows of operation units. The operation array is used for operating the neural network model in a full-load operation mode.