Neural Network Weight Data Transmission Method
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning algorithms face challenges in efficiently loading and transmitting weight data, which affects calculation performance and user experience due to the large size of weight parameters, typically ranging from tens of thousands of bits to megabits, leading to prolonged calculation times and bandwidth inefficiencies.
Innovation Solution
A data transmission method that dynamically adjusts data reading policies by comparing weight specifications with write memory size, dividing the memory into first-in first-out and multiplexing write memories, and optimizing data loading strategies to reduce bandwidth usage and calculation time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If weight data is loaded from memory to calculation circuit, then calculation can be performed, but data transmission bandwidth is consumed and calculation time is prolonged
Solution Approach 1:
The patent divides weight data into hot weights (frequently used) and cold weights (less frequently used), and pre-loads hot weights into the write memory buffer before calculation operations begin. This preliminary action ensures that frequently accessed weight data is already available in the buffer, eliminating the need for repeated memory access during calculation and thereby reducing calculation time.
Solution Approach 2:
The patent implements a nested memory structure where the write memory buffer is divided into multiple levels: a first buffer for hot weights and a second buffer for cold weights. This nested arrangement allows the system to access hot weights from the faster first buffer while cold weights are stored in the second buffer, creating a hierarchical memory access pattern that improves calculation performance by minimizing access time for frequently used data.
2Productivity
If weight data is transmitted from memory to calculation circuit, then calculation can be performed, but data transmission bandwidth is consumed
Solution Approach 1:
The patent pre-loads hot weights into the write memory buffer before calculation operations begin. This preliminary action ensures that frequently accessed weight data is already available in the buffer, eliminating the need for repeated memory access during calculation and thereby reducing calculation time.
Solution Approach 2:
The patent segments weight data into hot weights and cold weights based on usage frequency, and stores them in different buffer regions. This segmentation allows the system to optimize bandwidth usage by only loading hot weights into the fast buffer, while cold weights remain in the second buffer, thereby reducing the total amount of data that needs to be transmitted over the bandwidth-constrained interface.
3Speed
If write memory is divided into first-in first-out and multiplexing write memories, then data reading efficiency is improved, but memory structure complexity increases
Solution Approach 1:
The patent segments the write memory buffer into multiple regions: a first buffer for hot weights and a second buffer for cold weights. This segmentation allows the system to optimize data access patterns by directing frequently accessed hot weights to the first buffer with faster access, while less frequently accessed cold weights are stored in the second buffer, thereby improving overall reading speed.
Solution Approach 2:
The patent pre-loads hot weights into the write memory buffer before calculation operations begin. This preliminary action ensures that frequently accessed weight data is already available in the buffer, eliminating the need for repeated memory access during calculation and thereby reducing calculation time.
Data Source
AI summary
Provided are a data transmission method for a neural network, and a related product. The method includes the following steps: acquiring a weight specification of weight data stored in a memory, comparing the weight specification with a specification of a write memory in terms of size and determining a comparison result; according to the comparison result, dividing the write memory into a first-in first-out write memory and a multiplexing write memory; according to the comparison result, determining data reading policies of the first-in first-out write memory and the multiplexing write memory; and according to the data reading policies, reading weights from the first-in first-out write memory and the multiplexing write memory and loading the weights to a calculation circuit. The technical solution provided by the present application has the advantages of low power consumption and short calculation time.