Neural Network Layer Serialization for Smaller Model Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The sheer amount of data required to represent neural networks, particularly the weights and biases, hampers efficient transmission and updating, leading to high data rates and computational complexity, which current lossy compression methods fail to adequately address.
Innovation Solution
A data stream representation of neural networks is structured into individually accessible portions, each representing a neural network layer, with parameters like serialization, numerical computation, layer type, and pointers encoded/decoded to facilitate efficient encoding, decoding, and inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If neural network parameters are stored in container formats with all data and properties, then the neural network can be fully reconstructed and executed, but the coded size of neural network representations becomes large
Solution Approach 1:
The patent segments the neural network representation into individually accessible portions, where each portion corresponds to a specific neural network layer. This segmentation allows the data stream to be divided into manageable chunks that can be independently processed, transmitted, and stored, thereby reducing the coded size while maintaining the ability to reconstruct and execute the full neural network.
Solution Approach 2:
The patent introduces parameters such as serialization parameters that indicate coding orders for neural network parameters, numerical computation parameters indicating data types and bit sizes, and layer type parameters. These parameter changes enable efficient encoding and decoding processes that reduce the overall coded size while preserving the neural network's functionality and reconstruction accuracy.
2Productivity
If frequent transmission and updates of neural networks are performed, then model updates can be applied quickly, but the data rate becomes a serious bottleneck
Solution Approach 1:
By segmenting the neural network data stream into individually accessible portions corresponding to different layers, the patent enables selective transmission and update of specific layers rather than requiring transmission of the entire neural network. This segmentation significantly reduces the data rate required for frequent transmissions and updates, thereby improving productivity without being constrained by data rate bottlenecks.
Solution Approach 2:
The patent extracts only the necessary portions of neural network data that need to be transmitted or updated, rather than transmitting the complete neural network representation. This extraction approach minimizes the data rate requirement while enabling frequent updates and transmissions of the most relevant model components.
3Quantity of substance
If lossy compression is applied to reduce coded size of neural network matrices, then transmission efficiency improves, but inference quality may deteriorate
Solution Approach 1:
The patent introduces numerical computation parameters that indicate the data type and bit size for representing neural network parameters during inference. By carefully managing these parameter representations and using efficient encoding schemes with proper quantization and precision control, the patent reduces coded size while maintaining inference quality through optimized parameter representation rather than simple lossy compression.
Data Source
AI summary
Data stream having a representation of a neural network encoded thereinto, the data stream including serialization parameter indicating a coding order at which neural network parameters, which define neuron interconnections of the neural network, are encoded into the data stream.