Vector Neural Network Dynamic Routing Elimination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The capsule network's reliance on repeated dynamic routing slows down arithmetic processing speed and requires more memory, limiting its efficiency in machine learning operations.
Innovation Solution
A vector neural network architecture that calculates output vectors by obtaining prediction vectors, sum vectors, normalization coefficients, and output vectors using output vectors from lower layers, eliminating the need for multiple dynamic routing iterations and reducing memory requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If capsule network uses repeated dynamic routing to obtain output vectors, then the machine learning model can achieve accurate vector neuron calculations, but the arithmetic speed becomes slow
Solution Approach 1:
The patent pre-calculates and stores prediction matrices between lower layer nodes and upper layer nodes before actual inference. These prediction matrices contain pre-computed transformation information that eliminates the need for repeated dynamic routing calculations during forward propagation, thereby maintaining calculation accuracy while significantly improving arithmetic speed
Solution Approach 2:
The patent transforms the static prediction matrix storage into a dynamic retrieval and application process. During inference, the system dynamically selects and applies appropriate prediction matrices based on the specific lower layer node inputs, enabling flexible and accurate vector neuron calculations without repeating the computationally expensive dynamic routing process
2Adaptability or versatility
If capsule network repeats dynamic routing multiple times, then the model can process complex vector relationships, but memory usage increases
Solution Approach 1:
The patent creates compact copies of transformation information in the form of prediction matrices that are stored in memory. These prediction matrices are condensed representations of the complex vector relationships that would otherwise require multiple iterative routing operations, thereby reducing memory usage while preserving the ability to process complex vector relationships
Solution Approach 2:
The patent changes the representation parameters from iterative routing states to static prediction matrix parameters. By transforming the dynamic routing process into a parameter-based matrix multiplication approach, the system reduces memory requirements while maintaining the capability to handle complex vector relationships through the structured parameters in the prediction matrices
Data Source
AI summary
An apparatus including: memory that stores a machine learning model of a vector neural; and one or more processors that execute an arithmetic operation. The machine model has a plurality of vector neuron layers each including a plurality of nodes. When one of the plurality of vector layers is referred to as an upper layer and a vector layer below is referred to as a lower layer, one or more processors execute outputting one output vector by using output vectors from the plurality of nodes of the lower layer as an input for each node of the upper layer, the outputting including: obtaining a prediction vector, obtaining a sum vector based on a linear combination of the vectors, obtaining a normalization coefficient, and obtaining the output vector of the target node by dividing the sum vector by the norm and multiplying the divided sum vector by the normalization coefficient.