Machine-Trained Networks With Configurable Activation Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep machine learning applications require significant computational resources, which are often unavailable in resource-constrained devices due to their limited computational and memory resources.
Innovation Solution
The development of machine-trained networks using novel processing nodes with adjustable activation functions that can emulate multiple logical operators and periodic functions, allowing for a richer set of mathematical expressions to be formulated with fewer layers, thus reducing the resource requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep machine learning applications with many layers of processing nodes are used, then computational power and accuracy are improved, but device complexity and resource consumption increase
Solution Approach 1:
The patent applies universality by designing a single processing node architecture that can perform multiple functions through configurable parameters. The processing nodes use a universal computation pattern that can be adjusted via bias and weight parameters to implement different logical operations and mathematical functions, eliminating the need for specialized hardware for each function type.
Solution Approach 2:
The patent employs parameter changes by making the processing nodes configurable through adjustable bias and weight parameters. By changing these parameters, the same hardware structure can adapt to different computational tasks, enabling a single node design to replace multiple specialized node types and reducing overall device complexity.
2Power
If deep machine learning applications with many layers of processing nodes are used, then computational power is improved, but resource consumption increases
Solution Approach 1:
The patent merges multiple computational functions into a single processing node design. By combining universal computation capabilities with configurable parameters within one node, the system reduces the total number of nodes required, thereby lowering energy consumption while maintaining computational power.
Solution Approach 2:
The universal processing node can perform multiple computational tasks through parameter configuration, reducing the need for separate specialized hardware components. This multi-functionality decreases the overall resource consumption while preserving high computational power through efficient resource utilization.
3Ease of manufacture
If traditional processing nodes are used, then implementation is simpler, but expressiveness and computational efficiency decrease
Solution Approach 1:
The patent uses parameter changes to provide configurability within a simple hardware structure. The processing nodes maintain implementation simplicity through a standardized design while achieving high expressiveness through adjustable bias and weight parameters that can be configured for different computational tasks during training and inference.
Data Source
AI summary
Some embodiments of the invention provide efficient, expressive machined-trained networks for performing machine learning. The machine-trained (MT) networks of some embodiments use novel processing nodes with novel activation functions that allow the MT network to efficiently define with fewer processing node layers a complex mathematical expression that solves a particular problem (e.g., face recognition, speech recognition, etc.). In some embodiments, the same activation function (e.g., a cup function) is used for numerous processing nodes of the MT network, but through the machine learning, this activation function is configured differently for different processing nodes so that different nodes can emulate or implement two or more different functions (e.g., two or more Boolean logical operators, such as XOR and AND). The activation function in some embodiments is a periodic function that can be configured to implement different functions (e.g., different sinusoidal functions).