Neural Processing Cell Contextual Field Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Leaky Integrate and Fire (LIF)-inspired multi-layer perceptron (MLP)-based deep neural networks (DNNs) are economically, technically, and environmentally unsustainable due to their energy-inefficient and fault-intolerant nature, which is exacerbated by the lack of dynamic cooperation and information sharing between neurons, making them unsuitable for real-time applications in low-energy electronics.
Innovation Solution
A computational neural layer comprising interconnected neural processing cells with a receptive field generator, transfer function, and activation circuit that integrates local and universal contextual fields to selectively process relevant information, reducing energy consumption and enhancing resilience by amplifying coherent signals and suppressing irrelevant ones.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If LIF-inspired MLP-based DNNs process every piece of information selfishly, then information processing capability is maintained, but energy consumption increases significantly
Solution Approach 1:
The patent merges local contextual fields from neighboring neurons with universal contextual fields from global memory states into a unified processing mechanism. This integration allows neurons to cooperate and share information, filtering out redundant computations and reducing overall energy consumption while maintaining processing capability through coordinated neural activity.
Solution Approach 2:
The patent implements feedback mechanisms where neurons receive contextual information from both local neighbors and global memory states, allowing dynamic adjustment of processing based on system state. This feedback enables neurons to suppress irrelevant information and focus on meaningful patterns, reducing unnecessary energy expenditure while preserving essential processing functions.
2Speed
If LIF cells fire continuously to maintain processing, then processing speed is maintained, but energy consumption becomes unsustainable
Solution Approach 1:
The patent enables neurons to fire periodically rather than continuously, using contextual fields to timing and coordination. Neurons can remain inactive during periods when information is not needed, then activate synchronously when processing is required, maintaining processing speed through coordinated periodic firing rather than continuous activity.
Solution Approach 2:
The patent introduces dynamic modulation of neural activity through contextual fields that adjust processing intensity based on system state. This allows the network to adapt its processing speed dynamically, firing rapidly when needed and remaining quiet during low-demand periods, thereby maintaining overall processing speed while significantly reducing average energy consumption.
3Device complexity
If LIF neurons operate independently without cooperation, then implementation simplicity is maintained, but fault tolerance and learning speed deteriorate
Solution Approach 1:
The patent implements universal contextual fields that serve multiple functions: providing global memory state information, enabling fault detection, facilitating coordinated processing, and supporting learning. This multi-functionality is achieved through a single global memory structure that all neurons access, adding fault tolerance and cooperation without proportionally increasing implementation complexity.
Solution Approach 2:
The patent introduces contextual fields as intermediary structures that mediate between individual neurons and the global system state. These fields act as messengers that carry information about local and global states, enabling neurons to cooperate and coordinate without requiring direct complex inter-neuron connections, thus maintaining implementation simplicity while improving reliability.
4Loss of information
If DNNs process all incoming information, then processing completeness is maintained, but energy efficiency decreases
Solution Approach 1:
The patent applies local contextual fields that provide neighborhood-specific information to each neuron, enabling differentiated processing based on local patterns. Neurons can identify and preserve locally relevant information while filtering out redundant or irrelevant data from neighboring regions, maintaining processing completeness for meaningful information while reducing energy consumption on redundant processing.
Solution Approach 2:
The patent dynamically changes processing parameters based on contextual field values, adjusting which information is processed and how intensely. When contextual fields indicate irrelevant patterns, neurons reduce processing intensity or suppress output, thereby maintaining completeness for important information while significantly reducing overall energy consumption through parameter-based selective processing.
Data Source
AI summary
An apparatus (800), computer program (808) and method for performing execution of a computational neural layer comprising interconnected neural processing cells each comprising: a receptive field generator (‘S’, 104) configured to generate a receptive field (St) based on inputs (x1t-xNt) to which synaptic weights (W1x-WNx) are applied; a transfer function (‘A’, 106) configured to generate a field variable (At); and an activation circuit (‘Y’, 108) configured to generate an output (Yt) for controlling an activation level of the neural processing cell, based at least in part on the field variable, wherein the transfer function is dependent on: the receptive field; a local contextual field (Ct) dependent on a plurality of receptive fields (S2t-SNt) of the other ones of the neural processing cells (102B, . . . ) of the computational neural layer; and a universal contextual field (Mt-1) indicative of a cross-cell memory state, based at least in part on previous output values of the neural processing cells.