Hierarchical Neural Network Feedback for Noisy Sequence Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional artificial neural networks lack the architecture to provide feedback in the form of semantic representations from higher levels to lower levels, which hinders their ability to tolerate noise and generalize well.
Innovation Solution
Implementing a hierarchical neural network with regions that include poolers, correlators, and sequence memories, where stable representations formed in upper regions are fed back through apical synapses to modulate the activity of lower regions, using Hebbian learning to strengthen or weaken connections based on predicted firing patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional artificial neural networks are used, then the network architecture is simple, but the ability to tolerate noise and generalize well deteriorates
Solution Approach 1:
The neural network is divided into multiple hierarchical levels with distinct functional regions. Each level processes information at different levels of abstraction, with lower levels handling basic feature detection and higher levels handling complex pattern recognition. This segmentation allows the network to maintain simplicity at each level while achieving robust noise tolerance and generalization through the hierarchical structure as a whole.
Solution Approach 2:
The patent introduces a temporal dimension to the network architecture by incorporating recurrent connections and sequence memory mechanisms. This allows the network to process sequences of inputs over time, maintaining information about past states and using it to improve current processing. The temporal dimension enables the network to tolerate noise better by recognizing patterns across time steps rather than relying solely on single-input accuracy.
2Reliability
If feedback mechanisms are added to provide semantic representations from higher levels to lower levels, then the ability to handle noisy inputs improves, but the device complexity increases
Solution Approach 1:
The patent implements feedback connections that allow higher-level regions to send semantic representations back to lower-level regions. This feedback mechanism enables higher-level context information to modulate lower-level processing, helping the network disambiguate noisy inputs by incorporating contextual knowledge from higher levels. The feedback is selectively applied based on the reliability and clarity of inputs, preventing unnecessary complexity from being introduced in all processing paths.
Solution Approach 2:
The feedback mechanism is applied selectively rather than uniformly across all network connections. Higher-level feedback is provided to specific lower-level regions based on their processing needs and the reliability of current inputs. This localized application of feedback reduces overall system complexity while maintaining the ability to handle noisy inputs in critical processing regions.
3Adaptability or versatility
If hierarchical regions with poolers, correlators, and sequence memory are implemented, then generalization ability improves, but the ease of manufacture deteriorates
Solution Approach 1:
The hierarchical network is segmented into modular functional units (poolers, correlators, sequence memory components) that can be independently designed, trained, and deployed. Each module performs a specific function in the processing hierarchy, making the overall complex system easier to manufacture through modular assembly. The segmentation also enables parallel training of different modules, reducing the practical difficulty of implementation.
Solution Approach 2:
The network structure is pre-configured with hierarchical relationships and connection patterns during the design phase. Pooling operations, correlation functions, and sequence memory structures are established in advance based on the expected data types and processing requirements. This preliminary structuring reduces the complexity of implementation by eliminating the need to optimize these fundamental relationships during deployment.
Data Source
AI summary
Feedback within hierarchical neural networks such as hierarchical temporal memories is provided. In various embodiments, a plurality of input streams is received at a first region of a hierarchical artificial neural network. The plurality of input streams is pooled in the first region. One or more recurrent features are identified in the pooled input streams in the first region. A temporal pattern is recognized in the one or more recurrent features in the first region. An indication of the temporal pattern is provided to at least a second region of the hierarchical neural network. A response to a further input is modulated based on feedback from the second region and at least one additional region.