Hierarchical Neural Network Temporal Pooling Method
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial neural networks, particularly recurrent neural networks, are not designed to form representations with clear semantic meaning from time-dependent data, such as 'ball approaching,' and require additional processing to achieve temporal correlation of firing activity for sequence representation.
Innovation Solution
A computational method for hierarchical artificial neural networks that pools and correlates two or more consecutive feed-forward inputs, using operations like logical OR or concatenation, to form sparse distributed representations that stabilize and predict longer sequences, with the ability to learn correlations between inputs from different time steps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If recurrent neural networks are used to process time-dependent data, then temporal sequences can be handled, but the networks cannot form representations with clear semantic meaning and require additional processing for temporal correlation
Solution Approach 1:
The network is divided into distinct functional regions: lower regions that detect static features and higher regions that specialize in temporal correlation. This segmentation allows each region to perform its specific function efficiently without requiring the entire network to handle all aspects of temporal processing.
Solution Approach 2:
The patent extends the hierarchical structure into the temporal dimension by having higher regions correlate patterns across multiple time steps. This adds a temporal dimension to the otherwise spatial hierarchy, enabling semantic representation of sequences like 'ball approaching' without complicating the base feature detection mechanisms.
2Measurement precision
If static feature detection is performed at lower levels, then simple feature recognition is achieved, but temporal dynamics and motion prediction cannot be captured
Solution Approach 1:
Higher regions act as intermediaries that receive static feature detections from lower regions and add temporal context. These intermediary regions correlate patterns across time steps and pass enriched representations upward, preserving both the precision of static feature detection and the dynamics of temporal sequences.
Solution Approach 2:
The patent replaces the need for complex recurrent connections at every level with a hierarchical structure where temporal processing is substituted by correlation mechanisms in higher regions. This substitution allows static feature detectors to remain simple while temporal dynamics are captured through pattern correlation across time.
3Adaptability or versatility
If hierarchical processing is used to build complex features, then feature composition is achieved, but temporal correlation of firing activity requires additional processing steps
Solution Approach 1:
The patent merges feature composition and temporal correlation into a unified hierarchical process. Higher regions simultaneously perform both functions by correlating pooled inputs from lower regions across multiple time steps, eliminating the need for separate temporal processing steps and improving processing efficiency.
Solution Approach 2:
The hierarchical structure enables continuous temporal correlation as higher regions constantly receive and process pooled inputs from lower regions. This continuous processing allows the network to build complex temporal representations without discrete additional processing steps, maintaining high productivity while achieving accurate temporal correlation.
Data Source
AI summary
A computational method is disclosed for the simulation of a hierarchical artificial neural network (ANN), wherein a single correlator pools, during a single time-step, two or more consecutive feed-forward inputs from previously predicted and now active neurons of one or more lower levels.