Hierarchical Temporal Memory Inference via Top-Down Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hierarchical Temporal Memory (HTM) networks face limitations in accurately inferring causes of input patterns due to the lack of feedback information and temporal history integration during the inference stage, leading to reduced accuracy in pattern recognition and prediction tasks.
Innovation Solution
The implementation of enhanced inference nodes within the HTM network that generate and send top-down messages to children nodes, maintaining a history of input patterns and co-occurrences to account for temporal relationships, thereby improving inference accuracy and enabling more precise predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional HTM networks are used for inference, then the system structure remains simple, but inference accuracy is reduced due to lack of feedback information and temporal history integration
Solution Approach 1:
The patent introduces top-down messages that carry feedback information from higher levels of the HTM hierarchy back to lower levels. This feedback mechanism allows the system to integrate temporal history and contextual information during inference, significantly improving inference accuracy by enabling nodes to consider both bottom-up sensory input and top-down contextual expectations
Solution Approach 2:
The patent adds a temporal dimension to the traditional HTM inference process by maintaining and utilizing temporal history of inputs and states. This is achieved through storing historical information in temporal memory structures and incorporating it into the inference process, transforming the system from purely spatial processing to spatio-temporal processing
2Measurement precision
If temporal history and feedback mechanisms are added to improve inference, then inference accuracy improves, but computational resources and processing time increase
Solution Approach 1:
The patent performs preliminary actions by pre-computing and storing temporal statistics and spatial co-occurrence information during the learning phase. This preprocessing allows the inference stage to operate more efficiently by retrieving pre-computed information rather than calculating everything from scratch, reducing real-time computational burden while maintaining high accuracy
3Adaptability or versatility
If top-down messages are generated and sent to children nodes, then the system can discriminate between spatial co-occurrences with different temporal histories, but the complexity of message generation and processing increases
Solution Approach 1:
The patent extracts essential temporal and contextual information from the complex state of the HTM network into condensed top-down messages. These messages carry only the most relevant information needed for discrimination tasks, separating the essential discriminative features from the full network state, thereby improving adaptability while managing processing complexity
Data Source
AI summary
A node, a computer program storage medium, and a method for a hierarchical temporal memory (HTM) network where at least one of its nodes generates a top-down message and sends the top-down message to one or more children nodes in the HTM network. The first top-down message represents information about the state of a node and functions as feedback information from a current node to its child node. The node may also maintain history of the input patterns or co-occurrences so that temporal relationships between input patterns or co-occurrences may be taken into account in an inference stage. By providing the top-town message and maintaining history of previous input patterns, the HTM network may, among others, (i) perform more accurate inference based on temporal history, (ii) make predictions, (iii) discriminate between spatial co-occurrences with different temporal histories, (iv) detect “surprising” temporal patterns, (v) generate examples from a category, and (vi) fill in missing or occluded data.