Hierarchical Temporal Memory Single-Shot Learning Synaptic Adaptation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Hierarchical Temporal Memory (HTM) systems face challenges in learning and recognizing temporal sequences due to the degradation of memory over time, inability to differentiate between similar sequences, and incorrect prediction handling, which leads to the loss of correlation between sequence observation frequency and permanence strength.

Innovation Solution

The system employs sparse distributed representations (SDRs) to encode time-ordered components and generates predictions to add synaptic connections between portions of the HTM, allowing for correct disambiguation of multiple predictions and maintaining accurate sequence recognition by storing instance numbers and synaptic weights without decreasing permanence values.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If HTM systems use traditional learning methods to recognize temporal sequences, then they can learn sequences over multiple exposures, but memory degrades over time and correlation between observation frequency and permanence strength is lost

Engineering Contradiction:

Improvesequence recognition accuracyVSAvoidmemory retention duration

Core Design Contradiction:

Measurement precisionVSDuration of action of stationary object

Solution Approach 1:

The system performs preliminary encoding of the first time-ordered component into sparse distributed representations before processing subsequent components. This pre-encoding preserves the initial state information and allows the system to maintain correlation between observation frequency and permanence strength throughout the learning process, preventing memory degradation over time

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses generated predictions about the third time-ordered component to add additional synaptic connections between portions of the HTM. This feedback mechanism reinforces correct predictions and maintains the correlation between observation frequency and permanence strength, ensuring accurate sequence recognition over extended periods

Inventive Principle:

Principle #23Feedback

2Adaptability or versatility

If HTM systems generate multiple predictions for temporal sequences, then they can handle uncertain predictions, but they cannot differentiate between similar sequences

Engineering Contradiction:

Improveprediction handling capabilityVSAvoidsequence differentiation accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The system divides the temporal sequence processing into distinct portions, with each portion handling specific time-ordered components. The first portion processes the first component, the second portion processes the second component, and the third portion generates predictions for the third component. This segmentation allows multiple predictions to be generated while maintaining the ability to differentiate between similar sequences through localized processing

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different portions of the HTM system are assigned different functional qualities - the first portion is optimized for encoding initial components, the second portion for processing intermediate components with existing synaptic connections, and the third portion for generating predictions. This local quality differentiation enables both versatile prediction handling and precise sequence differentiation simultaneously

Inventive Principle:

Principle #3Local quality

3Measurement precision

If HTM systems add synaptic connections based on predictions, then they can improve sequence recognition, but device complexity increases

Engineering Contradiction:

Improvesequence recognition accuracyVSAvoidsynaptic connection complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system adds synaptic connections selectively based on prediction outcomes rather than comprehensively updating all connections. Additional synapses are added only between portions where predictions were generated and where improvement is needed, rather than uniformly increasing connectivity throughout the entire system. This partial action improves recognition accuracy while limiting complexity growth

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11182673B2Temporal memory adapted for single-shot learning and disambiguation of multiple predictions

Publication Date: 2021.11.23 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Single-shot learning and disambiguation of multiple predictions in hierarchical temporal memory is provided. In various embodiments an input sequence is read. The sequence comprises first, second, and third time-ordered components. Each of the time-ordered components is encoded in a sparse distributed representation. The sparse distributed representation of the first time-ordered component is inputted into a first portion of a hierarchical temporal memory. The sparse distributed representation of the second time-ordered component is inputted into a second portion of the hierarchical temporal memory. The second portion is connected to the first portion by a first plurality of synapses. A plurality of predictions as to the third time-ordered component is generated within a third portion of the hierarchical temporal memory. The third portion is connected to the second portion by a second plurality of synapses. Based on the plurality of predictions, additional synaptic connections are added between the first portion and the second portion.