Condition calculation for continuous learning

Abstract

Aspects provide methods for learning, such as continuous learning, that support task incremental learning using a multi-head classification architecture. Various aspects may enable conditional calculations to support multi-head classification. Aspects provide methods for learning, such as continuous learning, that support class incremental learning using a single-head classification architecture. Various aspects may enable conditional calculations to support single head classification by predicting tasks associated with a given test input and selecting an associated classification head based at least in part on the task prediction.

Description

[0001] Related applications [0002] This application claims the benefit of priority to U.S. Provisional Application No. 62 / 935,147, filed November 14, 2019, entitled "Conditional Computation For Continual Learning," the entire contents of which are owned by all Purpose is incorporated herein by reference. Background technique [0003] Deep neural networks are heavily used in computing devices for a variety of tasks, including scene detection, facial recognition, image classification, and labeling. These networks use a multi-layered architecture, where each layer receives input, performs computation on the input, and generates an output or "activation". The outputs or activations of the nodes in the first layer become the inputs of the nodes in the second layer, the activations of the nodes in the second layer become the inputs of the nodes in the third layer, and so on. Therefore, the computation in the deep neural network is distributed over the entire processing nodes tha...

AI Technical Summary

Problems solved by technology

Since traditional continual learning occurs in a sequential manner, after the first task is learned, the data from this first task is not available for the next task

Images