Method and system for reducing deep neural network architectures

Abstract

A method includes deploying a neural network (NN) model on an electronic device. The NN model is generated by training a first NN architecture on a first dataset. A first function defines a first layer of the first NN architecture. The first function is constructed based on approximating a second function applied by a second layer of a second NN architecture. Retraining of the NN model is enabledon the electronic device using a second data set.

Description

technical field [0001] One or more embodiments relate generally to neural network modeling, and in particular to simplifying neural network structures. Background technique [0002] Deep neural networks (NNs) have become ubiquitous in machine learning, with applications ranging from computer vision to speech recognition and natural language processing. A deep NN defines a parameterized function from input to output as a combination of multiple layers of basis functions for both linear / affine transformations and nonlinear functions. The recent success of Convolutional Neural Networks (CNNs) in computer vision applications is due in part to recent progress in scaling these networks to have millions of parameters, multiple convolutional layers, and fully connected layer. As the number of parameters of these networks continues to increase, simplifying their storage and computational processing becomes crucial to meet the requirements of practical applications. While it is pos...

AI Technical Summary

Problems solved by technology

While it is possible to train and deploy these deep CNNs on modern clusters (cloud platforms), their storage, memory bandwidth, and computational requirements make them impractical for embedded mobile applications

Images