Training convolutional neural networks on graphics processing units

Abstract

A convolutional neural network is implemented on a graphics processing unit. The network is then trained through a series of forward and backward passes, with convolutional kernels and bias matrices modified on each backward pass according to a gradient of an error function. The implementation takes advantage of parallel processing capabilities of pixel shader units on a GPU, and utilizes a set of start-to-finish formulas to program the computations on the pixel shaders. Input and output to the program is done through textures, and a multi-pass summation process is used when sums are needed across pixel shader unit registers.

Description

Background technique [0001] Neural Networks [0002] Certain computer problems, such as character recognition and image recognition, can be solved well with machine learning techniques, mainly using neural networks. A neural network is a class of algorithms based on the concept of interconnected "neurons". In a typical neural network, neurons have multiple data values, and each data value affects the value of a connected neuron according to a predefined connection strength, and it is judged whether the total connection for each specific neuron reaches the predetermined value. defined threshold. By determining the appropriate connection strength and threshold (a process also known as "training"), the neural network can effectively recognize graphics and characters. Often, these neurons are grouped into multiple "layers" to make the connections between groups more apparent for each computation of a data value. [0003] Figure 1 illustrates a simplified neural network block d...

AI Technical Summary

Problems solved by technology

In addition, the computational complexity of non-fully connected neural networks is even greater

Images