Information processing device

Abstract

To provide an information processing device that performs calculation using a neural network and is easily applicable to SNNs and digital brain-type computers.SOLUTION: An information processing device according to an embodiment is the information processing device that includes an input layer, one or more intermediate layers and an output layer and performs calculation using a neural network. At least the input layer and each of the one or more intermediate layers include a quantization layer that finally outputs 0 or 1.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] An embodiment of the present invention relates to an information processing device. [Background technology]

[0002] In recent years, neural networks have been used in various technical fields. For example, convolutional neural networks (CNNs) are widely used in the field of image processing. CNNs can extract many features from input data.

[0003] In addition, by deepening the CNN layers, more complex features can be extracted. Among them, there is a famous CNN model called Residual Neural Networks (ResNet). This ResNet is a model that stacks intermediate layers that combine skip connections and residual blocks. Skip connections basically propagate the features of the input data to the subsequent layers without degradation, but information degradation occurs during downsampling. Residual blocks are set up in parallel with the skip connections to correct the above-mentioned information degradation. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, "Identity Mappings in Deep Residual Networks",arXiv:1603.05027v3 [cs.CV] 25 Jul 2016 Summary of the Invention [Problem to be solved by the invention]

[0005] ResNet has two types of model structures: Post-Activation ResNet and Pre-Activation ResNet. Of these, Pre-Activation ResNet is known to have better performance. However, when a binary signal is used as the input value, the input and output of the intermediate layer are real numbers (decimals), making it difficult to apply the Pre-Activation ResNet model to SNNs (Spiking Neural Networks) that handle information (spike signals) consisting of {0,1}. For the same reason, it is also difficult to apply the model to digital brain-type computers that process information consisting of {0,1}.

[0006] Therefore, one of the problems to be solved by the embodiments of the present invention is to provide an information processing device that performs calculations using a neural network, which can be easily applied to SNNs and digital brain-type computers. [Means for solving the problem]

[0007] An information processing device according to one embodiment of the present invention is an information processing device that performs calculations using a neural network including an input layer, one or more intermediate layers, and an output layer, and at least the input layer and each of the one or more intermediate layers finally have a quantization layer that outputs 0 or 1.

[0008] According to this configuration, by equipping the input layer and hidden layer in the neural network with a quantization layer that outputs 0 or 1, this technology can be easily applied to SNNs and digital brain-type computers.

[0009] In addition, in the information processing device, each of the one or more intermediate layers has a value set for weighting, and when the quantization layer outputs 0, nothing is output to the next layer, when the quantization layer outputs 1, 1 is output to the next layer, and when 1 is input from the previous layer, the weighting value is used in subsequent processing.

[0010] According to this configuration, since it is not necessary to perform multiplication when inputting and outputting data to the intermediate layer, calculation costs and power consumption can be significantly reduced.

[0011] In the information processing device, the quantization layer uses a predetermined step function to output 0 or 1 according to an input numerical value.

[0012] According to this configuration, the quantization layer can be easily realized by using a predetermined step function.

[0013] In the information processing device, the neural network is any one of a spiking neural network, a binary neural network, a quantized neural network, and a convolutional neural network.

[0014] With this configuration, the technologies using the various neural networks described above can be easily applied to SNNs and digital brain-type computers. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram showing a configuration of an image recognition device according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating a functional configuration of the NN circuit according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] An exemplary embodiment of the information processing device of the present invention is disclosed below. The configuration of the embodiment shown below, and the actions, results, and effects brought about by the configuration are merely examples. The present invention can be realized by configurations other than those disclosed in the following embodiments, and it is possible to obtain at least one of various effects and derivative effects based on the basic configuration.

[0017] Fig. 1 is a block diagram showing the configuration of an image recognition device 1 (information processing device) according to an embodiment of the present invention, Fig. 2 is a diagram showing a schematic functional configuration of an NN circuit 3 according to an embodiment of the present invention.

[0018] The image recognition device 1 is an example of an information processing device that performs calculations using a neural network including an input layer, one or more intermediate layers, and an output layer. Specifically, the NN circuit 3 configures a ResNet neural network in an intermediate layer (e.g., intermediate layer 52) including a residual block (e.g., residual block 523) and a skip connection (e.g., skip connection 522) that shortcuts the residual block. In the following, an example of the image recognition device 1 will be described in which image data acquired by a specific imaging device is analyzed and a process for identifying an object (e.g., a vehicle, a person, etc.) captured in the image data is performed.

[0019] The image recognition device 1 includes an input I / F (Interface) 2, a Neural Network (NN) circuit 3, and an output I / F 4.

[0020] The input I / F 2 is a device that inputs image data to be analyzed and outputs it to the NN circuit 3. The output I / F 4 is a device that outputs the recognition result generated by the NN circuit 3 in a predetermined format. There are various ways to use the recognition result, but for example, if it is a recognition result of streaming data acquired by an imaging device such as an event camera mounted on a vehicle, it can be used for vehicle driving support control, danger avoidance control, occupant monitoring control, etc.

[0021] The NN circuit 3 is, for example, an integrated circuit capable of information processing that mimics information transmission in brain neurons, and is capable of analyzing image data using a trained model whose parameters have been adjusted by machine learning (deep learning).

[0022] The specific configuration of the NN circuit 3 is not particularly limited, but for example, it is configured using a circuit in a digital brain-type computer, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The NN circuit 3 has a feature extraction unit 5 and a classification unit 6 as its functional configuration.

[0023] The feature extraction unit 5 extracts features of image data input from the input I / F 2. The feature extraction unit 5 includes an input layer 51, an intermediate layer 52, an intermediate layer 53, and an intermediate layer downstream of the intermediate layer 53 (not shown in FIG. 2).

[0024] The input layer 51 includes a convolution layer 511 and a quantization layer 512. The convolution layer 511 executes a predetermined convolution process on the input image data to generate feature data (feature map) indicating local features of each of a plurality of locations in the image data.

[0025] The quantization layer 512 outputs 0 or 1 according to the information input from the convolution layer 511. For example, the quantization layer 512 outputs 0 or 1 according to the input information using a predetermined step function (e.g., Heaviside step function, etc.).

[0026] If the quantization layer 512 outputs 0, the input layer 51 does not output anything to the next layer, the hidden layer 52, and if the quantization layer 512 outputs 1, the input layer 51 outputs 1 to the hidden layer 52.

[0027] In the intermediate layer 52, values ​​are set for weights, and when a value of 1 is input from the previous layer, the input layer 51, the weight value is used in subsequent processing. The intermediate layer 52 includes a branching unit 521, a skip connection 522, a residual block 523, an adder 528, and a quantization layer 529.

[0028] The residual block 523 includes an activation function 524, a convolutional layer 525, an activation function 526, and a convolutional layer 527.

[0029] The activation function 524 converts the input value into another numerical value and outputs it. The convolution layer 525 executes a predetermined convolution process and outputs the result of the process.

[0030] The activation function 526 converts the input value into another numerical value and outputs it. The convolution layer 527 executes a predetermined convolution process and outputs the result of the process.

[0031] In addition, the residual block 523 may further include, for example, a batch normalization layer (Batch Norm) before the activation function 524.

[0032] The adder 528 adds the input value from the skip connection 522 and the input value from the convolutional layer 527.

[0033] The quantization layer 529 outputs 0 or 1 according to the information input from the addition unit 528. For example, the quantization layer 529 outputs 0 or 1 according to the input information using a predetermined step function (e.g., Heaviside step function, etc.).

[0034] If the quantization layer 529 outputs 0, the hidden layer 52 does not output anything to the next layer, the hidden layer 53, and if the quantization layer 529 outputs 1, the hidden layer 52 outputs 1 to the hidden layer 53.

[0035] The intermediate layers after intermediate layer 53 have the same configuration as intermediate layer 52, and include a quantization layer at the final stage.

[0036] The recognition unit 6 includes a fully connected layer 61 and an output layer 62 . The fully connected layer 61 has a plurality of units defined by an activation function including preset bias values ​​and weight values, and performs a predetermined connection process using information input from the feature extraction unit 5, and outputs the connection information to the output layer 62.

[0037] The output layer 62 uses the connection information input from the fully connected layer 61 to perform processing to identify objects appearing in the image data, and outputs the identification results.

[0038] In this way, according to the image recognition device 1 of this embodiment, the input layer 51 and the intermediate layers (intermediate layers 52, 53, etc.) in the neural network are provided with quantization layers (quantization layers 512, 529, etc.) that output 0 or 1, so that this technology can be easily applied to SNNs and digital brain-type computers.

[0039] In other words, information propagation from the input layer 51 to the hidden layer 52, from the hidden layer 52 to the hidden layer 53, and from the hidden layer 53 to the next hidden layer is realized using spike signals, which are information consisting of {0, 1}. This makes it possible to apply the method to various network structures that include residual blocks, including ResNet.

[0040] In addition, a value (weight value) is set for the weight of each intermediate layer, and when the quantization layer outputs 0, nothing is output to the next layer, when the quantization layer outputs 1, 1 is output to the next layer, and when 1 is input from the previous layer, the weight value is used in the subsequent processing. This eliminates the need for multiplication during input and output to the intermediate layer, so that the calculation cost and power consumption can be significantly reduced. In general, the calculation cost and power consumption are significantly lower when both multiplication and addition are performed than when only addition is performed. Therefore, by implementing the image recognition device 1 using, for example, a digital brain-type computer, which is one of the post-Neumann type computers, it is possible to achieve extremely high power efficiency compared to conventional technologies.

[0041] Furthermore, even if spike signals with little information, such as binary values, are used for information propagation as described above, in the layer receiving the spike signal, if a 1 is input from the previous layer, a weight value is used in subsequent processing, so the accuracy of information processing does not decrease significantly compared to conventional technology.

[0042] In addition, if the quantization layer outputs 0, each intermediate layer can omit processing from the next layer onwards by not outputting anything to the next layer. This reduces computational costs and power consumption.

[0043] Moreover, the quantization layers (quantization layers 512, 529) can be easily realized by using a predetermined step function.

[0044] Also, for example, a predetermined step function (such as a Heaviside step function) may be used as the activation functions 524, 526 in the residual block 523. In this way, the state of neurons can be expressed as a binary state of active or inactive, and no calculations are generated in neurons (nodes) in the inactive state, so the overall amount of calculations can be reduced compared to Post-Activation ResNet (ResNet that processes all neurons as active because there is a convolutional layer at the beginning of the residual block).

[0045] The program for causing a computer to execute processes for realizing the above-mentioned functions may be provided as a computer program product by being stored in a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disk), or flexible disk (FD) in an installable or executable format file. The program may also be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. The program may also be provided or distributed via a network such as the Internet.

[0046] Although the embodiment and the modified examples of the present invention have been described above, the above-mentioned embodiment is presented as an example and is not intended to limit the scope of the invention. This new embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims.

[0047] For example, neural networks to which the present invention can be applied include, but are not limited to, spiking neural networks (including single time-step spiking neural networks), binary neural networks, quantized neural networks, and convolutional neural networks (CNNs). [Explanation of symbols]

[0048] 1...image recognition device, 2...input I / F, 3...NN circuit, 4...output I / F, 5...feature extraction unit, 6...recognition unit, 51...input layer, 52...intermediate layer, 53...intermediate layer, 61...fully connected layer, 62...output layer, 511...convolutional layer, 512...quantization layer, 521...branching unit, 522...skip connection, 523...residual block, 524...activation function, 525...convolutional layer, 526...activation function, 527...convolutional layer, 528...addition unit, 529...quantization layer

Claims

1. An information processing device that performs calculations using a neural network including an input layer, one or more intermediate layers, and an output layer, An information processing device, wherein at least the input layer and each of the one or more intermediate layers finally include a quantization layer that outputs 0 or 1.

2. Each of the one or more intermediate layers has a weight set to a value, If the quantization layer outputs 0, it does not output anything to the next layer; If the quantization layer outputs 1, output 1 to the next layer; The information processing apparatus according to claim 1 , wherein when a weight value of 1 is input from a previous layer, the weight value is used in subsequent processing.

3. The information processing device according to claim 1 , wherein the quantization layer uses a predetermined step function to output 0 or 1 according to an input numerical value.

4. The information processing device according to claim 1 , wherein the neural network is any one of a spiking neural network, a binary neural network, a quantized neural network, and a convolutional neural network (CNN).

Images