Computing device, computing method, and recording medium

Abstract

The neural network of the present disclosure has an input layer (10) into which input information is input, a plurality of blocks (111) for extracting a feature amount of the input information, and an output layer (12) that outputs the extracted feature amount, the plurality of blocks (111) each including a residual block composed of a combination of one or more first convolution layers and a connection, i.e., a skip connection, that spans the one or more first convolution layers, and a connection block having at least a second convolution layer and averaging outputs of the one or more first convolution layers and an output of the skip connection.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a neural network, a method of operation, and a recording medium. BACKGROUND

[0002] In recent years, a DL (Deep Learning) model using a convolutional network (CNN) has been exerting excellent performance on many visual problems such as image classification, object detection, and object recognition.

[0003] On the other hand, it is also known that the performance can be improved by deepening the layers of the DL model. However, in the DL model in which the layers are simply deepened, there is a problem that gradient vanishing or gradient exploding occurs.

[0004] In order to address this problem, a CNN model such as ResNet (Residual Network) incorporating residual learning (for example, refer to Non-Patent Literature 1) has been proposed. In ResNet, the above problem is solved by learning a residual F(x) = H(x) - x obtained by subtracting an input x from an output H(x), and a network structure of a depth of 152 layers is realized. In addition, in the ResNet disclosed in Non-Patent Literature 1, a residual block composed of a combination of a skip connection that bypasses an input to a certain layer and inputs to a layer deep from the layer, and a convolutional layer is stacked to have a network structure. In addition, the skip connection used in ResNet is sometimes referred to as a local residual skip connection.

[0005] PRIOR ART DOCUMENTS

[0006] NON-PATENT LITERATURE

[0007] Non-Patent Literature 1: He, K., Zhang, X., Ren, S. and Sun, J., 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778). SUMMARY

[0008] PROBLEMS TO BE SOLVED BY THE INVENTION

[0009] However, the model (network model) disclosed in Non Patent Literature 1 having the network structure of ResNet is constituted by deep layers, and requires a large number of parameters and computational load (FLOPs). In addition, the resource of a hardware platform such as a mobile terminal of a smartphone or a tablet, a home appliance such as a television, or an industrial device is limited, and the installable computational load and the model size are limited. That is, it is difficult to install the ResNet disclosed in Non Patent Literature 1 to a hardware platform with limited resources such as a mobile terminal and utilize it.

[0010] The present disclosure was completed in view of the above circumstances, and an object thereof is to provide a neural network, an operation method, and a recording medium capable of further reducing a model size and a computational load while maintaining accuracy.

[0011] Means for solving the problem

[0012] To solve the above problem, the neural network of one embodiment of the present disclosure includes: an input layer into which input information is input; a plurality of blocks for extracting a feature amount of the input information; and an output layer that outputs the extracted feature amount, the plurality of blocks each including: a residual block constituted by a combination of one or more first convolution layers and a connection, that is, a skip connection, that spans the one or more first convolution layers; and a connection block having at least a second convolution layer and averaging outputs of the one or more first convolution layers and an output of the skip connection.

[0013] In addition, these general or specific aspects can be implemented by a system, a method, an integrated circuit, a computer program, or a recording medium such as a CD-ROM readable by a computer, and can be implemented by any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0014] Effects of the Invention

[0015] According to the present disclosure, a neural network and the like capable of further reducing a model size and a computational load while maintaining accuracy can be realized. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a block diagram showing an example of the structure of an operation device of an embodiment.

[0017] Figure 2 is a diagram showing an example of the hardware structure of a computer that realizes the function of the operation device of the embodiment by software.

[0018] Figure 3 is a diagram showing the structure of a neural network of an embodiment.

[0019] Figure 4A is a conceptual diagram for explaining Figure 3 the network structure of the blocks shown.

[0020] Figure 4B is a conceptual diagram that conceptually represents the network structure of the ResNet10. Figure 4A is a diagram that conceptually represents the operation by the convolutional layer shown in

[0021] Figure 4C is a conceptual diagram that conceptually represents the network structure of the ResNet34. Figure 4A is a diagram that conceptually represents the detailed structure of the connection block shown in

[0022] Figure 5 is a diagram that conceptually represents the network structure of the block shown in Figure 4A is a diagram that conceptually represents one of the detailed ways of the network structure of the block shown in

[0023] Figure 6 is a diagram that conceptually represents another of the detailed ways of the network structure of the block shown in Figure 4A

[0024] Figure 7 is a diagram that conceptually represents an installation example of the network structure of the equivalent ResNet10.

[0025] Figure 8 is a diagram that conceptually represents an installation example of the neural network of the embodiment.

[0026] Figure 9A is a diagram that conceptually represents an installation example of the network structure of the ResNet34.

[0027] Figure 9B is a diagram that conceptually represents an installation example of the network structure of the equivalent ResNet18.

[0028] Figure 10 is a flowchart that represents an outline of the action of the operation device of the embodiment.

[0029] Figure 11 is a diagram that represents an outline of the database of the embodiment.

[0030] Figure 12 is a diagram that represents the experimental results of the embodiment. DETAILED DESCRIPTION

[0031] The neural network of one embodiment of the present disclosure includes an input layer to which input information is input, a plurality of blocks for extracting a feature amount of the input information, and an output layer that outputs the extracted feature amount, the plurality of blocks each including a residual block composed of one or more first convolutional layers and a connection, i.e., a skip connection, that connects across the one or more first convolutional layers, and a connection block having at least a second convolutional layer and averaging outputs of the one or more first convolutional layers and an output of the skip connection.

[0032] ​According to the configuration, it is possible to realize a neural network in which the model size and the amount of computation can be further reduced while the accuracy can be maintained.

[0033] Here, for example, the connection block can also include a second convolutional layer that inputs the output of the one or more first convolutional layers and the output of the skip connection, a first output layer that inputs the output of the second convolutional layer, a weighting layer that gives a pre-stored weight to the output of the first output layer, and a third convolutional layer that inputs the output of the weighting layer.

[0034] According to the configuration, it is possible to reduce the number of blocks required to maintain the accuracy, and thus reduce the model size and the amount of computation.

[0035] Further, for example, each of the plurality of blocks can output, in addition to the output of the connection block, the output of the skip connection, the connection block including a second convolutional layer that inputs the output of the one or more first convolutional layers and the output of the skip connection, a first output layer that inputs the output of the second convolutional layer, a weighting layer that gives a pre-stored weight to the output of the first output layer, a direct connection that directly connects the first output layer and the weighting layer, and a third convolutional layer that inputs the output of the weighting layer and the output of the direct connection.

[0036] According to the configuration, it is possible to not only reduce the number of blocks required to maintain the accuracy, but also avoid explosion or divergence of the output gradient of the connection block due to the direct connection, and thus further reduce the model size and the amount of computation while maintaining the accuracy.

[0037] Here, for example, the first output layer outputs a value obtained by using a Softmax function on the input of the output of the second convolutional layer.

[0038] Further, the operation method of one embodiment of the present disclosure is an operation method of a plurality of blocks in a neural network including an input layer that inputs input information, the plurality of blocks that extract a feature amount of the input information, and an output layer that outputs the extracted feature amount, in which the operation method includes a step of inputting first information to a residual block that is a residual block constituting the block and is composed of one or more first convolutional layers and a connection, i.e., a skip connection, that spans the one or more first convolutional layers, and a step of inputting a feature amount of the first information extracted by the one or more first convolutional layers and the first information output from the skip connection to a connection block that is a connection block constituting the block and has at least a second convolutional layer, and averaging the feature amount of the first information and the first information.

[0039] The embodiments described below each represent one specific example of the present disclosure. Thus, the numerical values, shapes, materials, components, component configurations, connection modes, and the like shown in the following embodiments are examples, and are not intended to limit the present disclosure. In addition, regarding components in the following embodiments, components that are not described in independent technical solutions that represent one specific example of the present disclosure are described as arbitrary components. The embodiments of the present disclosure are not limited to the current independent technical solutions, and can be expressed by other independent technical solutions.

[0040] (Embodiment)

[0041] Hereinafter, the embodiments will be described with reference to the drawings.

[0042] [1. Computing device 1]

[0043] Figure 1 is a block diagram that represents one example of the structure of the computing device 1 of the present embodiment.

[0044] The computing device 1 is realized by a computer or the like that uses a learned NN (Neural Networks), and is configured as shown in Figure 1 by an input unit 2, a processing unit 3, and an output unit 4. The computing device 1 outputs, as a computation result, a feature quantity extracted from input information when the input information is input. In addition, the input information is, for example, information for a visual task such as an image or a video.

[0045] The input unit 2 acquires input information, and inputs the acquired input information to the processing unit 3. In the present embodiment, the input unit 2 inputs the acquired input information to the processing unit 3.

[0046] The processing unit 3 extracts a feature quantity from input information using the neural network of the present embodiment. In the present embodiment, the processing unit 3 extracts a feature quantity from input information using a learned neural network 3a. The learned neural network 3a is stored in a storage unit (not shown) possessed by the processing unit 3. In addition, the learned neural network 3a can be stored on a cloud, and can be used by the processing unit 3 via a communication network such as the Internet. The structure of the neural network 3a is described later.

[0047] The output unit 4 outputs, as a computation result, a feature quantity extracted in the processing unit 3.

[0048] [1-1. Hardware structure]

[0049] Figure 2 is a diagram that represents one example of the hardware structure of a computer 1000 that realizes the functions of the computing device 1 of the present embodiment by software.

[0050] As shown in Figure 2As shown, the computer 1000 is a computer provided with an input device 1001, an output device 1002, a CPU 1003, a built-in memory 1004, a RAM 1005, a reading device 1007, a transmission-reception device 1008, and a bus 1009. The input device 1001, the output device 1002, the CPU 1003, the built-in memory 1004, the RAM 1005, the reading device 1007, and the transmission-reception device 1008 are connected by the bus 1009.

[0051] The input device 1001 is a device such as an input button, a touch pad, a touch panel display, or the like that becomes a user interface and accepts an operation of a user. In addition, the input device 1001 can be a structure that accepts an operation using a sound, a remote operation using a remote controller, or the like, in addition to accepting a contact operation of a user.

[0052] The built-in memory 1004 is a flash memory or the like. In addition, the built-in memory 1004 can pre-store at least one of a program for realizing a function of the arithmetic device 1 and an application (Application) that utilizes a functional structure of the arithmetic device 1. In addition, the built-in memory 1004 can store a learned neural network 3a.

[0053] The RAM 1005 is a random access memory (Random Access Memory) that is used for storage of data and the like when a program or an application is executed.

[0054] The reading device 1007 is a device that reads information from a recording medium such as a USB (Universal Serial Bus) memory. The reading device 1007 reads a program, a neural network 3a, and an application from a recording medium on which the program, the application are recorded as described above, and stores them in the built-in memory 1004.

[0055] The transmission-reception device 1008 is a communication circuit for performing communication by wireless or wired. The transmission-reception device 1008 can communicate with a server device connected to a network, for example, download the program, the neural network 3a, and the application as described above from the server device, and store them in the built-in memory 1004.

[0056] The CPU 1003 is a central processing unit (Central Processing Unit) that copies a program or an application stored in the built-in memory 1004 to the RAM 1005, and sequentially reads out commands included in the program or the application from the RAM 1005 and executes them.

[0057] [1-2. Neural network 3a]

[0058] Figure 3is a diagram showing the structure of the neural network 3a of the present embodiment.

[0059] As shown in Figure 3 , the neural network 3a of the present embodiment has: an input layer 10 to which input information is input; an intermediate layer 11 composed of a plurality of blocks 111 for extracting a feature amount of the input information; and an output layer 12 that outputs the extracted feature amount. Further, hereinafter, the neural network 3a of the present embodiment is also referred to as LVNet. Hereinafter, a model having a network structure of the block 111 will be described.

[0060] [1-2-1. Block 111]

[0061] Figure 4A is a conceptual diagram for explaining the network structure of the block 111 shown in Figure 3 .

[0062] The blocks 111 are stacked to constitute the neural network 3a. As shown in Figure 4A , the block 111 is composed of a connection block 112, a basic unit layer (fundamental unit) denoted by H i-1 , and a skip connection. Further, the block 111 is referred to as a building block in ResNet.

[0063] The basic unit layer is composed of one or more first convolution layers. In the example shown in Figure 4A , the basic unit layer denoted by H i-1 is composed of a convolution layer 1111 denoted by Conv (i-1) and a convolution layer 1112 denoted by Conv i . The convolution layer 1111 and the convolution layer 1112, when installed, are composed of a layer that performs Batch Normalization (BN in the figure), an activation function (ACT in the figure) such as relu, and a convolution layer (Weight in the figure) as shown in Figure 4A . Further, the order of operations is not limited to the order shown in Figure 4A .

[0064] The skip connection uses the basic unit layer denoted by H i-1 to bypass, and inputs an input denoted by H 0 to the connection block 112.

[0065] The connection block 112 is shown as CB i-1 (connection block) in Figure 4A , and averages the output of the basic unit layer denoted by H i-1 and the output of the skip connection.

[0066] Further, Hi-1 The basic unit layer and the skip connection constitute a residual block.

[0067] Figure 4B is a conceptual diagram illustrating an operation performed by the convolution layer shown in Figure 4A . In Figure 4B , as an example, an operation performed in the Conv i-1 of the basic unit layer shown in (i-1) is schematically illustrated.

[0068] As shown in Figure 4B , the input feature input to the Conv (i-1) is a tensor, which is represented by the following equation:

[0069] [Equation 1]

[0070]

[0071] The dimension of the input feature is represented by the following equation:

[0072] [Equation 2]

[0073] w (i-1) Xw (i-1) Xw (n-1)

[0074] The kernel size of the kernel performing the convolution is represented by the following equation:

[0075] [Equation 3]

[0076] b (i-1) Xb (i-1 )Xb n

[0077] The feature map obtained by the convolution operation using the kernel is represented by the following equation:

[0078] [Equation 4]

[0079]

[0080] This feature map becomes the input feature of the Conv (i) , and the dimension thereof is represented by the following equation:

[0081] [Equation 5]

[0082] w i Xw i Xw n

[0083] Figure 4C is a conceptual diagram illustrating an example of the structure of the connection block 112 shown in Figure 4A .

[0084] like Figure 4C As shown, the connecting block 112 is composed of Conv (i-1) (CB) Flexible maximum function, Weights W n CB and Conv (i) (CB) It consists of, etc.

[0085] [1-2-2. Specific methods of block 111 network structure]

[0086] The following uses Figure 5 ,right Figure 4A The specific method of the network structure of block 111 shown will be explained.

[0087] Figure 5 It means Figure 4A The diagram shows a specific configuration of the network structure of block 111.

[0088] Figure 5 The block 111 shown includes a residual block, a connecting block 112A, and an adder 113. Alternatively, the block 111 may not include the adder 113.

[0089] A residual block consists of a combination of one or more first convolutional layers and skip connections that span one or more first convolutional layers. Figure 5 In the example shown, the residual block is composed of convolutional layers 1111 and 1112, and a combination of skip connections (connections across convolutional layers 1111 and 1112). Furthermore, convolutional layers 1111 and 1112 are configured in... Figure 4A as well as Figure 4C The basic unit layer is described in the text.

[0090] Connector block 112A is a connector block having at least a second convolutional layer, and averages the output of one or more first convolutional layers with the output of skipped connections. Figure 5 In the example shown, the connection block 112A consists of a convolutional layer 1121, an output layer 1122, weights 1123, a weighted layer 1124, an adder 1125, and the convolutional layer 1121. Additionally, in Figure 5 In the example shown, since the adder 1125 does not function, the connecting block 112A may also not have the adder 1125.

[0091] Convolutional layer 1121 is an example of a second convolutional layer, which is input to the outputs of one or more first convolutional layers and the outputs of skip connections. Convolutional layer 1121 convolves the input outputs of one or more first convolutional layers and the outputs of skip connections to produce its output. In this embodiment, convolutional layer 1121, for example, in...Figure 5 The middle is represented by Conv (CB) The inputs are the outputs of convolutional layer 1112 and the skipped connections. Convolutional layer 1121 outputs the feature map obtained by convolving the inputs of convolutional layer 1112 and the skipped connections to output layer 1122. Furthermore, for example, in... Figure 4C In the middle, convolutional layer 1121 behaves as Conv (i-1) (CB) And are stacked and input using H i-1 The output of the basic unit layer and the output as a skip connection are represented by H. 0 The input is represented.

[0092] Output layer 1122 is an example of a first output layer, receiving the output of the second convolutional layer as input. More specifically, output layer 1122 outputs the value obtained by applying a flexible maximum function to the input output of the second convolutional layer. In this embodiment, output layer 1122, for example, uses a flexible maximum function to calculate the probability of the output based on the feature map output from convolutional layer 1112, and outputs it to weighted layer 1124. Output layer 1122, for example, in... Figure 5 This is represented as Softmax, for example in Figure 4C The probability of occurrence is represented by a curve obtained by using the flexible maximum value function.

[0093] Weights 1123 are pre-stored (a set of weights). The weights stored in weights 1123 can be obtained by learning connection block 112A using learning data. Weights 1123, for example, in... Figure 5 The middle is represented by Weights (CB) For example, in Figure 4C The middle is represented by Weights W n CB .

[0094] Weighting layer 1124 assigns pre-stored weights to the output of output layer 1122. In this embodiment, weighting layer 1124 assigns weights by multiplying the weights pre-stored in weights 1123 by the output of output layer 1122. This enables weighting of the outputs distributed across the residual block, thereby improving the nonlinearity of the input and output of the residual block.

[0095] Convolutional layer 1126, an example of a third convolutional layer, is input to the output of weighted layer 1124. In this embodiment, the output of convolutional layer 1126 is the feature map obtained by convolving the output of weighted layer 1124. By convolving the output of weighted layer 1124, convolutional layer 1126 can downsample the output of weighted layer 1124. Furthermore, convolutional layer 1126, for example, in... Figure 4CThe middle is represented by Conv i (CB) .

[0096] Figure 6 It means Figure 4A A diagram illustrating another specific embodiment of the network structure of block 111. (For...) Figure 4A , Figure 4C as well as Figure 5 Identical elements are labeled with the same reference numerals, and detailed descriptions are omitted.

[0097] Figure 6 Block 111B shown is relative to Figure 5 The difference between block 111 shown is that the structure of connecting block 112B and the connection destination of skipped connections are increased. The following explanation focuses on the differences.

[0098] In addition to the output of connection block 112B, block 111B also outputs the output of skip connections. More specifically, in block 111B, the skip connections not only connect to the convolutional layer 1121 of connection block 112, spanning convolutional layers 1111 and 1112, but also connect to the adder 113 at the output destination of connection block 112B, spanning connection block 112B. Thus, the input to the skip connections is output not only to the convolutional layer 1121 of connection block 112, but also to the next block 111B after block 111B. Therefore, block 111B also functions as a residual block, improving the nonlinearity of the input and output of block 111, avoiding gradient loss, and thus further improving accuracy.

[0099] Connector block 112B and Figure 5 Compared to the connection block 112A shown, it also has direct connections. Connection block 112B, like connection block 112A, is a connection block with at least a second convolutional layer, and averages the output of one or more first convolutional layers and the output of skipped connections.

[0100] The direct connection directly connects the first output layer and the weighting layer. In this embodiment, the direct connection directly connects the output of the convolutional layer 1121, the output layer 1122, and the weighting layer 1124, and connects the output of the convolutional layer 1121 to the addition unit 1125.

[0101] The adder 1125 adds the output of the convolutional layer 1121, which is output via a direct connection, to the output of the weighted layer 1124. As a result, the output of the convolutional layer 1121 is forced to be nonlinear by the weighted layer 1124. If the output is directly output to the convolutional layer 1126, gradient explosion or divergence can be avoided even if gradient explosion or divergence occurs due to instability.

[0102] The convolution layer 1126 is an example of a third convolution layer that is input with the output of the weighting layer 1124 and the output of the direct connection. In the present embodiment, the convolution layer 1126 outputs a feature map obtained by convolving the output of the adding section 1125. The convolution layer 1126 can down-sample the output of the adding section 1125 by convolving the output of the adding section 1125.

[0103] [1-3. Example of installation of neural network 3a]

[0104] Next, an example of installation of the neural network 3a will be described.

[0105] As described above, the neural network 3a includes the residual block in each of the stacked blocks 111. Therefore, the neural network 3a can be installed by using the network structure of the ResNet having the residual block. In addition, the neural network 3a includes the connection block 112A or 112B in each of the stacked blocks 111, and thus can maintain the same accuracy as the ResNet even when installed by using the network structure of the ResNet composed of shallower layers.

[0106] Figure 7 is a diagram conceptually showing an example of installation of the network structure of the equivalent ResNet10. As shown in Figure 7 , the equivalent ResNet10 has four Building Blocks shown by Building Block 1 to Building Block 4 stacked. In addition, the Building Blocks are each composed of a combination of two convolution layers and a skip connection. Therefore, the equivalent ResNet10 is the network structure of the ResNet composed of shallower layers disclosed in the above Non-Patent Literature 1, and has a network structure composed of ten layers of convolution layers.

[0107] Figure 8 is a diagram conceptually showing an example of installation of the neural network 3a of the present embodiment. In addition, Figure 8 , the neural network 3a shown in

[0108] More specifically, Figure 8 , the neural network 3a shown in Figure 7 has a network structure in which the connection block 112B described above is additionally provided in each of Building Block 1 to Building Block 4 of the equivalent ResNet10 shown in

[0109] In addition, the example of installation of the neural network 3a of the present embodiment is not limited to Figure 8The network structure having the connection block 112B added as described above can also be used in each of the Building Blocks of ResNet34, or in each of the Building Blocks of the equivalent ResNet18.

[0110] Here, the network structure of ResNet34 and the equivalent ResNet18 will be described.

[0111] Figure 9A is a diagram conceptually showing an example of the network structure of ResNet34. As shown in Figure 9A ResNet34 has stacked four Building Blocks indicated by Building Block 1 to Building Block 4. In addition, the Building Blocks are each composed of a combination of a plurality of convolution layers and a plurality of skip connections. In Building Block 1, the convolution layers are configured to be six layers, in Building Block 2, the convolution layers are configured to be eight layers, in Building Block 3, the convolution layers are configured to be twelve layers, and in Building Block 4, the convolution layers are configured to be six layers. The skip connections are configured to span two layers of the convolution layers. Thus, ResNet34 has a network structure configured using thirty-four layers of the convolution layers.

[0112] In addition, Figure 9B is a diagram conceptually showing an example of the network structure of the equivalent ResNet18. As shown in Figure 9B The equivalent ResNet18 has stacked four Building Blocks indicated by Building Block 1 to Building Block 4. In addition, the Building Blocks are each composed of a combination of four convolution layers and a skip connection spanning two layers of the convolution layers. That is, the equivalent ResNet18 is a network structure of ResNet configured by shallower layers disclosed in the above Non-Patent Literature 1, and has a network structure configured using eighteen layers of the convolution layers.

[0113] [2. Operation of the arithmetic device 1]

[0114] Hereinafter, an example of the operation of the arithmetic device 1 configured as described above will be described.

[0115] Figure 10 is a flowchart showing an outline of the operation of the arithmetic device 1 in the present embodiment. Figure 10 The operation processing performed by one of the plurality of blocks 111 configuring the neural network 3a is shown.

[0116] First, the operation device 1 inputs the first information as input information to the residual block of the block 111 constituting the neural network 3a (S10). More specifically, the operation device 1 inputs the first information to the residual block composed of one or more first convolution layers and a connection, that is, a skip connection, across one or more first convolution layers. Further, in a case where the block 111 conforms to Building Block 1, the first information is the input information. In a case where the block 111 does not conform to Building Block 1, the first information is the output of the first one of Building Blocks including the block 111.

[0117] Next, the operation device 1 inputs the feature amount of the first information extracted by one or more first convolution layers of the residual block and the first information output through the skip connection of the residual block to the connection block, and averages the feature amount of the first information and the first information (S11).

[0118] [3. Effects, etc.]

[0119] According to the present embodiment, by appending the connection block to the residual block known in ResNet or the like, it is possible to average the output of the residual block by performing weighting or the like dispersed in the output of the residual block, and it is possible to improve the nonlinearity of the input and output of the residual block. Therefore, it is possible to reduce the number of blocks required to maintain accuracy, and thus it is possible to realize a neural network that can further reduce the model size and the amount of computation while maintaining accuracy.

[0120] Further, according to the present embodiment, the connection block also has a direct connection, and thus it is possible to improve the nonlinearity of the input and output of the block and avoid loss of gradient. Therefore, it is possible not only to reduce the number of blocks required to maintain accuracy, but also to avoid explosion or divergence of the output gradient of the connection block by the direct connection, and thus it is possible to further reduce the model size and the amount of computation while maintaining accuracy.

[0121] Further, it is possible to further reduce the model size and the amount of computation while maintaining accuracy, and thus it has the effect that it becomes easy to use in a manner in which it is mounted on an integrated circuit and provided in a device main body, that is, so-called edge AI.

[0122] For example, in a case where a broadcast wave of a television or the like is received and the received broadcast wave is processed, a large amount of processing such as image recognition is required. Therefore, in a case where a conventional neural network such as ResNet is used, the conventional neural network is installed on a cloud (so-called cloud AI), and communication via a network such as the Internet is required. Also, in a case where the conventional neural network is installed on the cloud, a delay based on the communication amount or communication of the broadcast occurs. On the other hand, in a case where the neural network of the present embodiment such as LVNet is used, the neural network of the present embodiment can further reduce the model size and the amount of computation while maintaining the accuracy, and thus is suitable for installation in an integrated circuit and can be provided in a device main body of a television or the like. Therefore, the neural network of the present embodiment can reduce the communication amount even in a case where communication via a network such as the Internet is required, and thus can suppress a delay caused by the communication amount or communication.

[0123] (Embodiment)

[0124] The effectiveness of the neural network 3a of the present embodiment used by the computing device 1 was verified, and thus the experimental results thereof will be described as an embodiment.

[0125] In the present embodiment, two pieces of experimental data, Test A and Test B, which were collected, were used to evaluate the performance of the neural network 3a used by the computing device 1.

[0126] <Experimental Data>

[0127] Figure 11 is a diagram showing an outline of the database of the present embodiment.

[0128] In the database of the present embodiment, images collected for four categories of Figure 11 shown in FIG. 1. In addition, categories such as weather, scenery, and conversation are included in the other category.

[0129] The images saved in the database are, for example, generated from more than 1 million high-resolution images collected from various resources such as YouTube (registered trademark), Google (registered trademark), and the like. In addition, the images saved in the database are divided into 96K images as training data (Training) and 7K images as evaluation data (Validation).

[0130] In addition, Test A is composed of an image group having a small variance within a class, and Test B is composed of an image group having a small variance between classes.

[0131] <Experimental Conditions>

[0132] In this experiment, learning of the object neural network was performed for 8 hours using a GPU (Graphics Processing Unit). The learning in this experiment was performed with the following parameters. That is, the learning rate was set to 0.025, and was halved every 10 epochs. In addition, the learning in this experiment was performed in a batch gradient descent method of an optimizer using momentum, which was set to 0.9.

[0133] <Experiment Results>

[0134] Figure 12 is a graph showing the experimental results of the example. More specifically, Figure 12 is a graph showing the experimental results of evaluating the accuracy of the neural network 3a of the present embodiment and the accuracy of the neural networks of the comparative examples using Test A and Test B of the present embodiment. In Figure 12 , the model size, the amount of computation (complexity of the model), the parameters, and the memory access of the neural networks of the present embodiment and the comparative examples are shown together. In addition, the neural networks of the comparative examples are ResNet34 and ResNet18. The ResNet34 corresponds to the ResNet34 shown in Figure 9A , and the ResNet18 corresponds to the equivalent ResNet18 shown in Figure 9B . The neural network 3a of the present embodiment is shown as LR10_CB128 and LR10_CB128_SH. The LR10_CB128_SH corresponds to the LVNet, that is, a model having the network structure shown in Figure 8 . The LR10_CB128 corresponds to a model having the network structure in which the connection block 112B of the LVNet shown in Figure 8 is changed to the connection block 112A shown in Figure 5 .

[0135] As shown in Figure 12 , it is known that the accuracy of the ResNet34 and the ResNet18 of the neural networks of the comparative examples is the same as that of the LR10_CB128 and the LR10_CB128_SH of the neural network 3a of the present embodiment.

[0136] In addition, as shown in Figure 12 , it is known that the memory size and the amount of computation (complexity of the calculation) of the neural network 3a of the present embodiment are greatly reduced compared to the memory size and the amount of computation (complexity of the calculation) of the neural networks of the comparative examples. Furthermore, it is also known that the LR10_CB128_SH, that is, Figure 8The model having a network structure, that is, the LVNet, exhibits the highest performance compared to all other neural networks. For example, the model size and the operation amount (FLOPs) of the LR10_CB128_SH are 8.2% and 6.3% of the model size and the operation amount of the ResNet18. That is, it is known that the model size of the LR10_CB128_SH is reduced by 91.8% and the operation amount (FLOPs) is reduced by 92.2% compared to the ResNet18.

[0137] As described above, according to the present embodiment, it is possible to realize the neural network 3a that can further reduce the model size and the operation amount while maintaining the accuracy equivalent to the ResNet.

[0138] (Possibilities of other embodiments)

[0139] The above describes the neural network, the operation device, and the operation method according to one embodiment of the present disclosure based on the embodiments, but the present disclosure is not limited to these embodiments. As long as the spirit of the present disclosure is not deviated, various modifications that can be thought of by those skilled in the art are applied to the embodiments, or a configuration element in different embodiments is combined to construct a mode, and the mode is also included in the scope of the present disclosure. For example, the following cases are also included in the present disclosure.

[0140] (1) Specifically, a part or all of the configuration elements that configure the above-described operation device can also be a computer system configured by a microprocessor, a ROM, a RAM, a hard disk unit, a display unit, a keyboard, a mouse, and the like. A computer program is stored in the RAM or the hard disk unit. The microprocessor acts according to the computer program, and thereby each device realizes its function. Here, the computer program is configured by combining a plurality of instruction codes that indicate instructions to a computer in order to realize a prescribed function.

[0141] (2) A part or all of the configuration elements that configure the above-described operation device can also be configured by one system LSI (Large Scale Integration). The system LSI is a super multifunctional LSI in which a plurality of configuration units are integrated on one chip, and specifically, is a computer system including a microprocessor, a ROM, a RAM, and the like. A computer program is stored in the RAM. The microprocessor acts according to the computer program, and thereby the system LSI realizes its function.

[0142] (3) A part or all of the components that constitute the above-described operation device can be constituted by an IC card or a single module that can be attached to and detached from each device. The IC card or the module is a computer system constituted by a microprocessor, a ROM, a RAM, and the like. The IC card or the module can also be provided with the above-described super multifunction LSI. The microprocessor acts in accordance with a computer program, and thereby the IC card or the module realizes its function. The IC card or the module can also have tamper resistance.

[0143] Industrial applicability

[0144] The present disclosure can be utilized for a neural network, an operation method, and a recording medium, and particularly, can be utilized for a neural network, an operation method, and a recording medium that can exert excellent performance on a visual problem even when installed in a hardware platform with limited resources.

[0145] Explanation of reference numerals

[0146] 1 Operation device

[0147] 2 Input section

[0148] 3 Processing section

[0149] 3a Neural network

[0150] 4 Output section

[0151] 10 Input layer

[0152] 11 Intermediate layer

[0153] 12, 1122 Output layer

[0154] 111, 111B Block

[0155] 112, 112A, 112B Connection block

[0156] 113, 1125 Addition section

[0157] 1111, 1112, 1121, 1126 Convolution layer

[0158] 1123 Weight

[0159] 1124 Weighted layer

Claims

1. A computing device, wherein, Including a neural network, the neural network having: The input layer receives input information, which may be image information or video information. Multiple blocks are used to extract feature quantities from the input information; as well as The output layer outputs the extracted feature values. The plurality of blocks respectively include: A residual block is composed of a combination of one or more first convolutional layers and skip connections that span the one or more first convolutional layers; and The connection block averages the outputs of the one or more first convolutional layers and the outputs of the skipped connections. The connecting block has: The second convolutional layer is fed into the output of the more than one first convolutional layer and the output of the skip connection; The first output layer is fed into the output of the second convolutional layer; A weighted layer assigns pre-stored weights to the output of the first output layer; and The third convolutional layer is fed into the output of the weighted layer. in, Each of the plurality of blocks, in addition to the output of the connection block, also outputs the skipped connection output. The connection block also includes a direct connection that directly connects the first output layer and the weighting layer. In addition to being input to the output of the weighted layer, the third convolutional layer is also input to the output of the direct connection.

2. The computing device according to claim 1, wherein, The first output layer outputs a value obtained by applying a flexible maximum function to the output of the second convolutional layer.

3. A computational method, which is a computational method for multiple blocks in a neural network, the neural network comprising: an input layer, into which input information is input, the input information being image information or video information; and multiple blocks, used to extract feature quantities from the input information; And the output layer, which outputs the extracted features, where, The calculation method includes: The step of inputting first information to the residual block, wherein the residual block is a residual block constituting the block, and is composed of a combination of one or more first convolutional layers and connections spanning the one or more first convolutional layers, i.e., skip connections; and The steps of inputting the feature values ​​of the first information extracted by the one or more first convolutional layers and the first information output by the skipped connections into the connection block, and averaging the feature values ​​of the first information and the first information, are described. The connecting block is a connecting block constituting the block, and has the following features: The second convolutional layer is fed into the output of the more than one first convolutional layer and the output of the skip connection; The first output layer is fed into the output of the second convolutional layer; A weighted layer assigns pre-stored weights to the output of the first output layer; and The third convolutional layer is fed into the output of the weighted layer. in, Each of the plurality of blocks, in addition to the output of the connection block, also outputs the skipped connection output. The connection block also includes a direct connection that directly connects the first output layer and the weighting layer. In addition to being input to the output of the weighted layer, the third convolutional layer is also input to the output of the direct connection.

4. A non-transitory computer-readable recording medium storing a program for executing a method of operating multiple blocks in a neural network, the neural network comprising an input layer receiving input information as image information or video information, the multiple blocks for extracting feature quantities from the input information, and an output layer outputting the extracted feature quantities, wherein... The program causes the computer to execute: The step of inputting first information to the residual block, wherein the residual block is a residual block constituting the block, and is composed of a combination of one or more first convolutional layers and connections spanning the one or more first convolutional layers, i.e., skip connections; and The steps of inputting the feature values ​​of the first information extracted by the one or more first convolutional layers and the first information output by the skipped connections into the connection block, and averaging the feature values ​​of the first information and the first information, are described. The connecting block is a connecting block constituting the block, and has the following features: The second convolutional layer is fed into the output of the more than one first convolutional layer and the output of the skip connection; The first output layer is fed into the output of the second convolutional layer; A weighted layer assigns pre-stored weights to the output of the first output layer; as well as The third convolutional layer is fed into the output of the weighted layer. in, Each of the plurality of blocks, in addition to the output of the connection block, also outputs the skipped connection output. The connection block also includes a direct connection that directly connects the first output layer and the weighting layer. In addition to being input to the output of the weighted layer, the third convolutional layer is also input to the output of the direct connection.

5. A computer program product comprising a computer program for causing a processor to execute the arithmetic method of claim 3.

Images