Inference processing device, inference processing method, and inference processing program

Abstract

This inference processing device comprises: a division unit that divides a convolutional neural network layer into a plurality of sublayers in the channel direction; a convolution unit that outputs a convolution result by performing convolution processing for each of the sublayers; an addition unit that, each time convolution processing is performed, uses an adder for adding a bias to the convolution result to add, to the convolution result, an intermediate value obtained by cumulatively adding the convolution results up to the preceding sublayer, thereby outputting an addition result; and an activation unit that inputs, to an activation function, an addition result obtained by adding the convolution result for the last sublayer on which convolution processing was performed last.

Description

[Technical Field]

[0001] The disclosed technology relates to an inference processing device, an inference processing method, and an inference processing program that perform convolution processing in a neural network. [Background technology]

[0002] In a CNN (Convolutional Neural Network), the network model is composed of multiple layers, and convolution processing is performed in the convolutional layer. In the convolution processing, the input is an input feature map output from the previous layer, etc., and kernel data, which is a weighting coefficient. In the convolution processing, a bias is added to the product-sum operation of the input feature map and kernel data, and activation function processing is performed to obtain the output feature map as the output.

[0003] When performing CNN inference or learning processing, if the data size of the kernel data of the network model is relatively large, it may not be possible to load all of the kernel data into the memory of a computer or dedicated hardware at once. Therefore, the network model may be divided and processed. Specifically, by dividing the network model, the kernel data is divided so that each divided kernel data can be loaded into memory at once.

[0004] For example, Non-Patent Document 1 discloses a technology in which a feature map of each layer is divided into two in the channel direction and two pieces of hardware are operated in parallel to perform learning. The divided network model and kernel data may be processed in parallel by separate hardware, or may be processed sequentially by the same hardware. For example, when CNN inference processing is performed sequentially by the same hardware, if the network model is divided into n pieces in the input channel direction, and only one piece of kernel data out of the n pieces is stored in memory, and convolution processing is performed sequentially by the same hardware, the size of the kernel data that needs to be stored simultaneously in memory can be reduced to 1 / n compared to when the network model is not divided.

[0005] Here, hardware that executes CNN inference processing often has a multi-stage memory configuration consisting of high-speed, expensive, and low-capacity memory and low-speed, inexpensive, and large-capacity memory. For example, dedicated hardware often includes high-speed, expensive, and low-capacity internal memory such as SRAM (Static Random Access Memory) inside a large-scale integration (LSI) or similar device. It also often includes low-speed, inexpensive, and large-capacity external memory such as DRAM (Dynamic Random Access Memory) outside the LSI or similar device. In this case, the size of the internal memory can be reduced by storing all kernel data in the external memory and appropriately reading only the kernel data required for the current processing, which is 1 / nth the size, from the external memory. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Krizhevsky, Alex; Sutskever, Ilya; Hinton, Geoffrey E. (2017-05-24). “ImageNet classification with deep convolutional neural networks”. Communications of the ACM 60 (6): 84-90. doi:10.1145 / 3065386. ISSN 0001-0782. Summary of the Invention [Problem to be solved by the invention]

[0007] However, when dividing a network model into multiple input channels and sequentially processing the convolutional layers using the same hardware, it is necessary to add an adder circuit to finally integrate the convolution results for the number of divided channels. It is also necessary to apply an activation function to the convolution results obtained by adding all input channels. Furthermore, the convolution results for the number of divided channels, which are stored in external memory, must be read again and then added in the adder circuit, which may increase processing time.

[0008] The disclosed technology has been made in consideration of the above points, and aims to provide an inference processing device, an inference processing method, and an inference processing program that can perform general-purpose convolution processing while suppressing increases in hardware resources and processing time. [Means for solving the problem]

[0009] A first aspect of the present disclosure is an inference processing device including: a division unit that divides a layer of a convolutional neural network into a plurality of sublayers in a channel direction; a convolution unit that performs a convolution process for each of the sublayers and outputs a convolution result; an addition unit that is an adder for adding a bias to the convolution result each time the convolution process is performed, and that outputs an addition result by adding an intermediate value obtained by cumulatively adding the convolution results up to the previous sublayer to the convolution result; and an activation unit that inputs the addition result, obtained by adding the convolution result of the last sublayer on which the convolution process was last performed, to an activation function.

[0010] A second aspect of the present disclosure is an inference processing method, in which a division unit divides a layer of a convolutional neural network into a plurality of sublayers in a channel direction, a convolution unit performs a convolution process for each of the sublayers and outputs a convolution result, an addition unit uses an adder for adding a bias to the convolution result each time the convolution process is performed, and outputs an addition result by adding an intermediate value obtained by accumulating the convolution results up to the previous sublayer to the convolution result, and an activation unit inputs the addition result, obtained by adding the convolution result of the last sublayer on which the convolution process was last performed, to an activation function.

[0011] A third aspect of the present disclosure is an inference processing program for causing a computer to function as each part of the inference processing device of the first aspect. [Effects of the Invention]

[0012] According to the disclosed technology, it is possible to perform convolution processing that can be applied to general purposes while suppressing increases in hardware resources and processing time. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 2 is a schematic diagram illustrating an example of a hardware configuration of an inference processing device according to an embodiment. [Figure 2]FIG. 2 is a block diagram illustrating an example of a functional configuration of an inference processing device according to an embodiment. [Figure 3] FIG. 10 is a schematic diagram showing an example of the flow of convolution processing executed on each sublayer when one layer according to the embodiment is divided into three sublayers. [Figure 4] FIG. 10 is a schematic diagram showing an example of a processing flow for adding convolution results in a bias adder according to the embodiment. [Figure 5] FIG. 10 is a schematic diagram illustrating an example of a function to which an addition result is input according to the embodiment. [Figure 6] FIG. 10 is a schematic diagram illustrating an example of a processing flow in which addition results are added by a bias adder according to the embodiment. [Figure 7] FIG. 10 is a schematic diagram for explaining an example of bit precision according to the embodiment. [Figure 8] FIG. 10 is a schematic diagram illustrating an example of a setting of an undivided layer according to the embodiment. [Figure 9] FIG. 10 is a schematic diagram illustrating an example of the setting of divided layers according to the embodiment. [Figure 10] 10 is a flowchart illustrating an example of an inference process in the inference processing device according to the embodiment. [Figure 11] FIG. 10 is a schematic diagram showing an example of processing executed in the prior art when data for 32 channels can be stored in the internal memory. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of the disclosed technology will be described below with reference to the drawings. Note that the same reference numerals are used to designate identical or equivalent components and parts in each drawing. Also, the dimensional proportions in the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0015] First, the hardware configuration of an inference processing device 10 according to this embodiment will be described. As shown in Fig. 1, the inference processing device 10 has an LSI 20 and an external memory 30. Each component is connected to each other via a bus 19 so that they can communicate with each other.

[0016] The external memory 30 as a storage unit is an external memory of the LSI 20, and is, for example, a DRAM.

[0017] The LSI 20 includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, an internal memory 23, a convolution calculator 24, a bias adder 25, and an activation calculator 26.

[0018] The CPU 21 is a central processing unit that executes various programs and controls each component. That is, the CPU 21 reads programs from the ROM 22 and executes the programs using the internal memory 23 as a work area. The CPU 21 controls the above components and performs various arithmetic processing in accordance with the programs stored in the ROM 22. In this embodiment, an inference processing program is stored in the ROM 22. The inference processing program may be a single program, or may be a group of programs composed of multiple programs or modules.

[0019] The ROM 22 stores various programs and various data. The internal memory 23 temporarily stores programs or data as a working area. For example, an SRAM is used as the internal memory 23.

[0020] The convolution calculator 24 is a calculator that executes convolution processing. The bias adder 25 is an adder that adds a bias to the convolution result. The activation calculator 26 is a calculator that applies an activation function to the input value.

[0021] Next, we will explain the functional configuration of the inference processing device 10. As shown in Fig. 2, the inference processing device 10 includes a division unit 101, a convolution unit 102, an addition unit 103, and an activation unit 104. Each functional configuration is realized by the CPU 21 reading out an inference processing program stored in the ROM 22, expanding it in the internal memory 23, and executing it.

[0022] The dividing unit 101 divides a layer of a convolutional neural network into multiple sublayers in the channel direction. Specifically, the dividing unit 101 divides a layer stored in the external memory 30 into multiple sublayers in the channel direction. The dividing unit 101 divides the layer by dividing the number of input channels that can be stored in the internal memory 23 into units. For example, if the capacity of the internal memory 23 is 4 MBytes, a certain layer has a 3×3 kernel, 2048 input channels, and 1024 output channels, and the precision of the layer is 8 bits, the kernel data is 3*3*2048*1024*8 bits, which is 18 MBytes. Therefore, the dividing unit 101 divides the data into five or more sublayers by 18 / 4. Furthermore, since (4*8 bits*1024*1024) / (3*3*1024*8 bits) is 455, the dividing unit 101 limits the number of input channels per sublayer to 455 or less. Then, the division unit 101 passes one of the divided sub-layers to the convolution unit 102.

[0023] The convolution unit 102 outputs the convolution result by performing a convolution process on each sublayer passed from the division unit 101. Specifically, the convolution unit 102 stores one sublayer read from the external memory 30 in the internal memory 23, and performs a convolution process on the stored sublayer using the convolution calculator 24. Then, the convolution unit 102 passes the convolution result (i.e., a feature map that is an intermediate output output from the sublayer) to the addition unit 103.

[0024] Figure 3 shows the flow of convolution processing performed on each sublayer when one layer is divided into three sublayers. Below, the sublayer on which convolution processing is performed first is called the first sublayer, the sublayer on which convolution processing is performed last is called the last sublayer, and sublayers other than the first and last sublayers are called intermediate sublayers.

[0025] As shown in FIG. 3 , the convolution unit 102 reads out sublayer 0 as the first sublayer from the external memory 30 and stores it in the internal memory 23. Then, the convolution unit 102 performs convolution processing on sublayer 0 using the convolution calculator 24, and passes the result to the bias adder 25. The convolution unit 102 also reads out sublayer 1 as the middle sublayer from the external memory 30 and stores it in the internal memory 23. Then, the convolution unit 102 performs convolution processing on sublayer 1 using the convolution calculator 24, and passes the result to the bias adder 25. The convolution unit 102 also reads out sublayer 2 as the last sublayer from the external memory 30 and stores it in the internal memory 23. Then, the convolution unit 102 performs convolution processing on sublayer 2 using the convolution calculator 24, and passes the result to the bias adder 25. When dividing section 101 divides one layer into five sublayers, the first sublayer will have one sublayer, the middle sublayer will have three sublayers, and the last sublayer will have one sublayer.

[0026] Hereinafter, the sublayer that the convolution unit 102 reads from the external memory 30 will be referred to as the current sublayer. The sublayer that the convolution unit 102 reads from the external memory 30 immediately before the current sublayer will be referred to as the previous sublayer.

[0027] Each time convolution processing is performed by convolution unit 102, adder 103 adds an intermediate value obtained by accumulating and adding the convolution results up to the previous sub-layer using bias adder 25 to the convolution result, and outputs the addition result. Specifically, when the convolution result of the first sub-layer is delivered to adder 103, bias adder 25 adds a bias read from external memory 30 to the convolution result of the first sub-layer. Then, adder 103 delivers the addition result of the convolution result of the first sub-layer and the bias to activation unit 104.

[0028] After receiving the convolution result of the second sublayer, which has been subjected to the second convolution process, the adder 103 adds the convolution result of the current sublayer to the addition results of the previous sublayer stored in the external memory 30. The addition results of the previous sublayer are an intermediate value obtained by accumulating the convolution results of the previous sublayer as addition results stored in the external memory 30 by the activation unit 104 (described later). The adder 103 then passes the addition result of the addition results of the previous sublayer and the convolution result of the current sublayer to the activation unit 104. Specifically, the adder 103 sets the addition results of the previous sublayer by overwriting the location where the bias is originally set in the bias adder 25. This is because a bias only needs to be added once, and therefore, if a bias can be added to the convolution result of the first sublayer, there is no need to add a bias for the second or subsequent sublayers. By performing addition between sublayers using the existing bias adder 25, convolution results can be obtained without adding additional hardware resources.

[0029] 4 shows the flow of processing for adding convolution results in the bias adder 25. As shown in FIG. 4, the adder 103 adds a bias read out from the external memory 30 to the convolution result of the first sublayer in the bias adder 25. Then, the adder 103 adds the addition results of the sublayers up to the immediately preceding sublayer read out from the external memory 30 to the convolution result of the intermediate sublayer in the bias adder 25. The adder 103 also adds the addition results of the sublayers up to the immediately preceding sublayer read out from the external memory 30 to the convolution result of the last sublayer in the bias adder 25.

[0030] When the activation unit 104 receives the addition result obtained by adding the convolution result of the last sub-layer, it inputs the addition result to an activation function (e.g., a relu function) and stores the calculated feature map in the external memory 30. Hereinafter, the feature map output by the activation unit 104 is referred to as an output feature map (ofmap).

[0031] Furthermore, the activation unit 104 does not apply an activation function until the addition result obtained by adding the convolution result of the last sub-layer is passed from the adder 103, and inputs the passed addition result to a linear function (Y=X) whose proportionality constant is 1 and whose intercept is 0. As a result, the activation unit 104 stores the addition result passed from the adder 103 directly in the external memory 30 without actually inputting the addition result to the activation function.

[0032] 5 shows a function that inputs the addition result. As shown in FIG. 5, the activation unit 104 inputs, in the activation calculator 26, a value obtained by adding the convolution result of the first sublayer and the bias to a linear function with a proportionality constant of 1 and an intercept of 0, and stores the addition result in the external memory 30. Then, in the activation calculator 26, the activation unit 104 inputs, in the activation calculator 26, a value obtained by adding the convolution result of the intermediate sublayer and the addition results up to the immediately preceding sublayer to the linear function, and overwrites the addition result in the external memory 30. Then, in the activation calculator 26, the activation unit 104 inputs, in the activation function, a value obtained by adding the convolution result of the last sublayer and the addition results up to the immediately preceding sublayer, and stores the calculated final output feature map in the external memory 30.

[0033] 6 shows the flow of processing in which the bias adder 25 adds the addition results. As shown in FIG. 6, when the activation unit 104 receives the addition result obtained by adding the convolution results of the first sublayer or the addition result obtained by adding the convolution results of the intermediate sublayer, the activation unit 104 stores the addition result as is in the external memory 30. Then, the addition unit 103 adds the addition result read from the external memory 30 to the convolution result of the intermediate sublayer or the convolution result of the last sublayer in the bias adder 25. Then, the activation unit 104 inputs the addition result obtained by adding the convolution result of the last sublayer to the activation function, and stores the final output feature map in the external memory 30.

[0034] Next, the bit precision of each sub-layer will be described.

[0035] In this embodiment, until the convolution result of the last sublayer is input from the convolution unit 102, the adder 103 outputs a result with higher bit precision than the value calculated by the activation unit 104 by inputting an activation function to the result of addition obtained by adding the convolution result of the last sublayer. Specifically, the adder 103 outputs the result with the same bit precision as input from the convolution unit 102. Furthermore, until the addition result of adding the convolution result of the last sublayer is input, the activation unit 104 stores the result with higher bit precision in the external memory 30 than the value calculated by inputting an activation function to the result of addition obtained by adding the convolution result of the last sublayer. Specifically, the activation unit 104 outputs the result with the same bit precision as input from the adder 103. Until the addition result of adding the convolution result of the last sublayer is input, the activation unit 104 stores the result of addition input from the adder 103 in the external memory 30, instead of the actual output feature map. This is because if the bit precision of the output of sublayers other than the last sublayer were made the same as the bit precision of the output of the last sublayer, the calculation precision would be lower than if the layers were not divided. This eliminates the need to change the bit precision at the expense of calculation precision, and makes it possible to reduce the amount of data transferred to the external memory 30.

[0036] For example, if the input feature map is 8 bits and the kernel data is 8 bits, multiplication as is results in 16 bits, so the inference processing device 10 stores the intermediate results of the convolution process as 16 bits or more instead of 8 bits. This is because if the inference processing device 10 were to reduce 16 bits to 8 bits after each convolution and then perform cumulative addition as 8 bits, the calculation accuracy would be significantly degraded. Then, after the cumulative addition is completed (or even later, after adding a bias or inputting to an activation function), the inference processing device 10 reduces the intermediate results of the convolution process to the bit precision of the output feature map. Specifically, in the example described above, the inference processing device 10 reduces the intermediate results of the convolution process from 16 bits to 8 bits.

[0037] Fig. 7 is a schematic diagram illustrating bit precision. As shown in Fig. 7, the adder 103 passes the addition result obtained by adding the convolution results of the first sublayer and the convolution results of the intermediate sublayers from the bias adder 25 to the activation calculator 26 while leaving the bit precision unchanged. Then, the activation unit 104 stores the addition result obtained by adding the convolution results of the first sublayer and the convolution results of the intermediate sublayers from the activation calculator 26 to the external memory 30 while leaving the bit precision unchanged. Then, the adder 103 adds the addition result of the immediately preceding sublayer, which was stored by the activation unit 104 while leaving the bit precision unchanged, to the convolution result of the last sublayer.

[0038] Next, the settings set for each sublayer will be described.

[0039] Figure 8 shows an example of the settings for unsplit layer 0 and layer 1. Figure 9 shows an example of the settings for sublayers 0, 1, and 2 split from layer 0 and layer 1. Sublayer 0 is the first sublayer, sublayer 1 is the middle sublayer, and sublayer 2 is the last sublayer. As shown in Figure 9, each sublayer has a function, bias, etc. set in it, just like an unsplit layer. Therefore, CNN hardware can treat the sublayers as a single layer and process them in a general-purpose manner. This makes it possible to perform general-purpose convolutional processing while suppressing increases in hardware resources and processing time.

[0040] Regarding the number of input channels, if layer 1 before division is set to 3000 input channels as shown in Fig. 8, sublayer 0, sublayer 1, and sublayer 2 are all set to 1000 input channels as shown in Fig. 9. Note that in the example shown in Fig. 9, the input channels are divided equally into three, but this is not limiting. The number of input channels for one sublayer does not have to be divided equally as long as the data size can be stored in internal memory 23.

[0041] Regarding kernel data, when Kernel 1 is set in Layer 1 before division as shown in Figure 8, data corresponding to the first 1 / 3 of the input channels of Kernel 1 is set for Sublayer 0, data corresponding to the middle 1 / 3 of the input channels of Kernel 1 is set for Sublayer 1, and data corresponding to the last 1 / 3 of the input channels of Kernel 1 is set for Sublayer 2 as shown in Figure 9.

[0042] Regarding the bias, when bias 1 is set for layer 1 before division as shown in Fig. 8, bias 1, which is the actual bias, is set for sublayer 0, the sum of the results up to sublayer 0 is set for sublayer 1, and the sum of the results up to sublayer 1 is set for sublayer 2 as shown in Fig. 9. Note that, for actual settings in the inference processing device 10, the address of the corresponding data on the external memory 30 may be designated as the read address.

[0043] As shown in FIG. 8, when Y=f1(x) is set as the activation function for layer 1 before division, a linear function of Y=X is set for sublayer 0 and sublayer 1, and Y=f1(x) is set as the activation function for sublayer 2, as shown in FIG. 9.

[0044] As shown in Fig. 8, when the bit precision of the output feature map is set to b2 for layer 1 before division, b2 is also set for sublayer 2 as shown in Fig. 9. Then, b_tmp is set for sublayer 0 and sublayer 1. In order to suppress deterioration of calculation precision, b_tmp is set to the precision during convolution processing, rather than the precision of the original output feature map.

[0045] Next, the operation of the inference processing device 10 according to this embodiment will be described.

[0046] 10 is a flowchart showing the flow of inference processing by the inference processing device 10. The CPU 21 reads out the inference processing program from the ROM 22, expands it in the internal memory 23, and executes it, thereby performing the inference processing.

[0047] In step S100, the CPU 21 functions as the dividing unit 101 to divide a layer into a plurality of sub-layers in the channel direction.

[0048] In step S101, the CPU 21 functions as the convolution unit 102 to perform convolution processing on one of the passed sub-layers and output the convolution result.

[0049] In step S102, CPU 21, functioning as adder 103, determines whether the output convolution result is the convolution result of the first sub-layer. If the output convolution result is the first sub-layer (step S102: YES), CPU 21 proceeds to step S103. On the other hand, if the output convolution result is not the first sub-layer (step S102: NO), CPU 21 proceeds to step S104.

[0050] In step S103, the CPU 21 functions as the adder 103 and adds a bias to the output convolution result of the first sub-layer.

[0051] In step S104, the CPU 21 functions as the adder 103 and adds the addition result to the output convolution results of the sub-layers other than the first sub-layer.

[0052] In step S105, CPU 21, functioning as activation unit 104, determines whether the output addition result is the addition result obtained by adding the convolution result of the last sub-layer. If the output addition result is the addition result obtained by adding the convolution result of the last sub-layer (step S105: YES), CPU 21 proceeds to step S106. On the other hand, if the output addition result is not the addition result obtained by adding the convolution result of the last sub-layer (step S105: NO), CPU 21 proceeds to step S108.

[0053] In step S106, the CPU 21 functions as the activation unit 104 and inputs the result of adding the convolution result of the last sub-layer to the activation function.

[0054] In step S107, the CPU 21, functioning as the activation unit 104, stores the final output feature map in the external memory 30, and ends this inference process.

[0055] In step S108, the CPU 21 functions as the activation unit 104 and inputs the received addition result to a linear function with a proportional constant of 1 and an intercept of 0.

[0056] In step S109, the CPU 21, functioning as the activation unit 104, stores the addition result in the external memory 30, and the process returns to step S101.

[0057] Next, the effects of the inference processing device 10 according to this embodiment will be described.

[0058] Figure 11 shows the processing performed by the conventional technology when 32 channels of data can be stored in the internal memory. As shown in Figure 11, in the conventional technology, kernel data is divided into 32 channels by input channel, and the data is sequentially read into the internal memory to perform convolution processing. The convolution results are then stored in external memory. In this case, the 32 convolution results stored in the external memory must be read into the hardware and then finally integrated. Therefore, an adder circuit for integrating the convolution results must be added to the hardware. Furthermore, to obtain the final output feature map, an activation function must be applied to the convolution results obtained by adding all input channels. Furthermore, this may increase processing time.

[0059] To address this issue, the inference processing device 10 according to this embodiment divides a layer into multiple sublayers and calculates the final calculation result using existing calculation circuits, thereby enabling general-purpose convolution processing to be performed while suppressing increases in hardware resources and processing time.

[0060] <Modification> The present disclosure is not limited to the above-described embodiments, and various modifications and applications are possible within the scope of the gist of the present disclosure.

[0061] Furthermore, various processes executed by the CPU after reading software (programs) in the above-described embodiments may be executed by various processors other than the CPU. Examples of such processors include programmable logic devices (PLDs) such as field-programmable gate arrays (FPGAs), whose circuit configuration can be changed after fabrication, and dedicated electrical circuits such as application-specific integrated circuits (ASICs), which are processors with circuit configurations specifically designed to execute specific processes. The inference process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0062] In the above embodiment, the inference program is pre-stored (installed) in the ROM 22, but the present invention is not limited to this. The program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.

[0063] The following additional notes are provided regarding the above-described embodiments.

[0064] (Additional note 1) 1. An inference device, comprising: Memory and at least one processor coupled to said memory; Including, The processor: Divide the layers of the convolutional neural network into multiple sublayers in the channel direction, performing a convolution process for each sublayer to output a convolution result; an adder for adding a bias to a convolution result each time the convolution process is executed adds an intermediate value obtained by accumulating and adding the convolution results up to the previous sublayer to the convolution result, and outputs the addition result; Finally, the convolution result of the last sublayer on which the convolution process has been performed is added, and the resulting sum is input to an activation function. An inference device configured to:

[0065] (Additional note 2) A non-transitory storage medium storing a program executable by a computer to perform an inference process, The inference process includes: Divide the layers of the convolutional neural network into multiple sublayers in the channel direction, performing a convolution process for each sublayer to output a convolution result; an adder for adding a bias to a convolution result each time the convolution process is executed adds an intermediate value obtained by accumulating and adding the convolution results up to the previous sublayer to the convolution result, and outputs the addition result; Finally, the convolution result of the last sublayer on which the convolution process has been performed is added, and the resulting sum is input to an activation function. Non-transitory storage medium. [Explanation of symbols]

[0066] 10 Inference Processing Unit 19 Bus 20 LSI 21 CPU 22 ROM 23 Internal Memory 24 Convolution Calculator 25 Bias adder 26 Activation arithmetic unit 30 External Memory 101 Split section 102 Convolution section 103 Addition section 104 Activation part

Claims

1. a division unit that divides a layer of the convolutional neural network into a plurality of sublayers in a channel direction; a convolution unit that performs convolution processing for each sublayer and outputs a convolution result; an adder that adds a bias to a convolution result each time the convolution process is executed, and that adds an intermediate value obtained by accumulating and adding the convolution results up to one sublayer to the convolution result, thereby outputting the addition result; an activation unit that adds the convolution result of the last sublayer on which the convolution process has been finally performed and inputs the addition result to an activation function; An inference processing device comprising:

2. 2. The inference processing device according to claim 1, wherein the activation unit does not apply the activation function until the addition result obtained by adding the convolution result of the last sublayer is input from the adder, and stores the input addition result in a storage unit as is.

3. 3. The inference processing device according to claim 2, wherein the adder adds the bias to the convolution result for a first sublayer on which the convolution process is first performed, and after the convolution result for a second sublayer on which the convolution process is second performed is input from the convolution unit, the adder adds the addition result read out from the storage unit to the convolution result.

4. 3. The inference processing device according to claim 2, wherein the activation unit inputs the input addition result to a linear function having a proportional constant of 1 and an intercept of 0 until the addition result obtained by adding the convolution result of the last sublayer is input from the addition unit.

5. 2. The inference processing device according to claim 1, wherein the adder increases the bit precision of the addition result to be output until the convolution result of the last sublayer is input from the convolution unit, higher than the bit precision of a value calculated by the activation unit by inputting the activation function to the addition result obtained by adding the convolution result of the last sublayer.

6. 3. The inference processing device according to claim 2, wherein the activation unit keeps the bit precision of the addition result stored in the storage unit higher than the bit precision of a value calculated by inputting the activation function to the addition result obtained by adding the convolution result of the last sublayer until the addition result obtained by adding the convolution result of the last sublayer is input.

7. A division unit divides a layer of the convolutional neural network into a plurality of sublayers in a channel direction, a convolution unit that performs a convolution process for each sublayer and outputs a convolution result; an adder that adds a bias to a convolution result each time the convolution process is executed, and adds an intermediate value obtained by accumulating and adding the convolution results up to the previous sublayer to the convolution result, thereby outputting the addition result; an activation unit inputs the sum of the convolution result of the last sublayer on which the convolution process was last performed into an activation function; Inference processing method.

8. An inference processing program for causing a computer to function as each part of the inference processing device according to claim 1.

Images