Information processing device, information processing method and program

Abstract

To provide an integrated circuit capable of effectively using the bandwidth of an external memory in processing a layer algorithm.SOLUTION: An information processing apparatus comprises a first chip including one or more first calculation units, a second chip including one or more second calculation units and different from the first chip, and one or more non-volatile memories connected to the one or more first calculation units. The one or more second calculation units perform a backward process of a neural network to calculate a gradient of the weight of the neural network, and transfer the calculated gradient from the one or more second calculation units to the one or more non-volatile memories. The one or more first calculation units retrieve the calculated gradient from the one or more non-volatile memories and update the weight of the neural network based on the calculated gradient.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus.

Background Art

[0002] Recently, image recognition, speech recognition, etc. have been accurately performed by deep learning using neural networks. For example, a processor that executes deep learning uses image data, weight data, etc. read from a main storage device, which is an external memory, into an internal memory, and causes a plurality of arithmetic units to execute arithmetic operations in parallel. At this time, by converting the arrays of image data, weight data, etc. stored in the internal memory and inputting them to the processor core, the arithmetic operations are efficiently executed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Generally, in deep learning, the number of parameters used in each layer of a neural network is often smaller than the number of data supplied to each layer. Therefore, when inputting and outputting parameters to and from an external memory in the processing of a layer algorithm, there is a timing when the bandwidth of the external memory cannot be fully utilized, and the memory resources may not be fully utilized. Further, when executing parameter update processing in a semiconductor device including a large number of arithmetic units implemented to execute the processing of a layer algorithm, if the usage efficiency of the arithmetic units is low, the arithmetic resources may not be fully utilized.

Means for Solving the Problems

[0005] The information processing apparatus according to an embodiment of the present invention includes a first chip including one or more first arithmetic units, a second chip including one or more second arithmetic units, the second chip being different from the first chip, and one or more non-volatile memories connected to the one or more first arithmetic units. The one or more second arithmetic units execute backward processing of a neural network to calculate a gradient of weights of the neural network, transfer the calculated gradient from the one or more second arithmetic units to the one or more non-volatile memories, and the one or more first arithmetic units acquire the calculated gradient from the one or more non-volatile memories and update the weights of the neural network based on the calculated gradient.

Brief Description of Drawings

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Embodiments for Carrying Out the Invention

[0007] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0008] FIG. 1 is a block diagram showing an example of a semiconductor device in an embodiment of the present invention. The semiconductor device 10 shown in FIG. 1 is, for example, a processor used for training a deep neural network (DNN) including a plurality of layers, and executes forward processing, backward processing, and parameter update processing. The forward processing, backward processing, and parameter update processing will be described with reference to FIGS. 3 and 4.

[0009] The semiconductor device 10 has a first arithmetic unit 30 and a second arithmetic unit 20. The first arithmetic unit 30 has a first arithmetic unit 31, a first memory 32, an instruction decoder 33, a first data transfer control unit 34, and a second data transfer control unit 35. The second arithmetic unit 20 has a second arithmetic unit 21 and a second memory 22. In the present embodiment, the first arithmetic unit 30 is an optimization arithmetic unit that executes parameter update processing, and the second arithmetic unit 20 is a vector arithmetic unit that executes forward processing and backward processing of a deep neural network. The first arithmetic unit 30 is an example of a first integrated circuit, and the second arithmetic unit 20 is an example of a second integrated circuit.

[0010] The second arithmetic unit 21 has a number of arithmetic units for executing vector operations or matrix operations or the like. Note that the second arithmetic unit 20 may have an instruction buffer that holds instructions such as arithmetic instructions and data transfer instructions, an instruction decoder that decodes instructions, registers that hold data and arithmetic results used in the arithmetic operation, and the like. Further, the second arithmetic unit 20 may have an instruction memory, an instruction cache, or a data cache in addition to the second memory 22.

[0011] Examples of the types of various arithmetic units included in the second arithmetic unit 21 include, but are not limited to, a multiply-accumulate unit, a multiplier, an adder, a divider, and the like. Further, the second arithmetic unit 21 of the present embodiment has a floating-point arithmetic unit and a fixed-point arithmetic unit, and the floating-point data may be, for example, any of half-precision, single-precision, and double-precision.

[0012] The second memory 22 has a plurality of storage areas identified by addresses, and holds data, parameters, and the like used by the second arithmetic unit 21 in the arithmetic operation. The second memory 22 is, for example, a high-speed SRAM, but may also be an eDRAM (embedded DRAM), an MRAM (Magnetoresistive Random Access Memory), a PCM (Phase-change memory), or a ReRAM (Resistive Random Access Memory).

[0013] The first arithmetic unit 31 of the first arithmetic section 30 has, for example, an adder, a multiplier, a divider, a logical arithmetic unit (OR logic, AND logic, NOT logic, exclusive OR logic), a square root calculator, and the like. The square root calculator may use a method of obtaining an approximate value by a look-up table. When the first arithmetic unit 31 has an arithmetic unit that calculates other functions such as a logarithmic arithmetic unit, these arithmetic units may also use a method of obtaining an approximate value by a look-up table. Further, instead of the square root calculator, the first arithmetic unit 31 may have a reciprocal square root calculator.

[0014] Furthermore, when causing the first arithmetic unit 31 to execute the parameter update process in the training of the deep neural network, the first arithmetic unit 31 may have a dedicated arithmetic unit that calculates the value of an expression for obtaining the updated value of the parameter or the value of a predetermined term in the expression. For example, when obtaining the updated value Wt of the parameter by ADAM (Adaptive moment estimation) in FIG. 6, a dedicated arithmetic unit that calculates the second term on the right side may be provided in the first arithmetic unit 31. Thereby, the arithmetic efficiency can be improved.

[0015] For example, the various arithmetic units included in the first arithmetic unit 31 are floating point arithmetic units, and the type of floating point numbers to be handled is one or more of half precision, single precision, and double precision. The various arithmetic units included in the first arithmetic unit 31 may be SIMD (Single Instruction Multiple Data) arithmetic units. For example, a double precision SIMD arithmetic unit can execute the arithmetic operations of two single precision data or four half precision data in parallel. Note that the register in which the data to be executed by the SIMD arithmetic unit is stored may have a fixed length or a variable length whose size changes based on the information included in the instruction. The first arithmetic unit 31 may include an arithmetic unit that calculates fixed point data. Further, the first arithmetic unit 31 may have an arithmetic unit that converts the type of floating point data, an arithmetic unit that converts a floating point number to a fixed point number, or an arithmetic unit that converts a fixed point number to a floating point number.

[0016] Note that the first arithmetic unit 31 can implement the calculation of logarithmic functions and exponential functions by combining bit operations, floating-point operations, integer operations, etc. The calculation of logarithmic functions and exponential functions may be executed by combining a plurality of operation instructions, or may be executed by a dedicated arithmetic unit that combines a plurality of arithmetic units.

[0017] Each arithmetic unit included in the first arithmetic unit 31 reads the data held in the first memory 32 based on the control signal from the instruction decoder 33, executes the operation, and stores the operation result in the first memory 32. Hereinafter, each arithmetic unit included in the first arithmetic unit 31 is also referred to as the first arithmetic unit 31. Further, the first arithmetic unit 31 may have an optimal number of arithmetic units for each type of arithmetic unit to execute the parameter update process. Thereby, in the first arithmetic unit 30, the B / F ratio, which is the ratio of the bandwidth of the external memory 40 to the arithmetic performance at the time of executing the parameter update process, can be optimally set.

[0018] The first memory 32 has a plurality of storage areas identified by addresses. The first memory 32 is connected to the second memory 22 of the second arithmetic unit 20 via the first bus IBUS that connects the first arithmetic unit 30 and the second arithmetic unit 20, and is connected to the external memory 40 via the second bus EBUS. In this embodiment, the first bus IBUS and the second bus EBUS are independent of each other. Therefore, the transfer of data from the external memory 40 to the second memory 22 is performed via the first memory 32. However, as will be described later, by providing a bypass switch in the first memory 32 that mutually connects the first bus IBUS and the second bus EBUS, the penalty of access delay due to passing through the first memory 32 can be reduced. The first memory 32 is composed of high-speed SRAM, but may be eDRAM, MRAM, PCM, ReRAM, or the like.

[0019] The instruction decoder 33 decodes, for example, an instruction supplied from the outside of the semiconductor device 10, and outputs a control signal generated according to the decoding result to one or more of the first arithmetic unit 31, the first memory 32, and the first and second data transfer control units 34 and 35. When the instruction decoder 33 decodes an arithmetic instruction for causing the first arithmetic unit 31 to execute an operation, the instruction decoder 33 outputs information indicating the arithmetic unit for executing the operation in the first arithmetic unit 31 and information indicating the type of the arithmetic instruction to the first arithmetic unit 31. Further, based on the decoding of the arithmetic instruction, the instruction decoder 33 outputs read information (such as a read address) for reading data used in the operation from the first memory 32 and write information (such as a write address) for writing the operation result to the first memory 32 to the first memory 32. Hereinafter, the control signal and the like output from the instruction decoder 33 to the first arithmetic unit 31 are also referred to as instructions.

[0020] In the first arithmetic unit 31, the arithmetic unit corresponding to the instruction from the instruction decoder 33 executes an operation on the data loaded from the first memory 32 and stores the operation result in the first memory 32. Based on information such as an address from the instruction decoder 33, the first memory 32 reads data used by the first arithmetic unit 31 in the operation from the storage area and outputs it to the first arithmetic unit 31, and writes the received operation result to the storage area.

[0021] Note that the instruction supplied to the instruction decoder 33 may be an instruction for executing one operation, a SIMD instruction for executing a plurality of operations, or an instruction for specifying the number of operations. The first arithmetic unit 31 that has received an instruction specifying a plurality of times may execute the specified number of operations while sequentially changing the data load source and the operation result store destination in the first memory 32. By including a SIMD instruction or an instruction for specifying the number of operations in the instruction set, the supply efficiency of instructions to the first arithmetic unit 30 can be improved.

[0022] When the instruction decoder 33 decodes a data transfer instruction for transferring data between the second memory 22 and the first memory 32, for example, it outputs a source address, a destination address, transfer amount information, and input / output information to the first data transfer control unit 34. Also, when the instruction decoder 33 decodes a data transfer instruction for transferring data between the external memory 40 and the first memory 32, for example, it outputs a source address, a destination address, transfer amount information, and input / output information to the second data transfer control unit 35. The source address indicates the transfer source of the data, the destination address indicates the transfer destination of the data, the transfer amount information indicates the data transfer amount such as the number of transfer bytes, and the input / output information indicates the input / output direction (read / write) of the data.

[0023] Based on the source address, destination address, transfer amount information, and input / output information received from the instruction decoder 33, the first data transfer control unit 34 accesses the second memory 22 and the first memory 32 via the first bus IBUS, respectively. For example, when the data transfer instruction indicates data transfer from the second memory 22 to the first memory 32, the first data transfer control unit 34 issues read commands to the second memory 22 the number of times corresponding to the data transfer amount. Also, the first data transfer control unit 34 issues write commands to the first memory 32 the number of times corresponding to the data transfer amount. Then, an amount of data corresponding to the transfer amount information is transferred from the second memory 22 to the first memory 32.

[0024] The second data transfer control unit 35 accesses the external memory 40 and the first memory 32 via the second bus EBUS based on the source address, destination address, transfer amount information, and input / output information received from the instruction decoder 33. For example, when the data transfer instruction indicates data transfer from the first memory 32 to the external memory 40, the second data transfer control unit 35 issues read commands to the first memory 32 for the number of times corresponding to the data transfer amount. Also, the second data transfer control unit 35 issues write commands to the external memory 40 for the number of times corresponding to the data transfer amount. Then, data in an amount corresponding to the transfer amount information is transferred from the first memory 32 to the external memory 40.

[0025] Note that the first data transfer control unit 34 may issue a packet-form data transfer request to the second memory 22 and the first memory 32 based on the data transfer instruction from the instruction decoder 33. Similarly, the second data transfer control unit 35 may issue a packet-form data transfer request to the first memory 32 and the external memory 40 based on the data transfer instruction from the instruction decoder 33.

[0026] For example, the external memory 40 is a main memory device such as an SDRAM memory module. The external memory 40 may be a stacked type memory module such as HBM (High Bandwidth Memory) or HMC (Hybrid Memory Cube).

[0027] Note that, for example, the semiconductor device 10 preferably has a form of a single chip. By including the second arithmetic unit 20 and the first arithmetic unit 30 in one chip, the first bus IBUS can be formed as an in-chip wiring. Also, taking advantage of the high-speed access performance of the second memory 22 and the first memory 32, data transfer between the second memory 22 and the first memory 32 can be performed at high speed.

[0028] In contrast, the second arithmetic unit 20 and the first arithmetic unit 30 may be provided on different semiconductor chips. In this case, the semiconductor device including the second arithmetic unit 20 and the semiconductor device including the first arithmetic unit 30 are designed respectively. Thereby, since the chip size of each semiconductor chip can be made smaller than the chip size of a single semiconductor chip, the yield of the semiconductor chip can be improved. In this case, it is preferable to arrange the input / output buffers connected to the respective signal lines of the first bus IBUS, external terminals such as pads and bumps, and protection elements and the like on both chips.

[0029] The first arithmetic unit 30 may have an instruction buffer for holding instructions to be supplied to the instruction decoder 33, a program counter, registers for holding data and operation results used in the operation, and the like. For example, by storing a plurality of instructions in the instruction buffer in parallel, the supply efficiency of instructions to the first arithmetic unit 30 can be improved. The instructions held in the instruction buffer may be transferred from the external memory 40, or may be transferred from the external memory 40 to the instruction buffer based on an instruction from the CPU 210 described in FIG. 2.

[0030] Also, by providing a program counter in the first arithmetic unit 30, a conditional branch instruction can be added to the instruction set, and loop processing can be executed by adding the conditional branch instruction. As a result, the supply efficiency of instructions can be improved. Also, by changing the function to be executed according to the branch destination of the conditional branch instruction, the value of a complex function can be obtained approximately. Also, conditional execution may be performed by referring to the conditional bit.

[0031] Furthermore, the first arithmetic unit 30 may have a data address register. In this case, indirect reference of an address becomes possible, and different processes can be executed by changing the register value without changing the application program. Also, by rewriting the data address register according to the operation result, for example, the reference destination of the lookup table used for approximation of the operation can be changed, and the data can be divided into a plurality of sections, and approximation can be performed by different functions for each section.

[0032] FIG. 2 is a block diagram showing an example of an information processing apparatus on which the semiconductor device 10 of FIG. 1 is mounted. The information processing apparatus 100 shown in FIG. 2 includes a predetermined number of substrates 200, a CPU 210, an external storage device 220, and a network interface (I / F) 230 that are interconnected by a system bus SBUS. For example, the predetermined number of substrates 200 and the CPU 210 are mounted on a system board. Note that the information processing apparatus 100 according to the present disclosure can be in various forms. For example, a main storage device accessed by the CPU 210 may be connected to the system bus SBUS, or a GPU may be connected to the system bus SBUS.

[0033] The system bus SBUS may be, for example, a PCIe (Peripheral Component Interconnect express; registered trademark) bus. Further, the information processing apparatus 100 may function as a server that executes training of a deep neural network by causing a plurality of semiconductor devices 10 to cooperate.

[0034] Each substrate 200 includes a plurality of semiconductor devices 10, a plurality of external memories 40 respectively connected to the plurality of semiconductor devices 10, a communication interface 50, and an instruction control unit 60. The instruction control unit 60 may be provided in each of the plurality of semiconductor devices 10. Also, the number of semiconductor devices 10 mounted on the substrate 200 is not limited to four. The plurality of semiconductor devices 10, the communication interface 50, and the instruction control unit 60 are interconnected via a bus BUS provided on the substrate 200. Note that the substrate 200 may be in the form of a bare substrate connected to a connector provided on the system board, or may be in the form of a package soldered to the system board.

[0035] The CPU 210 manages the overall operation of the information processing apparatus 100 and functions as a host computer that, for example, allocates data used in deep learning to each of a plurality of semiconductor devices. The external storage device 220 may hold, for example, data and parameters used in the deep learning operations executed by each semiconductor device 10. Further, the external storage device 220 may hold a management program and an application program executed by the CPU 210, and may hold instructions executed by each semiconductor device 10. The external storage device 220 may be a hard disk, an SSD (Solid State Drive), or other storage device. The network interface 230 connects the system bus SBUS to an external network. Information such as programs and data held by the external storage device 220 may be transferred from the external network via the network interface 230.

[0036] On each board 200, the communication interface 50 controls communication between the CPU 210, the communication interfaces 50 of other boards 200, or the network interface 230 via the system bus SBUS. The instruction control unit 60 controls instructions issued to each of the semiconductor devices 10 connected by the bus BUS based on an instruction from the CPU 210. That is, the instruction control unit 60 issues various instructions for causing the semiconductor device 10 to execute training to each of the semiconductor devices 10 based on an instruction output from the CPU 210 according to a predetermined deep neural network training procedure. The instructions issued from the instruction control unit 60 are supplied to the instruction decoder 33 in the semiconductor device 10 shown in FIG. 1.

[0037] The instruction control unit 60 may output a timing signal for synchronizing the processing by the second arithmetic unit 20 and the processing by the first arithmetic unit 30 to one or both of the second arithmetic unit 20 and the first arithmetic unit 30. Note that when the CPU 210 can manage instructions issued to each of the semiconductor devices 10, the instructions may be directly issued from the CPU 210 to each semiconductor device 10. In this case, the instruction control unit 60 may not be mounted on the board 200.

[0038] Note that, for example, the plurality of semiconductor devices 10 mounted on the substrate 200 may be mounted on the substrate 200 in the form of a semiconductor module (package). For example, in FIG. 2, four semiconductor devices 10 are included in one semiconductor module. Further, when the first arithmetic unit 30 and the second arithmetic unit 20 are each designed as a semiconductor device (chip), a semiconductor module in which a plurality of semiconductor devices including the first arithmetic unit 30 are mounted may be mounted on the substrate 200. Furthermore, for example, the information processing apparatus 100 may construct a cluster with a plurality of semiconductor devices 10 connected via a network.

[0039] FIG. 3 is a flowchart showing an example of forward processing in the training of a deep neural network. In the forward processing, data and parameters such as weights are input to each of the input layer and a predetermined number of intermediate layers. In the input layer, the input data and parameter 1 are calculated, and intermediate data 1 is generated. In the intermediate layer next to the input layer, the intermediate data 1 and parameter 2 are calculated, and intermediate data 2 is generated. Also in the subsequent intermediate layers, the intermediate data generated by the previous intermediate layer and the parameters set for each intermediate layer are calculated, and the intermediate data generated by the calculation is output to the next intermediate layer. Note that there may be an intermediate layer that does not use parameters. Examples of the intermediate layer include a convolutional layer, a pooling layer, and a fully connected layer.

[0040] In the output layer, output data is obtained using the intermediate data N generated by the intermediate layer N (the Nth layer) immediately preceding the output layer. In the output layer for obtaining an error in a classification problem, for example, the softmax function is used as an activation function, and the cross entropy is used as an error function to obtain the output data (solution). In the output layer, as will be described with reference to FIG. 4, the error from the correct answer is obtained by comparing the output data with the teacher data (correct answer data).

[0041] In this way, in the forward process, in each layer of the neural network, the input data and parameters are calculated to obtain the data to be input to the next layer, and the output data is output from the final layer (forward propagation). Note that the forward process is used not only for training the neural network but also for inference using the neural network.

[0042] Figure 4 is a flowchart showing an example of the backward process and parameter update process in the training of a deep neural network. In the backward process, error backpropagation is performed to propagate the error in the reverse order of the forward process. In Figure 4, the symbol Δ indicates the error of the data or the error of the parameters. The parameter update process is indicated by the dashed arrow.

[0043] First, in the backward process, in the layer where the error is calculated (output layer), the output data generated in the forward process is compared with the teacher data, and Δ intermediate data N, which is the error with respect to the intermediate data N input to the output layer, is generated. Δ intermediate data N is also the error of the output data output by the Nth intermediate layer.

[0044] Next, in each intermediate layer, starting from the intermediate layer closer to the output layer in order, the error with respect to the output data (Δ intermediate data) and the intermediate data, which is the input data, are calculated, and Δ parameter, which is the error with respect to the parameters of the intermediate layer, is generated. Δ parameter indicates the gradient of the parameter in the curve showing the change of the error with respect to the change of the parameter. For example, in intermediate layer 2, Δ intermediate data 2 and intermediate data 1 are calculated to obtain Δ parameter 2.

[0045] Also, in each intermediate layer, an error (Δintermediate data) with respect to the output data and the parameters of the intermediate layer are calculated, and Δintermediate data, which is the error with respect to the input data of the intermediate layer, is generated. The error (Δintermediate data) with respect to the input data of the intermediate layer is also the error of the output data of the previous intermediate layer (or the input layer). For example, in intermediate layer 2, Δintermediate data 2 and parameter 2 are calculated to obtain Δintermediate data 1.

[0046] Similarly to the intermediate layer, in the input layer, Δintermediate data 1 and the input data may be calculated to obtain Δparameter 1, and Δintermediate data 1 and parameter 1 may be calculated to obtain Δinput data, which is the error with respect to the input data.

[0047] In the parameter update process, in each intermediate layer and the input layer, the parameters are corrected using the Δparameters (error gradients) obtained by the backward process. That is, the parameters are optimized. The optimization of the parameters is performed using a gradient descent method such as Momentum-SGD (Stochastic Gradient Descent) or ADAM.

[0048] In this way, in the backward process, the error of the data input to the output layer (the output data of the intermediate layer immediately preceding the output layer) is calculated from the output data and the teacher data. Then, the process of calculating the error of the input data using the calculated error of the data and the process of calculating the error of the parameters using the error of the input data are performed in order from the output-side layer (error backpropagation). In the parameter update process, the parameters are optimized based on the error of the parameters obtained by the backward process.

[0049] Figure 5 is an explanatory diagram showing an example of a layer algorithm in forward processing and backward processing. Figure 5 shows an example of a fully connected layer which is a basic layer algorithm in a neural network. In the following, an example of image data processing will be described, but the layer algorithm shown in Figure 5 is also applicable when processing data other than image data.

[0050] To simplify the explanation, it is assumed that the batch size processed by one layer algorithm is the same as the image size, and the layer algorithm shown in Figure 5 is implemented for each image. On the other hand, when each of the divided images obtained by dividing the image into a plurality of parts is used as the batch size, the number of nested loops increases compared to Figure 5. Note that a convolutional layer frequently used in image processing can also be considered as a fully connected layer with a large number of elements if the convolutional structure is ignored.

[0051] In forward processing, the symbol N indicates the order of the layers (layer number), and indicates that the layer with a larger layer number is located on the output side. The symbol D N+1 indicates the output data output from the layer, and the symbol D N indicates the input data input to the layer, and the symbol W N indicates the parameter input to the layer. The symbol B indicates the number of elements of the data (the number of pixels to be processed), and the symbol O C indicates the number of output channels (that is, the number of units included in the layer), and the symbol I C indicates the number of input channels (that is, the number of units included in the previous layer). In forward processing, as described in Figure 3, in the input layer and the intermediate layer, the process of calculating the output data D N+1 (intermediate data) is performed for each output channel while changing the input data D N and the parameter W N .

[0052] In backward processing, the symbol ΔD N+1 indicates the error of the data received from the output-side layer, and the symbol ΔDN indicates the error of the data output to the input - side layer. Here, the output - side indicates the side where data is output in the forward process, and the input - side indicates the side where data is input in the forward process. The symbol ΔW N indicates the error of the parameter, and the symbol W N indicates the parameter. Other symbols are the same as those in the forward process.

[0053] In the backward process, as described in FIG. 4, the process of calculating the input error ΔD N (Δ intermediate data or Δ input data) is performed for each output channel while changing the output error ΔD N+1 and the parameter W N . Also, the process of calculating the parameter error ΔW N (Δ parameter) is performed for each output channel while changing the output error ΔD N+1 and the input data D N .

[0054] Here, when training a neural network such as the forward process and the backward process is executed in parallel on a plurality of calculation nodes (for example, a pair of the semiconductor device 10 and the external memory 40), the following three methods can be considered as the method of allocating calculation nodes. (Allocation method 1) Allocate calculation nodes to each image or each divided image obtained by dividing an image. That is, allocate data to calculation nodes element - by - element (batch / image division). (Allocation method 2) Divide data in the channel (unit of each layer) direction and allocate it to calculation nodes (channel division). (Allocation method 3) Allocate data to calculation nodes layer - by - layer (layer division).

[0055] In channel partitioning, since it is necessary to communicate all-to-all the data input and output in layer units, it is likely to become a communication bottleneck. In layer partitioning, since it is necessary to move the data input and output between layers among the computing nodes, it is likely to become a communication bottleneck and the management of the data to be moved becomes complicated. In batch / image partitioning, it is necessary to reduce or broadcast the data regarding the parameters among the computing nodes. However, in most neural network configurations, the size of the parameters used in the operations is smaller than the size of the data such as the input data used in the operations, so it is less likely to become a communication bottleneck. Therefore, in order to suppress the communication volume between the computing nodes, it is often more efficient to train by batch / image partitioning (assignment method 1) that assigns relatively large-sized data to each computing node element by element.

[0056] The bandwidth of the external memory 40 needs to be designed considering relatively large-sized data. For example, in the training of a neural network by batch / image partitioning, since the data input and output between layers are different for each arithmetic unit that executes the operations, when the data is stored in the external memory 40, it is not necessary to widen the bandwidth.

[0057] On the other hand, in the training of a neural network by batch / image partitioning, when only relatively small-sized parameters are input and output to and from the external memory 40, there is a possibility that the bandwidth of the external memory 40 cannot be fully utilized. For example, the same value of the parameter is broadcast from the external memory 40 to each arithmetic unit, and the error of the parameter is reduced and stored in the external memory 40. Broadcasting and reduction are paired communications with each other, and although the input / output direction of the data is reversed, the order of the bandwidth does not change. Note that since the number of operations for the parameter update process for optimizing the parameter is smaller than the process of the layer algorithm such as the convolution operation, if this is performed in the second arithmetic unit 20 equipped with a large number of arithmetic units, the usage efficiency of the arithmetic units may decrease. In the present embodiment, these problems can be solved.

[0058] In addition, in the training of a neural network, communications such as reduction and broadcast are executed for a large number of arithmetic units. The bandwidth of this type of communication may be limited by the width of the communication path in the vicinity of each arithmetic unit, and in this case, there is a risk that the bandwidth of the external memory 40 cannot be fully utilized. For example, when performing an operation in the second arithmetic unit 20, communication for parameter reduction and broadcast is required during the calculation of the layer algorithm, and there may be a case where the bandwidth of the external memory 40 cannot be fully utilized at the timing of calculating the layer algorithm. On the other hand, in the present embodiment, by executing an additional calculation of parameter update processing in the first arithmetic unit 30 in parallel with the calculation of the layer algorithm by the second arithmetic unit 20, the bandwidth of the external memory 40 can be effectively utilized.

[0059] FIG. 6 is an explanatory diagram showing an example of the gradient descent method for optimizing parameters. In FIG. 6, the parameter with the symbol t-1 indicates the parameter before update, and the parameter with the symbol t indicates the parameter after update. The symbol ΔWt indicates the gradient of the error of the parameter immediately before update.

[0060] In Momentum-SGD, two multiplications and two additions and subtractions are performed to calculate a new parameter Wt from the parameter error ΔWt. In ADAM, six multiplications, six additions and subtractions, one square root operation, and one division are performed to calculate a new parameter Wt from the parameter error ΔWt. On the other hand, in the calculation of the layer algorithm shown in FIG. 5, a sum-of-products calculation is executed a number of times proportional to the product of the parameter size and the data size. As described above, the number of operations executed at the time of parameter update is smaller than the number of operations executed by a layer algorithm such as a convolution operation. Therefore, when only the operation of updating the parameter is executed by the arithmetic unit, the B / F ratio, which is the ratio of the memory bandwidth per operation performance, becomes high, and when using a vector arithmetic unit or the like with a large number of arithmetic units, most of the arithmetic units will be idle.

[0061] FIG. 7 is a sequence diagram showing an example of executing backward processing and parameter update processing by the semiconductor device 10 of FIG. 1. That is, FIG. 7 shows an example of a control method of the semiconductor device 10. FIG. 7 shows an example of executing, in parallel, execution by the second arithmetic unit 20 of error backpropagation of a deep neural network and parameter update processing by the first arithmetic unit 30. In FIG. 7, among a plurality of layers of the deep neural network, processing of layer N-1, a part of processing of layer N which is one layer behind layer N-1, and a part of processing of layer N-2 which is one layer before layer N-1 are shown.

[0062] The sign T attached to the parameters W, M, V indicates that it is before update, and the sign T+1 attached to the parameters W, M, V indicates that it is after update. For example, the parameter W is updated together with the parameters M, V by the ADAM method.

[0063] The pentagon indicates data transfer, and the protruding corner indicates the transfer destination. The pentagon does not necessarily indicate that data transfer is performed all at once, and may indicate that data transfer is performed in multiple times. When data transfer is performed in multiple times, the first data transfer control unit 34 or the second data transfer control unit 35 issues a data transfer command for each data transfer. Hereinafter, a data transfer command for transferring data between the external memory 40 and the first memory 32 is referred to as an external data transfer command, and a data transfer command for transferring data between the first memory 32 and the second memory 22 is referred to as an internal data transfer command. When the instruction decoder 33 shown in FIG. 1 decodes an external data transfer command, it outputs a control signal to the second data transfer control unit 35, and when it decodes an internal data transfer command, it outputs a control signal to the first data transfer control unit 34.

[0064] Similar to data transfer, the calculation process of data error ΔD and the calculation process of parameter error ΔW by the second calculation unit 20, and the calculation process of parameter W by the first calculation unit 30 may be performed in multiple times. In these calculation processes, the second calculator 21 and the first calculator 31 may execute calculations on a plurality of data with one instruction (SIMD method). At this time, the SIMD instruction supplied to the instruction decoder 33 may be able to specify the number of data to be calculated (variable-length SIMD method).

[0065] Based on the reception of the internal data transfer instruction, the first calculation unit 30 transfers the parameter W held in the first memory 32 N,T to the second memory 22 of the second calculation unit 20. Therefore, the semiconductor device 10 can transfer the parameter W N,T to the second memory 22 without using the second bus EBUS. The parameter W N,T is the parameter of layer N used during the forward process, and has been transferred from the external memory 40 to the first memory 32 during the update process of the parameter of layer N. As shown in FIG. 5, the second calculation unit 20 uses the data error ΔD N+1 and the transferred parameter W N,T to calculate the data error ΔD N , and stores the calculated data error ΔD N in the second memory 22. Note that the data error ΔD N+1 is calculated during the backward process of layer N+1 and is held in the second memory 22.

[0066] During the calculation of the data error ΔD N of layer N by the second calculation unit 20, the first calculation unit 30 receives an external data transfer instruction, transfers the data D N-1 held in the external memory 40 to the first memory 32, and further transfers it from the first memory 32 to the second memory 22. Since the external memory 40 is not accessed during the calculation of the data error ΔD N , the data D N-1 can be transferred to the first memory 32 by using the free bandwidth of the second bus EBUS.

[0067] Next, the second arithmetic unit 20 uses the data error ΔD calculated in the processing of layer N N and the data D transferred from the external memory 40 N-1 to calculate the parameter error ΔW N-1 and stores the calculated parameter error ΔW N-1 in the second memory 22. The first arithmetic unit 30 sequentially transfers the parameter error ΔW N-1 from the second memory 22 to the first memory 32 based on the reception of the internal data transfer instruction. For example, the instruction control unit 60 in FIG. 2 outputs a timing signal to the second arithmetic unit 20 so that the timing of calculating the parameter error ΔW N-1 by the second arithmetic unit 20 coincides with the timing of the internal data transfer instruction for transferring the parameter error ΔW N-1 to the first memory 32.

[0068] Based on the reception of the external data transfer instruction, the first arithmetic unit 30 transfers the parameters W N-1,T M N-1,T V N-1,T from the external memory 40 to the first memory 32. Also, the first arithmetic unit 31 of the first arithmetic unit 30 performs update processing of the parameter W N-1 based on the arithmetic instruction decoded by the instruction decoder 33. That is, the first arithmetic unit 31 executes the ADAM arithmetic shown in FIG. 6 to calculate new parameters W N-1,T+1 M N-1,T+1 V N-1,T+1 and stores them in the first memory 32. Based on the reception of the external data transfer instruction, the first arithmetic unit 30 transfers the updated parameters W N-1,T+1 M N-1,T+1 V N-1,T+1 held in the first memory 32 to the external memory 40.

[0069] As shown in FIG. 7, the calculation processing of the parameters W N-1,T+1 M N-1,T+1 V N-1,T+1 by the first arithmetic unit 30 is executed in parallel with the calculation of the parameter error ΔW by the second arithmetic unit 20. Note that the parameter error ΔWN-1 is only used for calculating the parameters W of the layer N-1 N-1,T+1、 M N-1,T+1 and V N-1,T+1 and thus does not need to be stored in the external memory 40

[0070] Next, similar to the calculation of the data error ΔD of layer N, the data error ΔD of layer N-1 is calculated using the second calculator 21 of the second calculation unit 20 N The first calculation unit 30 transfers the parameter W used for calculating the data error ΔD from the first memory 32 to the second memory 22 of the second calculation unit 20 based on the reception of the internal data transfer instruction. Since the parameter W has been transferred from the external memory 40 to the first memory 32 during the previous parameter update process, there is no need to transfer it from the external memory 40 again, and the bandwidth of the second bus EBUS can be saved N-1 N-1 N-1,T N-1,T

[0071] The second calculator 21 of the second calculation unit 20 calculates the data error ΔD using the data error ΔD and the parameter W held in the second memory 22, and stores the calculated data error ΔD in the second memory 22. The data error ΔD is calculated during the backward processing of layer N and is held in the second memory 22. During the calculation of the data error ΔD of layer N-1 by the second calculation unit 20, the first calculation unit 30 transfers the data D from the external memory 40 to the second memory 22 via the first memory 32 based on the external data transfer instruction N N-1,T N-1 N-1 N N-1 N-2

[0072] Next, similar to the calculation of the parameter error ΔW of layer N-1, the parameter error ΔW of layer N-2 is calculated using the second calculator 21 of the second calculation unit 20 N-1 N-2 N-1 ​​​​​​​​​​​​​and the data D transferred from the external memory 40 to the second memory 22 N-2 to calculate the parameter error ΔW N-2 and store the calculated parameter error ΔW N-2 in the second memory 22. Based on the reception of the internal data transfer instruction, the first arithmetic unit 30 sequentially transfers the parameter error ΔW N-2 from the second memory 22 to the first memory 32. Based on the reception of the external data transfer instruction, the first arithmetic unit 30 transfers the parameters W N-2,T , M N-2,T , V N-2,T from the external memory 40 to the first memory 32. Then, the first arithmetic unit 31 of the first arithmetic unit 30 updates the parameters by calculating new parameters W N-2,T+1 , M N-2,T+1 , V N-2,T+1 . Based on the reception of the external data transfer instruction, the first arithmetic unit 30 transfers the updated parameters W N-2,T+1 , M N-2,T+1 , V N-2,T+1 held in the first memory 32 to the external memory 40.

[0073] In this embodiment, during the calculation of the parameter error ΔW by the second arithmetic unit 20, the first arithmetic unit 30 can execute the update process of the parameter W. That is, the backward process and the update process of the parameter W can be executed in parallel by different second arithmetic unit 21 and first arithmetic unit 31. Thereby, the time required for the backward process and the parameter update process shown in FIG. 4 can be shortened. Further, since the update process of the parameter W is executed during the calculation of the parameter error ΔW, the parameter error ΔW used only for the update process of the parameter W does not need to be saved in the external memory 40. Therefore, the bandwidth of the second bus EBUS can be designed without considering the transfer of the parameter error ΔW. Since the unnecessary data transfer using the second bus EBUS can be eliminated, the power consumption of the semiconductor device 10 can be reduced.

[0074] Furthermore, as described with reference to FIG. 6, the number of operations executed in the process of updating the parameters used in the neural network is smaller than the number of operations in the forward process and the backward process. Therefore, when the parameter update process is executed by the second arithmetic unit 21 of the second arithmetic unit 20, the B / F ratio increases and the utilization efficiency of the second arithmetic unit 21 decreases. However, by executing the parameter update process by the first arithmetic unit 31 of the first arithmetic unit 30 during the calculation of the parameter error ΔW by the second arithmetic unit 21, it is possible to prevent the utilization efficiency of the second arithmetic unit 21 from decreasing.

[0075] In addition, since the first arithmetic unit 31 can be designed as a number of arithmetic units corresponding to the number of operations in the parameter update process, the utilization efficiency of the first arithmetic unit 31 of the first arithmetic unit 30 can be optimized during the parameter update process. Furthermore, when calculating the data error ΔD by the second arithmetic unit 20, it is not necessary to transfer the parameter W from the external memory 40, so the bandwidth of the second bus EBUS can be designed without considering the amount of the parameter W.

[0076] Although FIG. 7 appears to be executed collectively, actually, as shown in FIG. 8, the parameter update process, the accompanying transfer operations of parameters and the like, and the calculation process of the parameter error ΔW are executed in multiple divisions.

[0077] FIG. 8 is an explanatory diagram showing details of the calculation process of the parameter error ΔW of layer N-1 in FIG. 7 N-1 and the update process of the parameter W N-1 , M N-1 , V N-1 . The calculation process of the parameter error ΔW N-1 and the update process of the parameter W N-1 , M N-1 , V N-1 can be executed in parallel by dividing them into any number. For example, FIG. 8 shows an example of dividing the process into four, or shows four processes out of 100 divisions of the entire process. Hereinafter, an example of dividing the entire process into four and executing them in parallel will be described.

[0078] In process (1), first, the second arithmetic unit 20 uses the data error ΔD N (one - quarter) held in the second memory 22 and the data D N-1 (one - quarter) transferred to the second memory 22 to calculate the parameter error ΔW N-1 (one - quarter). The second arithmetic unit 20 stores the calculated parameter error ΔW N-1 in the second memory 22.

[0079] In process (1), the first quarter of the parameters W N-1,T , M N-1,T , V N-1,T of layer N - 1 is transferred from the external memory 40 to the first memory 32. Also, the first quarter of the parameter error ΔW N-1 calculated by the second arithmetic unit 20 and stored in the second memory 22 is transferred from the second memory 22 to the first memory 32.

[0080] In process (1), the first arithmetic unit 30 uses the parameters W N-1,T , M N-1,T , V N-1,T (one - quarter) transferred to the first memory 32 and the parameter error ΔW N-1 (one - quarter) to calculate the updated parameters W N-1,T+1 , M N-1,T+1 , V N-1,T+1 (one - quarter). The calculated parameters W N-1,T+1 , M N-1,T+1 , V N-1,T+1 are transferred to the external memory 40. Also, the parameter W N-1,T is transferred from the first memory 32 to the second memory 22 of the second arithmetic unit 20.

[0081] In processes (2), (3), and (4) as well, similar to process (1), one - quarter of the data error ΔD N and one - quarter of the data D N-1 are used to calculate one - quarter of the parameter error ΔW N-1 . Also, the parameters W N-1,T , M N-1,T , VN-1,T One-fourth of it and the parameter error ΔW N-1 One-fourth of it and are used to update the parameter W N-1,T+1 M N-1,T+1 V N-1,T+1 One-fourth of is calculated. The calculated parameter W N-1,T+1 M N-1,T+1 V N-1,T+1 is transferred to the external memory 40, and the parameter W N-1,T is transferred from the first memory 32 to the second memory 22 of the second arithmetic unit 20.

[0082] Note that after the calculation of the parameter error ΔW is completed, the second arithmetic unit 20 starts the calculation of the data error ΔD N-1 After the calculation of the parameter error ΔW is completed, the data error ΔD N-1 The calculation of the data error ΔD N-1 The number of divisions of the calculation process of the data error ΔD may be different from the number of divisions of the calculation process of the parameter error ΔW N-1 and may be executed without division or executed collectively without division.

[0083] Figure 9 is a flowchart showing an example of the operations of the second arithmetic unit 20 and the first arithmetic unit 30 that execute the sequence of Figure 7. That is, Figure 9 shows an example of a control method for the semiconductor device 10. For operations similar to those in Figures 4 and 7, detailed descriptions are omitted. The backward process by the second arithmetic unit 20 is executed after the forward process by the second arithmetic unit 20 is completed. The parameter update process by the first arithmetic unit 30 is executed in accordance with the progress of the backward process by the second arithmetic unit 20. In the example shown in Figure 9, the parameter update process is executed by the ADAM method, but it may also be executed by the Momentum-SGD or other methods.

[0084] First, in step S21, the second calculation unit 20 calculates a data error ΔD using the output data calculated by the output layer of the backward process and the teacher data prepared in advance. Next, in step S22, the second calculation unit 20 calculates a parameter error ΔW using the data error ΔD (on the output layer side) and the input data D of the layer. The calculated parameter error ΔW is used for the parameter update process by the first calculation unit 30.

[0085] Next, in step S23, the second calculation unit 20 calculates the data error ΔD on the input side using the data error ΔD on the output side and the parameter W. Next, in step S24, when the process of the input layer is executed, the second calculation unit 20 ends the operation, and when the process other than the input layer is executed, it returns to step S22 to execute the operation for the previous layer (on the input layer side).

[0086] On the other hand, in step S31, the first calculation unit 30 transfers the input data D of the layer from the external memory 40 to the first memory 32, and further transfers it to the second memory 22. Note that the first step S31 is an operation for the layer before the output layer. The input data D is used in step S22. Next, in step S32, the first calculation unit 30 controls the first data transfer control unit 34 to transfer the parameter error ΔW calculated by the second calculation unit 20 from the second memory 22 to the first memory 32.

[0087] In step S33, the first calculation unit 30 controls the second data transfer control unit 35 to transfer the parameters W, M, V before update from the external memory 40 to the first memory 32. Next, in step S34, the first calculation unit 30 calculates the parameters W, M, V after update using the parameters W, M, V before update and the parameter error ΔW. That is, the first calculation unit 30 executes the parameter update process. Next, in step S35, the first calculation unit 30 transfers the parameters W, M, V after update from the first memory 32 to the external memory 40.

[0088] Here, the transfer of the parameter error ΔW from the second memory 22 to the first memory 32 in step S32 and the transfer of the parameters W, M, and V before update from the external memory 40 to the first memory 32 in step S33 are executed independently of each other. That is, the first data transfer control unit 34 and the second data transfer control unit 35 transfer different information from each other. The bandwidth of the external memory 40 is large. Further, in the parameter update process by the first arithmetic unit 30 in step S34, the second bus EBUS and the first bus IBUS are not used. For this reason, some of the operations in steps S32, S33, S34, and S35 can be executed with overlapping timings. However, the data and parameters used for the parameter update process need to be stored in the first memory 32 before the execution of the update process.

[0089] Also, as described with reference to FIG. 8, the operation of step S22 by the second arithmetic unit 20 and the operations of S32, S33, S34, and S35 by the first arithmetic unit 30 may be executed in multiple divisions by separating them into groups of a predetermined number of data and a predetermined number of parameters. Further, the transfer of the data D from the external memory 40 via the second bus EBUS N-1 may be executed in multiple divisions.

[0090] Next, in step S36, the first arithmetic unit 30 transfers the parameter W before update from the first memory 32 to the second memory 22. Next, in step S37, if the first arithmetic unit 30 executes the update process of the parameters of the input layer, the operation ends. Also, if the first arithmetic unit 30 executes the update process of the parameters other than the input layer, it returns to step S31 to execute the operation for the previous layer (input layer side).

[0091] In FIGS. 7 to 9, an example was described in which the first arithmetic unit 30 executes the update process of the parameter W instead of the second arithmetic unit 20, thereby shortening the total processing time of the backward process and the parameter update process shown in FIG. 4 and reducing the bandwidth of the second bus EBUS. However, the process executed by the first arithmetic unit 30 is not limited to the update process of the parameter W. For example, the first arithmetic unit 30 may execute other processes for reducing the calculation load of the second arithmetic unit 20 and reducing the amount of data transferred from the external memory 40 to the second memory 22. Further, the type of arithmetic unit mounted on the first arithmetic unit 31 of the first arithmetic unit 30 can be appropriately changed according to the process executed by the first arithmetic unit 30. Further, the first arithmetic unit 30 may be designed to execute the parameter update process and other processes instead of the second arithmetic unit 20, and the first arithmetic unit 31 may be equipped with an arithmetic unit necessary for executing these processes.

[0092] The semiconductor device 10 that executes the operation shown in FIG. 7 can efficiently execute the backward process and the parameter update process while suppressing the bandwidth of the second bus EBUS. Further, as shown in FIG. 8, by executing the parameter update process in multiple stages, it is possible to prevent a situation where the bandwidth of the external memory 40 becomes surplus at the time of parameter reduction / broadcast. Furthermore, in the semiconductor device 10, since the second arithmetic unit 20 does not execute the parameter update process, the second arithmetic unit 20 and the first bus IBUS can be designed without considering the B / F ratio at the time of executing the parameter update process.

[0093] In the present embodiment, the first arithmetic unit 30 and the second arithmetic unit 20 are each configured to sequentially execute a single instruction sequence, and the layer algorithm and the parameter update process can be executed in parallel. Therefore, the present embodiment includes a configuration for executing the layer algorithm and the parameter update process in parallel.

[0094] As another example of a configuration for executing a layer algorithm and parameter update processing in parallel, a configuration can be considered in which a single type of arithmetic unit capable of executing two or more instruction streams is connected to an external memory via a global cache. In this configuration, since it is necessary to design the arithmetic unit so that it can handle both calculations with significantly different B / F ratios, such as a layer algorithm and parameter update processing, there is a risk of increased redundancy in circuit design. For example, a configuration in which the arithmetic unit is hierarchically structured and has internal memories such as a primary cache and a secondary cache in each layer may be installed because it is not necessary for parameter update processing but has a great effect on performance improvement in the layer algorithm. However, in such a configuration, since the data path between the global cache and the individual arithmetic units becomes long, there is a risk that the overall processing time will increase. On the other hand, in the first arithmetic unit 30 of the present embodiment, since the first arithmetic unit 31 and the first memory 32 can be physically installed in a close location, parameter update processing can be executed while reducing the occurrence of delays.

[0095] Furthermore, as yet another example, a configuration can also be considered in which the first arithmetic unit 30 and the second arithmetic unit 20 exchange data via an external memory. In such a configuration, it is necessary for the second arithmetic unit 20 to write the parameter error ΔW to the external memory, and for the first arithmetic unit 30 to read the value of this parameter error ΔW from the external memory and execute parameter update processing. For this reason, compared with the present embodiment, the frequency of accessing the external memory and the amount of data increase. In such a configuration, it is considered that the overall processing time increases compared with the present embodiment.

[0096] Further, it is also conceivable to hold the parameter error ΔW in the second arithmetic unit 20 without saving it in the external memory 40, calculate an updated value of the parameter using the held parameter error ΔW, and store the calculated updated value in the external memory 40. For example, when optimizing parameters using ADAM, in addition to the parameter W used in the processing of the layer algorithm, parameters M and V are required, and these parameters M and V also need to be held in the second arithmetic unit 20. In this case, there is a possibility that the capacity of the second memory 22 of the second arithmetic unit 20 is insufficient, and it is not efficient to save only the parameters M and V in the external memory 40 in order not to cause a buffer capacity shortage.

[0097] Also, even when it is possible to avoid a shortage of the capacity of the second memory 22 by obtaining an updated value of the parameter W only from the parameter error ΔW without using the parameters M and V, there is a possibility that the convergence to the optimal value of the parameter W will be delayed and the parameter update processing time will become long.

[0098] Furthermore, it is conceivable to reduce the bandwidth of the external memory 40 by providing a cache memory between the second arithmetic unit 20 and the external memory 40. For example, while the second arithmetic unit 20 is transmitting and receiving the parameter W and the parameter error ΔW to and from the cache memory, image data D and the like are communicated between the cache memory and the external memory 40.

[0099] Thereby, even when there is a communication bandwidth bottleneck on the second arithmetic unit 20 side in the communication of the parameter W and the parameter error ΔW, it is possible to create a situation where the cache memory and the external memory 40 are always communicating. However, in this method, even when a high-speed external memory 40 can be used, the bandwidth is deliberately reduced, and for example, in a process with a high B / F ratio such as parameter update processing, the bandwidth of the external memory 40 may become a bottleneck and the processing time may increase.

[0100] In contrast, in this embodiment, the parameter update process by the first arithmetic unit 30 can be executed in parallel with the layer algorithm process by the second arithmetic unit 20. Therefore, a situation can be created in which the first arithmetic unit 30 and the external memory 40 are always communicating, and the bandwidth of the external memory 40 can be effectively utilized.

[0101] Furthermore, in this case, even if the bandwidth between the second arithmetic unit 20 and the cache memory can be secured, the bandwidth between the cache memory and the external memory 40 cannot be secured, and for example, the update process of parameters with a high B / F ratio may become a bottleneck for the entire sequence.

[0102] Note that the B / F ratios of the layer algorithm process and the parameter update process are very different, and the B / F ratios between different layers may also be very different. This is because the balance of the computational amount of the layer algorithm, the input / output data size, and the parameter size changes depending on the image size, the number of channels, etc.

[0103] For example, in ResNet, which is a typical configuration of deep learning for image processing, a total of four poolings are executed. For each pooling, the image size (the area of the XY plane) becomes 1 / 4, and the number of channels becomes twice. In ResNet, since the computational amount of each layer is designed to be generally constant, in the layer closer to the input layer, the communication of image data is likely to become a bottleneck, and in the layer closer to the output layer, the calculation or the communication of weights is likely to become a bottleneck. In particular, when allocating computational nodes by batch / image division, in order to improve the usage efficiency of the arithmetic unit, the image size and the batch size of the image are often increased. That is, in a typical configuration of deep learning for image processing, the B / F ratio of the layer algorithm is generally proportional to the image size input / output by the layer, and tends to increase for the layer closer to the input layer and decrease for the layer closer to the output layer.

[0104] Even between layer algorithms that have a dominant computational load in deep learning calculations like this, there are non-negligible differences in the B / F ratio. For this reason, it is necessary to design the communication bandwidth of the external memory 40 so that sufficient communication bandwidth can be ensured even for layers with a high B / F ratio. Therefore, it is difficult to reduce the bandwidth of the external memory 40 itself.

[0105] Furthermore, by dynamically changing the clock frequency of the external memory 40 according to the B / F ratio of the layer algorithm, it is also possible to create a situation where the external memory 40 and the cache memory are always communicating. However, such a design may complicate the system configuration and hinder the stable operation of the semiconductor device. Also, since the parallel execution of the parameter update process and the layer algorithm process is not performed, it is necessary to execute the parameter update process at a timing different from that of the layer algorithm process.

[0106] On the other hand, in the configuration of this embodiment, it is possible to execute the parameter update process and the layer algorithm calculation in parallel, and furthermore, whether to execute them in parallel can be controlled by software. For this reason, for example, it is possible to implement countermeasures such as not executing in parallel for layer algorithms with a high B / F ratio and executing in parallel for layer algorithms with a low B / F ratio. Therefore, in the configuration of this embodiment, the processing of the deep neural network can be efficiently executed according to different B / F ratios for each layer algorithm.

[0107] Note that although this embodiment has advantages compared to the above example, the present invention may include part or all of the configuration of the above example.

[0108] FIG. 10 is a block diagram showing an example of the first memory 32 in FIG. 1. The first memory 32 has an operation control unit CNTL, an address decoder ADEC, a data input / output unit DIO, and a memory cell array ARY including a plurality of banks BK (four in this example).

[0109] The address decoder ADEC selects one of the banks BK to access based on the bank number received from either the first and second data transfer control units 34, 35 or the instruction decoder 33 in FIG. 1, and selects a memory cell in the selected bank BK based on the address. The operation control unit CNTL controls the operations of the memory cell array ARY and the data input / output unit DIO based on control signals such as the port number, read enable, and write enable received from either the first and second data transfer control units 34, 35 or the instruction decoder 33.

[0110] The data input / output unit DIO connects one of the input / output ports P (P0, P1, P2) to the memory cell array ARY according to the received port number, and reads and writes data to and from the memory cell array ARY. The input / output port P0 is connected to the first arithmetic unit 31, the input / output port P1 is connected to the external memory 40 via the second bus EBUS, and the input / output port P2 is connected to the second memory 22 of the second arithmetic unit 20 via the first bus IBUS. Although not particularly limited, the data input or output by the data input / output unit DIO each time the memory cell array ARY is accessed may be, for example, a power of 2 (n is an integer of 4 or more). When error detection and correction data is added to the data processed by the first arithmetic unit 31, the number of bits of the data may also be a power of 2 + m bits (m is the number of bits of the error detection and correction data).

[0111] The data input / output unit DIO outputs the data read from one of the banks BK based on the read enable to one of the input / output ports P selected according to the port number. The data input / output unit DIO outputs the data received at the input / output port P selected according to the port number based on the write enable to the bank BK selected by the bank number, and writes it to the memory cell in the bank BK.

[0112] Each of the port number, address, and bank number is composed of multiple bits, and the OR gate is provided for each signal line. Read enable and write enable are 1-bit signals indicating valid or invalid by logic level. The read enable becomes the valid level when reading data from the memory cell array ARY, and the write enable becomes the valid level when writing data to the memory cell array ARY.

[0113] The instruction decoder 33 shown in FIG. 1 outputs an address, a bank number, a port number for selecting the input / output port P0, and a read enable to the first memory 32 in order to read data used by the first arithmetic unit 31 from the first memory 32 based on an arithmetic instruction. Further, the instruction decoder 33 outputs an address, a bank number, a port number for selecting the input / output port P0, and a write enable to the first memory 32 in order to write the arithmetic result to the first memory 32 based on the completion of the arithmetic operation in the first arithmetic unit 31.

[0114] The instruction decoder 33 outputs the read start address of the external memory 40, the write start address of the first memory 32, the transfer size, and the input / output information to the second data transfer control unit 35 based on a data transfer instruction for transferring data from the external memory 40 to the first memory 32. The second data transfer control unit 35 outputs a write address, a bank number, a port number for selecting the input / output port P1, and a write enable to the first memory 32 based on the information received from the instruction decoder 33.

[0115] The instruction decoder 33 outputs the read start address of the first memory 32, the write start address of the external memory 40, the transfer size, and the input / output information to the second data transfer control unit 35 based on a data transfer instruction for transferring data from the first memory 32 to the external memory 40. The second data transfer control unit 35 outputs a read address, a bank number, a port number for selecting the input / output port P1, and a read enable to the first memory 32 based on the information received from the instruction decoder 33.

[0116] Based on a data transfer instruction for transferring data from the second memory 22 to the first memory 32, the instruction decoder 33 outputs the read start address of the second memory 22, the write start address of the first memory 32, the transfer size, and the input / output information to the first data transfer control unit 34. Based on the information received from the instruction decoder 33, the first data transfer control unit 34 outputs the write address, the bank number, the port number for selecting the input / output port P2, and the write enable to the first memory 32.

[0117] Based on a data transfer instruction for transferring data from the first memory 32 to the second memory 22, the instruction decoder 33 outputs the read start address of the first memory 32, the write start address of the second memory 22, the transfer size, and the input / output information to the first data transfer control unit 34. Based on the information received from the instruction decoder 33, the first data transfer control unit 34 outputs the write address, the bank number, the port number for selecting the input / output port P2, and the write enable to the first memory 32.

[0118] FIG. 11 is a block diagram showing another example of the first memory 32 in FIG. 1. Detailed descriptions of the same elements as in FIG. 10 are omitted. The first memory 32 shown in FIG. 11 has a single-port type data input / output unit DIO and a switch circuit SW connected to the data input / output unit DIO. Other configurations of the first memory 32 are the same as those in FIG. 10.

[0119] In the first memory 32 shown in FIG. 11, since the data input / output unit DIO is of the single-port type, the operation control unit CNTL does not receive the port number, but receives the read enable and the write enable to control the data transfer direction in the data input / output unit DIO.

[0120] The switch circuit SW connects the data terminal of the data input / output unit DIO to the first arithmetic unit 31, the second bus EBUS, or the first bus IBUS based on a switch control signal received from any one of the first and second data transfer control units 34 and 35 or the instruction decoder 33 in FIG. 1. The switch control signal is generated by the instruction decoder 33 shown in FIG. 1 instead of the port numbers shown in FIG. 10. The operation of the instruction decoder 33 and the operation of the first memory 32 are the same as the operations described in FIG. 10.

[0121] FIG. 12 is a block diagram showing another example of the first memory 32 in FIG. 1. Detailed descriptions of the same elements as in FIGS. 10 and 11 are omitted. The first memory 32 has a plurality of memory cores MCORE (two in this example, MCORE1 and MCORE2) and a switch circuit SW. Each memory core MCORE may have a memory cell array ARY including an operation control unit CNTL (not shown), an address decoder ADEC, a data input / output unit DIO, and a plurality of banks BK, similar to FIG. 10. However, the data input / output unit DIO is of the single-port type, similar to FIG. 11, and each memory core MCORE does not receive a port number, similar to FIG. 11. The data terminals of each memory core MCORE are connected to the switch circuit SW.

[0122] The switch circuit SW receives a switch control signal from any one of the first and second data transfer control units 34 and 35 or the instruction decoder 33 in FIG. 1, similar to the switch circuit SW in FIG. 11. Then, based on the switch control signal, the switch circuit SW connects any one of the data terminals of the memory core MCORE to the first arithmetic unit 31, the second bus EBUS, or the first bus IBUS. The switch control signal is generated by the instruction decoder 33 shown in FIG. 1 and includes the logic of the address for selecting the memory core MCORE.

[0123] For example, the switch circuit SW may be provided with a function of simultaneously connecting the memory core MCORE1 to the external memory 40 and connecting the memory core MCORE2 to the first arithmetic unit 31. In this case, the logic of the switch control signal indicates the combination of connections between the memory cores MCORE1 and MCORE2 and the first arithmetic unit 31, the second bus EBUS, and the first bus IBUS. The instruction decoder 33 decodes instructions (arithmetic instructions and data transfer instructions) corresponding to the number of connection combinations of the switch circuit SW. As a result, a plurality of memory cores MCORE can be accessed simultaneously and independently.

[0124] FIG. 13 is a block diagram showing another example of the first memory 32 in FIG. 1. Detailed descriptions of the same elements as in FIG. 12 are omitted. The first memory 32 shown in FIG. 13 is the same as the first memory 32 shown in FIG. 12, except that the switch circuit SW is different from the switch circuit SW in FIG. 12. The switch circuit SW is the same as the switch circuit SW in FIG. 12, except that it has a path for mutually connecting the second bus EBUS and the first bus IBUS.

[0125] The instruction decoder 33 in FIG. 1 has a function of decoding a direct data transfer instruction for directly transferring data between the external memory 40 and the second memory 22 of the second arithmetic unit 20. In other words, the semiconductor device 10 has an instruction set including a direct data transfer instruction.

[0126] When the instruction decoder 33 decodes a direct data transfer instruction, it outputs a switch control signal for connecting the second bus EBUS and the first bus IBUS to the first memory 32. Further, the instruction decoder 33 outputs information for accessing the external memory 40 to the second data transfer control unit 35, and outputs information for accessing the second memory 22 to the first data transfer control unit 34.

[0127] The second data transfer control unit 35 issues a read command or a write command to the external memory 40 based on the information from the instruction decoder 33. The first data transfer control unit 34 issues a read command or a write command to the second memory 22 based on the information from the instruction decoder 33. Thereby, data can be transferred between the external memory 40 and the second memory 22 of the second arithmetic unit 20 without writing data to the memory core MCORE of the first memory 32. For example, the data D N-1 , D N-2 can be transferred from the external memory 40 to the second memory 22 without writing to the memory core MCORE.

[0128] FIG. 14 is a block diagram showing another example of the first memory 32 in FIG. 1. Detailed descriptions of the same elements as in FIGS. 11 and 12 are omitted. In the first memory 32 shown in FIG. 14, switch circuits SW (SW1, SW2) are provided for each memory core MCORE, and each switch circuit SW operates in response to different switch control signals received from the instruction decoder 33. The switch circuit SW is the same as the switch circuit SW shown in FIG. 11 and connects a data input / output unit DIO (not shown) of the memory core MCORE to the first arithmetic unit 31, the second bus EBUS, or the first bus IBUS.

[0129] The switch control signal includes the logic of the address for selecting the memory core MCORE. Thereby, the first memory 32 can operate as two independent memories. That is, the memory core MCORE1 can be operated without being affected by the operation of the memory core MCORE2, and the memory core MCORE2 can be operated without being affected by the operation of the memory core MCORE1.

[0130] As described above, in this embodiment, the forward processing and backward processing in the neural network are executed by the second arithmetic unit 20, and the parameter update processing is executed by the first arithmetic unit 30 in the background of the backward processing. Also, the first arithmetic unit 30 is equipped with arithmetic units having specifications and numbers suitable for the parameter update processing, and the parameter W used in the backward processing is transferred from the first memory 32 instead of the external memory 40. As a result, without reducing the usage efficiency of the arithmetic units of both the second arithmetic unit 20 and the first arithmetic unit 30, it is possible to efficiently execute the backward processing and the parameter update processing in accordance with the bandwidth of the external memory 40.

[0131] Also, in order to execute the backward processing and the parameter update processing in parallel, the time from the start of the backward processing to the update of the parameters can be shortened. Further, by performing the calculation process of the parameter error ΔW of the backward processing and the parameter update processing in parallel, the bandwidth of the second bus EBUS can be efficiently utilized.

[0132] The parameter update processing can be executed by the first arithmetic unit 30 without storing the parameter error ΔW generated by the second arithmetic unit 20 in the external memory 40. Also, the second arithmetic unit 20 can calculate the data error ΔD by reading the updated value of the parameter W generated by the first arithmetic unit 30 from the first memory 32 without reading it from the external memory 40. As a result, the data transfer to the external memory 40 can be reduced, and the bandwidth of the second bus EBUS can be effectively used. Also, the power consumption of the semiconductor device 10 can be reduced.

[0133] Note that the first arithmetic unit 30 may execute other additional processes for reducing the calculation load of the second arithmetic unit 20 and reducing the amount of data transferred from the external memory 40 to the second memory 22 based on an instruction. That is, the process executed by the first arithmetic unit 30 is not limited to the parameter update process. Even in this case, the above effects can be obtained.

[0134] FIG. 15 is a block diagram showing an example of a semiconductor device according to another embodiment of the present invention. The same elements as those in FIG. 1 are denoted by the same reference numerals, and detailed description thereof will be omitted. The semiconductor device 10A shown in FIG. 15 has an instruction decoder 33A and first and second data transfer control units 34A and 35A instead of the instruction decoder 33 and the first and second data transfer control units 34 and 35 in FIG. 1. Other configurations of the semiconductor device 10A are the same as those of the semiconductor device 10 shown in FIG. 1. The information processing apparatus on which the semiconductor device 10A is mounted is the same as the information processing apparatus shown in FIG. 2.

[0135] The instruction decoder 33A has an instruction queue 331 (instruction buffer) that holds the received instruction, and decodes the instruction held in the instruction queue 331. The instruction decoder 33A of the present embodiment can decode a wait instruction that waits for the output of an instruction (decoding result). When the instruction decoder 33A decodes a wait instruction, the instruction decoder 33A stops decoding the instruction and does not execute the decoding of the instruction following the wait instruction until it receives a start signal STT (STT1, STT2) from either or both of the first and second data transfer control units 34A and 35A.

[0136] By providing the instruction queue 331, the instruction following the wait instruction can be held in the queue, and a circuit (such as the instruction control unit 60 in FIG. 2) that supplies an instruction to the instruction decoder 33A can supply an instruction to the instruction decoder 33A without being synchronized with the start signal STT. Note that the instruction queue may be disposed outside the instruction decoder 33A.

[0137] The first data transfer control unit 34A can output a start signal STT1 in response to a completion signal CPLT1 received from the second arithmetic unit 20, or can output a start signal STT1 in response to the completion of data transfer. For example, the second arithmetic unit 20 has a function of outputting a completion signal CPLT1 based on the completion of data transfer instructed in advance from the first data transfer control unit 34A.

[0138] The second data transfer control unit 35A outputs a start signal STT2 in response to a completion signal CPLT2 received from the external memory 40, or can output a start signal STT2 in response to the completion of data transfer. For example, the external memory 40 has a function of outputting a completion signal CPLT2 based on the completion of data transfer instructed in advance from the second data transfer control unit 35A.

[0139] FIG. 16 is a flowchart showing an example of the operation when the semiconductor device 10A of FIG. 15 executes a parameter update process. That is, FIG. 16 shows an example of a control method of the semiconductor device 10A. FIG. 16 shows the operations corresponding to the processes from step S32 to step S34 shown in FIG. 8.

[0140] First, the instruction decoder 33A decodes, in step S41, a data transfer instruction for loading the parameter error ΔW calculated by the second arithmetic unit 20 and held in the second memory 22 into the first memory 32. The instruction decoder 33A issues the decoded data transfer instruction (first data transfer instruction) to the first data transfer control unit 34A. The first data transfer control unit 34A controls, in step S42, the second memory 22 and the first memory 32 based on the first data transfer instruction, and executes a process of transferring the parameter error ΔW from the second memory 22 to the first memory 32. Steps S41 and S42 are processes corresponding to step S32 shown in FIG. 9.

[0141] Also, in step S43, the instruction decoder 33A decodes a data transfer instruction for loading the parameters W, M, and V held in the external memory 40 into the first memory 32, and issues the data transfer instruction (second data transfer instruction) to the second data transfer control unit 35A. In step S44, the second data transfer control unit 35A controls the external memory 40 and the first memory 32 based on the second data transfer instruction, and executes a process of transferring the parameters W, M, and V from the external memory 40 to the first memory 32. Steps S43 and S44 are processes corresponding to step S33 shown in FIG. 9. Note that the order of steps S41, S42 and steps S43, S44 may be reversed, or they may be executed in parallel.

[0142] Next, in step S45, the instruction decoder 33A decodes a wait instruction corresponding to the first and second data transfer instructions in steps S41 and S43, and stops the instruction decoding operation. The instruction decoder 33A may receive two wait instructions corresponding to the two data transfer instructions, or may receive a wait instruction common to the two data transfer instructions. Note that the instruction decoder 33A can decode a new wait instruction even while the instruction decoding is stopped. Therefore, even when the instruction decoder 33A receives two wait instructions continuously and stops the instruction decoding based on the first wait instruction, it can decode the second wait instruction.

[0143] Next, in step S46, the second data transfer control unit 35A waits for the reception of the completion signal CPLT2, and when the completion signal CPLT2 is received, in step S47, it outputs the start signal STT2 to the instruction decoder 33A. In step S48, the first data transfer control unit 34A waits for the reception of the completion signal CPLT1, and when the completion signal CPLT1 is received, in step S49, it outputs the start signal STT1 to the instruction decoder 33A. Steps S45, S46 and steps S47, S48 may be executed in the reverse order, or may be executed in parallel.

[0144] In addition, when the second data transfer control unit 35A can determine by itself the completion of the transfer of the parameters W, M, and V to the first memory 32, based on the completion of the transfer, it may output the start signal STT2 without receiving the completion signal CPLT2. Similarly, when the first data transfer control unit 34A can determine by itself the completion of the transfer of the parameter error ΔW to the first memory 32, based on the completion of the transfer, it may output the start signal STT1 without receiving the completion signal CPLT1.

[0145] In step S50, the instruction decoder 33A that has received the start signals STT1 and STT2 corresponding to the two data transfer instructions determines that the transfer operations by the two data transfer instructions have been completed, and resumes the decoding of the instructions. For example, the instruction decoder 33A receives and decodes an arithmetic instruction for updating the parameters, and issues the arithmetic instruction to the first arithmetic unit 31. Then, in step S51, the first arithmetic unit 31 uses the parameters W, M, and V and the parameter error ΔW transferred to the first memory 32 to execute the update process of the parameters W, M, and V. Step S51 corresponds to the process of step S34 shown in FIG. 9. Note that, as shown in FIG. 8, when the parameter update process is divided and executed in a plurality of parts, the operation shown in FIG. 16 is executed for each of the divided parameter update processes.

[0146] As described above, also in this embodiment, similar to the above-described embodiment, without reducing the usage efficiency of the arithmetic units of both the second arithmetic unit 20 and the first arithmetic unit 30, backward processing and parameter update processing can be executed in accordance with the bandwidth of the external memory 40. Further, in this embodiment, even when the backward processing and the parameter update processing are respectively executed by the second arithmetic unit 20 and the first arithmetic unit 30 that operate independently of each other, the processing can be synchronized by a rendezvous instruction. That is, the parameter update processing can be started in synchronization with the completion of the transfer of the parameter error ΔW and the parameters W, M, V used in the update processing. Therefore, no wasted waiting time is inserted before the parameter update processing, and the backward processing and the parameter update processing can be efficiently executed. As a result, the time from the start of the backward processing to the update of the parameters can be further shortened.

[0147] The present invention is not limited to the specifically disclosed above embodiments, and various modifications and changes are possible without departing from the scope of the claims.

Explanation of Signs

[0148] 10 Semiconductor device 20 Second arithmetic unit 21 Second arithmetic unit 22 Second memory 30 First arithmetic unit 31 First arithmetic unit 32 First memory 33, 33A Instruction decoder 34, 34A First data transfer control unit 35, 35A Second data transfer control unit 40 External memory 50 Communication interface 60 Instruction control unit 100 Information processing device 200 Substrate 210 CPU 220 External storage device 230 Network interface 331 command queue EBUS second bus IBUS first bus

Claims

1. a first chip including one or more first computing units; a second chip including one or more second arithmetic units and having a different configuration from the first chip; the one or more second computing units perform backward processing of the neural network to calculate gradients of the weights of the neural network; transferring the calculated gradients from the one or more second computing units to one or more non-volatile memories; the one or more first computing units retrieve the calculated gradients from the one or more non-volatile memories and update the weights of the neural network based on the calculated gradients. Information processing device.

2. The one or more first calculation units update one or more parameters based on the calculated gradient; updating the weights of the neural network based on the updated one or more parameters; The information processing device according to claim 1 .

3. The one or more first computing units transfer the updated weights to the second chip.

3. The information processing device according to claim 1.

4. The one or more parameters include at least a parameter M and a parameter V.

4. The information processing device according to claim 2 or 3.

5. The one or more second calculation units do not update the parameter M and the parameter V. The information processing device according to claim 4 .

6. The one or more first calculation units do not transfer the updated parameters M and the updated parameters V to the one or more second calculation units.

6. The information processing device according to claim 4 or claim 5.

7. The one or more second calculation units further perform forward processing of the neural network. The information processing device according to claim 1 .

8. The forward processing, the backward processing, and the calculation of the gradient by the one or more second arithmetic units, and the updating of the weights by the one or more first arithmetic units are repeatedly executed. The information processing device according to claim 7 .

9. The method of claim 8, wherein the second chip transfers at least a portion of the calculated gradient to the first chip before the gradient calculation is completed. The information processing device according to claim 1 .

10. The method of claim 1, wherein the first chip transmits at least a portion of the updated weights to the second chip before the weight update is completed. The information processing device according to claim 1 .

11. The method of claim 10, wherein the backward processing of the neural network and the updating of the neural network weights are performed in parallel at least part of the time. The information processing device according to any one of claims 1 to 10.

12. The first chip and the second chip have different designs. The information processing device according to any one of claims 1 to 11.

13. The one or more first calculation units update the weights using a SIMD instruction. The information processing device according to any one of claims 1 to 12.

14. A portion of training data obtained by dividing a batch constituting training data of the neural network into a plurality of batches is assigned to the one or more second calculation units. The information processing device according to any one of claims 1 to 13.

15. Each of the training data divided in the unit direction in each layer of the neural network is assigned to each of a plurality of calculation nodes including the one or more second calculation units. The information processing device according to any one of claims 1 to 14.

16. Each of the multiple layers of the neural network is assigned to a multiple of computation nodes including the one or more second computation units. The information processing device according to any one of claims 1 to 15.

17. The one or more first calculation units update the weights of the neural network based on a gradient obtained by condensing the calculated gradient [KY1][TA2] and a gradient calculated by one or more other second calculation units. The information processing device according to any one of claims 1 to 15.

18. Using the information processing device according to any one of claims 1 to 17, updating the weights of a neural network. Information processing methods.

19. A first chip including one or more first arithmetic units; [KY3][TA4] and causing an information processing device including a second chip that includes one or more second arithmetic units and is different from the first chip to execute the information processing method according to claim 18. program.

Images