Information processing device, information processing method, and program

Abstract

To provide an integrated circuit capable of effectively using the band of an external memory in processing of a layer algorithm.SOLUTION: An information processing apparatus includes a first chip including one or more first arithmetic processing units, and a second chip including one or more second arithmetic processing units. The one or more second arithmetic processing units execute a backward operation of a neural network and calculate a gradient of the weight of the neural network. The one or more second arithmetic processing units transfer the calculated gradient to the first chip. The one or more first arithmetic processing units update the weight of the neural network based on the calculated gradient. The one or more first arithmetic processing units transfer the updated weight to the second chip. The one or more first arithmetic processing units update the weight of the neural network in a method using multiple types of parameters.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing device , INFORMATION PROCESSING METHOD AND PROGRAM - Patent application Regarding. [Background technology]

[0002] Recently, image recognition, voice recognition, and the like have become more accurate through deep learning using neural networks. For example, a processor that performs deep learning uses image data and weight data, etc., read from a main storage device, which is an external memory, to an internal memory, and causes multiple computing units to perform calculations in parallel. At this time, the arrays of the image data, weight data, etc. stored in the internal memory are converted and input to the processor core, thereby allowing the calculations to be performed efficiently. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-120547 A Summary of the Invention [Problem to be solved by the invention]

[0004] In general, in deep learning, the number of parameters used in each layer of a neural network is often smaller than the number of data supplied to each layer. Therefore, when inputting and outputting parameters to an external memory in processing a layer algorithm, there are times when the bandwidth of the external memory is not fully used, and memory resources may not be fully utilized. In addition, when updating parameters in a semiconductor device including a large number of arithmetic units implemented to execute processing of a layer algorithm, if the efficiency of use of the arithmetic units is low, the arithmetic resources may not be fully utilized. [Means for solving the problem]

[0005] According to an embodiment of the present invention, an information processing device includes a first chip including one or more first arithmetic units and a second chip including one or more second arithmetic units, the one or more second arithmetic units execute backward processing of a neural network to calculate gradients of weights of the neural network, the one or more second arithmetic units transfer the calculated gradients to the first chip, and the one or more first arithmetic units perform a calculation based on the calculated gradients. Using multiple types of parameters updating weights of the neural network, and the one or more first computing units transferring the updated weights to the second chip; do . [Brief description of the drawings]

[0006] [Figure 1] 1 is a block diagram showing an example of a semiconductor device according to an embodiment of the present invention; [Diagram 2] 2 is a block diagram showing an example of an information processing device in which the semiconductor device of FIG. 1 is mounted. [Diagram 3] FIG. 1 is a flow diagram showing an example of forward processing in training a deep neural network. [Figure 4] FIG. 1 is a flow diagram illustrating an example of backward processing and parameter update processing in training a deep neural network. [Diagram 5] FIG. 11 is an explanatory diagram showing an example of a layer algorithm in forward processing and backward processing. [Figure 6] FIG. 1 is an explanatory diagram showing an example of a gradient descent method for optimizing parameters. [Figure 7] 1. FIG. 4 is a sequence diagram showing an example in which a backward process and a parameter update process are executed by the semiconductor device of FIG. [Figure 8] 8 is an explanatory diagram showing details of the parameter error calculation process and the parameter update process of the layer N-1 in FIG. 7. [Figure 9] 8 is a flow chart showing an example of the operation of a main arithmetic unit and a sub-arithmetic unit which execute the sequence of FIG. 7. [Figure 10]FIG. 2 is a block diagram showing an example of an internal memory of FIG. 1. [Figure 11] 1. FIG. 4 is a block diagram showing another example of the internal memory of FIG. [Figure 12] 1. FIG. 4 is a block diagram showing another example of the internal memory of FIG. [Figure 13] 1. FIG. 4 is a block diagram showing another example of the internal memory of FIG. [Figure 14] 1. FIG. 4 is a block diagram showing another example of the internal memory of FIG. [Figure 15] FIG. 11 is a block diagram showing an example of a semiconductor device according to another embodiment of the present invention. [Figure 16] 16 is a flowchart showing an example of an operation performed by the semiconductor device of FIG. 15 when updating a parameter. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0008] Fig. 1 is a block diagram showing an example of a semiconductor device according to an embodiment of the present invention. The semiconductor device 10 shown in Fig. 1 is, for example, a processor used for training a deep neural network (DNN) including multiple layers, and executes forward processing, backward processing, and parameter update processing. The forward processing, backward processing, and parameter update processing will be described with reference to Figs. 3 and 4.

[0009] The semiconductor device 10 has a first arithmetic unit 30 and a second arithmetic unit 20. The first arithmetic unit 30 has a first arithmetic unit 31, a first memory 32, an instruction decoder 33, a first data transfer control unit 34, and a second data transfer control unit 35. The second arithmetic unit 20 has a second arithmetic unit 21 and a second memory 22. In this embodiment, the first arithmetic unit 30 is an optimization arithmetic unit that executes parameter update processing, and the second arithmetic unit 20 is a vector arithmetic unit that executes forward processing and backward processing of a deep neural network. The first arithmetic unit 30 is an example of a first integrated circuit, and the second arithmetic unit 20 is an example of a second integrated circuit.

[0010] The second arithmetic unit 21 has a number of arithmetic units for executing vector or matrix operations, etc. The second arithmetic unit 20 may have an instruction buffer for holding instructions such as arithmetic instructions and data transfer instructions, an instruction decoder for decoding instructions, and a register for holding data used in the operation and the operation results, etc. The second arithmetic unit 20 may have an instruction memory, an instruction cache, or a data cache in addition to the second memory 22.

[0011] The types of various arithmetic units included in the second arithmetic unit 21 include, but are not limited to, a multiply-and-accumulate unit, a multiplier, an adder, a divider, etc. Furthermore, the second arithmetic unit 21 of this embodiment has a floating-point arithmetic unit and a fixed-point arithmetic unit, and the floating-point number data may be, for example, any of half precision, single precision, and double precision.

[0012] The second memory 22 has a plurality of storage areas identified by addresses, and holds data, parameters, etc. used for calculations by the second arithmetic unit 21. The second memory 22 is, for example, a high-speed SRAM, but may also be an embedded DRAM (eDRAM), a Magnetoresistive Random Access Memory (MRAM), a Phase-change memory (PCM), a Resistive Random Access Memory (ReRAM), etc.

[0013] The first calculator 31 of the first calculation unit 30 has, for example, an adder, a multiplier, a divider, a logical calculator (OR logic, AND logic, NOT logic, exclusive OR), a square root calculator, etc. The square root calculator may be of a type that obtains an approximate value using a lookup table. When the first calculator 31 has calculators that calculate other functions such as a logarithm calculator, these calculators may also be of a type that obtains an approximate value using a lookup table. Moreover, the first calculator 31 may have a reciprocal square root calculator instead of a square root calculator.

[0014] Furthermore, when the first calculator 31 is made to execute the parameter update process in training the deep neural network, the first calculator 31 may have a dedicated calculator for calculating the value of an equation for determining the parameter update value or the value of a predetermined term in the equation. For example, when determining the parameter update value Wt by the ADAM (Adaptive Moment Estimation) of FIG. 6, the first calculator 31 may be provided with a dedicated calculator for calculating the second term on the right side. This can improve the calculation efficiency.

[0015] For example, the various arithmetic units included in the first arithmetic unit 31 are floating-point arithmetic units, and the type of floating-point numbers handled is one or more of half-precision, single-precision, and double-precision. The various arithmetic units included in the first arithmetic unit 31 may be SIMD (Single Instruction Multiple Data) arithmetic units. For example, a double-precision SIMD arithmetic unit can execute operations on two single-precision data or four half-precision data in parallel. Note that the register in which the data executed by the SIMD arithmetic unit is stored may be fixed-length or variable-length, the size of which changes based on information included in the instruction. The first arithmetic unit 31 may include an arithmetic unit that operates on fixed-point data. Furthermore, the first arithmetic unit 31 may have an arithmetic unit that converts the type of floating-point data, or an arithmetic unit that converts a floating-point number to a fixed-point number, or a arithmetic unit that converts a fixed-point number to a floating-point number.

[0016] The first arithmetic unit 31 can realize the calculation of logarithmic functions and the calculation of exponential functions by combining bit operations, floating-point operations, integer operations, etc. The calculation of logarithmic functions and the calculation of exponential functions may be executed by combining a plurality of calculation instructions, or may be executed by a dedicated arithmetic unit that combines a plurality of arithmetic units.

[0017] Each of the computing units included in the first computing unit 31 reads data held in the first memory 32, executes an operation, and stores the operation result in the first memory 32 based on a control signal from the instruction decoder 33. Hereinafter, each of the computing units included in the first computing unit 31 is also referred to as the first computing unit 31. The first computing unit 31 may have an optimal number of computing units for each type of computing unit to execute the parameter update process. This allows the first computing unit 30 to optimally set the B / F ratio, which is the ratio of the bandwidth of the external memory 40 to the computing performance when executing the parameter update process.

[0018] The first memory 32 has a plurality of storage areas identified by addresses. The first memory 32 is connected to the second memory 22 of the second arithmetic unit 20 via a first bus IBUS that connects the first arithmetic unit 30 and the second arithmetic unit 20, and is connected to the external memory 40 via a second bus EBUS. In this embodiment, the first bus IBUS and the second bus EBUS are independent of each other. For this reason, data transfer from the external memory 40 to the second memory 22 is performed via the first memory 32. However, as described later, by providing a bypass switch that connects the first bus IBUS and the second bus EBUS to each other in the first memory 32, the penalty of access delay due to passing through the first memory 32 can be reduced. The first memory 32 is composed of a high-speed SRAM, but may be eDRAM, MRAM, PCM, ReRAM, or the like.

[0019] The instruction decoder 33 decodes, for example, an instruction supplied from the outside of the semiconductor device 10, and outputs a control signal generated according to the decoded result to one or more of the first arithmetic unit 31, the first memory 32, and the first and second data transfer control units 34 and 35. When the instruction decoder 33 decodes an arithmetic instruction for causing the first arithmetic unit 31 to execute an arithmetic operation, the instruction decoder 33 outputs to the first arithmetic unit 31 information indicating the arithmetic unit that executes the arithmetic operation in the first arithmetic unit 31 and information indicating the type of the arithmetic operation instruction. In addition, based on the decoding of the arithmetic operation instruction, the instruction decoder 33 outputs to the first memory 32 read information (read address, etc.) for reading data used in the arithmetic operation from the first memory 32 and write information (write address, etc.) for writing the arithmetic operation result to the first memory 32. Hereinafter, the control signal, etc. output by the instruction decoder 33 to the first arithmetic unit 31 will also be referred to as an instruction.

[0020] In the first arithmetic unit 31, the arithmetic unit corresponding to the instruction from the instruction decoder 33 executes an operation on the data loaded from the first memory 32, and stores the operation result in the first memory 32. Based on information such as an address from the instruction decoder 33, the first memory 32 reads data to be used in the operation by the first arithmetic unit 31 from a storage area, outputs the data to the first arithmetic unit 31, and writes the received operation result in the storage area.

[0021] The instruction supplied to the instruction decoder 33 may be an instruction for executing one operation, a SIMD instruction for executing multiple operations, or an instruction specifying the number of operations. The first arithmetic unit 31 that receives an instruction specifying the number of operations may execute the specified number of operations while sequentially changing the data load source and the storage destination of the operation result in the first memory 32. By including a SIMD instruction or an instruction specifying the number of operations in the instruction set, it is possible to improve the efficiency of supplying instructions to the first arithmetic unit 30.

[0022] When the instruction decoder 33 decodes a data transfer instruction for transferring data between the second memory 22 and the first memory 32, it outputs, for example, a source address, a destination address, transfer amount information, and input / output information to the first data transfer control unit 34. When the instruction decoder 33 decodes a data transfer instruction for transferring data between the external memory 40 and the first memory 32, it outputs, for example, a source address, a destination address, transfer amount information, and input / output information to the second data transfer control unit 35. The source address indicates the source of data transfer, the destination address indicates the destination of data transfer, the transfer amount information indicates the amount of data transfer such as the number of transferred bytes, and the input / output information indicates the input / output direction of data (read / write).

[0023] The first data transfer control unit 34 accesses the second memory 22 and the first memory 32 via the first bus IBUS, based on the source address, destination address, transfer amount information, and input / output information received from the command decoder 33. For example, when the data transfer command indicates data transfer from the second memory 22 to the first memory 32, the first data transfer control unit 34 issues a read command to the second memory 22 the number of times corresponding to the data transfer amount. In addition, the first data transfer control unit 34 issues a write command to the first memory 32 the number of times corresponding to the data transfer amount. Then, data of an amount corresponding to the transfer amount information is transferred from the second memory 22 to the first memory 32.

[0024] The second data transfer control unit 35 accesses the external memory 40 and the first memory 32 via the second bus EBUS, based on the source address, destination address, transfer amount information, and input / output information received from the command decoder 33. For example, when the data transfer command indicates data transfer from the first memory 32 to the external memory 40, the second data transfer control unit 35 issues a read command to the first memory 32 the number of times corresponding to the data transfer amount. In addition, the second data transfer control unit 35 issues a write command to the external memory 40 the number of times corresponding to the data transfer amount. Then, data in an amount corresponding to the transfer amount information is transferred from the first memory 32 to the external memory 40.

[0025] The first data transfer control unit 34 may issue a data transfer request in packet format to the second memory 22 and the first memory 32 based on a data transfer command from the command decoder 33. Similarly, the second data transfer control unit 35 may issue a data transfer request in packet format to the first memory 32 and the external memory 40 based on a data transfer command from the command decoder 33.

[0026] For example, the external memory 40 is a main storage device such as a memory module of SDRAM, etc. The external memory 40 may be a stacked type memory module such as a High Bandwidth Memory (HBM) or a Hybrid Memory Cube (HMC).

[0027] For example, the semiconductor device 10 is preferably in the form of a single chip. By including the second arithmetic unit 20 and the first arithmetic unit 30 in one chip, the first bus IBUS can be formed as wiring within the chip. Furthermore, by taking advantage of the high-speed accessibility of the second memory 22 and the first memory 32, data transfer between the second memory 22 and the first memory 32 can be performed at high speed.

[0028] On the other hand, the second arithmetic unit 20 and the first arithmetic unit 30 may be provided on different semiconductor chips. In this case, a semiconductor device including the second arithmetic unit 20 and a semiconductor device including the first arithmetic unit 30 are designed separately. This allows the chip size of each semiconductor chip to be smaller than the chip size of a single semiconductor chip, thereby improving the yield of the semiconductor chips. In this case, it is preferable to arrange input / output buffers connected to each signal line of the first bus IBUS, external terminals such as pads and bumps, and protective elements on both chips.

[0029] The first arithmetic unit 30 may have an instruction buffer and a program counter for holding instructions to be supplied to the instruction decoder 33, and a register for holding data used in the calculation and the calculation result. For example, by storing a plurality of instructions in parallel in the instruction buffer, it is possible to improve the efficiency of supplying instructions to the first arithmetic unit 30. Note that the instructions held in the instruction buffer may be transferred from the external memory 40, or may be transferred from the external memory 40 to the instruction buffer based on an instruction from the CPU 210 described in FIG. 2.

[0030] Furthermore, by providing a program counter in the first arithmetic unit 30, a conditional branch instruction can be added to the instruction set, and loop processing can be executed by adding the conditional branch instruction. As a result, the efficiency of instruction supply can be improved. Furthermore, by changing the function to be executed depending on the branch destination of the conditional branch instruction, the value of a complex function can be approximated. Moreover, conditional execution can be performed by referring to a condition bit.

[0031] Furthermore, the first calculation unit 30 may have a data address register. In this case, indirect reference of the address becomes possible, and different processes can be executed by changing the register value without changing the application program. Also, by rewriting the data address register according to the calculation result, for example, it is possible to change the reference destination of the lookup table used for approximation of the calculation, and it is possible to divide the data into a plurality of intervals and perform approximation by a different function for each interval.

[0032] Fig. 2 is a block diagram showing an example of an information processing device in which the semiconductor device 10 of Fig. 1 is mounted. The information processing device 100 shown in Fig. 2 has a predetermined number of boards 200, a CPU 210, an external storage device 220, and a network interface (I / F) 230, which are interconnected by a system bus SBUS. For example, the predetermined number of boards 200 and the CPU 210 are mounted on a system board. Note that the information processing device 100 according to the present disclosure can be in various forms, and for example, a main storage device accessed by the CPU 210 may be connected to the system bus SBUS, and a GPU may be connected to the system bus SBUS.

[0033] The system bus SBUS may be, for example, a PCIe (Peripheral Component Interconnect express; registered trademark) bus. The information processing device 100 may also function as a server that causes a plurality of semiconductor devices 10 to cooperate with each other and execute training of a deep neural network.

[0034] Each board 200 has a plurality of semiconductor devices 10, a plurality of external memories 40 connected to the plurality of semiconductor devices 10, a communication interface 50, and an instruction control unit 60. The instruction control unit 60 may be provided in each of the plurality of semiconductor devices 10. The number of semiconductor devices 10 mounted on the board 200 is not limited to four. The plurality of semiconductor devices 10, the communication interface 50, and the instruction control unit 60 are connected to each other via a bus BUS provided on the board 200. The board 200 may be in the form of a bare board connected to a connector provided on a system board, or in the form of a package soldered to the system board.

[0035] The CPU 210 manages the overall operation of the information processing device 100 and functions as a host computer that allocates data used in deep learning to each of the multiple semiconductor devices. The external storage device 220 may hold, for example, data and parameters used in the deep learning calculations performed by each semiconductor device 10. The external storage device 220 may hold a management program and an application program executed by the CPU 210, and may hold instructions executed by each semiconductor device 10. The external storage device 220 may be a hard disk, a solid state drive (SSD), or other storage device. The network interface 230 connects the system bus SBUS to an external network. Information such as programs and data held by the external storage device 220 may be transferred from an external network via the network interface 230.

[0036] In each board 200, the communication interface 50 controls communication with the CPU 210, the communication interface 50 of the other board 200, or the network interface 230 via the system bus SBUS. The instruction control unit 60 controls instructions to be issued to each of the semiconductor devices 10 connected by the bus BUS based on instructions from the CPU 210. That is, the instruction control unit 60 issues various instructions to each of the semiconductor devices 10 for causing the semiconductor devices 10 to execute training based on instructions output from the CPU 210 according to a predetermined procedure for training the deep neural network. The instructions issued by the instruction control unit 60 are supplied to the instruction decoder 33 in the semiconductor device 10 shown in FIG. 1.

[0037] The instruction control unit 60 may output a timing signal to one or both of the second and first calculation units 20 and 30 for synchronizing the processing by the second calculation unit 20 and the processing by the first calculation unit 30. When the CPU 210 can manage the instructions to be issued to each of the semiconductor devices 10, the instructions may be issued directly from the CPU 210 to each of the semiconductor devices 10, and in this case, the instruction control unit 60 may not be mounted on the board 200.

[0038] For example, the multiple semiconductor devices 10 mounted on the substrate 200 may be mounted on the substrate 200 in the form of a semiconductor module (package). For example, in FIG. 2, four semiconductor devices 10 are included in one semiconductor module. When the first arithmetic unit 30 and the second arithmetic unit 20 are each designed as a semiconductor device (chip), a semiconductor module having multiple semiconductor devices including the first arithmetic unit 30 mounted thereon may be mounted on the substrate 200. Furthermore, for example, the information processing device 100 may construct a cluster by multiple semiconductor devices 10 connected via a network.

[0039] FIG. 3 is a flow diagram showing an example of forward processing in training a deep neural network. In forward processing, data and parameters such as weights are input to each of an input layer and a predetermined number of intermediate layers. In the input layer, the input data and parameter 1 are calculated to generate intermediate data 1. In the intermediate layer next to the input layer, intermediate data 1 and parameter 2 are calculated to generate intermediate data 2. In the subsequent intermediate layers, the intermediate data generated by the previous intermediate layer and the parameters set for each intermediate layer are also calculated, and the intermediate data generated by the calculation is output to the next intermediate layer. Note that there may also be intermediate layers that do not use parameters. Examples of intermediate layers include a convolution layer, a pooling layer, and a fully connected layer.

[0040] In the output layer, output data is calculated using intermediate data N generated by the intermediate layer N (Nth layer) immediately before the output layer. In the output layer, which calculates the error in a classification problem, output data (solution) is calculated, for example, by using a softmax function as the activation function and cross entropy as the error function. In the output layer, as explained in Figure 4, the error from the correct answer is calculated by comparing the output data with training data (correct answer data).

[0041] In this way, in forward processing, input data and parameters are calculated in each layer of the neural network to obtain data to be input to the next layer, and output data is output from the final layer (forward propagation). Note that forward processing is used not only for training neural networks, but also for inference using neural networks.

[0042] FIG. 4 is a flow diagram showing an example of backward processing and parameter update processing in training a deep neural network. In backward processing, error backpropagation is performed to propagate errors in the reverse order of forward processing. In FIG. 4, the symbol Δ indicates data error or parameter error. The parameter update processing is indicated by a dashed arrow.

[0043] First, in the backward processing, in the layer (output layer) for calculating the error, the output data generated in the forward processing is compared with the teacher data, and Δintermediate data N is generated, which is the error for the intermediate data N input to the output layer. Δintermediate data N is also the error of the output data output by the Nth intermediate layer.

[0044] Next, in each intermediate layer, starting from the intermediate layer closest to the output layer, the error (Δ intermediate data) for the output data and the intermediate data that is the input data are calculated, and a Δ parameter that is the error for the parameter of the intermediate layer is generated. The Δ parameter indicates the gradient of the parameter in the curve that indicates the change in error relative to the change in parameter. For example, in intermediate layer 2, Δ intermediate data 2 and intermediate data 1 are calculated to obtain Δ parameter 2.

[0045] In addition, in each intermediate layer, the error (Δ intermediate data) for the output data and the parameters of the intermediate layer are calculated, and Δ intermediate data, which is the error for the input data of the intermediate layer, is generated. The error (Δ intermediate data) for the input data of the intermediate layer is also the error of the output data of the previous intermediate layer (or input layer). For example, in intermediate layer 2, Δ intermediate data 2 and parameters 2 are calculated to obtain Δ intermediate data 1.

[0046] In the input layer, similarly to the intermediate layer, Δintermediate data 1 and input data may be calculated to obtain Δparameter 1, and Δintermediate data 1 and parameter 1 may be calculated to obtain Δinput data, which is an error with respect to the input data.

[0047] In the parameter update process, the parameters are corrected in each intermediate layer and input layer using the Δ parameter (gradient of error) obtained in the backward process. In other words, the parameters are optimized. Parameter optimization is performed using gradient descent methods such as Momentum-SGD (Stochastic Gradient Descent) and ADAM.

[0048] In this way, in the backward processing, the error of the data input to the output layer (the output data of the intermediate layer immediately before the output layer) is calculated from the output data and the teacher data. Then, the process of calculating the error of the input data using the calculated data error and the process of calculating the parameter error using the input data error are performed in order starting from the output side layer (error backpropagation). In the parameter update process, the parameters are optimized based on the parameter error obtained in the backward processing.

[0049] Fig. 5 is an explanatory diagram showing an example of a layer algorithm in forward processing and backward processing. Fig. 5 shows an example of a fully connected layer, which is a basic layer algorithm in a neural network. Below, an example of processing image data will be described, but the layer algorithm shown in Fig. 5 can also be applied to processing data other than image data.

[0050] For simplicity, the batch size processed in one layer algorithm is assumed to be the same as the image size, and the layer algorithm shown in Fig. 5 is executed for each image. In contrast, if an image is divided into multiple parts and each of the divided images is used as the batch size, the number of nested loops increases compared to Fig. 5. Note that convolutional layers, which are frequently used in image processing, can also be considered as fully connected layers with a large number of elements if the convolutional structure is ignored.

[0051] In forward processing, the symbol N indicates the order of layers (layer number), and the larger the layer number, the closer the layer is to the output side. N+1 indicates the output data from the layer, and D N indicates the input data input to the layer, and the symbol W N indicates the parameters input to the layer. The symbol B indicates the number of data elements (the number of pixels to be processed), and the symbol O C indicates the number of output channels (i.e., the number of units included in the layer), and the symbol I C indicates the number of input channels (i.e., the number of units in the previous layer). In forward processing, as explained in Figure 3, the output data D N+1 The process of calculating (intermediate data) is performed for each output channel using the input data D N and parameter W N This is carried out in various ways.

[0052] In the backward processing, the code ΔD N+1 indicates the error of the data received from the output layer, and the symbol ΔDN indicates the error of the data output to the input layer. Here, the output side indicates the side from which data is output in the forward processing, and the input side indicates the side to which data is input in the forward processing. The symbol ΔW N indicates the error of the parameter, and the symbol W N indicates a parameter. The other symbols are the same as in the forward process.

[0053] In the backward processing, as explained in Fig. 4, the input error ΔD N The process of calculating (Δintermediate data or Δinput data) is performed for each output channel to calculate the output error ΔD N+1 and parameter W N The parameter error ΔW N The process of calculating the output error ΔD (Δ parameter) for each output channel N+1 and the input data D N This is carried out in various ways.

[0054] Here, when neural network training such as forward processing and backward processing is executed in parallel on a plurality of computation nodes (for example, a pair of the semiconductor device 10 and the external memory 40), the following three methods of allocating computation nodes are considered. (Allocation method 1) Data is allocated to a computing node for each image or for each divided image obtained by dividing an image. In other words, data is allocated to a computing node for each element (batch / image division). (Allocation method 2) Data is divided into channels (units of each layer) and allocated to computational nodes (channel partitioning). (Allocation method 3) Allocate data to computation nodes by layer (layer division).

[0055] In channel partitioning, data input and output on a layer-by-layer basis must be communicated all-to-all, which can easily become a communication bottleneck. In layer partitioning, data input and output between layers must be moved between computation nodes, which can easily become a communication bottleneck and makes management of the moved data complicated. In batch / image partitioning, parameter-related data must be reduced or broadcast between computation nodes. However, in most neural network configurations, the size of the parameters used in the calculation is smaller than the size of the input data and other data used in the calculation, so communication bottlenecks are unlikely to occur. Therefore, in order to reduce the amount of communication between computation nodes, it is often more efficient to train using batch / image partitioning (allocation method 1), which assigns relatively large data to computation nodes on an element-by-element basis.

[0056] The bandwidth of the external memory 40 needs to be designed taking into consideration the relatively large size of data. For example, in training a neural network using batch / image division, the data input / output between layers differs for each computing unit that executes the calculation, so when saving the data to the external memory 40, the bandwidth needs to be widened.

[0057] On the other hand, in the training of a neural network by batch / image division, when only parameters having a relatively small size are input / output to the external memory 40, there is a risk that the bandwidth of the external memory 40 cannot be fully used. For example, the same parameter value is broadcast from the external memory 40 to each computing unit, and the parameter error is contracted and stored in the external memory 40. Broadcasting and contraction are paired communications, and the data input / output directions are opposite, but the order of the bandwidth does not change. Note that the parameter update process for optimizing the parameters requires fewer calculations than the processing of layer algorithms such as convolution calculations, so that if this process is performed in the second computing unit 20 equipped with a large number of computing units, the utilization efficiency of the computing units may decrease. In this embodiment, these problems can be solved.

[0058] In addition, in training a neural network, a communication called contraction and broadcasting is performed for a large number of computing units. The bandwidth of this type of communication may be limited to the width of the communication path in the vicinity of each computing unit, and in this case, the bandwidth of the external memory 40 may not be fully used. For example, when performing a calculation in the second computing unit 20, a communication of contraction and broadcasting of parameters is required when calculating a layer algorithm, and the bandwidth of the external memory 40 may not be fully used at the timing of calculating the layer algorithm. On the other hand, in this embodiment, an additional calculation called a parameter update process is performed in the first computing unit 30 in parallel with the calculation of the layer algorithm by the second computing unit 20, so that the bandwidth of the external memory 40 can be effectively used.

[0059] Fig. 6 is an explanatory diagram showing an example of the gradient descent method for optimizing parameters. In Fig. 6, parameters with the symbol t-1 indicate parameters before updating, and parameters with the symbol t indicate parameters after updating. The symbol ΔWt indicates the gradient of the error of the parameter immediately before updating.

[0060] In Momentum-SGD, two multiplications and two additions and subtractions are performed to calculate a new parameter Wt from the parameter error ΔWt. In ADAM, six multiplications, six additions and subtractions, one square root operation, and one division are performed to calculate a new parameter Wt from the parameter error ΔWt. In contrast, in the calculation of the layer algorithm shown in Figure 5, a number of multiplication and addition calculations proportional to the product of the parameter size and the data size are performed. In this way, the number of operations performed when updating parameters is less than the number of operations performed in layer algorithms such as convolution operations. For this reason, if only the operation to update parameters is performed in the arithmetic unit, the B / F ratio, which is the ratio of memory bandwidth per calculation performance, becomes high, and if a vector arithmetic unit with a large number of arithmetic units is used, most of the arithmetic units will be idle.

[0061] Fig. 7 is a sequence diagram showing an example of a case where the semiconductor device 10 of Fig. 1 executes backward processing and parameter update processing. That is, Fig. 7 shows an example of a control method for the semiconductor device 10. Fig. 7 shows an example of executing error backpropagation of a deep neural network by the second calculation unit 20 and parameter update processing by the first calculation unit 30 in parallel. Fig. 7 shows processing of layer N-1, part of processing of layer N that is one layer after layer N-1, and part of processing of layer N-2 that is one layer before layer N-1, among multiple layers of the deep neural network.

[0062] The symbol T attached to the parameters W, M, and V indicates that they are before updating, and the symbol T+1 attached to the parameters W, M, and V indicates that they are after updating. For example, the parameter W is updated together with the parameters M and V by the ADAM method.

[0063] A pentagon indicates data transfer, and a protruding corner indicates a transfer destination. A pentagon does not necessarily indicate data transfer all at once, but may indicate data transfer in multiple parts. When data transfer is performed in multiple parts, the first data transfer control unit 34 or the second data transfer control unit 35 issues a data transfer command for each data transfer. Hereinafter, a data transfer command for transferring data between the external memory 40 and the first memory 32 is referred to as an external data transfer command, and a data transfer command for transferring data between the first memory 32 and the second memory 22 is referred to as an internal data transfer command. When the command decoder 33 shown in FIG. 1 decodes an external data transfer command, it outputs a control signal to the second data transfer control unit 35, and when the command decoder 33 decodes an internal data transfer command, it outputs a control signal to the first data transfer control unit 34.

[0064] As with data transfer, the calculation process of the data error ΔD and the calculation process of the parameter error ΔW by the second calculation unit 20 and the calculation process of the parameter W by the first calculation unit 30 may be performed multiple times. In these calculation processes, the second calculation unit 21 and the first calculation unit 31 may execute calculations on multiple data with one instruction (SIMD method). In this case, the SIMD instruction supplied to the instruction decoder 33 may be capable of specifying the number of data to be calculated (variable-length SIMD method).

[0065] The first calculation unit 30, upon receiving the internal data transfer command, calculates the parameter W N,T to the second memory 22 of the second calculation unit 20. Therefore, the semiconductor device 10 can transfer the parameter W N,T can be transferred to the second memory 22. N,T are parameters of layer N used during forward processing, and have been transferred from the external memory 40 to the first memory 32 during the update processing of the parameters of layer N. As shown in FIG. N+1 and the transferred parameter W N,T and the data error ΔD using N Calculate the calculated data error ΔD N is stored in the second memory 22. Note that the data error ΔD N+1 is calculated during the backward processing of layer N+1 and is stored in the second memory 22.

[0066] Data error ΔD of layer N by the second calculation unit 20 N During the calculation of N-1 is transferred to the first memory 32, and then transferred from the first memory 32 to the second memory 22. N During the calculation of, the external memory 40 is not accessed, so the data D N-1 can be transferred to the first memory 32.

[0067] Next, the second calculation unit 20 calculates the data error ΔD N and data D transferred from external memory 40 N-1 and the parameter error ΔW N-1 Calculate the parameter error ΔW N-1 In response to the internal data transfer command, the first calculation unit 30 calculates the parameter error ΔW N-1 2 sequentially transfers the parameter error ΔW N-1 The timing of calculating the parameter error ΔW N-1 The second calculation unit 20 outputs a timing signal to the second calculation unit 20 so as to synchronize with the internal data transfer command for transferring the data to the first memory 32.

[0068] The first calculation unit 30 calculates a parameter W N-1,T , M N-1,T , V N-1,T The first calculator 31 of the first calculation unit 30 transfers the parameter W N-1 That is, the first calculator 31 executes the calculation of ADAM shown in FIG. N-1,T+1 , M N-1,T+1 , V N-1,T+1 The first calculation unit 30 calculates the updated parameter W held in the first memory 32 based on the reception of the external data transfer command. N-1,T+1 , M N-1,T+1 , V N-1,T+1 is transferred to the external memory 40.

[0069] As shown in FIG. 7, the parameter W N-1,T+1 , M N-1,T+1 , V N-1,T+1 The calculation process of is executed in parallel with the calculation of the parameter error ΔW by the second calculation unit 20.N-1 is the parameter W of the layer N-1. N-1,T+1、 M N-1,T+1 , V N-1,T+1 Since it is used only for the calculation of, it does not need to be saved in the external memory 40.

[0070] Next, the data error ΔD of layer N N Similarly to the calculation of N-1 The first calculation unit 30 calculates the data error ΔD based on the reception of the internal data transfer command. N-1 Parameter W used in the calculation of N-1,T The parameter W is transferred from the first memory 32 to the second memory 22 of the second calculation unit 20. N-1,T Since these have been transferred from the external memory 40 to the first memory 32 during the immediately preceding parameter update process, they do not need to be transferred again from the external memory 40, and therefore there is no need to use the bandwidth of the second bus EBUS.

[0071] The second calculator 21 of the second calculator 20 calculates the data error ΔD N and parameter W N-1,T and the data error ΔD using N-1 Calculate the calculated data error ΔD N-1 is stored in the second memory 22. Note that the data error ΔD N is calculated during backward processing of layer N and stored in the second memory 22. The data error ΔD of layer N-1 by the second calculation unit 20 N-1 During the calculation of N-2 Transfer.

[0072] Next, the parameter error ΔW N-1 Similarly to the calculation of N-2 The second calculation unit 20 calculates the data error ΔD N-1and data D transferred from the external memory 40 to the second memory 22 N-2 and the parameter error ΔW N-2 Calculate the parameter error ΔW N-2 In response to the internal data transfer command, the first calculation unit 30 calculates the parameter error ΔW N-2 The first calculation unit 30 sequentially transfers the parameter W from the second memory 22 to the first memory 32 based on the reception of the external data transfer command. N-2,T , M N-2,T , V N-2,T The first calculator 31 of the first calculation unit 30 transfers the new parameter W N-2,T+1 , M N-2,T+1 , V N-2,T+1 Upon receiving an external data transfer command, the first calculation unit 30 updates the parameters by calculating the updated parameters W N-2,T+1 , M N-2,T+1 , V N-2,T+1 is transferred to the external memory 40.

[0073] In this embodiment, the first arithmetic unit 30 can execute the update process of the parameter W while the second arithmetic unit 20 is calculating the parameter error ΔW. That is, the backward process and the update process of the parameter W can be executed in parallel by the different second arithmetic unit 21 and the first arithmetic unit 31. This can shorten the time required for the backward process and the parameter update process shown in FIG. 4. In addition, since the update process of the parameter W is executed while the parameter error ΔW is being calculated, the parameter error ΔW used only for the update process of the parameter W does not need to be saved in the external memory 40. Therefore, the band of the second bus EBUS can be designed without considering the transfer of the parameter error ΔW. Since it is possible to eliminate unnecessary data transfer using the second bus EBUS, it is possible to reduce the power consumption of the semiconductor device 10.

[0074] 6, the number of calculations performed in the process of updating the parameters used in the neural network is smaller than the number of calculations in the forward process and the backward process. Therefore, when the parameter update process is performed by the second calculator 21 of the second calculator 20, the B / F ratio increases, and the utilization efficiency of the second calculator 21 decreases. However, by performing the update process of the parameter W by the first calculator 31 of the first calculator 30 while the second calculator 21 is calculating the parameter error ΔW, it is possible to prevent the utilization efficiency of the second calculator 21 from decreasing.

[0075] Moreover, since the first computing unit 31 can be designed to have a number of computing units corresponding to the number of calculations in the parameter update process, it is possible to optimize the usage efficiency of the first computing unit 31 of the first computing unit 30 during the parameter update process. Furthermore, since it is not necessary to transfer the parameter W from the external memory 40 when the second computing unit 20 calculates the data error ΔD, the bandwidth of the second bus EBUS can be designed without considering the amount of the parameter W.

[0076] Although FIG. 7 shows that these steps appear to be executed together, in reality, as shown in FIG. 8, the parameter update process, the associated parameter transfer operation, and the calculation process of the parameter error ΔW are executed in multiple separate steps.

[0077] Figure 8 shows the parameter error ΔW N-1 Calculation process and parameter W N-1 , M N-1 , V N-1 FIG. 1 is an explanatory diagram showing details of the update process of the parameter error ΔW N-1 Calculation process and parameter W N-1 , M N-1 , V N-1 The update process can be divided into any number of parts and executed in parallel. For example, FIG. 8 shows an example in which the process is divided into four parts and executed, or shows four parts out of 100 parts in which the entire process is divided. The following describes an example in which the entire process is divided into four parts and executed in parallel.

[0078] In the process (1), first, the second calculation unit 20 calculates the data error ΔD held in the second memory 22. N (one fourth) and the data D transferred to the second memory 22 N-1 (1 / 4) and the parameter error ΔW N-1 The second calculation unit 20 calculates the calculated parameter error ΔW N-1 is stored in the second memory 22.

[0079] In process (1), the parameter W N-1,T , M N-1,T , V N-1,T The first quarter of the parameters ΔW calculated by the second calculation unit 20 and stored in the second memory 22 is transferred from the external memory 40 to the first memory 32. N-1 is transferred from the second memory 22 to the first memory 32.

[0080] In process (1), the first calculation unit 30 calculates the parameter W N-1,T , M N-1,T , V N-1,T (1 / 4) and the parameter error ΔW N-1 (1 / 4) and the updated parameters W N-1,T+1 , M N-1,T+1 , V N-1,T+1 Calculate the parameter W (1 / 4). N-1,T+1 , M N-1,T+1 , V N-1,T+1 is transferred to the external memory 40. Also, the parameter W N-1,T is transferred from the first memory 32 to the second memory 22 of the second calculation unit 20.

[0081] In processes (2), (3), and (4), the data error ΔD N A quarter of and Data D N-1 Using a quarter of and the parameter error ΔW N-1 The parameter W N-1,T , M N-1,T , VN-1,T and the parameter error ΔW N-1 Using a quarter of and the updated parameters W N-1,T+1 , M N-1,T+1 , V N-1,T+1 The calculated parameter W N-1,T+1 , M N-1,T+1 , V N-1,T+1 is transferred to the external memory 40, and the parameter W N-1,T is transferred from the first memory 32 to the second memory 22 of the second calculation unit 20.

[0082] The second calculation unit 20 calculates the parameter error ΔW N-1 After the calculation is completed, the data error ΔD N-1 Start calculating the data error ΔD N-1 The number of divisions in the calculation process of the parameter error ΔW N-1 The number of divisions may be different from that of the calculation process of (a) or may be executed all at once without division.

[0083] FIG. 9 is a flow diagram showing an example of the operation of the second calculation unit 20 and the first calculation unit 30 which execute the sequence of FIG. 7. That is, FIG. 9 shows an example of a control method of the semiconductor device 10. Detailed description of the same operations as those of FIG. 4 and FIG. 7 will be omitted. The backward processing by the second calculation unit 20 is executed after the forward processing by the second calculation unit 20 is completed. The parameter update processing by the first calculation unit 30 is executed in accordance with the progress of the backward processing by the second calculation unit 20. In the example shown in FIG. 9, the parameter update processing is executed by the ADAM method, but may be executed by the Momentum-SGD or other methods.

[0084] First, in step S21, the second calculation unit 20 calculates a data error ΔD using output data calculated in the output layer of the backward processing and teacher data prepared in advance. Next, in step S22, the second calculation unit 20 calculates a parameter error ΔW using the data error ΔD (output layer side) and the input data D of the layer. The calculated parameter error ΔW is used in the parameter update process by the first calculation unit 30.

[0085] Next, in step S23, the second calculation unit 20 calculates the input-side data error ΔD using the output-side data error ΔD and the parameter W. Next, in step S24, the second calculation unit 20 ends the operation if it has executed processing of the input layer, and returns to step S22 to execute an operation on the previous layer (input layer side) if it has executed processing of a layer other than the input layer.

[0086] Meanwhile, in step S31, the first calculation unit 30 transfers the input data D of the layer from the external memory 40 to the first memory 32, and further transfers it to the second memory 22. Note that the first step S31 is an operation for the layer immediately before the output layer. The input data D is used in step S22. Next, in step S32, the first calculation unit 30 controls the first data transfer control unit 34 to transfer the parameter error ΔW calculated by the second calculation unit 20 from the second memory 22 to the first memory 32.

[0087] In step S33, the first calculation unit 30 controls the second data transfer control unit 35 to transfer the pre-update parameters W, M, and V from the external memory 40 to the first memory 32. Next, in step S34, the first calculation unit 30 calculates the updated parameters W, M, and V using the pre-update parameters W, M, and V and the parameter error ΔW. That is, the first calculation unit 30 executes a parameter update process. Next, in step S35, the first calculation unit 30 transfers the updated parameters W, M, and V from the first memory 32 to the external memory 40.

[0088] Here, the transfer of the parameter error ΔW from the second memory 22 to the first memory 32 in step S32 and the transfer of the pre-update parameters W, M, and V from the external memory 40 to the first memory 32 in step S33 are performed independently of each other. That is, the first data transfer control unit 34 and the second data transfer control unit 35 transfer different information from each other. The external memory 40 has a large bandwidth. Furthermore, in the parameter update process by the first calculation unit 30 in step S34, the second bus EBUS and the first bus IBUS are not used. For this reason, some of the operations in steps S32, S33, S34, and S35 can be performed with overlapping timing. However, the data and parameters used in the parameter update process must be stored in the first memory 32 before the update process is performed.

[0089] 8, the operation of step S22 by the second calculation unit 20 and the operations of S32, S33, S34, and S35 by the first calculation unit 30 may be divided into groups of a predetermined number of data and a predetermined number of parameters and executed in multiple times. N-1 The transfer may be performed in multiple instalments.

[0090] Next, in step S36, the first calculation unit 30 transfers the pre-update parameter W from the first memory 32 to the second memory 22. Next, in step S37, if the first calculation unit 30 has executed an update process for the parameters of the input layer, it ends the operation. Also, if the first calculation unit 30 has executed an update process for the parameters other than the input layer, it returns to step S31 to execute an operation for the previous layer (input layer side).

[0091] 7 to 9, an example has been described in which the first calculation unit 30 performs the update process of the parameter W instead of the second calculation unit 20, thereby shortening the total processing time of the backward process and the parameter update process shown in FIG. 4 and reducing the bandwidth of the second bus EBUS. However, the process performed by the first calculation unit 30 is not limited to the update process of the parameter W. For example, the first calculation unit 30 may perform other processes to reduce the calculation load of the second calculation unit 20 and to reduce the amount of data transferred from the external memory 40 to the second memory 22. In addition, the type of the calculator mounted on the first calculator 31 of the first calculation unit 30 can be changed as appropriate depending on the process to be performed by the first calculation unit 30. In addition, the first calculation unit 30 may be designed to perform the parameter update process and other processes instead of the second calculation unit 20, and the first calculation unit 31 can be equipped with calculators necessary for performing these processes.

[0092] The semiconductor device 10 that executes the operation shown in Fig. 7 can efficiently execute backward processing and parameter update processing by suppressing the bandwidth of the second bus EBUS. Also, as shown in Fig. 8, by executing the parameter update processing in multiple batches, it is possible to prevent the occurrence of a timing in which the bandwidth of the external memory 40 is left over when parameters are contracted and broadcast. Furthermore, in the semiconductor device 10, since the second arithmetic unit 20 does not execute the parameter update processing, it is possible to design the second arithmetic unit 20 and the first bus IBUS without considering the B / F ratio when the parameter update processing is executed.

[0093] In this embodiment, the first calculation unit 30 and the second calculation unit 20 are each configured to sequentially execute a single instruction sequence, and the layer algorithm and the parameter update process can be executed in parallel. Therefore, this embodiment has a configuration for executing the layer algorithm and the parameter update process in parallel.

[0094] As another example of a configuration for executing layer algorithms and parameter update processing in parallel, a configuration can be considered in which a single type of arithmetic unit capable of executing two or more instruction streams is connected to an external memory via a global cache. In this configuration, since it is necessary to design the arithmetic unit so that it can handle both calculations with significantly different B / F ratios, namely, layer algorithms and parameter update processing, there is a risk of increasing redundancy in the circuit design. For example, a configuration in which the arithmetic units are hierarchized and each hierarchical unit has internal memories such as a primary cache and a secondary cache, is not necessary for parameter update processing, but is sometimes installed because it has a significant effect on improving performance in layer algorithms. However, in such a configuration, the data path between the global cache and each arithmetic unit becomes longer, and there is a risk of the overall processing time being extended. In contrast, in the first arithmetic unit 30 of this embodiment, the first arithmetic unit 31 and the first memory 32 can be installed in physically close locations, so that the parameter update processing can be executed while reducing the occurrence of delays.

[0095] As another example, a configuration in which the first calculation unit 30 and the second calculation unit 20 exchange data via an external memory is also possible. In such a configuration, the second calculation unit 20 needs to write the parameter error ΔW to the external memory, and the first calculation unit 30 needs to read the value of this parameter error ΔW from the external memory and execute the parameter update process. This increases the frequency of accessing the external memory and the amount of data compared to this embodiment. In such a configuration, it is considered that the overall processing time will be longer than in this embodiment.

[0096] Also, it is possible to hold the parameter error ΔW in the second calculation unit 20 without saving it to the external memory 40, calculate an update value of the parameter using the held parameter error ΔW, and store the calculated update value in the external memory 40. For example, when optimizing parameters using ADAM, parameters M and V are required in addition to parameter W used in processing the layer algorithm, and these parameters M and V also need to be held in the second calculation unit 20. In this case, there is a risk that the capacity of the second memory 22 of the second calculation unit 20 will be insufficient, and it is not efficient to save only the parameters M and V in the external memory 40 in order to prevent a shortage of buffer capacity.

[0097] In addition, even if a shortage of capacity of the second memory 22 can be avoided by finding the update value of the parameter W only from the parameter error ΔW without using the parameters M and V, the convergence of the parameter W to the optimal value may be delayed, and the parameter update processing time may become long.

[0098] Furthermore, it is possible to reduce the bandwidth of the external memory 40 by providing a cache memory between the second calculation unit 20 and the external memory 40. For example, while the second calculation unit 20 transmits and receives the parameter W and the parameter error ΔW to and from the cache memory, the image data D and the like are communicated between the cache memory and the external memory 40.

[0099] This makes it possible to create a situation in which the cache memory and the external memory 40 are constantly communicating with each other, even if a bottleneck in the communication bandwidth due to the second calculation unit 20 exists in the communication of the parameter W and the parameter error ΔW. However, this method intentionally reduces the bandwidth even when a high-speed external memory 40 can be used, and there is a risk that the bandwidth of the external memory 40 will become a bottleneck in processing with a high B / F ratio, such as parameter update processing, and the processing time will be extended.

[0100] In contrast, in this embodiment, the parameter update process by the first calculation unit 30 can be executed in parallel with the process of the layer algorithm by the second calculation unit 20. This makes it possible to create a situation in which the first calculation unit 30 and the external memory 40 are constantly communicating with each other, and the bandwidth of the external memory 40 can be used effectively.

[0101] Furthermore, in this case, even if the bandwidth between the second calculation unit 20 and the cache memory can be secured, the bandwidth between the cache memory and the external memory 40 cannot be secured, and, for example, the update process of a parameter with a high B / F ratio may become a bottleneck for the entire sequence.

[0102] The B / F ratio between the layer algorithm processing and the parameter update processing may vary greatly, and the B / F ratio between different layers may also vary greatly. This is because the balance between the amount of calculation of the layer algorithm, the input / output data size, and the parameter size changes depending on the image size, the number of channels, etc.

[0103] For example, in ResNet, a typical configuration of deep learning for image processing, a total of four poolings are performed, and the image size (area on the XY plane) is reduced to 1 / 4 and the number of channels is doubled for each pooling. ResNet is designed so that the amount of calculation in each layer is roughly constant, so in layers close to the input layer, the communication of image data is likely to become a bottleneck, and in layers close to the output layer, the communication of calculations or weights is likely to become a bottleneck. In particular, when allocating calculation nodes by batch / image division, the image size and image batch size are often made large to improve the efficiency of the use of computing units. In other words, in a typical configuration of deep learning for image processing, the B / F ratio of a layer algorithm is roughly proportional to the image size input and output by the layer, and tends to increase in layers closer to the input layer and decrease in layers closer to the output layer.

[0104] In this way, there is a non-negligible difference in the B / F ratio even between layer algorithms that have a dominant computational amount in deep learning calculations. Therefore, it is necessary to design the communication bandwidth of the external memory 40 so that a sufficient communication bandwidth can be secured even in layers with a high B / F ratio, and it is difficult to reduce the bandwidth of the external memory 40 itself.

[0105] Furthermore, it is possible to dynamically change the clock frequency of the external memory 40 according to the B / F ratio of the layer algorithm, thereby creating a situation in which the external memory 40 and the cache memory are constantly communicating with each other. However, such a design may complicate the system configuration and may hinder the stable operation of the semiconductor device. In addition, since the parameter update process and the layer algorithm process are not executed in parallel, the parameter update process needs to be executed at a different timing from the layer algorithm process.

[0106] In contrast, in the configuration of this embodiment, the parameter update process and the calculation of the layer algorithm can be executed in parallel, and further, whether or not to execute them in parallel can be controlled by software. Therefore, for example, it is possible to realize a response such that the layer algorithm with a high B / F ratio is not executed in parallel, and the layer algorithm with a low B / F ratio is executed in parallel. Therefore, in the configuration of this embodiment, the processing of the deep neural network can be efficiently executed according to the B / F ratio that differs for each layer algorithm.

[0107] It should be noted that this embodiment has advantages over the above-mentioned example, but the present invention may include a part or all of the configurations of the above-mentioned example.

[0108] Fig. 10 is a block diagram showing an example of the first memory 32 in Fig. 1. The first memory 32 has an operation control unit CNTL, an address decoder ADEC, a data input / output unit DIO, and a memory cell array ARY including a plurality of banks BK (four in this example).

[0109] The address decoder ADEC selects one of the banks BK to be accessed based on a bank number received from either the first and second data transfer control units 34, 35 or the instruction decoder 33 in Fig. 1, and selects a memory cell in the selected bank BK based on an address. The operation control unit CNTL controls the operations of the memory cell array ARY and the data input / output unit DIO based on control signals such as a port number, a read enable, and a write enable received from either the first and second data transfer control units 34, 35 or the instruction decoder 33.

[0110] The data input / output unit DIO connects one of the input / output ports P (P0, P1, P2) to the memory cell array ARY according to the port number of the reception, and reads and writes data from / to the memory cell array ARY. The input / output port P0 is connected to the first arithmetic unit 31, the input / output port P1 is connected to the external memory 40 via the second bus EBUS, and the input / output port P2 is connected to the second memory 22 of the second arithmetic unit 20 via the first bus IBUS. Although not particularly limited, the data input or output by the data input / output unit DIO every time the memory cell array ARY is accessed may be, for example, 2 n (n is an integer equal to or greater than 4). When data for error detection and correction is added to the data processed by the first arithmetic unit 31, the number of bits of the data may be 2 n +m bits (m is the number of bits of data for error detection and correction).

[0111] The data input / output unit DIO outputs data read from one of the banks BK based on the read enable to one of the input / output ports P selected according to the port number. The data input / output unit DIO outputs data received at the input / output port P selected according to the port number to the bank BK selected by the bank number based on the write enable, and writes the data into a memory cell in the bank BK.

[0112] Each of the port number, address, and bank number consists of multiple bits, and an OR gate OR is provided for each signal line. Read enable and write enable are 1-bit signals that indicate valid or invalid depending on the logical level. Read enable is at an active level when reading data from the memory cell array ARY, and write enable is at an active level when writing data to the memory cell array ARY.

[0113] 1 outputs an address, a bank number, a port number selecting input / output port P0, and a read enable to the first memory 32 in order to read data to be used by the first arithmetic unit 31 from the first memory 32 based on an arithmetic instruction. Also, the instruction decoder 33 outputs an address, a bank number, a port number selecting input / output port P0, and a write enable to the first memory 32 in order to write the arithmetic result to the first memory 32 based on the completion of the arithmetic operation in the first arithmetic unit 31.

[0114] The instruction decoder 33 outputs a read start address of the external memory 40, a write start address of the first memory 32, a transfer size, and input / output information to the second data transfer control unit 35 based on a data transfer command for transferring data from the external memory 40 to the first memory 32. The second data transfer control unit 35 outputs a write address, a bank number, a port number for selecting the input / output port P1, and a write enable to the first memory 32 based on the information received from the instruction decoder 33.

[0115] The instruction decoder 33 outputs a read start address of the first memory 32, a write start address of the external memory 40, a transfer size, and input / output information to the second data transfer control unit 35 based on a data transfer command for transferring data from the first memory 32 to the external memory 40. The second data transfer control unit 35 outputs a read address, a bank number, a port number for selecting the input / output port P1, and a read enable to the first memory 32 based on the information received from the instruction decoder 33.

[0116] The instruction decoder 33 outputs a read start address of the second memory 22, a write start address of the first memory 32, a transfer size, and input / output information to the first data transfer control unit 34 based on a data transfer command for transferring data from the second memory 22 to the first memory 32. The first data transfer control unit 34 outputs a write address, a bank number, a port number for selecting the input / output port P2, and a write enable to the first memory 32 based on the information received from the instruction decoder 33.

[0117] The instruction decoder 33 outputs the read start address of the first memory 32, the write start address of the second memory 22, the transfer size, and input / output information to the first data transfer control unit 34 based on a data transfer command for transferring data from the first memory 32 to the second memory 22. The first data transfer control unit 34 outputs the write address, bank number, port number for selecting the input / output port P2, and write enable to the first memory 32 based on the information received from the instruction decoder 33.

[0118] Fig. 11 is a block diagram showing another example of the first memory 32 of Fig. 1. Detailed description of the same elements as those in Fig. 10 will be omitted. The first memory 32 shown in Fig. 11 has a single-port type data input / output unit DIO and a switch circuit SW connected to the data input / output unit DIO. Other configurations of the first memory 32 are the same as those in Fig. 10.

[0119] In the first memory 32 shown in FIG. 11, since the data input / output unit DIO is of the single port type, the operation control unit CNTL does not receive a port number, but receives read enable and write enable to control the data transfer direction in the data input / output unit DIO.

[0120] The switch circuit SW connects the data terminal of the data input / output unit DIO to the first arithmetic unit 31, the second bus EBUS, or the first bus IBUS based on a switch control signal received from either the first and second data transfer control units 34, 35 or the command decoder 33 in Fig. 1. The switch control signal is generated by the command decoder 33 shown in Fig. 1 in place of the port numbers shown in Fig. 10. The operation of the command decoder 33 and the operation of the first memory 32 are similar to the operations described in Fig. 10.

[0121] FIG. 12 is a block diagram showing another example of the first memory 32 in FIG. 1. Detailed description of the same elements as those in FIG. 10 and FIG. 11 is omitted. The first memory 32 has a plurality of memory cores MCORE (in this example, two, MCORE1 and MCORE2) and a switch circuit SW. Each memory core MCORE may have an operation control unit CNTL (not shown), an address decoder ADEC, a data input / output unit DIO, and a memory cell array ARY including a plurality of banks BK, as in FIG. 10. However, the data input / output unit DIO is of a single port type as in FIG. 11, and each memory core MCORE does not receive a port number as in FIG. 11. The data terminal of each memory core MCORE is connected to the switch circuit SW.

[0122] The switch circuit SW receives a switch control signal from either the first and second data transfer control units 34, 35 or the instruction decoder 33 in Fig. 1, similar to the switch circuit SW in Fig. 11. Then, the switch circuit SW connects one of the data terminals of the memory core MCORE to the first arithmetic unit 31, the second bus EBUS or the first bus IBUS based on the switch control signal. The switch control signal is generated by the instruction decoder 33 shown in Fig. 1, and includes the logic of an address for selecting the memory core MCORE.

[0123] For example, the switch circuit SW may have a function of simultaneously connecting the memory core MCORE1 to the external memory 40 and connecting the memory core MCORE2 to the first arithmetic unit 31. In this case, the logic of the switch control signal indicates a combination of connections between the memory cores MCORE1 and MCORE2 and the first arithmetic unit 31, the second bus EBUS, and the first bus IBUS. The instruction decoder 33 decodes instructions (operation instructions and data transfer instructions) corresponding to the number of combinations of connections of the switch circuit SW. This allows multiple memory cores MCORE to be accessed simultaneously and independently.

[0124] Fig. 13 is a block diagram showing another example of the first memory 32 in Fig. 1. Detailed description of the same elements as in Fig. 12 will be omitted. The first memory 32 shown in Fig. 13 is similar to the first memory 32 shown in Fig. 12 except that the switch circuit SW is different from the switch circuit SW in Fig. 12. The switch circuit SW is similar to the switch circuit SW in Fig. 12 except that it has a path that connects the second bus EBUS and the first bus IBUS to each other.

[0125] 1 has a function of decoding a direct data transfer instruction for directly transferring data between the external memory 40 and the second memory 22 of the second processing unit 20. In other words, the semiconductor device 10 has an instruction set including a direct data transfer instruction.

[0126] When the instruction decoder 33 decodes a direct data transfer instruction, it outputs a switch control signal for connecting the second bus EBUS and the first bus IBUS to the first memory 32. In addition, the instruction decoder 33 outputs information for accessing the external memory 40 to the second data transfer control unit 35, and outputs information for accessing the second memory 22 to the first data transfer control unit 34.

[0127] The second data transfer control unit 35 issues a read command or a write command to the external memory 40 based on the information from the instruction decoder 33. The first data transfer control unit 34 issues a read command or a write command to the second memory 22 based on the information from the instruction decoder 33. This makes it possible to transfer data between the external memory 40 and the second memory 22 of the second calculation unit 20 without writing data to the memory core MCORE of the first memory 32. For example, data D shown in FIG. N-1 , D N-2 can be transferred from the external memory 40 to the second memory 22 without being written to the memory core MCORE.

[0128] Fig. 14 is a block diagram showing another example of the first memory 32 in Fig. 1. Detailed description of the same elements as those in Fig. 11 and Fig. 12 will be omitted. The first memory 32 shown in Fig. 14 is provided with a switch circuit SW (SW1, SW2) for each memory core MCORE, and each switch circuit SW operates by receiving a different switch control signal from the instruction decoder 33. The switch circuit SW is the same as the switch circuit SW shown in Fig. 11, and connects a data input / output unit DIO (not shown) of the memory core MCORE to the first arithmetic unit 31, the second bus EBUS, or the first bus IBUS.

[0129] The switch control signal includes the logic of the address for selecting the memory core MCORE. This allows the first memory 32 to operate as two independent memories. That is, the memory core MCORE1 can operate without being affected by the operation of the memory core MCORE2, and the memory core MCORE2 can operate without being affected by the operation of the memory core MCORE1.

[0130] As described above, in this embodiment, the forward processing and backward processing in the neural network are executed by the second calculation unit 20, and the parameter update processing is executed by the first calculation unit 30 in the background of the backward processing. Also, the first calculation unit 30 is equipped with a number of calculators with specifications suitable for the parameter update processing, and the parameter W used in the backward processing is transferred from the first memory 32, not from the external memory 40. This makes it possible to execute the backward processing and the parameter update processing efficiently in accordance with the bandwidth of the external memory 40, without reducing the utilization efficiency of the calculators of both the second calculation unit 20 and the first calculation unit 30.

[0131] In addition, since the backward processing and the parameter update processing are executed in parallel, the time from the start of the backward processing to the update of the parameters can be shortened. Furthermore, by executing the calculation processing of the parameter error ΔW of the backward processing and the parameter update processing in parallel, the bandwidth of the second bus EBUS can be used efficiently.

[0132] The first calculation unit 30 can execute the parameter update process without storing the parameter error ΔW generated by the second calculation unit 20 in the external memory 40. Also, the second calculation unit 20 can calculate the data error ΔD by reading the update value of the parameter W generated by the first calculation unit 30 from the first memory 32 without reading it from the external memory 40. This makes it possible to reduce data transfer to the external memory 40 and to effectively use the bandwidth of the second bus EBUS. Also, the power consumption of the semiconductor device 10 can be reduced.

[0133] The first calculation unit 30 may execute other additional processes based on instructions to reduce the calculation load of the second calculation unit 20 and to reduce the amount of data transferred from the external memory 40 to the second memory 22. That is, the processes executed by the first calculation unit 30 are not limited to the parameter update process. In this case, the above-mentioned effects can also be obtained.

[0134] FIG. 15 is a block diagram showing an example of a semiconductor device according to another embodiment of the present invention. The same elements as those in FIG. 1 are given the same reference numerals, and detailed description thereof will be omitted. The semiconductor device 10A shown in FIG. 15 has an instruction decoder 33A and first and second data transfer control units 34A and 35A instead of the instruction decoder 33 and the first and second data transfer control units 34 and 35 in FIG. 1. The other configurations of the semiconductor device 10A are the same as those of the semiconductor device 10 shown in FIG. 1. The information processing device on which the semiconductor device 10A is mounted is the same as the information processing device shown in FIG. 2.

[0135] The instruction decoder 33A has an instruction queue 331 (instruction buffer) that holds received instructions, and decodes the instructions held in the instruction queue 331. The instruction decoder 33A of this embodiment can decode a wait instruction that waits for the output of an instruction (decoded result). When the instruction decoder 33A decodes a wait instruction, it stops decoding the instruction and does not execute decoding of the instruction following the wait instruction until it receives a start signal STT (STT1, STT2) from either or both of the first and second data transfer control units 34A, 35A.

[0136] By providing the instruction queue 331, the instruction following the waiting instruction can be held in the queue, and a circuit (such as the instruction control unit 60 in FIG. 2) that supplies instructions to the instruction decoder 33A can supply instructions to the instruction decoder 33A without synchronizing with the start signal STT. The instruction queue may be disposed outside the instruction decoder 33A.

[0137] The first data transfer control unit 34A can output the start signal STT1 in response to a completion signal CPLT1 received from the second calculation unit 20, or can output the start signal STT1 in response to the completion of the data transfer. For example, the second calculation unit 20 has a function of outputting the completion signal CPLT1 based on the completion of the data transfer instructed in advance by the first data transfer control unit 34A.

[0138] The second data transfer control unit 35A can output the start signal STT2 in response to a completion signal CPLT2 received from the external memory 40, or can output the start signal STT2 in response to the completion of the data transfer. For example, the external memory 40 has a function of outputting the completion signal CPLT2 based on the completion of the data transfer instructed in advance by the second data transfer control unit 35A.

[0139] Fig. 16 is a flow diagram showing an example of an operation when the semiconductor device 10A of Fig. 15 executes a parameter update process. That is, Fig. 16 shows an example of a control method for the semiconductor device 10A. Fig. 16 shows an operation corresponding to the processes from step S32 to step S34 shown in Fig. 8.

[0140] First, in step S41, the instruction decoder 33A decodes a data transfer instruction for loading the parameter error ΔW, which is calculated by the second calculation unit 20 and held in the second memory 22, into the first memory 32. The instruction decoder 33A issues the decoded data transfer instruction (first data transfer instruction) to the first data transfer control unit 34A. In step S42, the first data transfer control unit 34A controls the second memory 22 and the first memory 32 based on the first data transfer instruction, and executes a process of transferring the parameter error ΔW from the second memory 22 to the first memory 32. Steps S41 and S42 correspond to step S32 shown in FIG. 9.

[0141] Furthermore, in step S43, the instruction decoder 33A decodes a data transfer instruction for loading the parameters W, M, and V held in the external memory 40 into the first memory 32, and issues the data transfer instruction (second data transfer instruction) to the second data transfer control unit 35A. In step S44, the second data transfer control unit 35A controls the external memory 40 and the first memory 32 based on the second data transfer instruction, and executes a process of transferring the parameters W, M, and V from the external memory 40 to the first memory 32. Steps S43 and S44 correspond to step S33 shown in FIG. 9. The order of steps S41, S42 and steps S43, S44 may be reversed, or steps may be executed in parallel.

[0142] Next, in step S45, the instruction decoder 33A decodes a queuing instruction corresponding to the first and second data transfer instructions of steps S41 and S43, and stops the instruction decoding operation. The instruction decoder 33A may receive two queuing instructions corresponding to the two data transfer instructions, or may receive a queuing instruction common to the two data transfer instructions. Note that the instruction decoder 33A can decode a new queuing instruction even while the instruction decoding is stopped. Therefore, even if the instruction decoder 33A receives two queuing instructions in succession and stops the instruction decoding based on the first queuing instruction, it can decode the second queuing instruction.

[0143] Next, in step S46, the second data transfer control unit 35A waits for receipt of a completion signal CPLT2, and when the completion signal CPLT2 is received, outputs a start signal STT2 to the instruction decoder 33A in step S47. In step S48, the first data transfer control unit 34A waits for receipt of a completion signal CPLT1, and when the completion signal CPLT1 is received, outputs a start signal STT1 to the instruction decoder 33A in step S49. Steps S45, S46 and steps S47, S48 may be executed in the reverse order or in parallel.

[0144] When the second data transfer control unit 35A can determine by itself the completion of the transfer of the parameters W, M, and V to the first memory 32, it may output the start signal STT2 based on the completion of the transfer without receiving the completion signal CPLT2. Similarly, when the first data transfer control unit 34A can determine by itself the completion of the transfer of the parameter error ΔW to the first memory 32, it may output the start signal STT1 based on the completion of the transfer without receiving the completion signal CPLT1.

[0145] In step S50, the instruction decoder 33A receives start signals STT1 and STT2 corresponding to the two data transfer instructions, determines that the transfer operations by the two data transfer instructions are completed, and resumes decoding of the instructions. For example, the instruction decoder 33A receives and decodes an operation instruction for updating a parameter, and issues the operation instruction to the first calculator 31. Then, in step S51, the first calculator 31 executes an update process of the parameters W, M, and V using the parameters W, M, and V and the parameter error ΔW transferred to the first memory 32. Step S51 is a process corresponding to step S34 shown in FIG. 9. Note that, as shown in FIG. 8, when the parameter update process is divided into a plurality of parts and executed, the operation shown in FIG. 16 is executed for each of the divided plurality of parameter update processes.

[0146] As described above, in this embodiment, as in the above-mentioned embodiment, the backward processing and the parameter update processing can be executed in accordance with the bandwidth of the external memory 40 without reducing the utilization efficiency of the computing units of both the second computing unit 20 and the first computing unit 30. Furthermore, in this embodiment, even when the backward processing and the parameter update processing are executed by the second computing unit 20 and the first computing unit 30, which operate independently of each other, respectively, the processing can be synchronized by a waiting command. That is, the parameter update processing can be started in synchronization with the completion of the transfer of the parameter error ΔW and the parameters W, M, and V used in the update processing. Therefore, no unnecessary waiting time is inserted before the parameter update processing, and the backward processing and the parameter update processing can be executed efficiently. As a result, the time from the start of the backward processing to the update of the parameters can be further shortened.

[0147] The present invention is not limited to the above specifically disclosed embodiments, but various modifications and variations are possible without departing from the scope of the claims. [Explanation of symbols]

[0148] 10 Semiconductor device 20 Second Calculation Unit 21 Second arithmetic unit 22 The Second Memory 30 First Calculation Unit 31 First arithmetic unit 32 The First Memory 33, 33A Instruction Decoder 34, 34A First data transfer control section 35, 35A Second data transfer control section 40 External Memory 50 Communication Interface 60 Command control section 100 Information processing device 200 boards 210 CPU 220 External storage device 230 Network Interface 331 Command Queue EBUS 2nd bus IBUS No. 1 bus

Claims

1. a first chip including one or more first computing units; a second chip including one or more second computing units; the one or more second computing units perform backward processing of the neural network to calculate gradients of the weights of the neural network; the one or more second computing units forward the calculated gradients to the first chip; the one or more first calculation units update weights of the neural network by a method using a plurality of types of parameters based on the calculated gradient; the one or more first computing units transfer the updated weights to the second chip; Information processing device.

2. The information processing device according to claim 1 , wherein the one or more second calculation units further execute a forward process of the neural network.

3. The information processing device according to claim 1 , wherein the first chip and the second chip are different in design from each other.

4. The information processing apparatus according to claim 2 , wherein the forward processing of the neural network, the backward processing of the neural network, the calculation of the gradient of the weights of the neural network, and the updating of the weights of the neural network are performed repeatedly.

5. The information processing device according to claim 1 , wherein the one or more second arithmetic units include a single instruction multiple data (SIMD) arithmetic unit.

6. The information processing device according to claim 1 , wherein the one or more first calculation units are synchronized with the one or more second calculation units.

7. The information processing device according to claim 1 , wherein the backward processing of the neural network and the updating of the weights of the neural network are executed in parallel.

8. The information processing device according to claim 1 , wherein the one or more second processing units communicate with an external memory via the first chip.

9. The information processing device according to claim 1 , wherein the gradients are not transferred to an external memory.

10. The information processing device according to claim 8 , wherein the external memory is a High Bandwidth Memory (HBM).

11. The information processing device according to claim 1 , wherein only the updated weights of the plurality of types of parameters are transferred to the one or more second calculation units.

12. An information processing method executed by a first chip including one or more first calculation units and a second chip including one or more second calculation units, comprising: the one or more second computing units perform backward processing of the neural network to calculate gradients of the weights of the neural network; the one or more second computing units forward the calculated gradients to the first chip; the one or more first calculation units update weights of the neural network by a method using a plurality of types of parameters based on the calculated gradient; the one or more first computing units transfer the updated weights to the second chip; Information processing methods.

13. The information processing method according to claim 12 , wherein the one or more second calculation units further execute a forward process of the neural network.

14. The information processing method according to claim 12 or 13, wherein the first chip and the second chip are different in design from each other.

15. 14. The method of claim 13, wherein the forward processing of the neural network, the backward processing of the neural network, the calculation of gradients of the neural network weights, and the updating of the neural network weights are performed iteratively.

16. The information processing method according to claim 12 , wherein the one or more second arithmetic units comprise a single instruction multiple data (SIMD) arithmetic unit.

17. The information processing method according to claim 12 , wherein the one or more first computing units are synchronized with the one or more second computing units.

18. 18. The method of claim 12, wherein the backward processing of the neural network and the updating of the weights of the neural network are performed in parallel.

19. The information processing method according to claim 12 , wherein the one or more second processing units communicate with an external memory via the first chip.

20. 19. The method of claim 12, wherein the gradients are not transferred to an external memory.

21. The information processing method according to claim 19 , wherein the external memory is a High Bandwidth Memory (HBM).

22. The information processing method according to claim 12 , wherein only the updated weights of the plurality of types of parameters are transferred to the one or more second calculation units.

23. A program for causing a computer to function as an information processing device according to any one of claims 1 to 11.

Images