Processing Unit

Abstract

Provided is a calculation processing device (10) comprising: a calculation circuit unit (500) that repeatedly performs calculation by using first and second data having been read; a first storage unit (200) that stores therein the first data; a second storage unit (202) that stores therein the second data; a register unit (400) that reads, from the first storage unit or the second storage unit, data for repeated use in the calculation circuit unit among the first and second data and that temporarily holds the readout data; a first selector unit (300) that selects, from the first and second data, data to be held in the register unit; and a control unit (600) that controls the first selector unit on the basis of at least one of a calculation type, a calculation amount, types of the first and second storage units, the capacities of the first and second storage units, and a type of the calculation circuit unit.

Description

[Technical field]

[0001] The present disclosure relates to a processor. [Background technology]

[0002] In recent years, deep neural networks, which are mathematical models that mimic the mechanisms of the nervous system, have been developed. These deep neural networks have achieved high performance by increasing the amount of calculations and coefficient capacity. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-26027 A [Non-patent literature]

[0004] [Non-Patent Document 1] Squeeze-and-Excitation Network J. Hu, L. Shen, and G. Sun, arXiv:1709.01507,2017(https: / / arxiv.org / pdf / 1709.01507.pdf) Summary of the Invention [Problem to be solved by the invention]

[0005] For a processing device that performs arithmetic processing in a deep neural network, it is required to suppress the increase in power consumption even when the amount of calculation increases. Therefore, various methods have been proposed to reduce the processing load of calculations in a deep neural network and suppress the increase in power consumption, and one of such proposals is the method disclosed in the above Patent Document 1. However, even when the technology disclosed in the above Patent Document 1 is applied, the effect of suppressing the increase in power consumption is still insufficient, and further improvement is strongly required.

[0006] Therefore, the present disclosure proposes a processing device that can further suppress an increase in power consumption of a processing device that performs arithmetic processing in a deep neural network. [Means for solving the problem]

[0007] According to the present disclosure, there is provided an arithmetic processing device comprising: an arithmetic circuit unit that reads out first and second data and repeatedly performs an arithmetic operation using the read out first and second data; a first memory unit that stores the first data; a second memory unit that stores the second data; a register unit that reads out from the first memory unit or the second memory unit and temporarily holds data that is repeatedly used in the arithmetic circuit unit among the first and second data; a first selector unit that selects data to be held in the register unit among the first and second data; and a control unit that controls the first selector unit based on at least one of a type of arithmetic operation, an amount of arithmetic operation, a type of the first and second memory units, a capacity of the first and second memory units, and a type of the arithmetic circuit unit. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a conceptual diagram (part 1) for explaining an overview of an example of a deep neural network according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a conceptual diagram (part 2) for explaining an overview of an example of a deep neural network according to an embodiment of the present disclosure. [Diagram 3] FIG. 13 is a conceptual diagram (part 3) for explaining an overview of an example of a deep neural network according to an embodiment of the present disclosure. [Figure 4] 1 is an example of a functional block diagram of a calculation processing device 10 according to a first embodiment of the present disclosure. [Diagram 5] 2 is an example of a circuit block diagram of a calculation unit 120 according to the first embodiment of the present disclosure. [Figure 6]2 is an example of a circuit block diagram of a product adder 500 according to the first embodiment of the present disclosure. FIG. [Figure 7] 4 is a flowchart showing an example of a calculation processing method according to the first embodiment of the present disclosure. [Figure 8] 13 is an example of a circuit block diagram of a calculation unit 120a according to a second embodiment of the present disclosure. FIG. [Figure 9] 1 is a hardware configuration diagram illustrating an example of a computer 1000 that realizes the functions of a processor 10 according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] A preferred embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configuration are given the same reference numerals to avoid repeated explanation. In this specification and the drawings, similar components in different embodiments may be distinguished by adding different alphabets after the same reference numerals. However, when there is no particular need to distinguish between similar components, only the same reference numerals are given.

[0010] The explanation will be given in the following order. 1. How the embodiments of the present disclosure were created 1.1 Background 1.2 Background 1.3 Overview of the embodiments of the present disclosure 2. First embodiment 2.1. Example of Functional Configuration of the Processing Unit 10 2.2. Example of the configuration of the calculation unit 120 2.3. Example of the multiplier-accumulator 500 2.4. Calculation processing method 3. Second embodiment 4. Summary 5. Hardware Configuration 6. Supplementary Information

[0011] <<1. How the embodiments of the present disclosure were created>> <1.1 Background> First, before describing the details of the embodiment of the present disclosure, the background and process by which the present inventor created the embodiment of the present disclosure will be described. First, an overview of a fully connected layer type deep neural network, which is an example of a general deep neural network, will be described with reference to Fig. 1. Fig. 1 is a conceptual diagram for explaining the overview of an example of a deep neural network according to an embodiment of the present disclosure.

[0012] In detail, in the image of processing in the fully connected layer type deep neural network shown in Fig. 1, the fully connected layer type deep neural network has at least an input layer, a fully connected layer (middle layer), and an output layer, that is, performs hierarchical calculation processing. Note that in the fully connected layer type deep neural network, the fully connected layer is not limited to one, and multiple fully connected layers may be included in stages. In other words, the fully connected layer type deep neural network can have an input layer, multiple fully connected layers (middle layers), and an output layer.

[0013] In the fully connected deep neural network, multiple input data X1, X2, ..., X m Next, each input data X1, X2, ..., X m are weighted by different weighting coefficients and multiplied, so that each node Y1, Y2, ..., Y n-1 , Y n (Note that a weight matrix can be created by combining these multiple weight coefficients) (Affine operation). Similarly, each node Y1, Y2, ..., Y n-1 , Y n The values ​​of are weighted by different predetermined weighting coefficients and multiplied to become the output data Z of the output layer.

[0014] That is, in the example of the fully connected layer type deep neural network shown in Fig. 1, the calculation is repeatedly executed hierarchically so that the data propagates (unidirectionally) from the input layer through the fully connected layer to the output layer. Furthermore, in the fully connected layer type deep neural network, since all the nodes between the layers are connected as shown in Fig. 1, the calculation is repeatedly executed using the same data (same input data). In other words, in the fully connected layer type deep neural network (Affine calculation), the calculation is executed so that the data (input data) is reused.

[0015] Next, an overview of a convolutional deep neural network, which is an example of a general deep neural network, will be described with reference to Figures 2 and 3. Figures 2 and 3 are conceptual diagrams for explaining the overview of an example of a deep neural network according to an embodiment of the present disclosure.

[0016] In the above-mentioned fully connected deep neural network, data of each node in the same layer is handled independently of each other, in other words, the relationship between each node in the same layer is not taken into consideration when performing the calculation. In such a case, the accuracy of the output data obtained by the calculation may be poor depending on the characteristics of the input data handled (for example, when the input data is image data). Therefore, in order to perform the calculation while taking into consideration the relationship between each node in the same layer, a convolutional deep neural network has been devised, which performs convolution using a filter. The convolutional deep neural network will be described below by taking as an example a case where the convolutional deep neural network is applied to image processing (image recognition).

[0017] More specifically, in the image of processing in the convolutional deep neural network shown in Figure 2, features of the image data are extracted by applying a filter (shown on the left side of Figure 2) to the input image data (Input in Figure 2).

[0018] Therefore, referring to FIG. 3, a method of extracting features from image data using a filter will be described. Specifically, as shown in the left end of FIG. 3, the filter is divided into a matrix, and a weighting factor is assigned to each divided area in advance. First, as shown in the center of FIG. 3, the filter is applied to a predetermined area of ​​image data (Input in FIG. 3) (each feature amount in the predetermined area of ​​the image data is multiplied by the weighting factor of the filter at the corresponding position and integrated), to extract features in the predetermined area, and as shown in the right side of FIG. 3, the extracted features are assigned to a predetermined area of ​​output data (Output in FIG. 3). Furthermore, as shown in the center of FIG. 3, the filter is applied by sliding it sequentially on the image data, to extract features in each area of ​​the image data, and the extracted features are assigned to a predetermined area of ​​the output data. In this way, in a convolutional deep neural network, by using a filter, the feature amount of image data can be convoluted (aggregated) into output data while maintaining the positional relationship on the image data (Convolution operation).

[0019] Returning to Fig. 2, as shown in the upper part of Fig. 2, a filter is applied to the upper left region of the input image data to extract features of the upper left region, and the extracted features are generated as output data (Output in Fig. 2). As shown in Fig. 2, in the convolutional deep neural network, a filter is applied sequentially while sliding to each region of the image data, and the features of each region of the image data are repeatedly extracted. Furthermore, in the convolutional deep neural network, such extraction is repeatedly performed hierarchically.

[0020] That is, in the example of the convolutional deep neural network shown in Figures 2 and 3, a filter is used to perform calculations taking into account the vertical and horizontal positional relationship on the image data. Furthermore, in this example, calculations are performed using a repetitive filter (weighting coefficient). In other words, in the convolutional deep neural network (convolution calculation), calculations are performed so that coefficients are reused.

[0021] Furthermore, in this convolutional deep neural network, the extracted features are hierarchically combined and processed while maintaining the top, bottom, left, and right positional relationships on the image data, and therefore it can demonstrate excellent performance in processing such as image recognition.

[0022] In the embodiment of the present disclosure, the processing in the deep neural network may be performed in combination with the above-mentioned Affine operation (fully connected deep neural network) and Convolution operation (convolution deep neural network), and is not particularly limited.

[0023] <1.2 Background> In recent years, deep neural networks as described above have achieved high performance by increasing the amount of calculation and the coefficient capacity. However, for a processing device that performs such processing, even if the amount of calculation increases, it is required to suppress the increase in power consumption.

[0024] In detail, a fully connected deep neural network and a convolutional deep neural network, which are examples of deep neural networks, are realized by a combination of product-sum operations called Affine and Convolution operations, as described above. In order to execute such a large number of product-sum operations, many memory accesses are executed to read data used in the operations from a storage unit (memory), but in order to suppress an increase in power consumption of the arithmetic processing device, it is necessary to suppress an increase in the number of memory accesses.

[0025] As described above, in the Affine operation (fully connected deep neural network) and the Convolution operation (convolution deep neural network), the same input data or coefficients are repeatedly used for the operation. Therefore, if the input data or coefficients once read from the storage unit (memory) can be reused when the Affine operation and the Convolution operation are executed, the increase in the number of memory accesses can be suppressed.

[0026] Therefore, in the technology disclosed in Patent Document 1, input data and coefficients are read from a storage unit (memory) one row at a time, each of which is temporarily stored in a register unit, and the stored input data and coefficients are output to a calculation unit. Specifically, in the technology disclosed in Patent Document 1, when performing a convolution calculation, the coefficients stored in the register unit are slid to perform multiplication and accumulation on the first row of input data read from the storage unit (memory) to the register unit, and the intermediate results are stored in yet another register unit. Furthermore, in this technology, the multiplication and accumulation of the second row of input data and coefficients, the multiplication and accumulation of the third row of input data and coefficients, and so on are sequentially read from the register unit and the same calculation is repeatedly performed.

[0027] In addition, in the technology disclosed in the above Patent Document 1, a crossbar switch is provided between the register unit and the arithmetic unit, so that the input data and coefficients held in the register unit can be shared among multiple arithmetic units, thereby reducing the number of times the register unit is rewritten.

[0028] As described above, in the technology disclosed in Patent Document 1, the input data and coefficients read from the storage unit (memory) are temporarily stored in the register unit and reused, thereby suppressing an increase in the number of memory accesses. However, in the technology disclosed in Patent Document 1, three register units are provided for each calculation unit, so there is a concern that the circuit size will increase when there is a large amount of input data, coefficients, and output data for one row, or when the number of calculation units is increased to increase the number of parallel operations. In other words, in the technology disclosed in Patent Document 1, even if the increase in the number of memory accesses can be suppressed, it is expected that it will be difficult to suppress an increase in power consumption due to the increase in circuit size.

[0029] In view of the above-mentioned circumstances, the present inventor has created a processing device according to an embodiment of the present disclosure, which can further suppress an increase in power consumption of a processing device that performs calculations in a deep neural network.

[0030] <1.3 Overview of the embodiment of the present disclosure> Next, an overview of a processing device according to an embodiment of the present disclosure created by the present inventor will be described.

[0031] The arithmetic processing device according to the embodiment of the present disclosure has two storage units (memories) (first storage unit, second storage unit) that store data such as input data and coefficients (first data, second data), and a multiplier / accumulator (arithmetic circuit unit) that reads data from the storage units and repeatedly performs arithmetic operations using the read data. Furthermore, the arithmetic processing device has a register unit that reads data from the storage units and temporarily holds the data to be repeatedly used by the multiplier / accumulator, and a selector unit (first selector unit) that selects the data to be held in the register unit.

[0032] In the embodiment of the present disclosure, a selector unit provided before the register unit is controlled to select data (input data or coefficients) to be temporarily stored in the register unit so as to reduce the number of reads from the storage unit (the number of memory accesses). In the embodiment of the present disclosure, the input data or coefficients temporarily stored in the register unit are repeatedly reused by the multiply-accumulator.

[0033] Moreover, the arithmetic processing device according to the embodiment of the present disclosure may further include a storage unit (third storage unit) that stores intermediate results (partial sums) calculated by the product adder.

[0034] In addition, in the embodiment of the present disclosure, the multiplier-accumulator may have a plurality of multipliers, an adder, and an accumulator. Furthermore, in the embodiment of the present disclosure, the multiplier-accumulator may have a selector unit (second selector unit) that selects whether to use the accumulated result temporarily held in the accumulator or the intermediate result (partial sum) stored in the storage unit (third storage unit) for the next addition.

[0035] In an embodiment of the present disclosure, the selection operation of each of the selector units described above is controlled based on at least one of the type of operation (e.g., convolution operation, affine operation, etc.), the number of memory access bits determined from the amount of operation (e.g., kernel size, output size, address, number of maps, number of bits), the type of memory unit (e.g., DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), etc.), the capacity of the memory unit, and the type of multiplier / accumulator.

[0036] Specifically, when the type of operation to be executed is an Affine operation, a multiply-and-accumulate operation is executed between the input data and the coefficient, but in this multiply-and-accumulate operation, the same input data is used repeatedly as described above. Therefore, in the embodiment of the present disclosure, in the case of an Affine operation, the repeatedly used input data is temporarily stored in a register unit, thereby making it possible to suppress an increase in the number of memory accesses, and as a result, to further suppress an increase in power consumption.

[0037] Furthermore, when the type of operation to be executed is a convolution operation, a multiplication and accumulation operation between input data and a coefficient is executed, but in the multiplication and accumulation operation, as described above, the same input data and coefficient are repeatedly used. Therefore, in the case of a convolution operation, in the embodiment of the present disclosure, data that can be temporarily stored in a register unit and repeatedly used to minimize the number of times read from a storage unit (memory) (the number of memory accesses) is selected and temporarily stored in the register unit. As a result, in the embodiment of the present disclosure, an increase in the number of memory accesses can be suppressed, and an increase in power consumption can be further suppressed.

[0038] More specifically, when the type of calculation to be executed is a convolution calculation, in the embodiment of the present disclosure, data that can be temporarily stored in a register and repeatedly used to minimize the number of reads (number of memory accesses) is selected based on the amount of calculation (e.g., kernel size, output size, address, number of maps, number of bits), etc. According to the embodiment of the present disclosure, by doing so, it is possible to suppress an increase in the number of memory accesses, and as a result, it is possible to further suppress an increase in power consumption.

[0039] Furthermore, in the case where the type of operation to be executed is a convolution operation, in the embodiment of the present disclosure, data to be temporarily stored in the register unit is selected so that the capacity of the storage unit, which is determined based on the amount of operation (e.g., kernel size, output size, address, map number, bit number), is minimized. In this embodiment, for example, when the capacity of the storage unit is reduced by storing coefficients in the register unit, the coefficients are temporarily stored in the register unit. Also, in this embodiment, for example, when the capacity of the storage unit is reduced by storing input data in the register unit, the input data is temporarily stored in the register unit. According to the embodiment of the present disclosure, by doing so, it is possible to suppress an increase in the number of memory accesses and an increase in power consumption associated with memory access, and as a result, it is possible to further suppress an increase in power consumption.

[0040] Furthermore, in the case where the type of operation to be executed is a convolution operation, in the embodiment of the present disclosure, data to be temporarily stored in the register unit is selected so as to minimize the power consumed by accessing the storage unit. In this embodiment, for example, if the storage unit that stores the coefficients is a DRAM and the storage unit that stores the input data is an SRAM, the coefficients are temporarily stored in the register unit because the DRAM consumes more power by accessing it. According to the embodiment of the present disclosure, this can further suppress the increase in power consumption.

[0041] In other words, according to the embodiment of the present disclosure, it is possible to further suppress an increase in power consumption of a processing device that performs arithmetic processing in a deep neural network. Hereinafter, the details of the embodiment of the present disclosure created by the present inventor will be described in order.

[0042] <<2. First embodiment>> <2.1. Example of functional configuration of the arithmetic processing device 10> First, an example of the functional configuration of the arithmetic processing device 10 that realizes the arithmetic processing method according to the present embodiment will be described with reference to Fig. 4. Fig. 4 is an example of a functional block diagram of the arithmetic processing device 10 according to the present embodiment.

[0043] 4, the arithmetic processing device 10 according to the present embodiment mainly includes an input unit 110, a calculation unit 120, a storage unit 130, and an output unit 140. Each functional block of the arithmetic processing device 10 will be described below in order.

[0044] (Input unit 110) The input unit 110 has a function of detecting various input operations by an operator and receiving input data related to the input operations. For example, the input unit 110 can be realized by various devices for detecting input operations by an operator, such as various buttons, a keyboard, a touch panel, a mouse, and a switch.

[0045] (Computation unit 120) The calculation unit 120 has a function of performing calculations based on a plurality of input data and a plurality of coefficients corresponding to the input data, and calculating output data. Specifically, the calculation unit 120 according to this embodiment can perform calculations related to the deep neural network described above. The details of the calculation unit 120 will be described later.

[0046] (Storage unit 130) The storage unit 130 has a function of storing programs and the like used in each block of the arithmetic processing device 10.

[0047] (Output unit 140) The output unit 140 has a function of outputting various information to an operator. For example, the output unit 140 outputs output data which is a result of calculation by the calculation unit 120. In detail, the output unit 140 can be configured to include a display device which outputs visual information. Here, the above-mentioned display device can be, for example, a CRT (Cathode Ray Tube) display device, a Liquid Crystal Display (LCD) device, an OLED (Organic Light Emitting Diode) device, or the like.

[0048] An example of the functional configuration of the arithmetic processing device 10 according to the present embodiment has been described above. Note that the example of the functional configuration described above is merely an example, and the example of the functional configuration of the arithmetic processing device 10 according to the present embodiment is not limited to the above example. The arithmetic processing device 10 according to the present embodiment may further have a configuration other than that shown in FIG. 4. For example, the arithmetic processing device 10 may further have a communication unit or the like that performs information communication with an information processing terminal. That is, the design of the functional configuration of the arithmetic processing device 10 according to the present embodiment can be flexibly changed.

[0049] <2.2. Example of configuration of calculation unit 120> Next, the arithmetic unit 120 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is an example of a circuit block diagram of the arithmetic unit 120 according to this embodiment.

[0050] 5, the arithmetic unit 120 according to this embodiment mainly includes memory units 200, 202, and 204, selector units 300 and 302, a register unit 400 (REG in FIG. 5), a product-accumulator 500, and a control unit 600. Each circuit block of the arithmetic unit 120 will be described below in order.

[0051] (Storage section 200, 202) The storage unit (first storage unit) 200 can store, for example, coefficients (first data), and the storage unit (second storage unit) 202 can store, for example, input data (second data). In this embodiment, these storage units 200 and 202 may be volatile memories such as DRAM and SRAM, or non-volatile memories such as NAND. However, from the viewpoint of access speed, these storage units 200 and 202 are preferably volatile memories such as DRAM and SRAM. In this embodiment, the input data and coefficients stored in the storage units 200 and 202 are temporarily held in a register unit 400 described later, or directly read out by a product-adder 500 without being held in the register unit 400.

[0052] (Storage unit 204) The storage unit 204 can store intermediate results (partial sums) calculated by the product-adder 500. In this embodiment, the storage unit 204 may be a volatile memory such as DRAM or SRAM, or a non-volatile memory such as NAND, but from the viewpoint of access speed, a volatile memory such as DRAM or SRAM is preferable. In this embodiment, the intermediate results stored in the storage unit 204 are read out by the product-adder 500 and reused for calculation.

[0053] (Selector units 300, 302) The selector units 300 and 302 are configured with logic circuits and the like, and can select data to be temporarily held in the register unit 400 (described later) from, for example, the coefficients stored in the storage unit 200 or the input data stored in the storage unit 202, under the control of the control unit 600 (described later). For example, when a selection signal "0" is input from the control unit 600, the selector unit 300 outputs the coefficients stored in the storage unit 200 to the register unit 400, and the selector unit 302 outputs the input data stored in the storage unit 202 to the product-accumulator 500. On the other hand, when a selection signal "1" is input from the control unit 600, for example, the selector unit 300 outputs the input data stored in the storage unit 202 to the register unit 400, and the selector unit 302 outputs the coefficients stored in the storage unit 200 to the product-accumulator 500.

[0054] (Register unit 400) The register unit 400 can temporarily hold either the coefficients stored in the memory unit 200 or the input data stored in the memory unit 202, selected by the selector units 300 and 302. In this embodiment, the register unit 400 is configured by logic circuits (flip-flops, etc.) built into the CPU (Central Processing Unit) 1100 (see FIG. 9 ), and is a dedicated storage device with a smaller capacity but a faster operating speed than the above-mentioned storage units 200 and 202. Since the register unit 400 is a dedicated storage device, its circuit configuration is simplified and its power consumption is low. Therefore, in this embodiment, data to be repeatedly used in the product-adder 500 described later is temporarily held in the register unit 400 and repeatedly reused in the calculations of the product-adder 500 described later, thereby suppressing an increase in power consumption.

[0055] (Multiplier-accumulator 500) The product-accumulator 500 can read input data, coefficients, and other data from the storage units 200, 202 and the register unit 400, and repeatedly perform calculations using the read data. Furthermore, the product-accumulator 500 can output calculation results (intermediate results) to the storage units 202, 204, etc. In this embodiment, the product-accumulator 500 may be a digital circuit or an analog circuit depending on the calculation process, etc., and is not particularly limited. Details of the product-accumulator 500 will be described later.

[0056] (Control unit 600) The control unit 600 is composed of the CPU 1100 (see FIG. 9 ) and the like, and can control the selector units 300, 302 based on the type of calculation, the amount of calculation (e.g., kernel size, output size, address, number of maps, number of bits), the type and capacity of the storage units 200, 202, and the type of the product adder 500. The control unit 600 can also control the storage units 200, 202.

[0057] Specifically, in this embodiment, for example, when the type of calculation to be executed is an Affine calculation, the control unit 600 performs control so that repeatedly used input data is temporarily stored in the register unit 400 .

[0058] In addition, in this embodiment, in the case of convolution calculation, for example, the control unit 600 selects data that can reduce the number of reads (memory accesses) from the storage units 200 and 202 as much as possible by temporarily storing the data in the register unit 400 and repeatedly using the data, and controls the selected data to be stored in the register unit 400. Alternatively, in this embodiment, for example, the control unit 600 selects data that can reduce the number of reads (memory accesses) as much as possible by temporarily storing the data in the register unit 400 and repeatedly using the data based on the amount of calculation (for example, kernel size, output size, address, map number, bit number), and controls the selected data to be stored in the register unit 400. Alternatively, the control unit 600 selects data to be temporarily stored in the register unit 400 so that the capacity used by the storage units 200 and 202, which is determined based on the amount of calculation (for example, kernel size, output size, address, map number, bit number), is reduced as much as possible, and controls the selected data to be stored in the register unit 400. Furthermore, the control unit 600 selects data to be temporarily stored in the register unit 400 based on the type of the memory units 200, 202 (e.g., DRAM, SRAM, etc.) so as to minimize the power consumed by accessing the memory units 200, 202, and performs control so as to store the selected data in the register unit 400.

[0059] In this embodiment, by controlling the selector units 300 and 302 by the control unit 600 in this way, it is possible to suppress an increase in the number of memory accesses, and as a result, it is possible to further suppress an increase in power consumption. Furthermore, according to this embodiment, since many register units are not provided, the circuit scale does not increase, and as a result, it is possible to further suppress an increase in power consumption.

[0060] An example of the functional configuration of the calculation unit 120 according to the present embodiment has been described above. Note that the above-described example of the functional configuration is merely an example, and the example of the functional configuration of the calculation unit 120 according to the present embodiment is not limited to the above example. For example, the calculation unit 120 according to the present embodiment may further have a configuration other than that shown in FIG. 5.

[0061] <2.3. Example of the configuration of the multiplier-accumulator 500> Next, an example of the product adder 500 according to this embodiment will be described with reference to Fig. 6. Fig. 6 is an example of a circuit block diagram of the product adder 500 according to this embodiment.

[0062] 6, the product-accumulator 500 according to this embodiment mainly includes a plurality of multipliers 502, a selector unit 504, an adder 506, and an accumulator 508. In this embodiment, it is assumed that a plurality of product-accumulators 500 are provided in the arithmetic unit 120 according to the number of processing layers of the arithmetic processing in the deep neural network, for example. Each circuit block of the product-accumulator 500 will be described below in order.

[0063] (Multiplier 502) Each multiplier 502 is configured with a logic circuit or the like, and can read input data or coefficients stored in the storage units 200, 202 and input data or coefficients temporarily held in the register unit 400, and multiply them sequentially. Furthermore, each multiplier 502 can output the obtained calculation result to an adder 506, which will be described later.

[0064] (Selector unit 504) The selector unit 504 is composed of a logic circuit or the like, and can select data (third data) to be output to the adder 506 described later from among the intermediate result (partial sum) stored in the storage unit 204 and the accumulated result temporarily held in the accumulator 508 described later, according to control (selection signal) from the control unit 600. For example, when a selection signal "0" is input from the control unit 600, the selector unit 504 outputs the intermediate result (partial sum) stored in the storage unit 204 to the adder 506. On the other hand, when a selection signal "1" is input from the control unit 600, the selector unit 504 outputs the accumulated result temporarily held in the accumulator 508 to the adder 506. In this embodiment, the selector unit 504 may be controlled by the control unit 600 depending on the type of calculation, or may be controlled by the control unit 600 to further suppress an increase in power consumption, similar to the above-mentioned selector units 300 and 302.

[0065] (Adder 506) The adder 506 is configured by a logic circuit or the like, and can add the multiplication results from each multiplier 502 described above to either the intermediate result (partial sum) stored in the memory unit 204 and the accumulated result temporarily held in the accumulator 508, selected by the selector unit 504 described above. Furthermore, the adder 506 can output the addition result to the accumulator 508 and the memory units 202 and 204 described below.

[0066] (accumulator 508) The accumulator 508 temporarily holds the addition result from the adder 506 described above, and can output the held accumulation result to the storage units 202, 204, or output it to the adder 506 according to the selection of the selector unit 504. According to this embodiment, the adder 506 can perform calculations using the accumulation result held in the accumulator 508, so that it is possible to reduce the number of times data is read out to the storage units 200, 202, etc., and perform closed calculations within the product-sum unit 500. In addition, a reset signal for resetting the accumulated value to 0 may be input to the accumulator 508 as appropriate.

[0067] The accumulator 508 is configured by logic circuits (flip-flops, etc.) built into the CPU 1100 (see FIG. 9) in the same manner as the register unit 400, and is a dedicated storage device with a smaller capacity but faster operation speed than the above-mentioned storage units 200, 202, etc. Since the accumulator 508 is a dedicated storage device, the circuit configuration is simplified and the power consumption is small. Therefore, by using the accumulator 508, it is possible to suppress an increase in power consumption.

[0068] The above describes an example of the functional configuration of the product-accumulator 500 according to the present embodiment. Note that the above-described example of the functional configuration is merely an example, and the functional configuration of the product-accumulator 500 according to the present embodiment is not limited to the above example. For example, the product-accumulator 500 according to the present embodiment may further include configurations other than those shown in FIG. 6.

[0069] <2.4. Calculation processing method> The above is a detailed description of exemplary configurations of the arithmetic processing device 10, the arithmetic unit 120, and the product adder 500 according to this embodiment. Next, the arithmetic processing method according to this embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the arithmetic processing method according to this embodiment.

[0070] In detail, as shown in Fig. 7, the arithmetic processing method according to this embodiment includes a plurality of steps from step S101 to step S116. Each step included in the arithmetic processing method according to this embodiment will be described in detail below.

[0071] First, the control unit 600 sets (resets) a counter for a processing layer of arithmetic processing in the deep neural network to 0 (step S101). Next, the control unit 600 initializes counters for various processes (step S102).

[0072] Next, the control unit 600 generates a control signal and a selection signal according to the type of operation to be executed, and outputs them to each block of the operation unit 120 (step S103). In detail, the control unit 600 generates a selection signal "0" or a selection signal "1" based on the type of operation, the amount of operation (for example, kernel size, output size, address, map number, bit number), the type and capacity of the storage units 200 and 202, and the type of the multiply-adder 500, and outputs them to the selector units 300 and 302. In this embodiment, the control unit 600 selects data that can be temporarily stored in the register unit 400 and repeatedly used to minimize the number of reads (memory access number) from the storage units 200 and 202, and generates a selection signal to store the selected data in the register unit 400. In this embodiment, the control unit 600 generates a selection signal for controlling the selector units 300 and 302 in this way, thereby making it possible to suppress an increase in the number of memory accesses, and as a result, to further suppress an increase in power consumption.

[0073] When the type of calculation to be executed is a convolution calculation, as described above, the control unit 600 may select data to be temporarily stored in the register unit 400 so as to minimize the capacity used by the storage unit, which is determined based on the amount of calculation (e.g., kernel size, output size, address, number of maps, number of bits), etc. Also, the control unit 600 may select data to be temporarily stored in the register unit 400 so as to minimize the power consumed by accessing the storage units 200 and 202.

[0074] Furthermore, under the control of the control unit 600, the coefficients and input data are read out from the storage units 200 and 202 (step S104).

[0075] Here, the selector units 300 and 302 determine whether the selection signal from the control unit 600 is "0" (step S105). If the selector units 300 and 302 determine that the selection signal is "0" (step S105: Yes), the process proceeds to step S106, and if the selector units 300 and 302 determine that the selection signal is not "0" (step S105: No), the process proceeds to step S110.

[0076] Next, based on the selection signal "0" being input from the control unit 600, the selector unit 300 outputs the coefficients stored in the storage unit 200 to the register unit 400, and the selector unit 302 outputs the input data stored in the storage unit 202 to the product-accumulator 500. Therefore, the register unit 400 temporarily holds the coefficients (step S106).

[0077] Then, the product-accumulator 500 performs a product-accumulation operation using the coefficients temporarily stored in the register unit 400 and the input data read from the storage unit 202 (step S107). Next, the product-accumulator 500 reads the partial sum from the storage unit 204 and accumulates it with the product-accumulation result of the above-mentioned step S107 (step S108). Furthermore, the product-accumulator 500 outputs the accumulation result of the above-mentioned step S108 to the storage unit 204 (step S109).

[0078] Next, based on the selection signal "1" being input from the control unit 600, the selector unit 300 outputs the input data stored in the storage unit 202 to the register unit 400, and the selector unit 302 outputs the coefficients stored in the storage unit 200 to the product-accumulator 500. Therefore, the register unit 400 temporarily holds the input data (step S110).

[0079] Then, the product-accumulator 500 performs a product-sum operation using the input data temporarily held in the register unit 400 and the coefficient read from the storage unit 200 (step S111). Next, the product-accumulator 500 reads the accumulation result (partial sum) from the accumulator 508, and adds it to the product-sum result of step S111 described above (step S112). According to this embodiment, since the adder 506 can perform an operation using the accumulation result held in the accumulator 508, it is possible to reduce the number of times reading from the storage units 200, 202, etc., and perform a closed operation within the product-accumulator 500.

[0080] Furthermore, the product-sum calculator 500 outputs the result of the accumulation in step S112 described above to the accumulator 508 (step S113).

[0081] Next, the control unit 600 judges whether all output data for the target processing layer (target layer) has been generated (step S114). If the control unit 600 judges that all output data for the target layer has been generated (step S114: Yes), it proceeds to the process of step S115, and if the control unit 600 judges that all output data for the target layer has not been generated (step S114: No), it returns to the process of step S103.

[0082] Next, the control unit 600 judges whether the processed target layer is the final layer (step S115). If the control unit 600 judges that the processed target layer is the final layer (step S115: Yes), it ends the calculation process, and if the control unit 600 judges that the processed target layer is not the final layer (step S115: No), it proceeds to the process of step S116.

[0083] Next, the control unit 600 refers to the processing layer counter and determines whether the processed target layer is the final layer (step S115). If the control unit 600 determines that the processed target layer is the final layer (step S115: Yes), it ends the calculation process, and if the control unit 600 determines that the processed target layer is not the final layer (step S115: No), it proceeds to the process of step S116.

[0084] Furthermore, the control unit 600 increments the counter for the processing layer by 1 (step S116), and the process returns to step S103.

[0085] As described above, according to the present embodiment, it is possible to suppress an increase in power consumption of the arithmetic processing device 10 that performs arithmetic processing in a deep neural network.

[0086] In detail, in this embodiment, the selector units 300, 302 installed before the register unit 400 are controlled to select data (input data or coefficients) to be temporarily stored in the register unit 400 so as to reduce the number of reads (memory access count) from the storage units 200, 202. Therefore, according to this embodiment, it is possible to suppress an increase in the number of memory accesses, and as a result, it is possible to further suppress an increase in power consumption.

[0087] In addition, in this embodiment, the selector units 300 and 302 can share the data held in the register unit 400, so there is no need to provide many register units. Therefore, according to this embodiment, the circuit size does not increase, and as a result, the increase in power consumption can be further suppressed.

[0088] <<3. Second embodiment>> Incidentally, in the above-mentioned first embodiment, the arithmetic processing device 10 has been described as performing convolution calculations, affine calculations, and the like, but the arithmetic processing device 10 can also be applied to other calculations.

[0089] For example, as described above, in the convolutional deep neural network (convolution operation), the operation is performed by taking into consideration the relationship between each node in the same layer. However, in the convolutional deep neural network (convolution operation), the operation is to extract features by taking into consideration the local relationship by a filter and to convolute, and is not to take into consideration the relationship in the entire input data (for example, the entire image data). Therefore, in recent years, a method called squeeze-and-excitation network, which performs an operation by taking into consideration the relationship in the entire input data, has been considered (see Non-Patent Document 1). In the squeeze-and-excitation network, there is a phase (SE Block) in which the sum of products of input data is calculated. In detail, in the squeeze-and-excitation network, a scale factor that determines the degree of emphasis for each map of the input data is calculated from the input data, and the sum of products of the input data and the scale factor is further calculated.

[0090] Therefore, in the second embodiment, input data (scaling data) is stored in place of coefficients in the storage unit 200 of the calculation unit 120 according to the above-mentioned first embodiment, and the calculation unit 120 performs calculations, thereby making it possible to suppress an increase in the number of reads (memory access count) from the storage units 200, 202, and as a result, it is possible to further suppress an increase in power consumption. Such a second embodiment will be described below.

[0091] It should be noted that the configuration of the arithmetic processing device 10 according to this embodiment is similar to that of the arithmetic processing device 10 according to the above-described first embodiment, and therefore a description thereof will be omitted here.

[0092] The calculation unit 120a according to this embodiment will be described with reference to Fig. 8. Fig. 8 is an example of a circuit block diagram of the calculation unit 120a according to this embodiment. As shown in Fig. 8, the calculation unit 120a according to this embodiment includes storage units 200a, 202, and 204, selector units 300 and 302, a register unit 400 (REG in Fig. 8), a product-sum unit 500, and a control unit 600, similar to the first embodiment. However, unlike the first embodiment, the storage unit 200a stores at least a part of the input data (scaling data).

[0093] The configuration of the product-accumulator 500 according to this embodiment is similar to that of the product-accumulator 500 according to the first embodiment described above, and therefore a description thereof will be omitted here.

[0094] In this embodiment, for example, when a selection signal "0" is input from the control unit 600, the selector unit 300 outputs the input data (scaling data) stored in the storage unit 200 to the register unit 400, and the selector unit 302 outputs the input data stored in the storage unit 202 to the product-accumulator 500. Then, the product-accumulator 500 can calculate the product-accumulator of the input data (scaling data) and the input data.

[0095] In this embodiment as well, input data (scaling data) is stored in place of coefficients in the storage unit 200a of the calculation unit 120, and the stored input data (scaling data) is temporarily held in the register unit 400 for calculation, thereby suppressing an increase in the number of reads (memory access count) from the storage units 200, 202. As a result, according to this embodiment, an increase in power consumption can be further suppressed.

[0096] Furthermore, as can be seen from the above, the arithmetic processing device 10 according to the embodiment of the present disclosure is capable of handling various deep neural network calculations by controlling the selector units 300 and 302 according to the calculations.

[0097] <<4. Summary>> As described above, according to each of the above-described embodiments of the present disclosure, it is possible to suppress an increase in power consumption of the arithmetic processing device 10 that performs arithmetic processing in a deep neural network.

[0098] In detail, in each embodiment of the present disclosure, the selector units 300, 302 installed before the register unit 400 are controlled to select data (input data or coefficients) to be temporarily stored in the register unit 400 so as to reduce the number of reads (number of memory accesses) from the storage units 200, 202. Therefore, according to each embodiment of the present disclosure, it is possible to suppress an increase in the number of memory accesses, and as a result, it is possible to further suppress an increase in power consumption.

[0099] Furthermore, in each embodiment of the present disclosure, the selector units 300 and 302 can share the data held in the register unit 400, so there is no need to provide many register units. Therefore, according to each embodiment of the present disclosure, the circuit size does not increase, and as a result, the increase in power consumption can be further suppressed.

[0100] <<5. Hardware configuration>> The arithmetic processing device 10 according to each of the above-described embodiments is realized by a computer 1000 having a configuration as shown in FIG. 9, for example. The arithmetic processing device 10 of the present disclosure will be described below as an example. FIG. 9 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the arithmetic processing device 10. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. Each unit of the computer 1000 is connected by a bus 1050.

[0101] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processes corresponding to the various programs.

[0102] The ROM 1300 stores boot programs such as a basic input output system (BIOS) executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000.

[0103] The HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by the CPU 1100 and data used by the programs. Specifically, the HDD 1400 is a recording medium that records a computation program according to the present disclosure, which is an example of the program data 1450.

[0104] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0105] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading a program or the like recorded on a predetermined recording medium. The medium may be, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0106] For example, when the computer 1000 functions as the arithmetic processing device 10 according to an embodiment of the present disclosure, the CPU 1100 of the computer 1000 executes an arithmetic processing program loaded onto the RAM 1200 to realize functions of the arithmetic unit 120, etc. Also, the HDD 1400 stores the arithmetic processing program, etc. according to the embodiment of the present disclosure. Note that the CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, these programs may be obtained from other devices via an external network 1550.

[0107] Furthermore, the arithmetic processing device 10 according to the present embodiment may be applied to a system consisting of a plurality of devices, which is premised on connection to a network (or communication between devices), such as cloud computing, etc. In other words, the arithmetic processing device 10 according to the present embodiment described above can also be realized as an arithmetic processing system that performs the arithmetic processing according to the present embodiment using a plurality of devices, for example.

[0108] <<6. Supplementary Information>> The embodiment of the present disclosure described above may include, for example, a program for causing a computer to function as a processing device according to the present embodiment, and a non-transitory tangible medium on which the program is recorded. The program may also be distributed via a communication line (including wireless communication) such as the Internet.

[0109] In addition, each step in the computation process of each of the above-mentioned embodiments does not necessarily have to be processed in the order described. For example, each step may be processed in a different order as appropriate. Also, each step may be partially processed in parallel or individually instead of being processed in chronological order. Furthermore, the processing method of each step does not necessarily have to be processed in the manner described, and for example, each step may be processed in a different manner by another functional unit.

[0110] Although the preferred embodiment of the present disclosure has been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field of the present disclosure can conceive of various modified or amended examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure.

[0111] In addition, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that are apparent to a person skilled in the art from the description of this specification, in addition to or in place of the above effects.

[0112] The present technology can also be configured as follows. (1) an arithmetic circuit unit that reads out first and second data and repeatedly performs an arithmetic operation using the first and second data that have been read out; a first storage unit that stores the first data; a second storage unit that stores the second data; a register unit that reads out from the first storage unit or the second storage unit data that is repeatedly used in the arithmetic circuit unit and temporarily holds the data; a first selector unit that selects data to be held in the register unit from the first and second data; a control unit that controls the first selector unit based on at least one of a type of operation, an amount of operation, a type of the first and second storage units, a capacity of the first and second storage units, and a type of the arithmetic circuit unit; Equipped with Processing unit. (2) The processing device according to (1) above, wherein the amount of calculation includes at least one of a kernel size, an output size, an address, a map number, and a number of bits. (3) An input unit that accepts input data; an output unit that outputs a calculation result by the calculation circuit unit; The processing device according to claim 1 or 2, further comprising: (4) The arithmetic processing device according to (3) above, wherein the first data includes coefficient information and the second data includes the input data. (5) The arithmetic processing device according to (3) above, wherein the first and second data include at least a portion of the input data. (6) Further comprising a third storage unit for storing the partial sums calculated by the arithmetic circuit unit. The arithmetic processing device according to any one of (1) to (5) above. (7) The arithmetic circuit unit includes: a plurality of multipliers for multiplying the first and second data; an adder that adds the multiplication results of the multiple multipliers and third data; an accumulator for temporarily storing the sum of the adder; a second selector unit that selects either the accumulator or the third storage unit as an acquisition source of the third data; Has The second selector unit is controlled by the control unit. The processing device according to (6) above. (8) The arithmetic processing device according to any one of (1) to (7) above, wherein the first and second storage units are made of volatile memories. (9) The arithmetic processing device according to any one of (1) to (7) above, wherein the first and second storage units are made of non-volatile memories. (10) The arithmetic processing device according to any one of (1) to (9) above, wherein the arithmetic circuit section is formed of a digital circuit. (11) The arithmetic processing device according to any one of (1) to (9) above, wherein the arithmetic circuit section is formed of an analog circuit. [Explanation of symbols]

[0113] 10 Processing unit 110 Input section 120, 120a Calculation unit 130 Storage section 140 Output section 200, 200a, 202, 204 Storage section 300, 302, 504 Selector section 400 Register section 500 Multiplier and Accumulator 502 Multiplier 506 Adder 508 Accumulator 600 Control section

Claims

1. an arithmetic circuit unit that reads out first and second data and repeatedly performs an arithmetic operation using the first and second data that have been read out; a first storage unit that stores the first data; a second storage unit that stores the second data; a register unit that reads out from the first storage unit or the second storage unit data that is repeatedly used in the arithmetic circuit unit and temporarily holds the data; a first selector unit that selects data to be held in the register unit from the first and second data; a control unit that controls the first selector unit based on at least one of a type of operation and an amount of operation; a third storage unit for storing the partial sums calculated by the arithmetic circuit unit; Equipped with The arithmetic circuit unit includes: a plurality of multipliers for multiplying the first and second data; an adder that adds the multiplication results of the plurality of multipliers and third data; an accumulator for temporarily storing the sum of the adder; a second selector unit that selects either the accumulator or the third storage unit as an acquisition source of the third data; Has The second selector unit is controlled by the control unit. Processing unit.

2. An input unit that accepts input data; an output unit that outputs a calculation result by the calculation circuit unit; The processor of claim 1 further comprising:

3. The processor according to claim 2 , wherein the first data includes coefficient information and the second data includes the input data.

4. The processor according to claim 2 , wherein the first and second data include at least a portion of the input data.

5. 5. The processor according to claim 1, wherein the first and second storage units are made of volatile memories.

6. 5. The processor according to claim 1, wherein the first and second storage units are non-volatile memories.

7. 7. The arithmetic processing device according to claim 1, wherein the arithmetic circuit section is made up of a digital circuit.

8. 7. The arithmetic processing device according to claim 1, wherein the arithmetic circuit section is made up of an analog circuit.

Images