Semiconductor equipment

Abstract

To provide a semiconductor device in which a processing time of a neural network can be shortened.SOLUTION: A memory MEM2 has n banks BK[1] to BK[n] to which parallel access can be made, and stores pixel data Di / Do. An input DMA controller DMAC2i transfers the pixel data Di stored in the memory MEM2, to n product-sum computers MAC[1] to MAC[n] by using n input channels CHi[1] to CHi[n] respectively. A sequence controller 21 controls the input DMA controller DMAC2i to transfer pixel data of a first pixel space in an input bank to a first product-sum computer through a first input channel and to transfer pixel data of a second pixel space in the same input bank to a second product-sum computer through a second input channel.SELECTED DRAWING: Figure 2

Description

Technical Field

[0004] , , , ,

[0005] , , , ,

[0001] The present invention relates to a semiconductor device, for example, a semiconductor device that executes neural network processing.

Background Art

[0002] <000000⑨>Patent Document 1 discloses a method of performing operations on a plurality of intermediate layers constituting a convolutional neural network using a memory having a plurality of banks. In this method, for each of the plurality of intermediate layers, based on the transfer amount and transfer speed required for the input data and output data, the assignment of the bank serving as the storage source of the input data or the storage destination of the output data is appropriately determined.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For example, in neural network processing such as CNN (Convolutional Neural Network), enormous arithmetic processing is executed using a plurality of DMA (Direct Memory Access) controllers and a plurality of multiply-accumulate units mounted on a semiconductor device. Specifically, the DMA controller causes the multiply-accumulate unit to perform a multiply-accumulate operation by transferring, for example, pixel data and weight parameters of a certain convolutional layer stored in the memory to the multiply-accumulate unit. Further, the DMA controller transfers the operation result by the multiply-accumulate unit to the memory as pixel data of the next convolutional layer. The semiconductor device repeatedly executes such processing.

[0005] Such semiconductor devices often operate based on simple commands in order to flexibly accommodate diverse neural network configurations. For example, as shown in Patent Document 1, a command is used to specify the memory bank from which input data is stored and the memory bank from which output data is stored. However, when multiple multiply-accumulate units perform multiply-accumulate operations based on such commands, it is not always possible to efficiently utilize multiple multiply-accumulate units. As a result, the processing time of the neural network may increase more than expected.

[0006] The embodiments described later were made in view of these considerations, and other issues and novel features will become apparent from the description and accompanying drawings of this specification. [Means for solving the problem]

[0007] One embodiment of the semiconductor device includes a memory, n multiply-accumulate units, an input DMA controller, an output DMA controller, and a sequence controller, where n is an integer of 2 or more. The memory has n banks that can be accessed in parallel and stores pixel data. Each of the n multiply-accumulate units performs a multiply-accumulate operation on multiple pixel data and multiple weight parameters. The input DMA controller transfers the pixel data stored in the memory to the n multiply-accumulate units using n input channels. The output DMA controller transfers the pixel data from the n multiply-accumulate units to the memory using n output channels. The sequence controller controls the input DMA controller so that it transfers pixel data in the first pixel space in the input bank to the first multiply-accumulate unit via the first input channel, and transfers pixel data in the second pixel space in the same input bank to the second multiply-accumulate unit via the second input channel. [Effects of the Invention]

[0008] By using a semiconductor device according to one embodiment, it becomes possible to shorten the processing time of the neural network. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a schematic diagram showing an example of the configuration of the main parts of a semiconductor device according to Embodiment 1. [Figure 2] Figure 2 shows a detailed example of the neural network engine configuration in Figure 1. [Figure 3] Figure 3 shows a schematic configuration and operation example of the neural network engine in Figures 1 and 2. [Figure 4] Figure 4 shows a schematic configuration and operation example of the neural network engine in Figures 1 and 2, which differs from the case in Figure 3. [Figure 5] Figure 5 is a timing chart showing an example of comparing the operation of the neural network engine shown in Figure 4 and the neural network engine shown in Figure 9. [Figure 6] Figure 6 shows a detailed configuration example of the neural network engine in Figure 1 in the semiconductor device according to Embodiment 2. [Figure 7] Figure 7 is a schematic diagram showing an example of the configuration of the operation setting table in Figure 6. [Figure 8] Figure 8 is a flowchart showing an example of the operation of the sequence controller in Figure 6. [Figure 9] Figure 9 shows a schematic configuration and operation example of a neural network engine used as a comparative example. [Modes for carrying out the invention]

[0010] In the following embodiments, when necessary for convenience, the description will be divided into multiple sections or embodiments. Unless otherwise specified, these are not unrelated, and one may be a modification, detail, or supplementary explanation of part or all of the other. Furthermore, in the following embodiments, when referring to the number of elements (including number, numerical value, quantity, range, etc.), unless otherwise specified or clearly limited to a specific number in principle, it is not limited to that specific number, and may be greater than or less than that number.

[0011] Furthermore, in the following embodiments, it goes without saying that the constituent elements (including element steps, etc.) are not necessarily essential unless specifically stated or considered to be clearly essential in principle. Similarly, in the following embodiments, when referring to the shape, positional relationship, etc., of constituent elements, etc., it shall include those that substantially approximate or resemble such shapes, etc., unless specifically stated or considered to be not in principle. The same applies to the numerical values ​​and ranges mentioned above.

[0012] The embodiments will be described in detail below with reference to the drawings. In all the drawings used to describe the embodiments, the same reference numerals are used for members having the same function, and repeated descriptions of them will be omitted. In addition, in the following embodiments, descriptions of the same or similar parts will not be repeated unless it is particularly necessary.

[0013] (Embodiment 1) <Overview of Semiconductor Device> Figure 1 is a schematic diagram showing an example of the main components of a semiconductor device according to Embodiment 1. The semiconductor device 10 shown in Figure 1 is, for example, a System on Chip (SoC) composed of a single semiconductor chip. Typically, this semiconductor device 10 is mounted in an Electronic Control Unit (ECU) of a vehicle and provides ADAS (Advanced Driver Assistance System) functions. The semiconductor device 10 comprises a system bus 15, a neural network engine 16, a processor 17, memories MEM1 and MEM2, and a memory bus 18.

[0014] The processor 17 is a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), etc. The system bus 15 connects the neural network engine 16, memory MEM1 and MEM2, and the processor 17 to each other. The neural network engine 16 performs processing of neural networks, such as CNNs. The processor 17, for example, executes a predetermined program stored in memory MEM1, thereby causing the semiconductor device 10 to perform predetermined functions, including the control of the neural network engine 16.

[0015] Memory MEM1 is a DRAM (Dynamic Random Access Memory), and memory MEM2 is a cache SRAM (Static Random Access Memory). Memory MEM1 stores, for example, image data DT consisting of multiple pixel data, command CMD1, and parameter PR. Parameter PR includes a kernel KR consisting of multiple weight parameters and a bias parameter BP. Command CMD1 is used to control the operation of the neural network engine 16.

[0016] Memory MEM2 is used as the high-speed cache memory of the neural network engine 16. Memory MEM2 has n banks BK[1] to BK[n] that can be accessed in parallel, where n is an integer greater than or equal to 2. Memory MEM2 stores pixel data Di to the neural network engine 16 and pixel data Do from the neural network engine 16.

[0017] For example, the image data DT in memory MEM1, that is, the pixel data, is first copied to predetermined banks BK[1] to BK[n] in memory MEM2 via the system bus 15 and then used by the neural network engine 16. The memory bus 18 connects the neural network engine 16 and memory MEM2. The memory bus 18 is, for example, an AXI (Advanced eXtensible Interface) bus or the like that enables parallel access to the n banks BK[1] to BK[n] by means of a crossbar switch or the like.

[0018] The neural network engine 16 includes a plurality of DMA controllers DMAC1, DMAC2, a MAC unit 20, and a sequence controller 21. The MAC unit 20 includes n multiply-accumulate units, details of which will be described later. The DMA controller DMAC1 controls data transfer between the memory MEM1 and the MAC unit 20 via the system bus 15. Specifically, the DMA controller DMAC1 transfers the kernel KR, that is, the weight parameters and the like, stored in the memory MEM1 to the MAC unit 20. Also, the DMA controller DMAC1 transfers the command CMD1 stored in the memory MEM1 to the sequence controller 21.

[0019] The DMA controller DMAC2 controls data transfer between memory MEM2 and MAC unit 20 via the memory bus 18. Specifically, the DMA controller DMAC2 transfers the pixel data Di stored in memory MEM2 to MAC unit 20 using n channels CH[1] to CH[n]. As a result, MAC unit 20 performs a multiply-accumulate operation between the kernel KR from DMA controller DMAC1 and the pixel data Di from DMA controller DMAC2. Furthermore, although details are omitted, MAC unit 20 may also perform operations such as adding a bias parameter BP to the multiply-accumulate operation result and calculating an activation function on the added result.

[0020] Through this calculation, the MAC unit 20 generates pixel data Do, which will be the feature data for each feature in the feature map. The DMA controller DMAC2 transfers the pixel data Do from the MAC unit 20 to the memory MEM2 using n channels CH[1] to CH[n]. The pixel data Do written to MEM2 will then become the pixel data Di that will be input to the MAC unit 20 in the next convolutional layer.

[0021] The sequence controller 21 controls the operation sequence of the neural network engine 16, etc., based on command CMD1 from the DMA controller DMAC1 or command CMD2 input from the processor 17 via the system bus 15. As one such control, the sequence controller 21 controls the DMA controller DMAC2.

[0022] <Details of the neural network engine> Figure 2 shows a detailed configuration example of the neural network engine in Figure 1. In Figure 2, the MAC unit 20 is equipped with n multiply-accumulate units MAC[1] to MAC[n]. The DMA controller DMAC1 reads information including command CMD1 and kernel KR from memory MEM1. The DMA controller DMAC1 transfers the read kernel KR to at least one of the n multiply-accumulate units MAC[1] to MAC[n] and writes the read command CMD1 to register REG. At this time, the DMA controller DMAC1 is set with the address range of the source memory MEM1 and the destination multiply-accumulate units MAC[1] to MAC[n], specifically the addresses of each input register.

[0023] The DMA controller DMAC2 shown in Figure 1, in more detail as shown in Figure 2, has an input DMA controller DMAC2i and an output DMA controller DMAC2o. The input DMA controller DMAC2i has n input channels CHi[1] to CHi[n], and the output DMA controller DMAC2o has n output channels CHo[1] to CHo[n].

[0024] The input DMA controller DMAC2i transfers the pixel data Di stored in memory MEM2 to n multiply-accumulate units MAC[1] to MAC[n] using n input channels CHi[1] to CHi[n]. That is, when m is an integer from 1 to n, the destination of input channel CHi[m] is determined to be the multiply-accumulate unit MAC[m], specifically its input register. The multiply-accumulate unit MAC[m] performs a multiply-accumulate operation on the multiple pixel data Di transferred using input channel CHi[m] and the kernel KR transferred using DMA controller DMAC1, specifically the multiple weight parameters that make up the kernel KR.

[0025] The output DMA controller DMAC2o transfers pixel data Do from n multiply-accumulate units MAC[1] to MAC[n] to memory MEM2 using n output channels CHo[1] to CHo[n]. That is, as with the input DMA controller DMAC2i, the source of the transfer for output channel CHo[m] is determined by the multiply-accumulate unit MAC[m], specifically its output register.

[0026] The sequence controller 21 controls the input DMA controller DMAC2i and the output DMA controller DMAC2o based on the command CMD1 written to the register REG, or the command CMD2 from the processor 17. Commands CMD1 and CMD2 specify, for example, the input bank (referred to as IBK) from which the input pixel data Di will be transferred, and the output bank (referred to as OBK) to which the output pixel data Do will be transferred.

[0027] Based on the commands CMD1 and CMD2, the sequence controller 21 sets the source of input channels CHi[1] to CHi[n], specifically input bank IBK and the address range within input bank IBK, using the setting signals ST-CHi[1] to ST-CHi[n]. Similarly, based on the commands CMD1 and CMD2, the sequence controller 21 sets the destination of output channels CHo[1] to CHo[n], specifically output bank OBK and the address range within output bank OBK, using the setting signals ST-CHo[1] to ST-CHo[n].

[0028] <Configuration and operation of a neural network engine (comparative example)> Figure 9 shows a schematic configuration and operation example of a neural network engine used as a comparative example. In the neural network engine used as a comparative example, as shown in Figure 9, once the bank BK[m], which is the input bank IBK of the pixel data Di, is determined, the input channel CHi[m], the multiply-accumulate unit MAC[m], and the output channel CHo[m] are uniquely determined. However, the input bank IBK and the output bank OBK must be different. For this reason, the output channel CHo[m] can arbitrarily select the output bank OBK, i.e., the destination bank, via the memory bus 18.

[0029] In the example shown in Figure 9, with k being an integer from 1 to n excluding m, the output bank OBK is defined as bank BK[k]. Furthermore, as described in Patent Document 1, if the amount of pixel data Do is large, it is also possible to specify two banks, BK[k] and BK[q], as the output bank OBK. In this case, the output channel CHo[m] writes the pixel data Do to bank BK[k], and then writes any remaining pixel data Do to bank BK[q].

[0030] Furthermore, if a high data transfer rate is required in the next convolutional layer, it is possible to specify two banks, BK[k] and BK[q], as the output bank OBK. In this case, the output channel CHo[m] writes, for example, pixel data Do, distributed across the two banks BK[k] and BK[q]. In the next convolutional layer, the two multiply-accumulate units MAC[k] and MAC[q] perform multiply-accumulate operations in parallel, using the two banks BK[k] and BK[q] as input bank IBK, respectively.

[0031] Figure 9 also shows an example of the specific processing content when a 3x3 kernel KR is used in a certain convolutional layer. The kernel KR consists of nine weight parameters W1 to W9. The input bank IBK, bank BK[m], in this example stores 6x6 pixel data Di11 to Di16, Di21 to Di26, ..., Di61 to Di66. Here, in the pixel data Dixy, "x" represents the row number and "y" represents the column number.

[0032] The input channel CHi[m] reads, for example, the pixel data Di11~Di13, Di21~Di23, and Di31~Di33 contained in the 3x3 pixel space AR22 in bank BK[m], in other words, the address space, and transfers it to the multiply-accumulate unit MAC[m]. The pixel space AR22 is a 3x3 pixel space centered on the pixel data Di22. The multiply-accumulate unit MAC[m] performs a multiply-accumulate operation on the nine pixel data Di contained in the pixel space AR22 and the nine weight parameters W1~W9 that constitute the kernel KR. The output channel CHo[m] writes the pixel data Do22, which is the result of the multiply-accumulate operation, to a predetermined location, i.e., a predetermined address, in bank BK[k], which is the output bank OBK.

[0033] Next, the input channel CHi[m] reads the pixel data Di12~Di14, Di22~Di24, and Di32~Di34 contained in the pixel space AR23 in bank BK[m] and transfers them to the multiply-accumulate unit MAC[m]. The pixel space AR23 is a space obtained by sliding the pixel space AR23 once in the column direction, and is a 3x3 pixel space centered on the pixel data Di23. The multiply-accumulate unit MAC[m] performs a multiply-accumulate operation on the pixel data Di contained in the pixel space AR23 and the kernel KR. The output channel CHo[m] writes the pixel data Do23, which is the result of the multiply-accumulate operation, to an adjacent address in bank BK[k], where it is adjacent to the pixel data Do22 in the column direction.

[0034] Subsequently, the same process is performed on pixel space AR24, which is the space obtained by sliding pixel space AR23 once, and then on pixel space AR25, which is the space obtained by sliding pixel space AR23 once. As a result, pixel data Do22 to Do25 are arranged in order in the column direction.

[0035] When the convolution operation described above is performed on the 6x6 pixel data Di in the input bank IBK, which is bank BK[m], the 6x6 pixel data Do is written to the output bank OBK, which is bank BK[k], in the arrangement shown in Figure 9. That is, the 6x6 pixel data Do11~Do16,Do21~Do26,...,Do61~Do66 in the output bank OBK are arranged to match the 6x6 pixel data Di in the input bank.

[0036] In this example, the 6x6 pixel data Do in the output bank OBK is generated using padding. For example, when generating pixel data Do11, a 3x3 pixel space centered on pixel data Di11 is defined in the input bank IBK, and any non-existent pixel data Di is padded with zeros. Then, in the next convolutional layer, for example, the pixel data Do from bank BK[k] is used as pixel data Di, that is, bank BK[k] is used as the input bank IBK, and the same processing as in Figure 9 is performed.

[0037] However, in this method, only one of the n input channels CHi[m], the multiply-accumulate unit MAC[m], and the output channel CHo[m] are active, which can reduce the utilization efficiency of the neural network engine 16. As a result, the processing time of the neural network may increase.

[0038] <Configuration and operation of the neural network engine (Embodiment 1)> Figure 3 shows a schematic configuration and operation example of the neural network engine in Figures 1 and 2. In the example in Figure 3, the sequence controller 21 causes the input channel CHi[m] to transfer pixel data Di in, for example, pixel space AR23 in bank BK[m], which is input bank IBK, to the multiply-accumulate unit MAC[m]. Furthermore, in parallel with this, unlike in Figure 9, the sequence controller 21 causes another input channel CHi[j] to transfer pixel data Di in the next pixel space AR24 in bank BK[m] to the multiply-accumulate unit MAC[j]. Note that j is an integer from 1 to n, excluding m.

[0039] The multiply-accumulate unit MAC[m] performs a multiply-accumulate operation on the pixel data Di12~Di14, Di22~Di24, Di32~Di34 in pixel space AR23 from the input channel CHi[m] and the weight parameters W1~W9 that constitute the kernel KR. In parallel with this, unlike in the case of Figure 9, the multiply-accumulate unit MAC[j] performs a multiply-accumulate operation on the pixel data Di13~Di15, Di23~Di25, Di33~Di35 in pixel space AR24 from the input channel CHi[j] and the weight parameters W1~W9 that constitute the kernel KR.

[0040] Then, the sequence controller 21 causes the output channel CHo[m] to transfer pixel data Do23, which is for pixel space AR23, from the multiply-accumulate unit MAC[m] to bank BK[k], which is output bank OBK. Furthermore, in parallel with this, unlike in the case of Figure 9, the sequence controller 21 causes the output channel CHo[j] to transfer pixel data Do24, which is for pixel space AR24, from the multiply-accumulate unit MAC[j] to bank BK[q], which is another output bank OBK.

[0041] Thus, the sequence controller 21 controls the input DMA controller DMAC2i so that, each time the pixel space is slid, the pixel data Di in the pixel space is distributed and transferred to multiple multiply-accumulate units, in this case two, MAC[m] and MAC[j]. Specifically, the sequence controller 21 uses the setting signal ST-CHi[m] to set the input channel CHi[m] to sequentially transfer data from pixel space AR11 (not shown) to pixel space AR65 (not shown) in bank BK[m] with a step width of one space skipped. Similarly, the sequence controller 21 uses the setting signal ST-CHi[j] to set the input channel CHi[j] to sequentially transfer data from pixel space AR12 (not shown) to pixel space AR66 (not shown) with a step width of one space skipped.

[0042] This makes it possible to operate multiple input channels, in this case two, CHi[m], CHi[j], multiply-accumulate units MAC[m], MAC[j], and output channels CHo[m], CHo[j] in parallel. As a result, the utilization efficiency of the neural network engine 16 can be increased, and the processing time of the neural network can be reduced.

[0043] Figure 4 shows a schematic configuration and operation example of the neural network engine in Figures 1 and 2, different from that in Figure 3. In Figure 3, the next convolutional layer needs to perform a sum-of-accumulate operation using, for example, the pixel space consisting of pixel data Do11~Do13, Do21~Do23, and Do31~Do33. However, in the method shown in Figure 3, the pixel data Do11~Do13, Do21~Do23, and Do31~Do33 are distributed and written to two output banks BK[k] and BK[q], which are output bank OBK. Therefore, data rearrangement or the like may be necessary prior to processing in the next convolutional layer.

[0044] Therefore, in Figure 4, the sequence controller 21, as in Figure 3, causes the output channel CHo[m] to transfer pixel data Do23, which targets pixel space AR23, from the multiply-accumulate unit MAC[m] to bank BK[k], which is output bank OBK. In parallel with this, unlike in Figure 3, the sequence controller 21 causes the output channel CHo[j] to transfer pixel data Do24, which targets pixel space AR24, from the multiply-accumulate unit MAC[j] to bank BK[k] instead of bank BK[q].

[0045] In this process, the sequence controller 21 controls the output DMA controller DMAC2o so that the pixel data Do23 from output channel CHo[m] and the pixel data Do24 from output channel CHo[j] are located adjacent to each other in the address space of bank BK[k]. As a specific example, let the first address of bank BK[k] be #A[1] and the last address be #A[x].

[0046] In this case, the sequence controller 21 uses the setting signal ST-CHo[m] to configure the output channel CHo[m] to sequentially transfer data from #A[1] to #A[x-1] with a step width of one every other. The sequence controller 21 also uses the setting signal ST-CHo[j] to configure the output channel CHo[j] to sequentially transfer data from #A[2] to #A[x] with a step width of one every other.

[0047] As a result, as shown in Figure 4, pixel data Do11~Do16,...,Do61~Do66 can be written to the output bank OBK, which is bank BK[k], in the same arrangement as in the input bank IBK, which is bank BK[m]. This eliminates the need for data rearrangement, and as a result, the processing time of the neural network can be further reduced.

[0048] Figure 5 is a timing chart showing an example of comparing the operation of the neural network engine shown in Figure 4 and the neural network engine shown in Figure 9. As shown in Figure 5, in the method shown in Figure 9, a series of processing cycles Tc2 using one input channel CHi[m], a multiply-accumulate unit MAC[m], and an output channel CHo[m] are repeatedly executed. On the other hand, in the method shown in Figure 4, a series of processing cycles Tc1 using two input channels CHi[m], CHi[j], multiply-accumulate units MAC[m], MAC[j], and output channels CHo[m], CHo[j] are repeatedly executed.

[0049] As a result, by using the method shown in Figure 4, it is possible to reduce the processing time of the neural network compared to using the method shown in Figure 9, ideally by about half. Here, we have shown a method that distributes the processing across two multiply-accumulate units MAC[m] and MAC[j], but it is also possible to distribute the processing across three or more multiply-accumulate units in a similar manner.

[0050] <Main effects of Embodiment 1> As described above, in the method of Embodiment 1, the processing time of the neural network can be shortened by controlling the input DMA controller DMAC2i so that each pixel space in one input bank IBK is processed in a distributed manner by multiple multiply-accumulate units. Furthermore, the processing time of the neural network can be shortened even further by controlling the output DMA controller DMAC2o so that each pixel data from multiple multiply-accumulate units is arranged adjacently within one output bank OBK.

[0051] (Embodiment 2) <Details of the neural network engine> Figure 6 shows a detailed configuration example of the neural network engine in Figure 1 in the semiconductor device according to Embodiment 2. The neural network engine 16 shown in Figure 6 differs slightly from the configuration example shown in Figure 2 in the configuration and operation of the sequence controller 21a. The sequence controller 21a receives command CMD1 from register REG or command CMD2 from processor 17, as in Figure 2.

[0052] Commands CMD1 and CMD2 include identifiers for one or more input banks IBK and identifiers for one or more output banks OBK. The sequence controller 21a determines one or more input channels based on the identifiers of the one or more input banks IBK. Then, the sequence controller 21a determines one or more input channels to be used for distribution from among the input channels excluding the one or more input channels determined.

[0053] As a concrete example, consider a case where commands CMD1 and CMD2 instruct the system to use bank BK[1] as input bank IBK and bank BK[2] as output bank OBK, and to use bank BK[3] as input bank IBK and bank BK[4] as output bank OBK. In this case, the sequence controller 21a uniquely determines two input channels CHi[1] and CHi[3] based on the identifiers of the two banks BK[1] and BK[3], which are identifiers of input bank IBK. Then, the sequence controller 21a determines two input channels for distribution from the input channels excluding these two input channels CHi[1] and CHi[3], for example, CHi[2] and CHi[4].

[0054] Furthermore, the sequence controller 21a receives an enable signal MD-EN, which determines whether the distributed mode, an operating mode as shown in Figure 4, is enabled or disabled. Here, for example, we assume that a command is input to set bank BK[m] as the input bank IBK and bank BK[k] as the output bank OBK. If the distributed mode is enabled, the sequence controller 21a performs the operation shown in Figure 4, and if the distributed mode is disabled, it performs the operation shown in Figure 9. Also, if the distributed mode is enabled, the sequence controller 21a determines the settings for the input DMA controller DMAC2i and the output DMA controller DMAC2o based on the operation setting table 23 that is stored in advance.

[0055] Figure 7 is a schematic diagram showing an example of the configuration of the operation setting table in Figure 6. The operation setting table 23 shown in Figure 7 has pre-registered input commands and the operation setting contents corresponding to those input commands. For example, when the sequence controller 21a receives a command that sets bank BK[m] as the input bank IBK and bank BK[k] as the output bank OBK, it outputs setting signals ST-CHi[m],ST-CHi[j] to the input channels and setting signals ST-CHo[m],ST-CHo[j] to the output channels based on the operation setting table 23.

[0056] Figure 8 is a flowchart showing an example of the operation of the sequence controller in Figure 6. In Figure 8, the sequence controller 21a inputs commands CMD1 and CMD2 (step S101). Specifically, the sequence controller 21a inputs a command that sets bank BK[m] as the input bank IBK and bank BK[k] as the output bank OBK. Subsequently, the sequence controller 21a determines whether the distributed mode is enabled or disabled based on the enable signal MD-EN (step S102).

[0057] If the distributed mode is disabled in step S102 ("No"), the sequence controller 21a sets up normal operation (step S107). That is, as shown in Figure 9, the sequence controller 21a causes the input channel CHi[m] to transfer, for example, the pixel data Di in pixel space AR23 and the pixel data Di in the next pixel space AR24 in bank BK[m], which is input bank IBK, to the same multiply-accumulate unit MAC[m]. The sequence controller 21a also causes the output channel CHo[m] to transfer the pixel data Do23 and Do24 from the multiply-accumulate unit MAC[m] to bank BK[k], which is output bank OBK.

[0058] If distributed mode is enabled in step S102 ("Yes"), the sequence controller 21a determines whether the command is registered in the operation setting table 23 by referring to the input command as the key (step S103) (step S104). If the command is not registered in the operation setting table 23 in step S104 ("No"), the sequence controller 21a sets the normal operation (step S107).

[0059] If the command is registered in the operation setting table 23 in step S104 ("Yes"), the sequence controller 21a determines the available channels in the input DMA controller DMAC2i and, consequently, the output DMA controller DMAC2o (step S105). That is, the sequence controller 21a considers all input channels except for the input channel CHi[m] which is uniquely determined based on the input bank IBK, for example, bank BK[m], as available, and determines the input channel CHi[j] for distribution from among the available input channels. Once the input channels are determined, the multiply-accumulate unit and output channels are also uniquely determined.

[0060] Then, the sequence controller 21a outputs setting signals to each input channel CHi[m], CHi[j] and each output channel CHo[m], CHo[j] based on the operation setting table 23 (step S106). In the example in Figure 7, a command specifying one input bank IBK and one output bank OBK was registered, but it is also possible to apply distributed mode to a command by registering a command specifying, for example, one input bank IBK and two output banks OBK, along with the corresponding operation setting.

[0061] <Main effects of Embodiment 2> As described above, the same effects as those described in Embodiment 1 can be obtained by using the method of Embodiment 2. Furthermore, it becomes possible to make the neural network engine 16 operate in distributed mode without providing special commands for distributed mode to the processor 17, etc. Moreover, it becomes possible to use distributed mode selectively, for example, by disabling it when power consumption is to be reduced and enabling it when processing time is to be shortened.

[0062] Although the present invention has been specifically described above based on embodiments, it goes without saying that the present invention is not limited to the above embodiments and can be modified in various ways without departing from its essence. [Explanation of Symbols]

[0063] 10 Semiconductor Devices 15 System bus 16 Neural Network Engines 17 Processors 18 memory bus 20 MAC units 21,21a Sequence controller AR pixel space BK Bank CHi Input Channel CHo Output Channel CMD command DMAC1, DMAC2 DMA controllers Di,Do Pixel data IBK Input Bank MAC (Multiply-Accumulate-Add-Accumulate Unit) MD-EN Enable Signal MEM1, MEM2 memory OBK Output Bank W weight parameter

Claims

1. A memory that stores pixel data, having n banks that can be accessed in parallel, where n is an integer of 2 or more. n multiply-accumulate units, each performing a multiply-accumulate operation on multiple pixel data and multiple weight parameters, An input DMA (Direct Memory Access) controller that transfers the pixel data stored in the memory to the n multiply-accumulate units using n input channels, An output DMA controller that transfers pixel data from the n multiply-accumulate units to the memory using n output channels, A sequence controller that controls the input DMA controller and the output DMA controller, Equipped with, If any one of the n banks is designated as an input bank, any two of the n input channels are designated as a first input channel and a second input channel, and the sum-accumulate units that are the transfer destinations for the first input channel and the second input channel are designated as a first sum-accumulate unit and a second sum-accumulate unit, The sequence controller controls the input DMA controller such that it causes the first input channel to transfer pixel data in the first pixel space in the input bank to the first multiply-accumulate unit, and the second input channel to transfer pixel data in the second pixel space in the input bank to the second multiply-accumulate unit. Semiconductor equipment.

2. In the semiconductor device according to claim 1, If any one of the n banks is designated as an output bank, and the output channels that originate from the first multiply-accumulate unit and the second multiply-accumulate unit are designated as the first output channel and the second output channel, respectively, The sequence controller controls the output DMA controller such that it causes the first output channel to transfer pixel data from the first multiply-accumulate unit to the output bank, and the second output channel to transfer pixel data from the second multiply-accumulate unit to the output bank. Semiconductor equipment.

3. In the semiconductor device according to claim 2, The aforementioned second pixel space is a pixel space obtained by sliding the aforementioned first pixel space once. The sequence controller controls the output DMA controller such that the pixel data from the first output channel and the pixel data from the second output channel are arranged adjacent to each other in the address space of the output bank. Semiconductor equipment.

4. In the semiconductor device according to claim 1, The sequence controller receives a command that includes the identifiers of one or more input banks, determines one or more first input channels based on the identifiers of the one or more input banks, and determines one or more second input channels from among the input channels excluding the one or more first input channels. Semiconductor equipment.

5. In the semiconductor device according to claim 1, The sequence controller described above is A command containing the identifier of the aforementioned input bank and an enable signal that determines whether the distributed mode is enabled or disabled are input. When the distribution mode is enabled, the first input channel is used to transfer pixel data in the first pixel space in the input bank to the first multiply-accumulate unit, and the second input channel is used to transfer pixel data in the second pixel space in the input bank to the second multiply-accumulate unit. If the distribution mode is disabled, the first input channel is used to transfer the pixel data in the first pixel space and the pixel data in the second pixel space in the input bank to the first multiply-accumulate unit. Semiconductor equipment.

6. A semiconductor device consisting of a single semiconductor chip, A neural network engine that performs processing on the neural network, A first memory for storing weight parameters, A second memory that stores pixel data, having n banks that can be accessed in parallel, where n is an integer of 2 or more, Processor and A system bus connecting the neural network engine, the first memory, and the processor to each other, A memory bus connecting the neural network engine and the second memory, Equipped with, The aforementioned neural network engine is n multiply-accumulate units, each performing a multiply-accumulate operation on multiple pixel data and multiple weight parameters, A first DMA (Direct Memory Access) controller that transfers the weight parameters stored in the first memory to the n multiply-accumulate units, A second input DMA controller that transfers the pixel data stored in the second memory to the n multiply-accumulate units using n input channels, A second output DMA controller transfers pixel data from the n multiply-accumulate units to the second memory using n output channels, A sequence controller that controls the second input DMA controller and the second output DMA controller, Equipped with, If any one of the n banks is designated as an input bank, any two of the n input channels are designated as a first input channel and a second input channel, and the sum-accumulate units that are the transfer destinations for the first input channel and the second input channel are designated as a first sum-accumulate unit and a second sum-accumulate unit, The sequence controller controls the input DMA controller such that it causes the first input channel to transfer pixel data in the first pixel space in the input bank to the first multiply-accumulate unit, and the second input channel to transfer pixel data in the second pixel space in the input bank to the second multiply-accumulate unit. Semiconductor equipment.

7. In the semiconductor device according to claim 6, If any one of the n banks is designated as an output bank, and the output channels that originate from the first multiply-accumulate unit and the second multiply-accumulate unit are designated as the first output channel and the second output channel, respectively, The sequence controller controls the output DMA controller such that it causes the first output channel to transfer pixel data from the first multiply-accumulate unit to the output bank, and the second output channel to transfer pixel data from the second multiply-accumulate unit to the output bank. Semiconductor equipment.

8. In the semiconductor device according to claim 7, The aforementioned second pixel space is a pixel space obtained by sliding the aforementioned first pixel space once. The sequence controller controls the output DMA controller such that the pixel data from the first output channel and the pixel data from the second output channel are arranged adjacent to each other in the address space of the output bank. Semiconductor equipment.

9. In the semiconductor device according to claim 6, The sequence controller receives a command that includes the identifiers of one or more input banks, determines one or more first input channels based on the identifiers of the one or more input banks, and determines one or more second input channels from among the input channels excluding the one or more first input channels. Semiconductor equipment.

10. In the semiconductor device according to claim 6, The sequence controller described above is A command containing the identifier of the aforementioned input bank and an enable signal that determines whether the distributed mode is enabled or disabled are input. When the distribution mode is enabled, the first input channel is used to transfer pixel data in the first pixel space in the input bank to the first multiply-accumulate unit, and the second input channel is used to transfer pixel data in the second pixel space in the input bank to the second multiply-accumulate unit. If the distribution mode is disabled, the first input channel is used to transfer the pixel data in the first pixel space and the pixel data in the second pixel space in the input bank to the first multiply-accumulate unit. Semiconductor equipment.

Images