Semiconductor Devices

Abstract

To provide a semiconductor device which can reduce a circuit area.SOLUTION: A semiconductor device executes processing of a neural network. A memory MEM1 holds a plurality of pixel values and j compressed weighting factors. A decompressor DCMP restores the j compressed weighting factors to uncompressed k (k≥j) weighting factors. A DMA controller DMAC1 reads the j compressed weighting factors from the memory MEM1 and transfers them to the decompressor DCMP. n (n>k) accumulators in an accumulator unit ACCU multiply a plurality of pixel values and k uncompressed weighting factors to accumulate and add the multiplication results in time series. A switch circuit SW1 provided between the decompressor DCMP and the accumulator unit ACCU transfers the k uncompressed weighting factors restored by the decompressor DCMP to the n accumulators based on a correspondence represented by an identifier.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a semiconductor device, for example, a semiconductor device that executes neural network processing. [Background technology]

[0002] Patent Document 1 (JP 2019-40403 A) discloses an image recognition device having a convolution processing circuit that performs calculations using an integrated coefficient table to reduce the amount of calculations required for convolution calculations in a convolutional neural network (CNN). The integrated coefficient table holds N×N data, each of which is composed of a coefficient and a channel number. The convolution processing circuit has a multiplication circuit that executes N×N multiplication operations between an input image and the coefficients in parallel, and a channel selection circuit that performs cumulative addition operations for each channel number on the results of the multiplication operations and stores the results of the addition operations in an output register for each channel number. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-40403 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, in neural network processing such as CNN, it is desirable to use a DMA (Direct Memory Access) controller to speed up the transmission of image data and weighting coefficient data stored in memory to multiple accumulators. However, the amount of weighting coefficient data, in particular, can become very large. Therefore, a method can be considered in which pre-compressed weighting coefficient data is stored in memory, and the data is decompressed using a decompressor before being transmitted to multiple accumulators.

[0005] In this case, the decompressor can be placed between the memory and the DMA controller, or between the DMA controller and multiple accumulators. The former method could result in insufficient utilization of the multiple accumulators. The latter method requires an decompressor for each of the multiple accumulators, which could result in increased circuit area and power consumption.

[0006] Other objects and novel features will become apparent from the description of this specification and the accompanying drawings. [Means for solving the problem]

[0007] According to one embodiment, a semiconductor device performs neural network processing and includes one or more memories, a decompressor, a first DMA controller, an accumulator unit, and a first switch circuit. The one or more memories store a plurality of pixel values ​​and j compressed weight coefficients. The decompressor restores the j compressed weight coefficients to k (k≧j) uncompressed weight coefficients. The first DMA controller reads the j compressed weight coefficients from the memory and transfers them to the decompressor. The accumulator unit includes n (n>k) accumulators, multiplies the plurality of pixel values ​​by the k uncompressed weight coefficients, and accumulates and adds the multiplication results in a time series. The first switch circuit is provided between the decompressor and the accumulator unit, and transmits the k uncompressed weight coefficients restored by the decompressor to the n accumulators based on the correspondence indicated by a first identifier. [Effects of the Invention]

[0008] By using the semiconductor device of the embodiment, it becomes possible to reduce the circuit area. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram showing an example of the configuration of a main part of a semiconductor device according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating a detailed configuration example of the neural network engine in FIG. [Figure 3] FIG. 3 is a diagram showing an example of the operation of the expander and its surroundings in FIG. [Figure 4] FIG. 4 is a diagram showing an example of the configuration around the switch circuit in FIG. [Figure 5] FIG. 5 is a schematic diagram showing an example of the processing content in a convolutional layer included in a CNN. [Figure 6] FIG. 6 is a schematic diagram illustrating an example of the operation when the neural network engine of FIG. 2 executes the process of FIG. [Figure 7] FIG. 7 is a schematic diagram showing a configuration example in which a part of the semiconductor device shown in FIGS. 1 and 2 is extracted. [Figure 8] FIG. 8 is a schematic diagram showing an example of the configuration of a part around a neural network engine in a semiconductor device according to a second embodiment of the present invention. [Figure 9] FIG. 9 is a schematic diagram showing an example of the configuration of a part around a neural network engine in a semiconductor device according to a third embodiment of the present invention. [Figure 10] FIG. 10 is a diagram showing a detailed configuration example of a neural network engine in a semiconductor device according to a fourth embodiment of the present invention. [Figure 11A] FIG. 11A is a schematic diagram showing a configuration example of a semiconductor device serving as a comparative example of the present invention. [Figure 11B] FIG. 11B is a schematic diagram showing a configuration example of a semiconductor device serving as a comparative example of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] In the following embodiments, the description will be divided into multiple sections or embodiments for convenience, as necessary. However, unless otherwise specified, they are not unrelated to one another, and one is a partial or complete modification, detail, supplementary explanation, etc., of the other. Furthermore, in the following embodiments, when the number of elements (including the number, numerical value, amount, range, etc.) is mentioned, it is not limited to the specific number, and may be more or less than the specific number, unless otherwise specified or clearly limited to a specific number in principle. Furthermore, in the following embodiments, it goes without saying that the components (including element steps, etc.) are not necessarily essential, unless otherwise specified or clearly considered essential in principle. Similarly, in the following embodiments, when the shape, positional relationship, etc. of components, etc. are mentioned, it is intended to include those that are substantially similar to or similar to the shape, etc., unless otherwise specified or clearly considered not to be essential in principle. The same applies to the above numerical values ​​and ranges.

[0011] Hereinafter, embodiments will be described in detail with reference to the drawings. In all drawings for explaining the embodiments, components having the same functions are designated by the same reference numerals, and repeated explanations thereof will be omitted. In the following embodiments, explanations of the same or similar parts will not be repeated unless particularly necessary.

[0012] (Embodiment 1) <Outline of semiconductor device> Fig. 1 is a schematic diagram showing a configuration example of a main part of a semiconductor device according to a first embodiment of the present invention. The semiconductor device DEV shown in Fig. 1 is, for example, an SoC (System on Chip) configured by one semiconductor chip. The semiconductor device DEV is typically mounted in an ECU (Electronic Control Unit) of a vehicle, and provides the functions of an ADAS (Advanced Driver Assistance System).

[0013] The semiconductor device DEV shown in Fig. 1 has a neural network engine NNE, a processor PRC such as a CPU (Central Processing Unit), one or more memories MEM1 and MEM2, and a system bus SBUS. The neural network engine NNE executes neural network processing, such as CNN. The memory MEM1 is a DRAM (Dynamic Random Access Memory) or the like, and the memory MEM2 is a SRAM (Static Random Access Memory) or the like for cache. The system bus SBUS connects the neural network engine NNE, the memories MEM1 and MEM2, and the processor PRC to one another.

[0014] The memory MEM1 stores image data IMD containing multiple pixel values ​​and compressed weighting factor data WFDC. The amount of weighting factor data can be very large. For this reason, the uncompressed weighting factor data WFD is converted into compressed weighting factor data WFDC using compression software or the like before being stored in the memory MEM1. The memory MEM2 is used as a high-speed cache memory for the neural network engine NNE. For example, the image data IMD in the memory MEM1 is copied to the memory MEM2 in advance.

[0015] The neural network engine NNE includes a plurality of DMA controllers DMAC1 to DMAC3, a register REG, a decompressor DCMP, a plurality of switch circuits SW1 and SW2, a switch control circuit SWCT, and an accumulator unit ACCU. The DMA controller DMAC1 reads compressed weighting factor data WFDC from a memory MEM1 and transfers it to the decompressor DCMP. The decompressor DCMP restores the compressed weighting factor data WFDC to uncompressed weighting factor data WFD.

[0016] The switch circuit SW1 is provided between the decompressor DCMP and the accumulator unit ACCU. As will be described in detail later, the switch circuit SW1 transmits the weighting coefficients included in the uncompressed weighting coefficient data WFD restored by the decompressor DCMP to the accumulators in the accumulator unit ACCU based on a predetermined correspondence relationship. The DMA controller DMAC3 reads the image data IMD from the memory MEM2 and transfers it to the accumulator unit ACCU.

[0017] The accumulator unit ACCU includes a plurality of accumulators that perform multiply-and-accumulate operations, and performs the multiply-and-accumulate operation on the image data IMD from the DMA controller DMAC3 and the uncompressed weighting coefficient data WFD from the switch circuit SW1. The switch circuit SW2 is provided between the accumulator unit ACCU and the DMA controller DMAC2. The switch circuit SW2, which will be described in detail later, transmits the outputs from the plurality of accumulators in the accumulator unit ACCU to a plurality of channels in the DMA controller DMAC2 based on a predetermined correspondence relationship.

[0018] The switch control circuit SWCT controls the switch circuits SW1 and SW2 based on the setting data stored in the register REG. Specifically, the switch control circuit SWCT controls the correspondence between the switch circuits SW1 and SW2. The register REG also stores setting data for the address ranges of the DMA controllers DMAC1 to DMAC3, setting data for the accumulator unit ACCU, and the like.

[0019] <Neural network engine details> FIG. 2 is a diagram showing a detailed configuration example of the neural network engine in FIG. 1. FIG. 3 is a diagram showing an example of the operation of the expander and its surroundings in FIG. 2. FIG. 4 is a diagram showing an example of the configuration of the switch circuit and its surroundings in FIG. 2. In FIG. 2, the DMA controller DMAC1 reads out the weighting coefficient data set WFDS as shown in FIG. 3 from the memory MEM1 for each control cycle based on a preset read address range. That is, the memory MEM1 in FIG. 1 holds the weighting coefficient data set WFDS as shown in FIG. 3 in each address of the read address range in advance.

[0020] The weighting coefficient data set WFDS shown in FIG. 3 includes compressed weighting coefficient data WFDC and a header HD. The weighting coefficient data WFDC includes compressed j weighting coefficients P[1] to P[j] and map data MPD. The header HD includes two uncompressed identifiers ID1 and ID2. As an example, each of the weighting coefficients P[1] to P[j] is composed of 8 bits, where "j" is 11. The map data MPD is composed of 28 bits. Furthermore, each of the two identifiers ID1 and ID2 is composed of 6 bits. In this case, the weighting coefficient data set WFDS is composed of 128 bits.

[0021] The DMA controller DMAC1 transfers the compressed weighting factor data WFDC included in the weighting factor data set WFDS, i.e., the map data MPD and the j compressed weighting factors P[1] to P[j], to the decompressor DCMP, as shown in Figures 2 and 3. Also, as shown in Figure 2, the header HD included in the weighting factor data set WFDS, i.e., the identifiers ID1 and ID2, are stored in the register REG.

[0022] As shown in Figures 2 and 3, the decompressor DCMP restores the compressed weighting factor data WFDC to uncompressed weighting factor data WFD. Specifically, the weighting factor data WFDC is compressed data using, for example, a zero run length method. The decompressor DCMP restores the compressed j weighting factors P[1] to P[j] to uncompressed k (k ≥ j) weighting factors W[1] to W[k] based on the locations of non-zero coefficients represented by the map data MPD. As an example, each of the weighting factors W[1] to W[k] is composed of 8 bits, and "k" is a maximum of 28, etc.

[0023] 2, the switch control circuit SWCT controls the switch circuits SW1 and SW2 based on the header HD stored in the register REG. Specifically, the switch control circuit SWCT generates a switch control signal SS1 based on an identifier ID1 included in the weighting coefficient data set WFDS of FIG. 3, and uses the switch control signal SS1 to control the correspondence relationship in the switch circuit SW1. Similarly, the switch control circuit SWCT generates a switch control signal SS2 based on an identifier ID2 included in the weighting coefficient data set WFDS, and uses the switch control signal SS2 to control the correspondence relationship in the switch circuit SW2.

[0024] 2 and 4, the switch circuit SW1 is provided between the expander DCMP and n (n>k) accumulators ACC[1] to ACC[n] included in the accumulator unit. The switch circuit SW1 transmits the k uncompressed weighting coefficients W[1] to W[k] restored by the expander DCMP to the n accumulators ACC[1] to ACC[n] based on the correspondence indicated by the switch control signal SS1 and, in turn, the identifier ID1.

[0025] The switch circuit SW1 is configured, for example, as shown in FIG. 4, by a crossbar switch or the like including k×n switches S[1,1] to S[k,n]. The switches S[1,1] to S[k,n] are provided at the intersections of k lines LNd[1] to LNd[k] and n lines LNa[1] to LNa[n]. The k lines LNd[1] to LNd[k] respectively transmit k weighting coefficients W[1] to W[k] from the expander DCMP. The n lines LNa[1] to LNa[n] are respectively connected to n accumulators ACC[1] to ACC[n].

[0026] The switch control circuit SWCT is preset with on / off combinations for the switches S[1,1] to S[k,n] for each value of the identifier ID1. The switch control circuit SWCT receives the identifier ID1 and generates corresponding "k×n" switch control signals SS[1,1] to SS[k,n] to control the on / off of the switches S[1,1] to S[k,n]. Although not shown, the switch circuit SW2 in FIG. 2 is also configured, for example, with a crossbar switch similar to that in FIG. 4.

[0027] 2, the DMA controller DMAC3 includes n channels CH[1] to CH[n]. Each of the n channels CH[1] to CH[n] reads pixel values ​​in the image data IMD from the memory MEM2 in FIG. 1 and transfers them to n accumulators ACC[1] to ACC[n] for each control cycle based on a read address range that is set for each channel.

[0028] Each of the n accumulators ACC[1] to ACC[n] has, for example, one multiplier and one accumulator. In addition, each of the n accumulators ACC[1] to ACC[n] may have, for example, a bias adder and an activation function calculator required in the processing of a neural network. Each of the n accumulators ACC[1] to ACC[n] multiplies, for each control cycle, the n pixel values from the DMA controller DMAC3 and the k uncompressed weight coefficients W[1] to W[k] transmitted from the switch circuit SW1.

[0029] Here, the correspondence between the n accumulators ACC[1] to ACC[n] and the k (k < n) weight coefficients W[1] to W[k] is determined by the switch circuit SW1. At this time, the switch circuit SW1 transmits at least one of the k weight coefficients W[1] to W[k] in parallel to two or more of the n accumulators ACC[1] to ACC[n]. Then, each of the n accumulators ACC[1] to ACC[n] accumulates the multiplication results of the pixel values and the weight coefficients obtained in this way in time series over a plurality of control cycles. As an example, for k = 28, n may be on the order of 100 to 1000.

[0030] The DMA controller DMAC2 includes m channels CH[1] to CH[m]. Each of the m channels CH[1] to CH[m] transfers, for each control cycle, the outputs of the n accumulators ACC[1] to ACC[n] to the write address in the memory, for example, the memory MEM2 in FIG. 1, based on the individually set write address.

[0031] The switch circuit SW2 is provided between the n accumulators ACC[1] to ACC[n] and the DMA controller DMAC2. The switch circuit SW2 transmits the outputs of the n accumulators ACC[1] to ACC[n] to the m channels CH[1] to CH[m] in the DMA controller DMAC2 based on the correspondence represented by the switch control signal SS2 from the switch control circuit SWCT and thus the identifier ID2.

[0032] <Example of neural network processing> Fig. 5 is a schematic diagram showing an example of processing content in a convolutional layer included in a CNN. In Fig. 5, a certain two-dimensional region A in image data IMD is composed of pixel value data XDa consisting of i pixel values ​​Xa[1] to Xa[i]. Similarly, another two-dimensional region B in image data IMD is composed of pixel value data XDb consisting of i pixel values ​​Xb[1] to Xb[i].

[0033] On the other hand, CNN uses k weighting coefficient data WFD[1] to WFD[k], also called kernels, for k output channels. The weighting coefficient data WFD[1] for output channel [1] is composed of i weighting coefficients W[1,1] to W[1,i]. Similarly, the weighting coefficient data WFD[k] for output channel [k] is composed of i weighting coefficients W[k,1] to W[k,i].

[0034] In the convolutional layer, k feature maps FMP[1] to FMP[k] are generated according to k output channels. In the feature map FMP[1] of output channel [1], the feature value Va[1] of the pixel corresponding to the two-dimensional region A in the image data IMD is calculated by a product-sum operation of the pixel value data XDa and the weighting coefficient data WFD[1] of the output channel [1]. Similarly, in the feature map FMP[1], the feature value Vb[1] of the pixel corresponding to the two-dimensional region B in the image data IMD is calculated by a product-sum operation of the pixel value data XDb and the weighting coefficient data WFD[1] of the output channel [1].

[0035] In the feature map FMP[k] of output channel [k], the feature amount Va[k] of a pixel corresponding to two-dimensional region A in image data IMD is calculated by a product-sum operation of pixel value data XDa and weighting coefficient data WFD[k] of output channel [k]. Similarly, in the feature map FMP[k], the feature amount Vb[k] of a pixel corresponding to two-dimensional region B in image data IMD is calculated by a product-sum operation of pixel value data XDb and weighting coefficient data WFD[k] of output channel [k]. Note that each feature amount may be calculated by adding a bias value for each output channel to the result of such product-sum operation and then calculating an activation function.

[0036] Fig. 6 is a schematic diagram illustrating an example of the operation when the neural network engine in Fig. 2 executes the process in Fig. 5. In the example in Fig. 6, the accumulators ACC[1], ..., ACC[r], ..., ACC[q], ... calculate the feature quantities Va[1], ..., Vb[1], ..., Va[k], ... in Fig. 5, respectively.

[0037] In this case, the switch circuit SW1 transmits i weighting coefficients W[1,1] to W[1,i] in output channel [1] in parallel to the multiple accumulators ACC[1], ..., ACC[r] in sequence over i control cycles. Similarly, the switch circuit SW1 transmits i weighting coefficients W[k,1] to W[k,i] in output channel [k] in parallel to the multiple accumulators ACC[q], ... in sequence over i control cycles.

[0038] Furthermore, prior to this processing, the expander DCMP receives, for example, in the first control cycle, the compressed j weighting coefficients P[1,1] to P[j,1] and expands them to output weighting coefficients W[1,1] to W[k,1] for k output channels. A header HD is added to these compressed weighting coefficients P[1,1] to P[j,1] as shown in Figure 3. The switch control circuit SWCT generates a switch control signal SS1 based on the identifier ID1 in this header HD.

[0039] The switch circuit SW1 receives weighting coefficients W[1,1] to W[k,1] for k output channels from the expander DCMP, and transmits each of the weighting coefficients W[1,1] to W[k,1] in parallel to a plurality of accumulators based on a switch control signal SS1 from the switch control circuit SWCT. That is, for example, in Figure 4, the switch control circuit SWCT generates a switch control signal SS1 that turns on all of the switches S[1,1], S[1,2], ... connected to the wiring LNd[1] for the weighting coefficient W[1].

[0040] Meanwhile, in the DMA controller DMAC3, channels CH[1] and CH[q] each read i pixel values ​​Xa[1] to Xa[i] in sequence from memory MEM2 in i control cycles and transfer them to accumulators ACC[1] and ACC[q] in sequence. Also, channel CH[r] reads i pixel values ​​Xb[1] to Xb[i] in sequence from memory MEM2 in i control cycles and transfers them to accumulator ACC[r] in sequence. As a result, accumulators ACC[1], ..., ACC[r], ..., ACC[q], ... perform the product-sum operation as shown in Figure 5.

[0041] In each channel in the DMA controller DMAC2, a correspondence relationship between the feature maps FMP[1] to FMP[k] for each output channel and the write addresses for a memory, for example, the memory MEM2 in FIG. 1, is determined in advance, as shown in FIG. 5. The switch circuit SW2 transmits the outputs of the accumulators ACC[1], ..., ACC[r], ..., ACC[q], ... to each channel in the DMA controller DMAC2 based on a switch control signal SS2 from the switch control circuit SWCT and, in turn, an identifier ID2. Then, each channel in the DMA controller DMAC2 writes the output from the switch circuit SW2 to a predetermined write address in the memory.

[0042] <Major Effects of the First Embodiment> Fig. 7 is a schematic diagram showing a configuration example of a portion extracted from the semiconductor device of Fig. 1 and Fig. 2. Fig. 11A and Fig. 11B are schematic diagrams showing a configuration example of a semiconductor device serving as a comparative example of the present invention. Fig. 7 shows a memory MEM1, a DMA controller DMAC1, an expander DCMP, a switch circuit SW1, a register REG, a switch control circuit SWCT, and an accumulator unit ACCU in a neural network engine NNE.

[0043] 11A includes a neural network engine NNE′a including an accumulator unit ACCU and a DMA controller DMAC1, and a memory MEM1, and a decompressor DCMP is provided between the DMA controller DMAC1 and the memory MEM1.

[0044] In the configuration example of Figure 11A, the number of data items varies between the input and output of the decompressor DCMP, which may complicate the control of the DMA controller DMAC1. As a result, it may not be possible to efficiently transmit uncompressed weighting coefficients from the decompressor DCMP to the n accumulators ACC[1] to ACC[n] in the accumulator unit ACCU. In other words, there is a risk that the resources of the n accumulators ACC[1] to ACC[n] may not be fully utilized.

[0045] 11B includes a neural network engine NNE'b including n accumulators ACC[1] to ACC[n], n expanders DCMP[1] to DCMP[n], and a DMA controller DMAC1, and a memory MEM1. The expanders DCMP[1] to DCMP[n] are provided between the DMA controller DMAC1 and the accumulator unit ACCU, as in the case of FIG.

[0046] However, in the configuration example of Figure 11B, n expanders DCMP[1] to DCMP[n] are provided to effectively utilize the n accumulators ACC[1] to ACC[n]. The n expanders DCMP[1] to DCMP[n] transmit uncompressed weighting coefficients to the n accumulators ACC[1] to ACC[n], respectively. However, in this case, the circuit area increases due to the n expanders DCMP[1] to DCMP[n], and therefore power consumption may also increase.

[0047] 7 differs from the configuration example of FIG. 11B in that a switch circuit SW1 is provided between the accumulator unit ACCU and the expander DCMP. As shown in FIGS. 4 and 6, the switch circuit SW1 can transmit one weighting coefficient from the expander DCMP to multiple accumulators. As a result, the configuration example of FIG. 7 only requires one expander DCMP, which reduces the circuit area and ultimately the power consumption.

[0048] Furthermore, compared to the configuration example of FIG. 11A, by appropriately defining the correspondence within the switch circuit SW1 using the identifier ID1 in the header HD of FIG. 3, the resources of the n accumulators ACC[1] to ACC[n] can be fully and effectively utilized. As a result, it is possible to speed up the processing of the neural network. From another perspective, by appropriately defining the identifier ID1 in advance and storing it in the memory MEM1 as the weighting coefficient data set WFDS of FIG. 3, it is possible to flexibly accommodate neural networks of various configurations. The same effect of flexibility applies to the identifier ID2.

[0049] (Embodiment 2) <Configuration around the neural network engine> Fig. 8 is a schematic diagram showing a configuration example of a part around a neural network engine in a semiconductor device according to a second embodiment of the present invention. Unlike the configuration example of Fig. 7, the neural network engine NNEa shown in Fig. 8 includes multiple sets of an expander DCMP, a switch circuit SW1, a switch control circuit SWCT, and an accumulator unit ACCU.

[0050] In the configuration example of Fig. 7, as shown in Fig. 3, for example, one expander DCMP outputs up to 28 uncompressed weighting coefficients W[1] to W[k]. These weighting coefficients W[1] to W[k] are then transmitted appropriately via a switch circuit SW1 to approximately several hundred to several thousand accumulators ACC[1] to ACC[n] in the accumulator unit ACCU. However, in the configuration example of Fig. 7, for example, if the number of output channels is more than 28, it may become necessary to perform processing in a time-division manner.

[0051] Therefore, by providing multiple sets of expanders DCMP, switch circuits SW1, switch control circuits SWCT, and accumulator units ACCU as shown in Figure 8, it is possible to process many output channels in parallel even when there are a large number of output channels.As a result, it is possible to speed up the processing of the neural network.In this case, for example, it is possible to expand the bit width of the weighting coefficient data set WFDS in Figure 3 by multiple times, or to provide multiple DMA controllers DMAC1.

[0052] (Embodiment 3) <Configuration around the neural network engine> 9 is a schematic diagram showing an example of the configuration of a part around a neural network engine in a semiconductor device according to a third embodiment of the present invention. The neural network engine NNEb shown in FIG. 9 is different from the configuration example of FIG. 7 in that the output path of the header HD from the DMA controller DMAC1 to the register REG is deleted. Instead, an output path of the header HD from the processor PRC to the register REG is formed.

[0053] That is, when the DMA controller DMAC1 transfers compressed weight coefficient data WFDC as shown in Fig. 3 to the decompressor DCMP, the processor PRC outputs an identifier ID1 to the register REG and, in turn, to the switch control circuit SWCT via the system bus SBUS. Specifically, the processor PRC generates the header HD shown in Fig. 3 at a timing according to the processing of the neural network engine NNEb, and outputs the identifiers ID1 and ID2 included in the header HD to the switch control circuit SWCT. Using such a configuration makes it possible to reduce the storage capacity of the memory MEM1 required for the header HD.

[0054] (Fourth embodiment) <Neural network engine details> Fig. 10 is a diagram showing a detailed configuration example of a neural network engine in a semiconductor device according to embodiment 4 of the present invention. Compared to the configuration example of Fig. 2, the neural network engine shown in Fig. 10 has a decompression unit DU3 provided between a DMA controller DMAC3 and an accumulator unit ACCU.

[0055] The decompression unit DU3 has the same configuration as the decompression unit DU1, except that the decompression unit DU1 includes a switch control circuit SWCT, a decompressor DCMP, a switch circuit SW1, and a register REG, which are used for processing the weighting coefficients described above. Specifically, the memory MEM1 stores pre-compressed image data. The decompression unit DU3 then decompresses the compressed image data and transmits it to the accumulator unit ACCU.

[0056] Typically, image data IMD has a smaller data volume than weighting factor data WFD, and is therefore used in a state where it is stored as uncompressed data in cache memory MEM2. However, for example, if the number of input channels of image data IMD increases, it may become difficult to secure sufficient storage capacity for the image data IMD in memory MEM2. Therefore, by using the configuration example shown in Figure 10, it becomes possible to handle even cases where the data volume of image data IMD is large.

[0057] The invention made by the inventor has been specifically described above based on an embodiment, but it goes without saying that the present invention is not limited to the above embodiment and can be modified in various ways without departing from the gist of the invention. [Explanation of symbols]

[0058] ACC accumulator DCMP stretcher DEV Semiconductor device DMAC1 to DMAC3 DMA controllers MEM1, MEM2 memory NNE Neural Network Engine PRC Processor SBUS system bus SW1, SW2 switch circuit SWCT Switch control circuit WFD Weighting Factor Data WFDC Compressed Weighting Factor Data

Claims

1. A semiconductor device that executes neural network processing, one or more memories for holding a plurality of pixel values ​​and the j compressed weighting coefficients; a decompressor for restoring the j compressed weighting coefficients to k uncompressed weighting coefficients (k≧j); a first DMA (Direct Memory Access) controller that reads the j compressed weight coefficients from the memory and transfers them to the decompressor; n (n>k) accumulators that multiply the pixel values ​​by the k uncompressed weighting coefficients and accumulate the multiplication results in time series; a first switch circuit provided between the expander and the n accumulators, for transmitting the k uncompressed weighting coefficients restored by the expander to the n accumulators based on a correspondence relationship represented by a first identifier; having Semiconductor device.

2. 2. The semiconductor device according to claim 1, Further, a switch control circuit is provided, the memory stores the compressed j weighting factors together with the first identifier as a weighting factor data set; the first DMA controller reads the weighting factor data set from the memory and transfers the j compressed weighting factors included in the weighting factor data set to the decompressor; the switch control circuit controls the correspondence in the first switch circuit based on the first identifier included in the weighting coefficient data set read by the first DMA controller. Semiconductor device.

3. 2. The semiconductor device according to claim 1, the first switch circuit transmits at least one of the k uncompressed weighting coefficients to two or more of the n accumulators; Semiconductor device.

4. 2. The semiconductor device according to claim 1, further comprising: a second DMA controller for transferring the outputs of the n accumulators to the memory; a second switch circuit provided between the n accumulators and the second DMA controller, the second switch circuit transmitting outputs of the n accumulators to a plurality of channels in the second DMA controller based on a correspondence relationship represented by a second identifier; having Semiconductor device.

5. 5. The semiconductor device according to claim 4, Further, a switch control circuit is provided, the memory stores the compressed j weighting factors together with the first identifier and the second identifier as a weighting factor data set; the first DMA controller reads the weighting factor data set from the memory and transfers the j compressed weighting factors included in the weighting factor data set to the decompressor; the switch control circuit controls the correspondence relationship in the first switch circuit and the correspondence relationship in the second switch circuit based on the first identifier and the second identifier included in the weighting coefficient data set read by the first DMA controller, respectively; Semiconductor device.

6. 2. The semiconductor device according to claim 1, further comprising a third DMA controller that reads the plurality of pixel values ​​from the memory and transfers them to the n accumulators; Semiconductor device.

7. A semiconductor device composed of one semiconductor chip, a neural network engine that executes neural network processing; one or more memories for holding a plurality of pixel values ​​and the j compressed weighting coefficients; a processor; a bus connecting the neural network engine, the memory, and the processor to one another; Equipped with The neural network engine a decompressor for restoring the j compressed weighting coefficients to k uncompressed weighting coefficients (k≧j); a first DMA (Direct Memory Access) controller that reads the j compressed weight coefficients from the memory and transfers them to the decompressor; n (n>k) accumulators that multiply the pixel values ​​by the k uncompressed weighting coefficients and accumulate the multiplication results in time series; a first switch circuit provided between the expander and the n accumulators, for transmitting the k uncompressed weighting coefficients restored by the expander to the n accumulators based on a correspondence relationship represented by a first identifier; a switch control circuit that controls the correspondence in the first switch circuit based on the first identifier; having Semiconductor device.

8. 8. The semiconductor device according to claim 7, the processor outputs the first identifier to the switch control circuit when the first DMA controller transfers the compressed j weight coefficients to the decompressor. Semiconductor device.

9. 8. The semiconductor device according to claim 7, the first switch circuit transmits at least one of the k uncompressed weighting coefficients to two or more of the n accumulators; Semiconductor device.

10. 8. The semiconductor device according to claim 7, The neural network engine further comprises: a second DMA controller for transferring the outputs of the n accumulators to the memory; a second switch circuit provided between the n accumulators and the second DMA controller, for transmitting outputs of the n accumulators to the second DMA controller based on a correspondence relationship represented by a second identifier; having Semiconductor device.

Images