Low bandwidth convolution parallelization structure

Abstract

A computational circuit (4) comprising at least one computational chain (6) comprising a plurality of computational blocks (7), each comprising a multiplier (9) and an accumulator (10), the computational circuit being arranged to apply to the first inputs of the multipliers a succession of primary data forming at least one primary vector, and to the second inputs of the multipliers a succession of secondary data series forming secondary vectors, the computational chain being thus arranged to successively produce results of multiplications of at least one primary vector by each secondary vector. ABRIDGED FIGURE: Fig. 5

Description

Title of the invention: Low bandwidth convolution parallelization structure

[0001] The invention relates to the field of calculation circuits used for example to calculate convolution products in neural networks.

[0002] BACKGROUND OF THE INVENTION

[0003] The use of Deep Learning models and, in particular, models of the convolutional neural network type, requires performing a significant number of convolution product calculations. The ability of hardware components to efficiently perform these calculations is a major challenge for the development of applications using this technology. This challenge concerns the training of the model, but also and above all the execution of the inference, which can be implemented in an embedded system and therefore with limited resources (computing power, data bandwidth, available energy, etc.).

[0004] We are considering developing a calculation circuit, implemented for example in an FPGA (for Field Programmable Gate Arrays) or an ASIC (for Application Specified Integrated Circuit), which is as efficient as possible to carry out these calculations of convolution products.

[0005] The basic operation of convolution is the combined MAC operation, for Multiplication-Accumulation. This operation consists of multiplying a data by a weight and adding the result of the multiplication with the results of the previous multiplications.

[0006] However, it has been found that, in prior art computing circuits designed to calculate convolutions of neural networks, the efficiency of the multipliers is quite low, sometimes only 10%. This means that the multipliers spend 90% of the time doing nothing useful. This is very problematic because it increases the processing time and the consumption of the component implementing the computing circuit.

[0007] This low efficiency is due to a "starvation" of input data: the memories integrated in or connected to the component implementing the calculation circuit cannot provide the data to the multipliers quickly enough. Increasing the efficiency, and therefore the effectiveness of the implementation of these calculations, absolutely requires reducing the bandwidth requirement.

[0008] With reference to [Fig. 1], the traditional structure for performing convolution calculations in neural networks is a systolic architecture, which is a 2D (two-dimensional) structure.

[0009] In the example of [Fig.l], a and b are 3x3 matrices. Each square represents an Elementary Processor (EP). At each clock cycle, each PE: - multiplies the two inputs (top and left); - accumulates the result of the multiplication locally; - transfers the coefficients a^ and bjj respectively to the right and downwards.

[0010] Thus, after two clock ticks, the first PE has performed two MACs (this is the case in [Fig. 1 ]). Its two neighbors have performed one MAC, because they receive the data with a shift of one clock cycle.

[0011] After three clock cycles, the first PE has calculated the equation:

[0012] a0>0 x bOjo + a0>ix b1>0 + a0>2 x b2jo

[0013] This equation corresponds to the multiplication of the vector aOjX with the vector by 0.

[0014] After seven clock cycles, all vector multiplications have been completed, which corresponds to a multiplication of two 3x3 matrices performed in seven clock cycles.

[0015] With this pipelined structure and by sending the data of the next matrix as early as possible, it is even possible to calculate a multiplication of two 3x3 matrices every three clock cycles.

[0016] The systolic architecture requires low input data bandwidth, as it maximizes the reuse of input data in calculations (each data item from one of the two inputs is multiplied by all the data from the other input). However, this architecture involves moving data within the structure, which may require a lot of resources (registers). Its implementation also requires a lot of bandwidth and / or a lot of resources to produce the output data.

[0017] Another commonly used structure is the Tensor Cores type structure, which is a 3D structure. This structure has the advantage of requiring (roughly) the same bandwidth at the input and output. This is an advantage in a fixed and general-purpose structure (which is the case, for example, of GPUs, for Graphics Processing Units, or multi-core processors), because it allows data of homogeneous sizes to be handled at the input and output, allowing simple and homogeneous sizing of transmission buses, registers, memories, etc. The disadvantage of this structure is that it requires higher bandwidth, because each input data is less reused (data from one input is only multiplied by a part of the data coming from the other input). The structure therefore does not maximize the calculations performed: it requires more bandwidth in total and has higher consumption (because of more movements).

[0018] SUBJECT OF THE INVENTION

[0019] The object of the invention is to carry out a large number of multiplication-accumulations, while limiting the bandwidth requirement and the resources used. Summary of the invention

[0020] With a view to achieving this aim, a calculation circuit is proposed comprising at least one calculation chain comprising a plurality of calculation blocks each comprising a multiplier comprising a first input and a second input, and an accumulator comprising a first input connected to an output of the multiplier,

[0021] the calculation circuit being arranged to, in a manner timed by clock cycles, apply to the first inputs of the multipliers of the calculation blocks of said calculation chain a succession of primary data forming at least one primary vector, and apply to the second inputs of the multipliers of the calculation blocks of said calculation chain a succession of series of secondary data forming secondary vectors,

[0022] each calculation block of said calculation chain being arranged to, at each clock cycle, add a result of a multiplication carried out by said calculation block with a value accumulated at the previous clock cycle by the accumulator of said calculation block or of another calculation block of said calculation chain, the calculation chain thus being arranged to successively produce results of multiplications of the at least one primary vector by each secondary vector.

[0023] The calculation circuit according to the invention makes it possible to maximize the parallelization of the calculations by minimizing the necessary quantity of input data. The input data (first, second and third data) are used as many times as possible (multiplication of each data item of the first vector and of the second vector by each data item of each third vector), which minimizes the required bandwidth of the input data.

[0024] The time shift in the availability of the results at the output of the calculation chain makes it possible to reduce the output bandwidth and / or the storage and / or routing resources required for the output data. The instantaneous output bandwidth is minimized, because there are no bandwidth “peaks” that would be complex to manage. The calculation circuit makes it possible to minimize the resources used and therefore, for the same application, either to implement the application with smaller components, or, for the same component size, to implement the application in a more efficient manner.

[0025] We further propose a calculation circuit as previously described, comprising a number N of calculation chains forming rows of a matrix structure, each calculation chain comprising a number M of calculation blocks, the second inputs of the multipliers of the calculation blocks of the same column of said matrix structure being connected to each other.

[0026] We further propose a calculation circuit as previously described, each block of calculation of each calculation chain comprising a first multiplexer comprising a first input and a second input, the first input of the multiplier of said calculation block being connected to an output of said first multiplexer, the first inputs of the first multiplexers of the calculation blocks of said calculation chain being connected to each other and the second inputs of the first multiplexers of the calculation blocks of said calculation chain being connected to each other,

[0027] the calculation circuit being arranged to apply to the first inputs of said first multiplexers a succession of first primary data forming a first primary vector, apply to the second inputs of said first multiplexers a succession of second primary data forming a second primary vector,

[0028] the calculation chain being arranged to successively and serially produce results of multiplications of the first primary vector by each secondary vector then of the second primary vector by each secondary vector.

[0029] We further propose a calculation circuit as previously described, each calculation block of each calculation chain comprising a second multiplexer arranged so that:

[0030] - an output of said second multiplexer is connected to a second input of the accumulator of said calculation block;

[0031] - an output of said accumulator being connected to a first input of said second multiplexer;

[0032] - an output of said accumulator being connected to a first input of a following block of said calculation chain.

[0033] A calculation circuit is further proposed as previously described, in which the calculation blocks of each calculation chain comprise an input block, an output block and intermediate blocks positioned between the input block and the output block, the calculation chain being arranged so that an output of the accumulator of each of the input blocks and intermediate blocks is connected to a second input of the accumulator of a following block of said calculation chain.

[0034] We further propose a calculation circuit as previously described, in which the output of the accumulator of the output block is connected to a second input of the accumulator of the input block.

[0035] A calculation circuit is further proposed as previously described, in which each calculation chain comprises an output multiplexer, said calculation chain being arranged so that, for each calculation block, the output of the accumulator of said calculation block is connected to a separate input of the output multiplexer.

[0036] A calculation circuit is further proposed as previously described, in which each calculation chain further comprises an output accumulator having a first input connected to the output of the accumulator of an output block of said calculation chain.

[0037] We further propose a calculation circuit as previously described, in which each calculation chain further comprises an output accumulator having a first input connected to the output of the output multiplexer of said calculation chain.

[0038] We further propose a calculation circuit as previously described, further comprising a first buffer connected to the first input of the first multiplexer of an input block of the calculation chain, and a second buffer connected to the second input of the first multiplexer of the input block of the calculation chain.

[0039] We further propose a calculation circuit as previously described, further comprising a third buffer having an input connected to an output of the output accumulator and an output connected to a second input of the output accumulator.

[0040] A calculation circuit is further proposed as previously described, comprising a primary RAM block for each calculation chain, and a plurality of secondary RAM blocks, the primary data and the secondary data being applied for each calculation chain respectively via the primary RAM block associated with said calculation chain and via the secondary RAM blocks, the primary RAM blocks and the secondary RAM blocks making it possible to provide the calculation chains with the same primary or secondary data several times without needing to access each time one or more data sources from which the primary data and the secondary data originate.

[0041] We further propose a calculation circuit as previously described, the calculation circuit being configured to perform a convolution product of an input tensor by a mask of size Nmask to obtain an output tensor, the calculation circuit being such that:

[0042] Nmask = KM,

[0043] where K is a positive integer.

[0044] We further propose a calculation circuit as previously described, the calculation circuit being arranged to:

[0045] - apply the secondary data series in a cyclical sequence predefined;

[0046] - start applying the first primary data to the first inputs first multiplexers then, after a predefined number of clock cycles, start the application of the second primary data to the second inputs of the first multiplexers;

[0047] - drive the first multiplexers so that the calculation chain, simultaneously, completes a multiplication of the first primary vector by each of the vectors secondary vectors and begins a multiplication of the second primary vector by each of the secondary vectors.

[0048] A calculation circuit is further proposed as previously described, the secondary vectors forming a matrix, the predefined cyclic sequence comprising cycles each comprising a first phase during which the series of secondary data comprise an increasing number of components of successive rows or columns of the matrix, a second phase during which the series of secondary data comprise a constant number of components of successive rows or columns of the matrix, and a third phase during which the series of secondary data comprise a decreasing number of components of successive rows or columns of the matrix.

[0049] We further propose a calculation circuit as previously described, in which the calculation circuit comprises several calculation chains and an additional multiplexer having one or more inputs connected to one or more accumulators of one or more calculation blocks of the same calculation chain and / or of another calculation chain.

[0050] We further propose a calculation circuit as previously described, comprising a plurality of calculation structures each comprising a plurality of calculation chains, the calculation structures being arranged according to a pipeline architecture.

[0051] A computing circuit as previously described is further provided, each computing structure being arranged to perform convolution products for a distinct convolution layer of a neural network model.

[0052] We further propose an electronic component in which the calculation circuit as previously described is produced.

[0053] An electronic component as previously described is further provided, the electronic component being an FPGA.

[0054] We further propose an electronic component as previously described, the FPGA comprising DSP blocks, each calculation block being implemented in a DSP block.

[0055] We further propose an electronic component as previously described, the calculation circuit comprising at least one calculation structure having a matrix architecture comprising a plurality of calculation chains forming lines of the matrix architecture, the calculation circuit comprising DSP blocks in each of which two calculation blocks of two successive lines are produced.

[0056] An electronic component as previously described is further provided, the electronic component being an ASIC.

[0057] The invention will be better understood in light of the following description of particular non-limiting embodiments of the invention. Brief description of the drawings

[0058] Reference will be made to the attached drawings, among which:

[0059] [Fig-1] [Fig.l] illustrates a matrix convolution product carried out in an ar systolic architecture;

[0060] [Fig.2] [Fig.2] represents the convolution product of an input tensor by a mask ;

[0061] [Fig.3] [Fig.3] represents an example of a DSP block;

[0062] [Fig.4] [Fig.4] represents a calculation structure;

[0063] [Fig.5] [Fig.5] represents a calculation chain according to a first mode of realization lization;

[0064] [Fig.6] [Fig.6] represents the calculation chain without the multiplexers;

[0065] [Fig.7] [Fig.7] illustrates the calculations carried out by a calculation chain which would not have of multiplexers;

[0066] [Fig.8] [Fig.8] corresponds to the third and last column of [Fig.7], but in a multiplexer structure;

[0067] [Fig.9] [Fig.9] represents a pipeline of computational structures;

[0068] [Fig. 10] [Fig. 10] represents a calculation chain according to a second embodiment;

[0069] [Fig. 11] [Fig. 11] represents a calculation chain according to a third embodiment;

[0070] [Fig. 12] [Fig. 12] represents an example of a DSP block for implementing a calculation block of the calculation chain of [Fig. 11]. DETAILED DESCRIPTION OF THE INVENTION

[0071] The calculation circuit according to the invention is used here to perform calculations of 3D convolution products, in a neural network comprising a plurality of convolution layers.

[0072] With reference to [Fig.2], the calculation circuit therefore calculates convolution products between an input tensor 1 and one or more masks 2 (or kernels) to obtain an output tensor 3.

[0073] In the case of the first convolution layer of a neural network, the input tensor 1 is for example an input image acquired by an image capture device (camera for example), possibly preprocessed, and having a width 1, a height h and a depth p. The depth p is for example equal to three for a color image, corresponding to the RGB (Red, Green, Blue) components. The number of channels of the input tensor 1 is equal to its depth p.

[0074] A pixel is called the "carrot" of the input tensor 1 with coordinates (x,y). Each pixel includes a number of sub-pixels equal to the number of channels of the tensor entry 1.

[0075] The sub-pixel is the elementary constituent of the pixel. It is a 1-dimensional data item, which is coded for example in INT8, and which has coordinates (x, y, c), where c is the coordinate of the corresponding channel of the input tensor.

[0076] It is the subpixel that is multiplied by a weight (scalar) in the convolution.

[0077] Each mask 2 generally has the same number of channels as the input tensor 1, except in the case of a so-called Depthwise convolution which consists of applying a unique mask to each channel of an input tensor (we are also starting to find masks of intermediate depths).

[0078] The number of channels of the output tensor 3 is equal to the number of masks 2.

[0079] We first describe an embodiment, in which the calculation circuit is implemented implemented in an FPGA.

[0080] The use of an FPGA is particularly relevant in the case of a specialized architecture, dedicated to a particular model of neural network. This architecture must be efficient for a single network, which is known at the design of the computing circuit. This is the principle on which FPGA design is traditionally based, because the architecture that can be implemented in the FPGA is free. Intrinsically, the FPGA is slower than an ASIC, and has an average operating frequency. But the specialization of the architecture for the functions to be performed makes it very efficient: rather than high raw power and low efficiency, the FPGA has average raw power (compared to that of an ASIC) and high efficiency.

[0081] The FPGA used here comprises a plurality of DSP blocks (DSP for Digital Signal Processing), which are elementary and identical blocks. The FPGA comprises, for example, 2928 DSP blocks similar to the one visible in [Fig.3].

[0082] With reference to [Fig.4], the calculation circuit 4 comprises a plurality of calculation structures 5 each comprising a plurality of calculation chains 6. In the same calculation structure 5, all the calculation chains 6 are identical and operate in the same way.

[0083] Each calculation structure 5 has a matrix architecture comprising N rows, and therefore N calculation chains 6, and M columns.

[0084] It is considered that the calculation structure 5 comprises two main inputs: a first main input EPI, through which data vectors are introduced horizontally into the calculation structure 5, and a second main input EP2, through which data vectors are introduced vertically into the calculation structure 5. The data vectors have predefined and potentially different sizes.

[0085] We will speak here of primary data to speak of the data introduced horizon tally, and secondary data to talk about data introduced vertically (the terms "primary" and "secondary" have no other meaning than this).

[0086] To maximize the number of operations performed, a 2D structure is used. Each data item in a data vector of one of the main inputs is multiplied by each data item in the data vector of the other main input. This structure minimizes the incoming bandwidth requirement by maximizing data reuse (unlike 3D structures).

[0087] For example, according to the structure of the invention, to carry out n MACs per clock cycle, it is necessary to provide 2x"Vn data. Thus, if n = 256 (256 multiplications per cycle), it is necessary to provide 2x16 operands per cycle. In the matrix architecture of the calculation structure 5, the number of rows N is therefore equal to "Vn (i.e. 16 in our example) and the number of columns M is also equal to "Vn (i.e. 16 in our example).

[0088] The secondary data introduced by the second main input EP2 are here weights, that is to say mask components.

[0089] The number of lines N is for example equal to the number of pixels of the input tensor 1 (but not necessarily), and the number of columns M is for example equal to the number of channels of the output tensor 3, and therefore to the number of masks 2 (but not necessarily).

[0090] At each clock cycle, a sub-pixel of an input channel is therefore provided for each of the N pixels of the input tensor, and a weight for each of the M output channels.

[0091] The calculation therefore requires a number of clock ticks equal to the number of weights of the mask to calculate NxM output sub-pixels, that is to say M sub-pixels of N pixels.

[0092] The fully calculated output subpixels come out at the end of the chain at each clock tick if the number of channels of the output tensor is equal to M. Otherwise, as will be seen below, a post-accumulation is performed. This is a systolic type structure but without the propagation of the input information from one to the next; this structure minimizes the need for incoming bandwidth.

[0093] Typically, the weights are stored internally in the FPGA and the data (sub-pixels of the input tensor) are “streamed” (i.e. transmitted successively in a continuous flow).

[0094] With reference to [Fig.5], each calculation chain 6 comprises a number M of calculation blocks 7. The calculation blocks 7 comprise an input block 7a, an output block 7b and intermediate blocks 7c positioned between the input block 7a and the output block 7b.

[0095] Each calculation block 7 comprises a first multiplexer 8, a multiplier 9 and an accumulator 10 (or adder).

[0096] The first multiplexer 8, the multiplier 9 and the accumulator 10 each comprise a first input El, a second input E2 and an output S.

[0097] The first input El of the multiplier 9 is connected to the output S of the first multiplexer 8. The first input El of the accumulator 10 is connected to the output S of the multiplier 9.

[0098] The first inputs El of the first multiplexers 8 of all the calculation blocks 7 are connected to each other, and the second inputs E2 of the first multiplexers 8 are connected to each other, which parallelizes the structure of the calculation chain 6.

[0099] The output S of the accumulator 10 of each of the input blocks 7a and intermediate blocks 7c is connected to the second input E2 of the accumulator 10 of the following block 7. The output S of the accumulator 10 of the output block 7b is connected to the second input E2 of the accumulator 10 of the input block 7a.

[0100] The calculation chain 6 further comprises an output accumulator 11 having a first input El connected to the output S of the accumulator 10 of the output block 7b.

[0101] The calculation chain 6 also comprises a first buffer 12 (or buffer memory) connected to the first input El of the multiplexer 8 of the input block 7a, a second buffer 14 connected to the second input E2 of the first multiplexer 8 of the input block 7a, and a third buffer 15 having an input connected to an output S of the output accumulator 11 and an output connected to a second input E2 of the output accumulator 11.

[0102] It is therefore understood that the calculation circuit 4 comprises a number N of calculation chains 6 forming rows of a matrix structure, and that each calculation chain 6 comprises a number M of calculation blocks 7. The second inputs E2 of the multipliers 9 of the calculation blocks 7 of the same column are connected to each other. The calculation circuit 4 (and therefore each calculation chain 6) is connected to a source 16 which provides the input data (primary and secondary data). The source 16 is here a source external to the FPGA, for example an external memory, not integrated in the FPGA (DDR4 type memory or other), but it could be another element, and for example a buffer internal to the FPGA, a calculation block, etc.

[0103] The calculation circuit 4 further comprises primary RAM blocks 17 and secondary RAM blocks 18 (RAM). The blocks 17 and the blocks 18 are here internal memories of the FPGA and are connected to the source 16.

[0104] The primary data and the secondary data are applied for each calculation chain 6 respectively via the block 17 associated with said calculation chain and via the blocks 18.

[0105] The blocks 17, 18 make it possible to provide the calculation chains 6 with the same primary or secondary data several times without needing to access the source 16 from which the primary data and the secondary data come each time.

[0106] Indeed, the external memory (DDR) is "slow" compared to the internal need. On the other hand, as we only calculate "M" channels at a time, it is necessary to do several sequences calculation if the number of channels is greater than M. In this case, the incoming data must be reread as many times as there are calculation sequences. Rather than using the external DDR (slow and consuming a lot of energy), this data has been "buffered" in internal RAM to resupply it.

[0107] This also applies to 3x3xn (or more) masks. The same pixel participates in the calculation of the 8 neighboring pixels (in addition to itself). It would therefore be necessary to read the same data 9 times x number of calculation sequences.

[0108] For blocks 18 (typically the weights), the weights must be reread for each new group of N pixels to be processed.

[0109] Each calculation chain 6 thus comprises a block 17 comprising an output connected to the input of the first buffer 12 and to the input of the second buffer 14.

[0110] The calculation circuit 4 comprises as many blocks 18 as calculation blocks 7 per calculation chain 6.

[0111] Thus, referring to the matrix architecture of the computing structure 5, the computing structure 5 comprises N p blocks 17 and M blocks 18.

[0112] The output of each block 18 is connected to the second inputs E2 of the multipliers 9 of the calculation blocks 7 of the corresponding column.

[0113] The primary data applied as input to each calculation chain 6, via the associated memory block 17, therefore come from the input tensor 1 (which is here a 3D image).

[0114] The secondary data applied via blocks 18 are weights of a mask 2.

[0115] It is noted that it would be possible to apply the weights at the input of the calculation chains 6, via the blocks 17, and the data from the input tensor 1 via the blocks 18.

[0116] A sub-pixel of the input tensor channel will therefore be read several times in block 17, the first buffer 12 and the second buffer 14, which reduces the need for DDR bandwidth.

[0117] Block 17 makes it possible to reduce the number of accesses to external memory 16 (DDR). Buffers 12 and 14 make it possible to reduce the number of accesses to block 17.

[0118] Thus, DDR 16 has a large memory capacity but a low bandwidth. Block 17 has an average memory capacity and an average bandwidth (typically realized in RAM blocks in an FPGA) and buffers 12 and 14 are high bandwidth but low capacity (typically realized in flip-flops or distributed RAM on Xilinx FPGAs). Note that it is not mandatory to have the same dimension for the number of lines N and the number of columns M in the calculation structure 5, but that it is the square which minimizes the bandwidth requirement.

[0119] To maximize efficiency, that is, for each multiplier to perform a useful multiplication at each clock cycle, the values ​​of N and M must respect certain constraints: - N must be less than or equal to the number of channels of each mask and the number of channels of the output tensor; - the number of pixels of the input tensor should be a multiple of M; - the number of elements (sub-pixels) of the mask must be a multiple of N.

[0120] We now explain how the calculations are carried out.

[0121] For each calculation chain 6, the calculation circuit 4 is arranged to, in such a way clocked by clock cycles, apply to the first inputs of the multipliers 9 of the calculation blocks 7 of said calculation chain 6 a succession of primary data forming at least one primary vector, and apply to the second inputs of the multipliers 9 of the calculation blocks 7 of said calculation chain 6 a succession of series of secondary data forming secondary vectors. Each calculation block 7 of said calculation chain 6 is arranged to, at each clock cycle, add a result of a multiplication carried out by said calculation block with a value accumulated at the previous clock cycle by the accumulator of said calculation block or of another calculation block of said calculation chain, the calculation chain thus being arranged to successively produce results of multiplications of the at least one primary vector by each secondary vector.

[0122] In [Fig.5], each calculation block 7 of the calculation chain 6, at each clock cycle, adds a result of a multiplication carried out by said calculation block with a value accumulated at the previous clock cycle by the accumulator of a previous calculation block of said calculation chain.

[0123] To illustrate the invention, we first consider a configuration in which the calculation blocks 7 do not include a first multiplexer 8. This configuration is illustrated in [Fig.6].

[0124] The calculation circuit 4 applies, in a manner timed by clock cycles, a succession of primary data to the input of each calculation chain 6 - and therefore, as there are no multiplexers 8, these data are all applied to the first inputs El of the multipliers 9: at each clock cycle, a new and same data item is therefore applied to all the first inputs El of the multipliers 9, via the first memory block 17 associated with said calculation chain 6.

[0125] Similarly, at each clock cycle, a different weight is applied to the second input E2 of each multiplier 9, via the second associated memory block 18.

[0126] At each clock cycle, each calculation block 7 performs a multiplication, an accumulation, and the transfer of the data contained in its accumulator to the accumulator of the following calculation block.

[0127] With reference to [Fig.7], we consider the case where the primary data comprises a vector of four components 10; 11; 12; 13, and the case where each calculation chain 6 comprises four calculation blocks 7.

[0128] Here we seek to perform the multiplication of a pixel of the input tensor, comprising four sub-pixels (10; 11; 12; 13), with six 1x1x4 masks, to produce a six-channel pixel at the output. [Fig.7] only illustrates the calculation of the first output channel and the start of the calculation of the second channel.

[0129] In the first clock cycle, component 10 is applied to the first inputs El of multipliers 9, and the weights applied (secondary data) are:

[0130] W00 for input block 7a, no weight on first intermediate block 7c, no weight on the second intermediate block 7c, no weight on the output block 7b, or a series of weights which is represented in the following way:

[0131] W00

[0132] Input block 7a therefore calculates IOxWOO.

[0133] In the second clock cycle, component II is applied to the first El inputs of multipliers 9, and the weights applied are:

[0134] Wll;W10;-;-.

[0135] Input block 7a therefore calculates IlxWll.

[0136] The first intermediate block 7c therefore calculates IlxW 10 and adds the result of this multiplication with the result of the previous multiplication (IOxWOO).

[0137] The different calculations carried out are visible in [Fig.7] for the clock cycles up to the ninth cycle.

[0138] We therefore note that at the end of the fourth clock cycle, the calculation of the multiplication of the primary vector comprising the components 10; 11; 12; 13 (row vector), by the secondary vector comprising the weights W00; W10; W20; W30 (column vector), is finished.

[0139] Similarly, at the end of the fifth clock cycle, the calculation of the multiplication of the vector comprising the components 10; 11; 12; 13, by the vector comprising the weights W01; W11; W21; W31, is completed.

[0140] At the end of the ninth clock cycle, the calculation chain has therefore calculated the product of the vector:

[0141] (JO / 1 72 / 3)

[0142] by each of the weight vectors:

[0143] IW0Q\, / won IWQ2\ IW03\, / W04\ 9 IWQ5\ W10 Wll W12 W13 W14 W15 W20 W21 W22 W23 W24 W25 \W30 / ' W31 / \W32I W33! \W34 / \W35i

[0144] Thus, the calculation chain 6, at each clock cycle and for each calculation block, accumulates the result of a multiplication carried out by said calculation block with an accumulation carried out by the preceding calculation block. For the input block 7a, the preceding calculation block is the output block 7b. This makes it possible to “physically” shift in the calculation chain 6 the intermediate results produced as and when by each calculation block 7, simultaneously with the calculations, to extract the results in series. The availability of the results is thus spread over time.

[0145] This simultaneity therefore makes it possible to maximize the useful calculation cycles, while minimizing the input and output bandwidth to “extract” the results.

[0146] We avoid congestion or a heavy structure as in a systolic architecture.

[0147] This structure, however, has the disadvantage of introducing “pauses” when changing the pixel to be processed (i.e. when the calculation moves from one data set to another), which leads to a loss of performance.

[0148] The lost cycles, visible in [Fig.7], correspond to clock cycles during which the corresponding calculation blocks are not working; they therefore cause a reduction in performance.

[0149] The first multiplexers 8 compensate for this weakness by allowing a “bevel” without loss of calculation cycle between two sets of data. The first multiplexers 8 are therefore optional and make it possible to maximize the calculation efficiency.

[0150] The calculation circuit 4, in a manner timed by clock cycles, applies to the first inputs El of the first multiplexers 8 a succession of first primary data forming a first primary vector, applies to the second inputs of the multiplexers 8 a succession of second primary data forming a second primary vector, and applies to the second inputs E2 of the multipliers 9 a succession of series of secondary data forming secondary vectors.

[0151] Thus, at each clock cycle, a new first primary data item is applied to all the first inputs El of the first multiplexers 8, a new second primary data item is applied to all the second inputs E2 of the first multiplexers 8 (except during a predefined number of clock cycles: see below), and a new series of secondary data items is applied to the second inputs E2 of the multipliers 9.

[0152] At each clock cycle, each calculation block 7 performs a multiplication, an accumulation, and the transfer of the data contained in its accumulator 10 to the accumulator 10 of the following calculation block. For the output block 7b, the following calculation block is the input block 7a. Note that this looping is optional. For example, it is not necessary in the example of Figures 7 and 8, where the calculation of an output is finished after 4 cycles (therefore does not require looping).

[0153] At each clock cycle and for each calculation block 7, the calculation chain 6 therefore accumulates a result of a multiplication carried out by said calculation block 7 with an accumulation carried out by the previous calculation block, so as to produce suc successively and serially the results of multiplications of the first primary vector by each secondary vector and then of the second primary vector by each secondary vector. For input block 7a, the previous calculation block is output block 7b.

[0154] By “secondary data series” is meant the data which, at a clock cycle, are applied to the different second inputs of the multipliers 9 of the calculation blocks 7 of the calculation chain 6 (and therefore to all the calculation chains 6 of the calculation structure 5).

[0155] The first primary data form a first primary vector: (70 II 72 73),

[0156] and the second data form a second primary vector:

[0157] (70' 71' 72' 73') •

[0158] The first primary vector and the second primary vector are portions of the 3D image (particular channel of the input image).

[0159] The secondary data form secondary vectors, which themselves form the weight vectors visible above.

[0160] With reference to [Fig.8], the calculation circuit 4 applies the secondary data series according to a predefined cyclic sequence.

[0161] This predefined cyclic sequence corresponds to the sequence of [Fig.7].

[0162] The secondary data series are therefore applied successively according to the predefined sequence:

[0163] W00

[0164] Wll;W10;-;-,

[0165] W22 ; W21 ; W20 ; -,

[0166] W33 ; W32 ; W31 ; W30,

[0167] W04 ; W03 ; W02 ; W01,

[0168] W15 ; W14 ; W13 ; W12,

[0169] - ; W25 ; W24 ; W23,

[0170] - ; - ; W35 ; W34,

[0171] -;-;-;W35,

[0172] The predefined cyclic sequence comprises cycles each comprising a first phase during which the secondary data series comprise an increasing number of components of successive rows or columns (here rows) of a matrix comprising the third vectors, a second phase during which the secondary data series comprise a constant number of components of successive rows or columns (here rows), and a third phase during which the secondary data series comprise a decreasing number of components of successive rows or columns (here rows) of said matrix.

[0173] This matrix is:

[0174] / woo VEO1 VE02 VE03 VE04 VE05\ VE10 Wll VE12 VE13 VE 14 VE 15 VE20 W21 VE22 VE23 ÏE24 VE25 \VE30 W31 W32 VE33 VE34 VE35 /

[0175] The first phase corresponds to clock cycles 1 to 4.

[0176] The second phase corresponds to clock cycles 5 and 6.

[0177] The third phase corresponds to clock cycles 7 to 9.

[0178] The calculation circuit 4 starts the application of the first primary data to the first inputs El of the first multiplexers 8 then, after a predefined number of clock cycles, starts the application of the second primary data to the second inputs E2 of the first multiplexers 8.

[0179] Here, the predefined number is equal to six (because this corresponds to the need to calculate 6 output channels per pixel): the calculation circuit introduces into the calculation chain 6 the second data 10' after six clock cycles, i.e. at the seventh clock cycle.

[0180] The calculation circuit 4 controls the first multiplexers 8 so that the calculation chain 6, simultaneously, completes a multiplication of the first primary vector by each of the secondary vectors and begins a multiplication of the second primary vector by each of the secondary vectors.

[0181] Thus, we see in Figure 8 that in the eighth cycle and in the ninth cycle, the calculation chain 6 finishes carrying out the multiplication of the first primary vector (Z0 ZI Z2 / 3 ) by the secondary vector ( VE04 1E14 VE24 W34) and by the secondary vector ( VE05 VE15 VE25 VE35 ), and begins the multiplication of the second primary vector ( / 0' / 1' / 2' / 3') by the secondary vector (W00 VE 10 W20 VE3O), by the secondary vector (VEO1 VE 11 VE21 VE31), by the secondary vector (VE02 VE 12 VE22 VE32) and by the secondary vector (VE03 VE13 VE23 VE33)-

[0182] Thus, the calculation chain 6 receives two different data vectors and it is possible to select which multiplier receives which vector. This makes it possible to “interleave” the calculations of consecutive pixels and to no longer pause between the processing of two successive pixels. This maximizes the efficiency of the calculation circuit 4. The highest possible efficiency is obtained if all the MACs are used for a calculation phase.

[0183] Note that the multiplexing mechanism is inserted here for the first main EPI input (i.e. for the data entering the calculation chain 6 via the block 17), but it would be possible to insert it for the second main input EP2 (i.e. for data entering calculation chain 6 via blocks 18).

[0184] Note that the first buffer 12, the second buffer 14 and the block 17 are optional and depend on the needs and characteristics of the memory architecture.

[0185] They allow to reduce the bandwidth requirement of RAM memory (for data) and DDR4. It may be possible to share RAM memory for several buffers.

[0186] It would be possible to have the same principle for the weights, an intermediate buffer and less RAM memory (for the weights), and / or to stream the weights from the DDR4 if the RAM for the weights is insufficient in size to store all the weights.

[0187] The output accumulator 11 is optional and essential only for accumulations too large for the DSP blocks.

[0188] Indeed, as we saw earlier, the fully calculated output sub-pixels come out at the end of the chain at each clock stroke if the number of channels is equal to M.

[0189] Otherwise, the third buffer 15 and the output accumulator 11 allow post-accumulation to be carried out. The calculated sub-pixel is then available at the output of the accumulator 11.

[0190] The calculation structure 5 described makes it possible to integrate two calculation blocks per DSP block.

[0191] In [Fig.4], the calculation structure 5 has a matrix architecture comprising a plurality of calculation chains 6 forming lines of the matrix architecture. The calculation circuit 4 may comprise DSP blocks in each of which two calculation blocks belonging to two successive lines are produced.

[0192] An example of such a DSP block 20 is seen in [Fig.3].

[0193] To integrate two calculation blocks per DSP block, the calculations must be carried out with INT8 inputs (this is the standard type for deep learning model inference), and one of the two operands must be identical for both multiplications. So you have to use the same weight with two data, or the same data with two weights.

[0194] It is also appropriate to group the calculations of two consecutive lines. It is also possible to group two columns, but this grouping is not compatible with the bevel between pixels (because the same value or the same weight would be required for two columns).

[0195] We see in [Fig.3] the chaining between DSP blocks: path 21 is taken by incoming data coming from another DSP block, and path 22 is taken by outgoing data to another DSP block.

[0196] It is further noted that the structure with multiplexers allows the multipliers to be used at a frequency twice that of the rest of the structure.

[0197] It would thus be possible to divide by two the number of multipliers used in doubling the frequency of the multipliers relative to the rest of the structure (the ratio of two being only a typical example). The two variants can be combined.

[0198] The calculation structure 5 which has just been described can be integrated into a pipelined architecture.

[0199] In fact, the outputs are made at the same rate as the inputs, which simplifies the bus structure and allows the structures to be chained together for a pipelined architecture.

[0200] It can be seen in [Fig.9] that the calculation circuit 4 comprises a series of three calculation structures 5a, 5b, 5c forming a pipeline of structures.

[0201] Structure 5a, structure 5b and structure 5c respectively perform the calculations of convolution products for a first convolution layer, a second convolution layer and a third convolution layer of the neural network.

[0202] The operation of each calculation structure 5, which has just been explained, is perfectly adapted to this pipelined architecture. Indeed, as soon as the result of the convolution product of a vector by a vector of the weight matrix, carried out by a particular structure, is available, this result can be used by the following structure without the calculations of the particular structure being completed.

[0203] It is noted that the neural network model may have intermediate layers between the convolution layers (ReLu type activation layer, pooling layer, elementwise layer, etc.). The structures representing these layers are not shown here but may be inserted between the structures 5, in accordance with the algorithmic needs.

[0204] In the case of an architecture developed for a particular network, it is possible to adapt the size of the structures to the layers produced (data sizes, mask sizes, etc.). In particular, it is possible to select these sizes so that the flow rate and order of the data at the inputs and outputs of each structure are the same. Thus, the outgoing vector of structure 5a is directly transmitted to structure 5b with the same flow rate and the same order (same from 5b to 5c). It is noted that the structures may need buffering of the incoming or outgoing data, depending on the size of the masks and the data (in particular if the mask uses several lines of pixels of the input image).

[0205] The pipelined architecture is of great interest for optimizing bandwidth. Indeed, the bandwidth of an external memory, such as DDR, is relatively low compared to the bandwidths of internal memories. Chaining the structures into a pipeline makes it possible to reduce the demands on the external memory by directly transmitting the data from one computing stage to another, without going through the DDR.

[0206] Less external memory bandwidth is then required to achieve the same amount of calculations. Alternatively, it is possible to perform more calculations in parallel for a given bandwidth.

[0207] It is noted that the pipelined architecture also reduces the electrical consumption of the component in which the calculation circuit 4 is implemented, since an external memory consumes significantly more than the internal memories.

[0208] The calculation structure 5 which has just been described is therefore very advantageous.

[0209] The structure requires few resources outside of the “DSP blocks”: the mul- T-folders, accumulators, multiplexers, shift chain, etc. are already included in the DSP blocks. The resource requirement is therefore minimal.

[0210] As seen, each component of a vector applied as input to the calculation chain 6 is multiplied by each data item of the weight vector (as in a systolic architecture). This structure (2D) makes it possible to minimize the need for incoming bandwidth by maximizing the reuse of data (unlike 3D structures).

[0211] The 2D architecture used indeed has an excellent bandwidth multiplier factor. With a 2D architecture, the multiplication factor is n: by providing 16 data and 16 weights, it is possible to perform 256 multiplications. By comparison, a Tensor Cores type architecture has a multiplication factor equal to "Vn": by providing 16 data and 16 weights, it is possible to perform only 64 multiplications.

[0212] The input bandwidth is therefore minimized, as is the output bandwidth. We obtain homogeneous incoming data flows (like 3D structures). Thus, the outputs are at the same rate as the inputs.

[0213] It is not necessary that the data vector applied as input to each calculation chain, and the data vector applied via the second memory blocks, have equal sizes.

[0214] In an FPGA, a constraint typically encountered is the size of the accumulators. These accumulators may not be large enough to store the accumulation results for large masks.

[0215] The output accumulator 11, accumulating the high weights, can therefore be added at the end of the shift chain to carry out the accumulation. The advantage of this configuration is that only one accumulator per calculation chain 6 is necessary (and not one per accumulator), which reduces the necessary resources. The size of the parallelized accumulators in the chain could be limited to:

[0216] nb bit operand A + nb bit operand B + rounding up (log2(L)),

[0217] where A is the vector applied via the first memory module 17 and B is the vector applied via the second memory modules 18, and where L is the length of the calculation chain 6, i.e. the number of calculation blocks 7 per chain 6 (equal to M).

[0218] This also has an advantage for an ASIC, where the dimensions are adjustable.

[0219] Conversely, if the accumulators of the calculation blocks are sufficiently large or if one seeks to minimize their size, it is possible to exploit the accumulators of the multiplier-accumulator chain to the maximum by reinjecting the calculations at the end of the chain to the input of the chain. This assumes appropriate control and organization of the data and weights.

[0220] Advantageously, to maximize the efficiency (the yield) of the structure, in the embodiment of [Fig.5] a dimensioning criterion is respected according to which:

[0221] Nmask = KM,

[0222] where Nmask is the size of a mask, K is a positive integer and M is the width of the computational structure 5 (i.e. the number of computational blocks 7 per computational chain 6). The size of a mask is equal to the product of the number of rows times the number of columns of a 2D mask, multiplied by the number of channels of the mask.

[0223] It is also advantageous to have:

[0224] Cost> M,

[0225] where Coutest is the number of channels of the output tensor. Indeed, as an input data is multiplied M times, it is therefore necessary to contribute to M results.

[0226] The number of pixels in the output tensor must also be a multiple of the number of rows in the matrix architecture of the computational structure.

[0227] As the output is only available at the end of the chain, it is appropriate to introduce a “periodicity” into the calculations.

[0228] Another embodiment of the invention is now described with reference to [Fig. 10]. This embodiment makes it possible to remove the constraint which has just been mentioned.

[0229] Each calculation chain 106 of the calculation circuit 104 again comprises a plurality of calculation blocks 107 comprising an input block 107a, an output block 107b and intermediate blocks 107c.

[0230] Each calculation block 107 comprises a first multiplexer 108, a multiplier 109 comprising a first input connected to an output S of the first multiplexer 108, and an accumulator 110 having a first input connected to an output of the multiplier 109. The first inputs of the first multiplexers 108 are connected to each other and the second inputs of the first multiplexers 108 are connected to each other.

[0231] The calculation circuit again comprises primary RAM blocks 117 (internal RAM) and secondary RAM blocks 118 (internal RAM). Blocks 117 and blocks 118 are memories internal to the FPGA and are connected to source 116 (memory external to the FPGA).

[0232] The calculation chain 106 optionally includes (this depends on the size of the batteries emulators and the size of the masks) an output accumulator 111 having a first input connected to the output of the accumulator 110 of the output block 107b.

[0233] The calculation chain 106 also comprises a first buffer 112 connected to the first input El of the first multiplexer 108 of the input block 107a, a second buffer 114 connected to the second input of the first multiplexer 108 of the input block 107a, and a third buffer 115 having an input connected to an output of the output accumulator 111 and an output connected to a second input of the output accumulator 111.

[0234] This time, however, the output S of each accumulator 110 is not connected to the accumulator of the next block; the physical shift from one accumulator to the other is therefore eliminated.

[0235] The accumulator 110 here has only one input. Each new input value is added to the internally stored value of the accumulator.

[0236] At each clock cycle, the calculation block 107 therefore adds the result of the multiplication carried out by said calculation block 107 with the value accumulated at the previous clock cycle by the accumulator 110 of said calculation block 107.

[0237] It is noted that it would be possible to implement block 110 with an adder comprising two inputs. One of these two inputs is then connected to the output of multiplier 109, and the other of these two inputs is connected to the output of said adder. Looping the output of the adder back to one of its inputs creates an accumulator. Each calculation block 107 also comprises, to implement this configuration, a flip-flop stage sequenced by a clock.

[0238] The calculation chain 106 this time comprises an output multiplexer 113. For each calculation block 107, the output S of the accumulator 110 of said calculation block 107 is connected to a separate input of the output multiplexer 113.

[0239] The output of the output multiplexer 113 is connected to the first input of the output accumulator 111.

[0240] The output multiplexer 113 therefore acquires as inputs all the outputs of the accumulators 110 and selects one.

[0241] The output multiplexer 113 performs a selection which replaces the physical offset of the calculation results of the calculation blocks 7.

[0242] It is noted that, in the two cases described earlier (block 110 produced by an accumulator or by an adder), it is appropriate to initialize the value resident in block 110 when the value of block 110 is read by the output multiplexer 113.

[0243] In this second embodiment, it is not necessary to respect the sizing criterion mentioned earlier to optimize the yield.

[0244] To obtain maximum efficiency, the number of pixels in the output tensor must again be a multiple of the number of rows in the matrix architecture of the computational structure.

[0245] The size of the mask per output subpixel must also be greater than or equal to M.

[0246] Each input must contribute at least M multiplications in parallel, and therefore an incoming data must contribute to M outputs. The number of outputs to which the input contributes depends on the number of channels per output pixel and the size of each mask. On a 3x3 mask, an incoming data contributes to 9 output pixels (x the number of channels per output).

[0247] It also depends on the stride (number of pixels the mask moves by after each operation), groups, etc.

[0248] The number of output channels must also be greater than or equal to M. Indeed, by having a single output multiplexer, there can only be one calculation that ends per cycle, in order to be able to extract it.

[0249] It is therefore possible to have high efficiency while using the same computing structure for several layers.

[0250] With the output multiplexer 113, the physical shifting of the calculation results in the chain is no longer necessary, because the output multiplexer 113 allows any accumulator 110 to be selected for the output. However, it is still necessary to start the calculations in a shifted manner to be able to have only one result per cycle.

[0251] Another embodiment of the invention will now be described with reference to [Fig. 11].

[0252] Each calculation chain 206 of the calculation circuit 204 again comprises a plurality of calculation blocks 207 comprising an input block 207a, an output block 207b and intermediate blocks 207c.

[0253] Each calculation block 207 comprises a first multiplexer 208 comprising a first input El, a second input E2 and an output S, and a second multiplexer 208' comprising a first input El, a second input E2 and an output S.

[0254] Each calculation block 207 further comprises a multiplier 209 comprising a first input connected to the output S of the first multiplexer 208, and an accumulator 210 having a first input connected to an output of the multiplier 109 and a second input connected to the output S of the second multiplexer 208'.

[0255] The first inputs El of the first multiplexers 208 of the calculation blocks 207 of the calculation chain 206 are connected to each other, and the second inputs E2 of the first multiplexers 208 of the calculation blocks 207 of the calculation chain 206 are connected to each other.

[0256] The output of the accumulator 210 of each calculation block 207 is connected to the second input E2 of the second multiplexer 208' of said calculation block, but also to the first input El of the second multiplexer 208' of the following calculation block 207 in the calculation chain 206.

[0257] The output of the accumulator 210 of the output block 207b is connected to the first input El of the second multiplexer 208' of the input block 207a.

[0258] The calculation circuit again comprises primary RAM blocks 217 (internal RAM) and secondary RAM blocks 218 (internal RAM, through which the primary and secondary data from the source 216 are introduced.

[0259] The calculation chain 206 optionally comprises (this depends on the size of the accumulators and the size of the masks) an output accumulator 211 having a first input connected to the output of the accumulator 210 of the output block 207b.

[0260] The calculation chain 206 also comprises a first buffer 212 connected to the first input El of the multiplexer 208 of the input block 207a, a second buffer 214 connected to the second input of the multiplexer 208 of the input block 207a, and a third buffer 215 having an input connected to an output of the output accumulator 211 and an output connected to a second input of the output accumulator 211.

[0261] This architecture, with two multiplexers 208, 208' in each calculation block 207, has a number of advantages.

[0262] It allows to efficiently manage more mask geometries and, in particular, depthwise convolutions (or calculations in which the mask is small). It simplifies the organization of the operands (in particular the weights). It is "free" (in terms of resources) on a Xilinx FPGA, because looping is natively supported by the DSP block, via dynamic parameterization. It is compatible with the other variants already described, and therefore allows to adapt the sequencing of the calculations to the characteristics of the mask, without modifying the structure.

[0263] [Fig. 12] shows the representation of the internal loopback of a DSP block. The PCIN (reference 23) corresponds to the input of the previous DSP block. The bold line 24 corresponds to the loopback. The Z (reference 25) is the second multiplexer 208'.

[0264] The embodiments of the calculation structure 6, 106 and 206 which have just been described are also compatible with unpruning options and, in particular, with structured pruning (for example 2 unharmed weights among 4) - and this, efficiently in an FPGA and very efficiently in an ASIC.

[0265] Pruning is a method of removing weight connections in a network to increase inference speed and reduce model storage size. This pruning is performed during training.

[0266] In the case of structure 6, to make said calculation structure compatible with pruning, each calculation block comprises an additional multiplexer having an output connected to the second input of the accumulator 10 of said calculation block.

[0267] The goal is not to multiply by 0, but to occupy the multiplier anyway. But, the input is the following, because it is useful for other multiplications. The idea is therefore to do the multiplication for the following cost, that is to say by using not the previous accumulation, but the accumulation between before.

[0268] This also assumes that the 0s in the weights are removed (which can be done in preprocessing, requiring additional memory (to store the position of the 0s and jumps) or on the fly, requiring higher resources and bandwidth).

[0269] In the case of structure 106, to make said calculation structure compatible with lepruning, the output remains locally in the accumulator (and it cannot be moved because it would overwrite another useful data). It is therefore necessary to provide different data at the input of the multiplier (it is also assumed that the weights at 0 are not provided to the structure). It would then be necessary to have a multiplexer before the multiplier and provide the following data to the multipliers (which will then be provided to the other multipliers of the same line).

[0270] The additional multiplexer has one or more inputs connected to one or more accumulators of one or more calculation blocks of the same calculation chain. For pruning, the additional multiplexer of each of the intermediate blocks and of the output block has inputs connected to the outputs of the accumulators of the preceding block(s) of the same calculation chain.

[0271] The additional multiplexers make it possible to select another accumulation result from the same calculation chain (including the direct precedent and the other precedents).

[0272] The computational structure may also be 0-skipping compatible.

[0273] The additional multiplexer has one or more inputs connected to one or more accumulators of one or more calculation blocks of another calculation chain.

[0274] We know that at the output of a ReLU activation function, all negative values ​​are rounded to 0.

[0275] The outputs of the ReLU functions are multiplied by the weights of the next neuron.

[0276] 0-skipping consists of "skipping" multiplications by values ​​that are 0.

[0277] This method has no impact on the performance of the algorithm, since only unnecessary calculations are "skipped".

[0278] The gain depends on the proportion of data at 0 and, potentially, on their spatial dispersion.

[0279] 0-skipping makes parallelization more difficult - but reduces the need for parallelization.

[0280] 0-skipping is achieved by positioning again, in each calculation block, an additional multiplexer at the input of the second input of the accumulator. The additional multiplexer this time has inputs connected to one or more block accumulators calculation of another calculation chain (above or below) or of the same calculation chain.

[0281] This implies having one or more inputs at 0. But, the same weights are always required (because other pixels may need them). So, if we always do the same number of multiplications with the same weights, but skipping data, we must calculate neighboring pixels. We must therefore shift the calculations from one line to another and memorize the accumulation(s) not calculated in this cycle.

[0282] It is noted that registers should be provided to store the accumulation results which are not used in the next stage, but which are used in a subsequent stage.

[0283] Implementations of pruning and 0-skipping can be combined, which is particularly advantageous, since no state-of-the-art structure can simultaneously and efficiently support pruning and 0-skipping.

[0284] The calculation structure, via adapted control and additional addition stages, is also compatible with the Winograd transform, which makes it possible to reduce the number of multiplications to be carried out for a 3x3 or higher mask convolution. We know that multiplication requires more resources than addition in a hardware architecture. The advantage of the Winograd transform is that it makes it possible to reduce the number of multiplications and therefore the resources required.

[0285] The principle of the Winograd transform is to factor multiplications for neighboring positions.

[0286] However, the Winograd transform increases the number of additions, because it requires performing additions before and after multiplications.

[0287] Thus, for a 3x3 convolution and a 2x2 neighborhood, in classical convolution, 9 multiplications and 8 additions are necessary for each pixel calculated, and therefore 36 multiplications and 32 additions. Using the Winograd transform, 16 multiplications and 56 additions are necessary. The Winograd transform therefore reduces the number of multiplications to be carried out by a factor equal to 2.25, but it increases the number of additions by a factor equal to 1.75.

[0288] The structure is also compatible with any mask size or any data size.

[0289] Many variations are possible.

[0290] It would be possible to use the column input multiplexing structure as well, allowing more use cases.

[0291] In the calculation structure, it would be possible to invert the rows and columns (multiplexing and shifting the intermediate results on the columns and on the rows).

[0292] For a convolution of a deep learning model, the data is typically provided from the side and the weights from the top. It is therefore possible to invert the data and the weights. This can be of interest depending on the dimensions of the data. Preferably, the information fed from the top comes from the internal memory and the data coming from the side is streamed from the external memory or from another structure (similar or not). If the weights are larger than the data, it is then more interesting to store the data in internal memory and stream the weights from the external memory, which allows minimizing the overall processing time.

[0293] It would also be possible to position a multiplexer on each of the two main inputs.

[0294] It is possible to implement the calculation circuit in any type of FPGA.

[0295] It is also possible to use little or no DSP blocks and to achieve certain or all parts of the computing circuit with FPGA logic.

[0296] It is also possible to implement the calculation circuit in an ASIC, which does not have a predefined structure (like DSP blocks). A greater number of variants would then be possible, allowing other compromises (for example increasing the output bandwidth to minimize the shift duration). Note that this variant is also feasible on FPGA. This would consist of "segmenting" the shift chain to create additional outputs, allowing other variable sizes to be efficiently managed. It is also possible to use more output multiplexers (for the variant without shifting), or even to combine shifting, fixed output at the end of the calculation chain, fixed output at an intermediate point and one or more multiplexers to select other parts of the chain.

[0297] If the computing circuit is made for a specific network, the dimensions of each structure (height, width, size of multipliers and accumulators) can be adjusted to maximize efficiency, or to maximize performance or minimize the resources used, or to obtain a performance / resources compromise, or according to any optimization criterion.

[0298] Of course, the invention is not limited to the embodiment described but encompasses any variant falling within the scope of the invention as defined by the claims.

Claims

Claims

1. Calculation circuit (4; 104; 204) comprising at least one calculation chain (6; 106; 206) comprising a plurality of calculation blocks (7; 107; 207) each comprising a multiplier (9; 109; 209) comprising a first input and a second input, and an accumulator (10; 110; 210) comprising a first input connected to an output of the multiplier, the calculation circuit being arranged to, in a manner clocked by clock cycles, apply to the first inputs of the multipliers of the calculation blocks of said calculation chain a succession of primary data forming at least one primary vector, and apply to the second inputs of the multipliers of the calculation blocks of said calculation chain a succession of series of secondary data forming secondary vectors, each calculation block of said calculation chain being arranged to, at each clock cycle,adding a result of a multiplication carried out by said calculation block with a value accumulated in the previous clock cycle by the accumulator of said calculation block or of another calculation block of said calculation chain, the calculation chain thus being arranged to successively produce results of multiplications of the at least one primary vector by each secondary vector.,

2. Calculation circuit according to claim 1, comprising a number N of calculation chains (6; 106; 206) forming rows of a matrix structure, each calculation chain comprising a number M of calculation blocks (7; 107; 207), the second inputs of the multipliers of the calculation blocks of the same column of said matrix structure being connected to each other.

3. Calculation circuit according to one of the preceding claims, each calculation block of each calculation chain comprising a first multiplexer (8; 108; 208) comprising a first input and a second input, the first input of the multiplier of said calculation block being connected to an output of said first multiplexer, the first inputs of the first multiplexers (8; 108; 208) of the calculation blocks of said calculation chain being connected to each other and the second inputs of the first multiplexers (8; 108; 208) of the calculation blocks of said calculation chain being connected to each other, the calculation circuit being arranged to apply to the first inputs of said first multiplexers a succession of first primary data forming a first primary vector, apply to the second inputs of said first multiplexers (8; 108; 208) a succession of second primary data forming a second primary vector, the calculation chain being arranged to produce successively and in series results of multiplications of the first primary vector by each secondary vector then of the second primary vector by each secondary vector.

4. Calculation circuit according to one of the preceding claims, each calculation block (207) of each calculation chain (206) comprising a second multiplexer (208') arranged so that: - an output of said second multiplexer is connected to a second input of the accumulator (210) of said calculation block; - an output of said accumulator being connected to a first input of said second multiplexer; - an output of said accumulator being connected to a first input of a following block of said calculation chain.

5. Calculation circuit according to one of the preceding claims, in which the calculation blocks (7) of each calculation chain comprise an input block (7a), an output block (7b) and intermediate blocks (7c) positioned between the input block and the output block, the calculation chain (6) being arranged so that an output of the accumulator (10) of each of the input block (7a) and intermediate blocks (7c) is connected to a second input of the accumulator of a following block of said calculation chain.

6. Calculation circuit according to claim 5, in which the output of the accumulator of the output block (7b) is connected to a second input of the accumulator of the input block (7a).

7. Calculation circuit according to one of the preceding claims, in which each calculation chain comprises an output multiplexer (113), said calculation chain being arranged so that, for each calculation block (107), the output of the accumulator (110) of said calculation block is connected to a separate input of the output multiplexer (113).

8. Calculation circuit according to one of the preceding claims, in which each calculation chain (6; 206) further comprises an output accumulator (11; 211) having a first input connected to the output of the accumulator (10; 210) of an output block (7b; 207b) of said calculation chain.

9. A computing circuit according to claim 7, wherein each computing chain further comprises an output accumulator (111) having a first input connected to the output of the output multiplexer (113) of said computing chain.

10. Calculation circuit according to claim 3, further comprising a first buffer (12; 112; 212) connected to the first input of the first multiplexer (8; 108; 208) of an input block (7a; 107a; 207a) of the calculation chain, and a second buffer (14; 114; 214) connected to the second input of the first multiplexer (8; 108; 207) of the input block of the calculation chain.

11. Calculation circuit according to one of claims 8 or 9, further comprising a third buffer (15; 115; 215) having an input connected to an output of the output accumulator (11; 111; 211) and an output connected to a second input of the output accumulator.

12. Calculation circuit according to one of the preceding claims, comprising a primary RAM block (17; 117; 217) for each calculation chain, and a plurality of secondary RAM blocks (18; 118; 218), the primary data and the secondary data being applied for each calculation chain respectively via the primary RAM block associated with said calculation chain and via the secondary RAM blocks, the primary RAM blocks and the secondary RAM blocks making it possible to provide the calculation chains several times with the same primary or secondary data without needing to access each time one or more data sources (16; 116; 216) from which the primary data and the secondary data originate.

13. A computing circuit according to claim 2 or claim 5, the computing circuit being configured to perform a convolution product of an input tensor by a mask of size Nmask to obtain an output tensor, the computing circuit being such that: N = KM mask iv.ivi, where K is a positive integer.

14. Calculation circuit according to claim 3, the calculation circuit being arranged to: - apply the secondary data series according to a predefined cyclic sequence; - start the application of the first primary data on the first inputs of the first multiplexers (8; 108) then, after a predefined number of clock cycles, start the application of the second primary data on the second inputs of the first multiplexers (8; 108); - control the first multiplexers so that the calculation chain (6; 106), simultaneously, finishes a multiplication of the first primary vector by each of the secondary vectors and begins a multiplication of the second primary vector by each of the secondary vectors.

15. A computing circuit according to claim 14, the secondary vectors forming a matrix, the predefined cyclic sequence comprising cycles each comprising a first phase during which the secondary data series comprise an increasing number of components of successive rows or columns of the matrix, a second phase during which the secondary data series comprise a constant number of components of successive rows or columns of the matrix, and a third phase during which the secondary data series comprise a decreasing number of components of successive rows or columns of the matrix.

16. Calculation circuit according to one of the preceding claims, in which the calculation circuit comprises several calculation chains and an additional multiplexer having one or more inputs connected to one or more accumulators of one or more calculation blocks of the same calculation chain and / or of another calculation chain.

17. Calculation circuit according to one of the preceding claims, comprising a plurality of calculation structures (5a, 5b, 5c) each comprising a plurality of calculation chains (6), the calculation structures being arranged according to a pipeline architecture.

18. A computational circuit according to claim 17, each computational structure being arranged to perform convolution products for a distinct convolution layer of a neural network model.

19. Electronic component in which the calculation circuit according to one of the preceding claims is produced.

20. An electronic component according to claim 19, the electronic component being an FPGA.

21. An electronic component according to claim 20, the FPGA comprising DSP blocks, each calculation block being carried out in a DSP block.

22. Electronic component according to claim 21, the calculation circuit comprising at least one calculation structure having a matrix architecture comprising a plurality of calculation chains forming lines of the matrix architecture, the calculation circuit comprising DSP blocks in each of which two calculation blocks of two successive lines are produced.

23. An electronic component according to claim 19, the electronic component being an ASIC.

Images