METHOD, AND DEVICE, FOR PROCESSING DATA TO BE PROVIDED AS INPUT TO A FIRST SHIFT REGISTER OF A SYSTOLIC NEURONAL ELECTRONIC CIRCUIT

Abstract

TITLE: Method, and device, for processing data to be provided as input to a first shift register of a systolic neuronal electronic circuit. Method for processing, in an electronic processing device, data to be provided as input to a first shift register of a systolic neuronal electronic circuit comprising a grid of elementary processors, said first shift register carrying a data vector of size K at each clock cycle to K columns of elementary processors of said grid, said processing method comprising the following steps implemented by said electronic processing device: obtaining a sequence of successive data D(0), D(1), D(2) …D(d) extracted by reading said data classified consecutively in a first storage memory and storing said data in a second memory of the processing device;generation of vectors W(u) as a function of said stored data, each vector W(u) having K components (W(u)0, …, W(u)K-1) such that W(u)k, k= 0 to K-1= D(K0 + k*S2 + u*S1), where K is an integer strictly greater than 1, K0 is a predefined constant, the steps S1 and S2 are predefined integer constants; supplying, as input to said shift register, the vector W(u), such that successive values ​​W(u) are supplied during successive clock cycles, with u an integer ranging from 0 to L1-1, and L1 an integer constant strictly greater than one. Figure for the abbreviation: Figure 7;

Description

Title of the invention: Method and device for processing data to be provided as input to a first shift register of a systolic neural electronic circuit

[0001] The invention lies in the field of implementing convolutional neural processing (CNN) on reconfigurable computing circuits in embedded equipment. The fields of application of embedded neural networks are numerous, for example in image processing, and / or for classifying data or detecting specific elements on signals, etc.

[0002] These are so-called inference treatments, requiring very high computing power (typically hundreds or thousands of Giga-Operations / s), but very repetitive and lending themselves well to parallelization.

[0003] An efficient way to implement these treatments is to implement a systolic architecture, which can be presented in the form of an at least two-dimensional array of elementary processors (PEs), reduced to their purely arithmetic part, which are fed column by column by an input data flow propagating progressively downwards (in a shift register). This organization, on an FPGA ("field-programmable gate array") or an ASIC ("application-specific integrated circuit"), simplifies the placement-routing operations and makes it possible to best exploit the circuit resources and integrate a large number (hundreds or thousands) of PEs.

[0004] Systolic architectures make it possible to obtain very compact implementations of PEs reduced to their simplest expression, which in particular process the data which pass within their reach over time.

[0005] There are a number of systolic implementations of CNN (Ref: https: / / www.sigarch.org / dnn-accelerator-architecture-simd-or-systolic), for example in CN107578098.

[0006] The range of functionalities offered by the systolic grid depends largely on the composition of the data streams sent to it. However, the data streams entering or leaving the circuit are most of the time connected to one or more DDR (Double Data Rate) type memories, which need to be read or written by sequences of consecutive addresses, which greatly reduces the choice of distribution of this data on the computing grid.

[0007] In [Fig. 1] are shown an external high-capacity DDR memory 10 and a systolic neural electronic circuit 20. The circuit 20 is implemented by a programmable logic circuit, for example FGPA, and the DDR 10 memory is external to the FGPA.

[0008] The circuit 20 comprises a grid 22, here in 2 dimensions (in other embodiments, the dimension is greater than two) of elementary processors PEs, arranged in columns and rows, a row 23 of memories, a data distribution system 27.

[0009] Each memory m is associated with a respective column of PE processors. A memory m receives, from the data distribution system 27, the data that the PEs of the column that is associated with this memory will process and returns their calculation results to the distribution system 27.

[0010] These data are initially provided to the data distribution system 27 by the DDR memory 10 and the results are delivered by the data distribution system 27 to the DDR memory 10

[0011] The distribution system 27 comprises two buses 28, 29 (which can in certain cases be joined into a circular ring bus) each adapted to carry K data (K integer, here equal to 4, K being able to take any value greater than or equal to 1) at each clock cycle.

[0012] Bus 28 is an input bus for data supplied by the DDR 10 memory and bus 29 is an output bus for results to the DDR 10 memory.

[0013] In the same spirit as the systolic architecture of the processors, to promote compactness and speed, each of these buses 28, 29 operates as a shift register. A stage of the register corresponding to the input bus 28 supplies in parallel K (here therefore 4) column memories with data and the addresses of the memories for which these data are intended. A stage of the register corresponding to the output bus 29 collects in parallel K (here therefore 4) data coming from the memories m to which it is connected, and provides the addresses where these data are written in the memories m. At the next clock tick, these data are part of the next stage of the register and are thus presented / collected to the next set of K memories m. The other vertical row of memories, which stores the coefficients of the CNN processing, has not been shown in [Fig.l].

[0014] The DDR 10 memory contains the input data for the processing, for example groups of images called feature maps. This 3D data is stored dimension by dimension in the memory, typically line by line. The data of one dimension (typically the line) is stored at consecutive addresses in the DDR 10 memory. The DDR 10 is read in sequences of consecutive addresses (bursts), with a boot delay at the start of each burst which slows down the useful throughput all the more the shorter the burst.

[0015] As a result, unless a significant loss of efficiency is accepted, DDR 10 readings produce long sequences of adjacent pixels of the same line. The prior art way of connecting the output port of the DDR 10 memory to the distribution system 27 is to use a counter that generates consecutive addresses. Manufacturers such as Xilinx provide IPs optimized for this type of function (CDMA IP for example).

[0016] Thus according to the prior art: - as said above, the dimension (the line) of the input table (or similarly a result) which is contiguous in DDR is found distributed identically on the column memories in the prior art; once this DDR flow is connected to the internal data distribution system of the FPGA, each of the internal memories receives the pixel following that of its neighbor in the read line.

[0017] The dominant operation in CNNs is convolution, which performs the weighted sum of consecutive or spaced input data with a fixed step (called dilation). Convolution can be applied to multi-dimensional input data (row, column, channel, etc.). In the systolic architecture of the neural circuit 20, the calculations are parallelized by distributing one of the dimensions (usually the row) over the columns of the grid.

[0018] A PE of a column then needs to access data that has been placed in neighboring columns. To achieve this, physical paths that provide access to some neighboring memories at the head of the column are generally added to the architecture; these extensions, beyond a certain width, are paid for in increased complexity and risk of limiting the clock frequency. In this extended configuration, however, the architecture can only execute convolutions of a width equal to the number of memories accessible from a column.

[0019] [Fig.2] shows the data distribution of a line of a data structure (consecutive data of a line named D1, D2, D3 ....) possibly multidimensional, presented at the input of the systolic grid 22 for a convolution of the same dimension as the data table stored in DDR 10. Only the line dimension can pose a problem, the other dimensions of the table being local to each memory m, and directly accessible to the PEs of the column associated with the memory m.

[0020] In the figures, the data above the columns are those stored in the columns by the distribution system 27, before the start of the calculations by the PE columns.

[0021] In [Fig.2], Di is thus supplied to the memory mi, for i = 1 to 5. In this corresponding example, a column sees 3 memories, which makes it possible to make convolutions of maximum width equal to 3.

[0022] A frequent case in CNNs is that of so-called convolution stages with "stride" of value Str, where the calculations to be done are no longer as in [Fig.2]: convolution of (D1,D2, D3), convolution of (D2, D3, D4), convolution of (D3,D4,D5), convolution of (D4,D5,D6), but, in the case where the stride Str is worth 2: convolution of (D1,D2,D3), (D3,D4,D5), (D5,D6,D7), ...

[0023] In these cases, in the prior art, even if the PEs perform the calculations, one column out of 2 of PEs (in fact (Str-1) columns all Str columns) calculates results that will not be used, which results in an underuse of resources.

[0024] Other types of operations are not feasible with the supply of data to the systolic neural circuit 20 by the DDR memory 10 as carried out by the prior art or even lead to a loss of computing resources.

[0025] To this end, according to a first aspect, the invention proposes a method for processing, in an electronic processing device, data to be provided as input to a first shift register of a systolic neural electronic circuit comprising a grid of elementary processors, said first shift register conveying a data vector of size K at each clock cycle to K columns of elementary processors of said grid, said processing method comprising the following steps implemented by said electronic processing device: - obtaining a sequence of successive data D(0), D(l), D(2) .. .D(d) extracted by reading said data classified consecutively in a first storage memory and storing said data in a second memory of the processing device; - generation of vectors W(u) as a function of said stored data, each vector W(u) comprising K components (W(u)0, ..., W(u)K4) such that W(u)k> ko àk-1= D(K0 + k*S2 + u*Sl), where K is an integer strictly greater than 1, KO is a predefined constant, the steps S1 and S2 are predefined integer constants; - supplying, at the input of said shift register, the vector W(u), such that successive values ​​W(u) are supplied during successive clock pulses, with u an integer ranging from 0 to Ll-1, and L1 an integer constant strictly greater than one.

[0026] The invention thus makes it possible, by this input / output data management solution, to implement adequate data flows and thus to reduce limitations of the prior art in the performance of a systolic neural circuit, such as convolution of width 1 to 3, loss of efficiency in the case of stride greater than 1, etc.

[0027] In embodiments, a treatment method according to the invention further comprises one or more of the following features: - the following steps implemented by said electronic processing device: • obtaining a data vector w of size K provided by a second shift register of the systolic neural circuit, said second shift register conveying at each clock cycle a data vector of size K corresponding to the results of K respective columns of elementary processors of said grid; • the K components of w being (w0, ..., wK-i), storage in the second memory, from wk ,k=o to K-i at the address indO +K*C*S2 + x*Sl + k*S2 , where indO is a predefined constant, C is the column number and x is an index ranging from 0 to Ll-1; • supplying to the first storage memory a series of successive data extracted by reading said data classified consecutively in the second memory; - for k= 0 to Kl, W(u)k = D(K0 + Y ui*Sli + k*S2), the steps Sli and S2 ^1=1 to n are predefined constants; the vector being provided as input to said shift register, with ui an integer ranging from 0 to Lli-1, and Lli a constant strictly greater than one; - the following steps are implemented by said electronic processing device: • obtaining a data vector w of size K provided by a second shift register of the systolic neural circuit, said second shift register conveying at each clock cycle a data vector of size K corresponding to the results of K respective columns of elementary processors of said grid; • the K components of w being (wO, ..., wK-1), the storage in the second memory of wk, k= 0 to Kl is carried out at the address indO+4*C*S2 + Y ( xi * S li ) + k*S2, where indO is one to Lli-1 predefined constant, C is the column number and xi is an integer index ranging from 0 to Lli-1; • supplying to the first storage memory a series of successive data extracted by reading said data classified consecutively in the second memory; - the second memory comprises at least K separate memory banks, and the storage of data in the second memory is carried out in a manner to verify that data which are components of the same vector W(u) cannot be stored in the same bank.

[0028] According to a second aspect, the present invention proposes a data processing device to be provided as input to a first shift register of a systolic neural electronic circuit comprising a grid of elementary processors, said first shift register being adapted to convey a data vector of size K at each clock cycle to K columns of elementary processors of said grid, said electronic processing device being adapted to obtain a sequence of successive data D(0), D(1), D(2) ... D(d) extracted by reading said data classified consecutively in a first storage memory and to store said data in a second memory of the processing device; said electronic processing device being adapted to generate vectors W(u) as a function of said stored data, each vector W(u) comprising K components (W(u)0, ..., W(u)K_i) such that W(u)k>k=OàK 1= D(K0 + k*S2 + u*Sl), where K is an integer strictly greater than 1, KO is a predefined constant, the steps SI and S2 are predefined integer constants; said electronic processing device being adapted to provide, at the input of said shift register, the vector W(u), such that successive values ​​W(u) are provided during successive clock ticks, with u an integer ranging from O to Ll-l, and Ll an integer constant strictly greater than one. .

[0029] These characteristics and advantages of the invention will appear on reading the description which follows, given solely by way of example, and made with reference to the appended drawings, in which:

[0030] [Fig-1] [Fig.l] represents a schematic view of a 2D grid for calculating a systolic neural electronic circuit;

[0031] [Fig.2] [Fig.2] illustrates an input data distribution for a circuit systolic neuronal;

[0032] [Fig.3] [Fig.3] illustrates another input data distribution for a circuit systolic neuronal, which can be implemented in one embodiment of the invention;

[0033] [Fig.4] [Fig.4] illustrates another input data distribution for a circuit systolic neuronal, which can be implemented in one embodiment of the invention;

[0034] [Fig.5] [Fig.5] illustrates another input data distribution for a circuit systolic neuronal, which can be implemented in one embodiment of the invention;

[0035] [Fig.6] [Fig.6] illustrates another input data distribution for a circuit systolic neuronal, which can be implemented in one embodiment of the invention;

[0036] [Fig.7] [Fig.7] illustrates another input data distribution for a circuit systolic neuronal, which can be implemented in one embodiment of the invention;

[0037] [Fig.8] [Fig.8] represents the steps of a method in one embodiment of the invention;

[0038] [Fig.9] [Fig.9] represents a systolic neural system in a mode of realization of the invention;

[0039] [Fig.9] represents a systolic neural system 1 in one embodiment of the invention. This system is for example on board an aircraft.

[0040] The system 1 comprises a memory, for example mass storage, an electronic circuit physically implementing a neural network and a processing device 40.

[0041] The memory is for example the DDR memory 10 as mentioned previously with reference to [Fig.l], the electronic circuit is the systolic neural circuit 20 on FGPA as described in relation to [Fig.l], the processing device 40 according to the invention is interposed between the DDR memory 10 and the circuit 20, for example in the FGPA.

[0042] The processing device 40 comprises a memory MP 41 and a control block 42 comprising in particular a unit 420 for timing the operations.

[0043] The control block 42 comprises for example a memory and a microprocessor (not shown), the memory comprising software instructions, which when executed on the microprocessor, implement the following operations, described in [Fig.8].

[0044] Thus in a step 101, the control block 42 obtains a sequence of successive data D(0), D(1), D(2) .. .D(d) extracted by reading, for example in burst, said data classified consecutively in the memory 10 (for example the data of an image line, d being for example a value included in the range going for example from 16 to 256 and stores said data in the memory MP 4L

[0045] For example the data D(n) are read by bursts from the DDR 10 in the form of vectors V(t), each vector containing, for K=4, the data D(4t), D(4t+1), D(4t+2), D(4t+3) which are written consecutively in memory MP 41: the data D(n) is written at the address ADO + n.

[0046] In a step 102, the control block 42 generates vectors W(u) as a function of said stored data, according to the formula (1) described later.

[0047] Each vector W(u) is made up of 4 components (W(u)0, ..., W(u)K_i) (we recall that in the example K is equal to 4, as explained above with reference to [Fig.l], but K can take any value).

[0048] In a step 103, the control block 42 successively provides, at the input of the shift register 28, a vector W(u), for u integer ranging from O to L1-1, and L1 predefined constant strictly greater than one (for example strictly greater than 2), at each clock stroke, accompanied by the indication of the addresses of the K consecutive columns to which the K components of W(u) are intended.

[0049] These vectors, applied as input to the internal distribution system 27, produce the expected data distribution.

[0050] In one embodiment, W(u)k= D(K0 +u*Sl + k*S2), k=0 to Kl,

[0051] KO is a predefined constant, the steps SI and S2 are predefined constants, non-zero or zero depending on the case (in the case of implementation based on a memory with a prime number of banks, S2 will not be a multiple of the number of banks).

[0052] And the memory mk thus contains the data D(K0 +u*Sl + k*S2), for each u from 0 to Ll-1, after the input shift register 28 has presented this data W(u)k to it accompanied by the indication of the memory mk.

[0053] The values ​​of SI, S2, Ll, KO are initially chosen so as to give rise to the desired distribution.

[0054] In a more general embodiment, with k = 0 to Kl:

[0055] W(u)k=D(K0 + yk Ui*Sli + k*S2) formula (1)

[0056] ranging from 0 to Ll-l, i = 1 to n (n integer greater than or equal to 2) and Ll; predefined constant not zero and for example strictly greater than one (for example strictly greater than 2) (Lü=1 amounts to adding the sum of Su to KO and removing the sum from the formula), the steps S h and S2 are predefined constants.

[0057] To supply data to circuit 20, the incrementation of the u; is done in the order of increasing i: i <j, pour chaque j en partant de 0, on incrémente i, de 0 à Lin : pour chaque valeur de i, on fournit les W(u) ; puis on incrémente j de 1, on remet i à 0 et on recommence.

[0058] And the memory mk thus contains the data D(K0 +u;*Sli + k*S2), for each u; from 0 to Ll;-1, i = 1 to n after the input shift register 28 has presented this data W(u)k to it accompanied by the indication of the memory mk.

[0059] The values ​​of Sl;, S2, Ll;, KO are initially chosen so as to give rise to the desired distribution.

[0060] If we consider all the columns of the grid (for example Ctot columns, with Ctot ranging from 1 to a few hundred, each column mC thus contains the data D(K0 +u; *Sli + C*S2),C=0àCtot-l

[0061] All of these operations carried out by the control block 41 are clocked by the operation sequencing unit 420 (all the operations are clocked at the rate of the clock 20, which is the clock or one of the clocks used by the FPGA).

[0062] Formula (1) is defined to obtain the desired distribution of data between the processor columns, depending on the definition of the convolutions to be calculated.

[0063] As an example, [Fig.7] shows the data recorded successively in the memory which is located below them, vertically, KO having been set to 0 for the following formula taken from general formula 1: W(u)k= D(K0 +u*Sl+ v*S3 + k*S2), with u ranging from 0 to L1 - 1 and v ranging from 0 to L3 -1, L1 = 3, SI = 2, L3 = 2 and S3 = 50, memory ml being associated with column no. 0 of the PEs, memory m2 with column no. 1, ... memory mK with column no. Kl.

[0064] This type of distribution introduces recopies, and cannot be done simply by loading the internal distribution system 27 in the order in which the data leaves the DDR 10. The processing device 40 according to the invention is adapted to write a sequence of data in the column memories in an order different from that in which they were read in the DDR 10 memory.

[0065] Similarly, the processing device 40 according to the invention is adapted to read a sequence of data in the column memories in an order different from that in which they are then written in DDR 10.

[0066] In a step 201, the processing device 40 reads the results delivered by the shift register 29.

[0067] The order in which the data in the memories m can be read is constrained by the operation of the bus 29, which reads the K (=4 here) data which are located at the same address in 4 consecutive columns.

[0068] The data to be exported by the processing device 40 to the DDR 10 are distributed in the memories m with a step equal to S2 from one column to another and a step SI within each column. The reading will produce vectors W(C,x) containing the data of index 4*C*S2 + x*Sl + k*S2 with 0<=k <K=4 (en supposant ici que l’indice commence à 0 (C étant un indice allant de 0 à Ll-1 ; si on considère que l’indice ne commence pas à zéro mais à indo, alors la formule donnant l’indice est indo +4*C*S2 + x*Sl + k*S2). Note : pour une formule générique, x, SI, et L1 deviennent des vecteurs de même taille n ; soit x; les composantes de l’indice vecteur x (x; entier allant de 0 à Ll; -1), on remplace le x*Sl par Y ( Xi* Sli ), (dans la référence ^«=0 to Lli -1 above in [Fig.7] where we use SI and S3 as steps in the column, SI and S3 of this example are then the two components of the vector SI).

[0069] In a step 202, the processing device 40 writes the K (=4) data of the vector W(C,x) to the addresses 4*C*S2 + x*Sl + k*S2 with 0 <k<K ; en mémoire MP 41, une séquence de données d’indices consécutifs est ainsi écrite, avec les données d’indice consécutif écrites à des adresses consécutives.

[0070] In a step 203, the processing device 40 writes these sequences extracted from the MP memory 41 in the form of long bursts in DDR 10 at consecutive addresses.

[0071] The processing device 40 according to the invention is adapted to carry out data distributions according to a predefined formula, in the memories m while remaining compatible with the constraints of DDR accesses and the internal distribution system.

[0072] On the DDR 10 side, the data stream read by the processing device 40 is the sequence of consecutive pixels on a line of the input image table

[0073] On the side of the internal distribution system 27, the sequence conveyed is a series of vectors of K (here 4) data, intended for 4 consecutive column memories, which makes it possible to use the buses 28, 29 as a shift register which reduces the interconnections and simplifies the clock distributions in the FPGA.

[0074] The processing device 40 is of a size independent of the size of the computing grid that it feeds and is also inexpensive in terms of resources. It is possible to load or read the column memories of the grid with data patterns conforming to formula (1) and this makes it possible to considerably broaden the scope of the architecture.

[0075] The storage and then re-reading functions in a different order are easily carried out in the processing device 40 with a dual-port MP memory 41 if it only produces one data item per cycle. The objective in one embodiment being to produce a vector with several (K) components (4 in the example), this requires a parallel memory functionality described below.

[0076] During a cycle, one vector is written and another vector is read in MP 41, under the timing of the timing unit 420.

[0077] In MP 41: - concerning the exchanges of data in reading or writing between the processing device 40 and the circuit 20, the vector is read or written in MP 41 at addresses separated by a step S, whereas - this step is set to 1 (consecutive addresses) in MP 41 for read or write exchanges between the processing device 40 and the DDR 10.

[0078] In one embodiment, the processing device 40 contains two pairs of address generators for respectively the transfers between DDR 10 and pivot memory MP 41 (address generators 421, 422) and the transfers between pivot memory MP 41 and circuit 20 (address generators 423, 424).

[0079] An address generator is a multi-dimensional counter that provides the sequence of addresses: AdO + nl*Sl + n2*S2 + ... with 0<=ni <Li, les ni s’incrémentant dans l’ordre lexicographique, et les Si étant des entiers donnant les pas d’incrémentation dans chaque dimension du compteur.

[0080] Since the MP 41 memory has a limited capacity, the transfer of data blocks between DDR 10 and circuit 20 is carried out, for example, in several consecutive phases. For reasons of efficiency, these transfers are carried out in a pipeline: sub-block N is read at the same time as sub-block N+1 is written. Both sides of the MP 41 memory are most of the time active simultaneously. The sequencing of these operations is controlled by the sequencing unit 420 which sequences the inputs / outputs and which executes a microprogram in which the parameters of the address generators are contained in particular.

[0081] The invention makes it possible to significantly extend the range of CNN operations of the systolic circuit making it possible to move away from the data distribution scheme of the prior art presented above, where the adjacent pixels are distributed in the same order in the column memories m.

[0082] The functionality limitations of the architecture (convolution of width 1 to 3, loss of efficiency in the case of stride greater than 1) are thus lifted thanks to an adapted distribution of the data of the line dimension.

[0083] Figures 3 to 5 give examples of data distribution in a system implementing the invention and which make possible processing operations which were not in the distribution of the prior art where the data of index i0+ i on the line is written in the memory of column i and only in this memory and for which the matrix of PEs is used at full efficiency.

[0084] [Fig. 3] illustrates an example of distribution in an embodiment (where S1 = 1, S2 = 2, in the case of a convolution of width 5 and stride 1; the result of the column associated with the memory m2 must be the convolution of (D1, D2, D3, D4, D5), the result of the column associated with the memory m3 must be the convolution of (D2, D3, D4, D5, D6), the result of the column associated with the memory m4 must be the convolution of (D3, D4, D5, D6, D7), etc. In this case of implementation of the invention, a data item (for example here the data item D2 or D3 or D4 or D5) can thus be stored in several memories while limiting the calculations of the non-useful PEs.

[0085] The distribution does not include a copy in the case of [Fig.4] also corresponding to a convolution of width 5, but with a stride Str equal to 2. The result of the column associated with memory m2 must be the convolution of (D1, D2, D3, D4, D5), the result of the column associated with memory m3 must be the convolution of (D3, D4, D5, D6, D7), the result of the column associated with memory m4 must be the convolution of (D5, D6, D7, D8, D9), etc. In this case of implementation of the invention (where S1=1, S2=2), the convolution carried out by the column of PEs associated with memory 3 will use the data inside the area delimited by dotted lines: the data D3 to D6 are in memory m3 and D7 is in m4.

[0086] [Fig.5] illustrates an example of distribution in an embodiment (where Ll=l, S2=2- and therefore any SI), in the case of so-called “dilation” stages where the gap, i.e. the expansion value, between the successive data of the same convolution is not 1. In the embodiment of the invention illustrated here, the convolution width is 3 and the expansion value is 2; the result of the column associated with the memory m2 must be the convolution of (D1, D3, D5), the result of the column associated with the memory m3 must be the convolution of (D3, D5, D7), the result of the column associated with the memory m4 must be the convolution of (D5, D7, D9).

[0087] Finally, [Fig. 6] illustrates a case where several stages of the CNN graph are chained to the grid 22, from an initial data distribution in an implementation mode of the invention. This operating mode, called fusion, has the advantage of avoiding storage outside the intermediate results and of drastically reducing the need for input-output bandwidth. Fusion is an important factor in improving the performance of the circuit 20: in fact, since the systolic allows for the integration of high computing power, this is accompanied by a need of the same order in input-output throughput, which generally becomes the limiting factor for at least part of the CNN processing. There may be, among the merged stages, stages with a stride value Str greater than 1, as illustrated in [Fig.6] (a convolution stage of width K = 5 and stride value Str equal to l and a convolution stage of width K = 3 and stride value Str equal to 2). If the data are distributed as shown in [Fig.6] (where Ll = 3 S 1 = 1 S2 = 2), the PEs can chain the calculations for the two stages without needing data redistribution at the end of the first stage. The fusion applies to any number of stages, the limitation in practice coming from other criteria, including the capacity of the column memories m. .

[0088] In summary, the invention makes it possible to significantly broaden the field of operations of the systolic neural circuit 20 while retaining the compactness of the systolic grid architecture and its internal distribution system 27. The intermediate processing, for input / output management, implemented by the processing device 40 between the DDR memory 10 and the distribution system 27 makes it possible to carry out particular data distributions in the grid 22 and to process convolutions with any characteristics (kernel sizes, stride, dilation), making it possible to cover a wide range of CNNs, while optimizing the efficiency of the grid.

[0089] In addition, the type of data distribution according to the invention opens the door to CNN processing by stage fusion, which divides the input / output throughput requirement from or to the DDR by the number of merged stages. This artifice avoids the capping of the circuit performance on a part of the graph (generally the upper part of the graph, where the number of feature maps is lower).

[0090] The resource cost of the processing device 40 of the invention remains low compared to the density of current FPGAs: less than 4% for an average-sized FPGA. (7Z045 from Xilinx). The device size is otherwise the same regardless of the size of the PE grid.

[0091] Furthermore, an individual memory is generally only capable of storing or producing a single data item per clock cycle (one for each port if it is a dual-access memory), the processing device 40 using a parallel memory MP 41, i.e. capable of manipulating a vector with several components at each clock cycle, and taking into account the data patterns to be manipulated (multi-dimensional affine addressing laws). The parallel memory 41 comprises several memory banks M 410 as illustrated in [Fig.9]. To access a vector in such a set of banks M 410 at each cycle, it must be ensured that the components of the vectors are always in separate banks.

[0092] One possible implementation is parallel memory composed of a prime number of banks. This parallel memory technique is known (described in particular by DH Lawrie and CR Vora, “The prime memory System for array access“ in 1982).

[0093] The pivot memory MP 41 is therefore a set of K identical dual-port M 410 memory banks (allowing read or write access to each of its ports at each cycle), numbered from 1 to K. These M 410 memories are seen collectively as a larger memory (MP), whose capacity is the sum of the individual capacities. The address A in the memory MP 41 corresponds to the address A / K in the memory bank M 410 with number A%K (i.e. A modulo K).

[0094] In the direction from the DDR memory 10 to the gate 22, the objective of the processing device 40 is to be able to read in one clock cycle a vector of K (here 4) components y(k) = MP( ADR_A + k*S ) with 0 <k<K-l (correspond à un cas où N0 =0, SI = 0, S=S2 et S3 = 0 ). Ceci n’est possible que si les K composantes se trouvent dans des bancs distincts, puisqu’un banc ne peut fournir qu’une valeur par cycle.

[0095] So (ADR_A+kl*S)%K must be different from (ADR_A+k2*S)%K if k2 is different from kl for kl and k2 between 0 and Kl.

[0096] The condition is verified if (kl-k2)*S %K is zero only if k2=kl, a condition always true if S and K are coprime. In the current implementation, K=5, a prime number guaranteeing that all address vectors ADR_A +k*S in virtual memory MP 41 can be accessed, as long as S is not a multiple of 5, a pathological case that can always be avoided or circumvented.

[0097] In addition to the memory banks themselves, the processing device 40 then further comprises on each side (i.e. DDR 10 memory side and circuit 20 side) and in each direction (reading and writing) logic blocks, called Shuffler, 430, 431, 432, 433, which connect each memory bank to the component of the vector that it must read or write:

[0098] if the vector targets the addresses ADO + n*S, 0<=n<4, the number of the bank concerned by the component n being (ADO +n*S)%5 stores the components of the vector.

[0099] It is recalled that there is also on each side an address converter 421, 422, 423, 424 which transforms a virtual address A into 5 addresses intended for each of the banks: if the component n of a vector is intended for bank b = (ADO +n*S)%5, the address in this bank will be equal to (ADO +n*S -b ) / 5.

[0100] The invention can be implemented in embedded systems of a very varied nature, for example in the processing of images by neural network on board a satellite, and / or configurable on a radiation-hardened FPGA, and / or more generally relating to the classification, detection or recognition on real-time images or signals.

Claims

Claims

1. Processing method, in an electronic processing device (40), of data to be provided as input to a first shift register (28) of a systolic neural electronic circuit (1) comprising a grid of elementary processors (22), said first shift register conveying a data vector of size K at each clock cycle to K columns of elementary processors (PE) of said grid, said processing method comprising the following steps implemented by said electronic processing device: - obtaining a sequence of successive data D(0), D(l), D(2) ...D(d) extracted by reading said data classified consecutively in a first storage memory and storing said data in a second memory (m) of the processing device; - generation of vectors W(u) as a function of said stored data, each vector W(u) comprising K components (W(u)o, ..., W(u)K1) such that W(u)k ,k=OàK-i= D(K0 + k*S2 + u*Sl), where K is an integer strictly greater than 1, KO is a predefined constant, the steps S1 and S2 are predefined integer constants; - providing, at the input of said shift register, the vector W(u), such that successive values ​​W(u) are provided during successive clock pulses, with u an integer ranging from 0 to Ll-l, and Ll an integer constant strictly greater than one.

2. A processing method according to claim 1, further comprising the following steps implemented by said electronic processing device (40): - obtaining a data vector w of size K supplied by a second shift register (29) of the systolic neural circuit (1), said second shift register conveying at each clock cycle a data vector of size K corresponding to the results of K respective columns of elementary processors of said grid; - the K components of w being (w0, ..., wK-i), storage in the second memory (m), from wk >k=o to K i at the address indO +K*C*S2 + x*Sl + k*S2 , where indO is a predefined constant, C is the column number and x is an index ranging from 0 to Ll-1; - supplying to the first storage memory (10) a sequence of successive data extracted by reading said data classified consecutively in the second memory (m).

3. Processing method according to claim 1 or 2, according to which, for k= 0 to Kl, W(u)k = D(K0 + Y , U;*SL + k*S2), the steps SL to n and S2 are predefined constants; the vector being provided at the input of said shift register, with u; integer ranging from 0 to L1L cl LL constant strictly greater than one.

4. Processing method according to claim 3, further comprising the following steps implemented by said electronic processing device (40): - obtaining a data vector w of size K supplied by a second shift register (29) of the systolic neural circuit (1), said second shift register conveying at each clock cycle a data vector of size K corresponding to the results of K respective columns of elementary processors of said grid; - the K components of w being (w0, ..., wK-i), the storage in the second memory of wk >k=O to K i is carried out at the address indO+4*C*S2 + Y (Xi* Sli) + k*S2, where indO is a predefined constant, C is the column number and x; is an integer index ranging from 0 to Lli-1; - supplying to the first storage memory (10) a series of successive data extracted by reading said data classified consecutively in the second memory (m).

5. Processing method according to one of the preceding claims, according to which the second memory (m) comprises at least K distinct memory banks, and the storage of the data in the second memory is carried out in such a way as to verify that data which are components of the same vector W(u) cannot be stored in the same bank.

6. Device (40) for processing data to be supplied as input to a first shift register (28) of a systolic neural electronic circuit (1) comprising a grid of elementary processors (22), said first shift register being adapted to convey a data vector of size K at each clock cycle to K columns of elementary processors (PE) of said grid, said electronic processing device (40) being adapted to obtain a sequence of successive data D(0), D(1), D(2) ... D(d) extracted by reading said data classified consecutively in a first storage memory (10) and to store said data in a second memory (m) of the processing device; said electronic processing device being adapted to generate vectors W(u) as a function of said stored data, each vector W(u) comprising K components (W(u)0, ..., W(u)K_i) such that W(u)kjk=oàK i= D(K0 + k*S2 + u*Sl), where K is an integer strictly greater than 1, KO is a predefined constant, the steps SI and S2 are predefined integer constants; said electronic processing device (40) being adapted to provide, at the input of said shift register (28), the vector W(u), such that successive values ​​W(u) are provided during successive clock ticks, with u integer ranging from O to Ll-l, and Ll integer constant strictly greater than.

7. a. Data processing device (40) according to claim 6, further adapted to obtain a data vector w of size K supplied by a second shift register (29) of the systolic neural circuit (1), said second shift register being adapted to convey at each clock cycle a data vector of size K corresponding to the results of K respective columns of elementary processors of said grid, the K components of w being (w0, ..., wK-i); said electronic processing device being adapted to store in the second memory (m), wk >k=oàK i at the address indO +K*C*S2 + x*Sl + k*S2 , where indO is a predefined constant, C is the column number and x is an index ranging from 0 to Ll-1 and to provide the first storage memory with a sequence of successive data extracted by reading said data classified consecutively in the second memory.

8. Processing device (40) according to claim 6 or 7, wherein, for k= 0 to Kl, W(u)k = D(K0 + Y u;*Sli + k*S2), the steps SI; and S2 are predefined constants; the vector being provided at the input of said shift register, with u; an integer ranging from 0 to Ll-l, and Lli a constant strictly greater than one.

9. Processing device (40) according to claim 8, adapted to obtain a data vector w of size K supplied by a second shift register (29) of the systolic neural circuit (1), said second shift register conveying at each clock cycle a data vector of size K corresponding to the results of K respective columns of elementary processors of said grid, the K components of w being (w0, ..., wK-i), said electronic processing device (40) being adapted to store in the second memory from wk k-O to K i at the address indO+4*C*S2 + Y (Xi* Sli) + k*S2, where indO is a predefined constant, C is the column number and x; is an integer index ranging from 0 to Lli-1; said electronic processing device being adapted to provide the first storage memory with a sequence of successive data extracted by reading said data classified consecutively in the second memory.

10. Processing device (40) according to one of claims 6 to 9, the second memory comprising at least K distinct memory banks, and said processing device being adapted to store the data in the second memory so as to verify that data which are components of the same vector W(u) cannot be stored in the same bank.

Images