System and method for calculations for the inference of convolutional neural networks

Abstract

System and Method of Computation for the Inference of Convolutional Neural Networks The present invention relates to a method of computation for the inference of a convolutional neural network implemented on a multi-core computing platform, each core comprising a local core memory of capacity 'Cm' and a computing unit, the cores being grouped into groups of cores, such that in a group, one core is assigned to the computation of neural operations and is connected via neighbor links to at least one immediate neighboring core.The method includes, during inference, steps consisting of preloading a subset of operands required to perform a neural operation into the local core memories of the core groups. The local core memories of a core group define a local group memory with a capacity 'CM' equal to the sum of the capacities 'Cm' of the local core memories of the cores belonging to the same group. The neural operation is then executed by transmitting to the core groups operands complementary to the operands preloaded into the local group memory, and by performing the neural operation with the set of preloaded and complementary operands using the processing unit of said assigned processing cores. Figure 5 illustrates this in the abstract.

Description

Title of the invention: System and method for calculations for the inference of convolutional neural networks

[0001] The present invention relates to the technical field of artificial intelligence (AI), and more particularly to that of inference accelerators for convolutional neural networks.

[0002] Recent years have seen the emergence of new topologies of convolutional neural networks (CNNs for "Convolutional Neural Network" in English) making it possible to solve classification, detection or regression problems for many application areas.

[0003] Some potential applications require convolutional neural networks comprising several dozen or even hundreds of layers (DNNs for "Deep Neural Networks"). These applications can be used in environments with high energy, heat dissipation, or even weight and volume constraints. This includes, among others, vision or intensive computing applications embedded on drones, vehicles, or even satellites. It is therefore essential to have solutions for running high-performance neural networks on hardware compatible with the constraints of embedded systems.

[0004] Convolutional neural networks consist of a succession of processing layers, mostly convolutions, which apply a kernel or filter of coefficients to an input array (most often a 3D matrix, called an "Input Feature Map" in English). The coefficients are organized into X, Y channels (or rows, columns), producing a 3D output array (3D output matrix, also called a "feature map" or "activation map," but also an "Output Feature Map" in English).

[0005] Once the network structure is established (i.e. the list of layers, their characteristics and the different dependencies), the network goes through a learning phase ("machine learning" in English) which is carried out offline and which produces sets of coefficients for the different layers.

[0006] The inference phase consists of running the different layers of the network that has been learned on input data (for example images), and then obtaining an output result that depends on the learned neural processing.

[0007] CNN algorithms, although conceptually simple, are very computationally expensive. In the case of embedded AI applied to images, the need is to be able to process tens of megapixels (or MP) per second, in order to to obtain inference results in real time (i.e., with minimal processing time, on the order of milliseconds or tens of milliseconds depending on the application). The expected performance is measured in teramulacs per second or Tmulacs / s, or 10¹² multiplication-accumulation operations (mulacs) per second, a mulac being the central operation of a convolution and being present in very high numbers in CNN algorithms.

[0008] To achieve such performance levels, given the limitations of current processors, it is necessary to use processing units that enable massively parallel calculations, such as GPUs (Graphics Processing Units), FPGAs (Field Programmable Gate Arrays), or specialized circuits. These processing units, which allow the parallel execution of massively parallel operations, act as accelerators.

[0009] Traditional FPGAs are equipped with a large number of units called DSP Blocks (“Digital Signal Processing” in English), including operators hardwired to efficiently perform certain mathematical operations such as mulacs, often used for CNN inference.

[0010] The theoretical computing power (i.e., the number of operators multiplied by the clock frequency) can reach a few Tmulac / s, through the use of several hundred DSP Blocks operating at 400 or 500 MHz. However, several factors limit the actual performance, primarily the data movement rates between memories.

[0011] Solutions for improving the performance of FPGAs, for example, consist of adding to the classic components of FPGAs (i.e. programmable logic, memories, DSP Blocks, scalar processor), a matrix of vector processors or computing cores (which can be referred to individually as "core" or "elementary processor" PE or "Compute Unit" CE), which are very powerful, are programmable (for example in assembler or in C++) and which are capable of increasing the theoretical computing power.

[0012] This is the case, for example, with AMD's Versai multi-core computing platform, where each computing core, called the "AI Engine" (AIE), is capable of executing 128 mulacs of 8-bit operands per 0.8 ns clock cycle, or a theoretical 160 Gmulacs / s. The platform contains an array of 400 AIEs, which raises its theoretical computing power to 64 Tmulacs / s, well beyond the theoretical power of the DSPs of a similarly sized and power-efficient FPGA. However, in practice, known implementations typically achieve effective performance between 1 and 10 Tmulacs / s for CNN inference, because not all 400 available AIEs are used efficiently for inference.

[0013] More generally, current accelerator architectures for the parallel execution of mulacs operations by multi-core computing matrices have limitations in obtaining the computing power necessary for the inference of convolutional neural networks, in particular for the use of such networks for applications in embedded environments.

[0014] There is then a need to increase the efficiency of calculations on the cores of a multi-core system during an inference for a convolutional neural network.

[0015] The aim of the invention is then to propose a method and a device for efficiently executing neural algorithms on a multi-core computing system.

[0016] Advantageously, the proposed solution makes it possible to optimize the computing capacity of each computing core of a multi-core computing platform, by increasing the number of operations performed during the execution phase of a computing core per clock cycle.

[0017] The general principle of the invention is based on a strategy of using only a portion of the computing cores of a multi-core system to perform the calculations of a neural operation, and an extension of the local memory capacity of the cores performing the calculations by exploiting the local memory of one or more immediate neighboring computing cores.

[0018] Advantageously, the computing power / consumption efficiency is increased and becomes greater than the efficiency that each core would have if it were responsible for the calculations.

[0019] Advantageously, the energy efficiency of the overall system is improved, because the computing cores that do not perform calculations but only contribute to an extended storage capacity by making their local memory available to a core dedicated to calculation, can be switched off (i.e. the dynamic consumption of electricity is zero, their computing units can also be put into standby mode, or put into "power gating", also eliminating static electricity consumption).

[0020] Increasing the local memory of a dedicated computing core (i.e., extending the memory capacity) from a single memory location m to a group M of memories (directly neighboring computing cores) allows for the processing of more data within the computing program. Advantageously, this reduces the need to divide a neural program into multiple iterations, and thus improves the efficiency of the neural program and the efficiency of the computations on the cores.

[0021] Thanks to the improved cycle efficiency, it becomes possible either (1) to increase the effective computing power of the system with reduced energy consumption, or (2) to limit the operating frequency of the multi-core system to maintain the same computing power as a standard system but with reduced energy consumption (even more reduced than in case (1)).

[0022] To this end, the invention relates to a computation method for inferring convolutional neural networks, the convolutional neural network being implemented on a multi-core computing platform comprising a plurality of computing cores, each core comprising a local core memory of capacity 'Cm' and a computing unit, the computing cores being grouped into groups of computing cores, such that in a group, one core is assigned to the computation of neural operations and is connected via neighbor links to at least one immediate neighboring computing core. The method comprises, during an inference, steps consisting of:

[0023] - preload a subset of operands required to perform an operation neuronal, in the local core memories of the groups of cores, the local core memories of a group of cores defining a local group memory with a capacity 'CM' equal to the sum of the capacities 'Cm' of the local core memories of the cores belonging to the same group;

[0024] - execute the neural operation, said execution comprising steps consisting of transmit to the groups of cores, operands complementary to the operands preloaded in local group memory; and to perform, by the computing unit of said assigned computing cores, the neural operation with the set of preloaded and complementary operands.

[0025] Since the local memories of the computing cores are initialized with at least some of the operands before the execution of a neural operation, the need for external data supply to the various cores of the multi-core system during execution becomes less significant. It can even become less than the maximum bandwidth of the various data buses. Thus, advantageously, the bandwidth of the data buses no longer becomes a limiting factor.

[0026] The reduced bandwidth requirement results in fewer limitations on the operation of the processing cores. Indeed, they can receive most, if not all, of their operands in time (i.e., the operands are already loaded before the execution of the calculation requiring them), and thus operate at their maximum potential. Advantageously, the invention allows a maximum number of operations per clock cycle to be achieved. This advantage is further enhanced by the use of logical groups with extended local group memory, thus enabling initialization with a larger amount of data than individual local memory.

[0027] According to other advantageous aspects of the invention, the method of the invention comprises one or more of the following features, taken individually or in all technically possible combinations:

[0028] - the preloading step in each local group memory consists of storing the operands of said subset of operands in all or part of the local heart memories of said group;

[0029] - the preloading step in each local group memory consists of preload a subset of neuronal layer coefficients obtained during a neural network training phase;

[0030] - the step of transmitting complementary operands consists of transmitting pixel-type data or to transmit "feature-map" values;

[0031] - the transmission step consists of transmitting data from a memory external to the multi-core computing platform via a communication bus network, said bus network being arranged on the multi-core platform to route data to all computing cores and between computing cores;

[0032] - the step of performing the neuronal operation includes a transfer step of data stored in the local memories of groups, via neighbor links between immediate neighboring cores;

[0033] - the execution step of a neural operation consists of performing operations of multiplication-accumulation by the computing units of said dedicated cores;

[0034] - the method further comprises a step consisting of transmitting a result from a neural operation to a memory external to the multi-core computing platform, said step being performed after the execution of a current neural operation to transmit a current result or performed during the execution of a neural operation to transmit a result of a previous neural operation;

[0035] - the operations of the execution of a neural operation are carried out in parallel or sequentially;

[0036] - the method further comprises, during the neural execution step, steps data processing performed on computing units of computing cores directly adjacent to a dedicated computing core;

[0037] - the method further includes a step consisting of putting the units into standby mode inactive calculations of the computing cores dedicated to storage or to put them in "power gating";

[0038] - the process includes an initial step consisting of creating groups of cores according to a predefined pattern, the grouping defines for each group, a computing core dedicated to calculations such that only the computing units of said core are active to perform calculations, and such that the local group memory of a group of cores is defined by a capacity 'CM' equal to the sum of the capacities 'Cm' of the local core memories of the cores of said group.

[0039] The invention also relates to a computing device for the inference of convolutional neural networks, the convolutional neural network being implemented on a multi-core computing platform comprising a plurality of computing cores, each core comprising a local core memory of capacity 'Cm' and a computing unit, the computing cores being grouped into groups of computing cores, such that in a group, one core is assigned to the computations of the neural operations and is connected via neighbor links to at least one immediate neighboring computing core, the device comprising means for implementing the steps of the method of the invention.

[0040] The invention integrates into a complete implementation of a convolutional neural network inference processor and provides a sovereign and customizable solution to user needs, particularly for high-performance, embedded, real-time AI industrial applications.

[0041] The invention will find advantageous applications in many fields, including land transport (driver assistance systems, autonomous driving), environmental monitoring (detection of fires, oil spills, vegetation cover, ...), general industry (perception systems, intelligent sensors, predictive maintenance) or agriculture (detection of diseases and parasites, determination of maturity level, ...).

[0042] The invention will become clearer upon reading the following description, given solely by way of non-limiting example, and made with reference to the drawings in which:

[0043] [Fig-1] [Fig.1] schematically represents a multi-core computing platform;

[0044] [Fig.2a] [Fig.2a] schematically illustrates part of a multi-core system according to one embodiment of the invention;

[0045] [Fig.2b] [Fig.2b] schematically details the communication links of a computing core of a multi-core system according to an embodiment of the invention;

[0046] [Fig.3] [Fig.3] schematically illustrates a unit computing core of a multi-core system according to an embodiment of the invention;

[0047] [Fig.4] [Fig.4] is a flowchart of the steps to define logical groups of computing cores according to an embodiment of the invention;

[0048] [Fig.5] [Fig.5] is a flowchart of the calculation steps carried out during an inference according to an embodiment of the invention;

[0049] [Fig.6a] - [Fig.6b] - [Fig.6c] Figures 6a to 6c illustrate examples of grouping patterns of computing cores according to the invention.

[0050] Fig. 1 schematically represents a computing platform 100 (constituting all or part of an electronic chip) comprising a multi-core system 102 and an interface 104 for establishing communications with an external global memory 106.

[0051] Interface 104 allows, among other things, adapting external buses with internal buses in terms of protocol, clock frequency, bus width and number of buses.

[0052] The external memory 106 is adapted to store all the initial data for performing a convolution calculation, such as the operands of the neural operation (e.g., convolution coefficients, pixels, or other input values), to store the intermediate results (e.g., the results, also called "activation maps" or "feature maps" of the different neural layers), and to store the output data obtained after a calculation. Both the input data and the output results are located in the external memory block 106, for example, of the DDR (Double Data Rate) type, which is located outside the computing platform 100. The input data consists, on the one hand, of the network input data (e.g., an image to be processed that changes at the rate of the video frames), and on the other hand, of the coefficients of the neural layers that were calculated during the neural network's training phase.

[0053] Optionally, communications between the external memory 106 and the platform 100 can be established via various intermediate processing and storage modules 108 of cache or FPGA type, for example to reorganize data or accelerate transfers.

[0054] One or more clocks regulate the operation of the multi-core system. With this architecture, each core is capable of executing several operations per clock cycle, e.g., 128 multiply-accumulate operations per cycle.

[0055] A multi-core computing platform organized according to [Fig.1] generally has the following limitations.

[0056] A first limitation concerns the size, i.e. the storage capacity Cm of each local memory cell 'm' of each core, which, most of the time, is insufficient to store all the data necessary for the complete execution of a neural program (even a partial one with a few layers of convolution) on a computing core.

[0057] Another limitation relates to the bandwidth of the data buses, which is not sufficient to support the different cores with data when each of them executes several instructions per cycle.

[0058] The consequence of the limitation on the size of local memory is that it is difficult, if not impossible, to preload into local memory all the data necessary for a neural operation. This, combined with the bandwidth limitation, means that the neural operation cannot be executed all at once on all the input data, and it must be executed iteratively on several portions of the data until all the data has been processed.

[0059] Typically, the execution of a program in a computing core involves three phases: an initialization phase covering the import of data and coefficients, a computation phase covering the execution of the program in the computing core, and a termination phase covering the output of the results. However, the execution times of the initialization and termination phases are often difficult to reduce. Therefore, the drawback of such a three-phase division of the processing of program execution with iterative execution of the neural program on several portions of the input data is that the initialization and termination phases are multiplied, which then reduces the overall efficiency of the computations on the cores.

[0060] Also, advantageously the process of the invention aims to optimize the computing capacity of each computing core of a multi-core computing platform, by increasing the number of operations performed during the computing phase of a computing core, which mathematically increases the number of operations performed per clock cycle and therefore the efficiency of the calculations on the cores.

[0061] A multi-core system according to the invention can be organized, for example as illustrated in [Fig.2a], into a two-dimensional matrix comprising: a plurality of computing cores (PE) connected by communication links or buses distributed throughout the matrix according to a pattern defined in the design or implementation of the multi-core matrix and allowing data exchange between the inside and outside of the core matrix; and neighbor links between immediate neighboring cores allowing each core direct access to the local memory of an immediate neighboring computing core.

[0062] For the sake of simplicity in the description, a single core of the matrix is ​​referred to by 202, pairs of communication buses allowing data transfer into and out of the matrix by (204, 206) and (208, 210), and neighbor links allowing data transfer between immediate neighboring cores by (214, 216).

[0063] According to this example of a two-dimensional multi-core regular matrix, the communication buses and neighbor links are organized into row / column link networks or horizontal / vertical link networks or link networks along a first dimension and a second dimension.

[0064] In one embodiment, a distribution component 212 (“switch” in English) can be coupled to each computing core 202 to distribute data from the communication buses to this computing core or distribute data from this computing core to the communication buses.

[0065] A multi-core matrix according to the invention further implements neighbor links between immediately neighboring cores which allow a core direct access to memory local of one or more immediate neighboring computing cores. Thus in the simplified example, neighbor links 214 are shown offering direct access between two immediate neighboring cores positioned on the same row (or horizontally), and neighbor links 216 are shown offering direct access between two immediate neighboring cores positioned on the same column (or vertically).

[0066] A person skilled in the art understands that any generalization of neighbor links can be made for a multi-core matrix, allowing unidirectional or bidirectional access between immediate neighboring cores, and / or allowing access to one or more immediate neighboring computing cores. Thus, [Fig. 2b] schematically illustrates an example of bidirectional communication links (220, 222) and unidirectional communication links (224, 226).

[0067] Fig. 3 schematically illustrates a unit computing core 300 of a multi-core system according to the invention.

[0068] Each computing core can be considered an elementary processor comprising at least one local memory 302 and one computing unit 304 composed of one or more scalar computing units 306 and one or more vector computing units 308. Each core executes its own computing program, which may differ from the program executed by other cores.

[0069] The local memory of a computing core is a memory cell 'm' having a storage capacity 'Cm' and allowing the storage of, among other things, operands and results of programs executed on the core.

[0070] In the context of the invention, each core has direct access to its own local memory but also to the local memory of at least one direct or immediate neighboring core through neighbor links.

[0071] For example, for a core having direct access to 'n = 3' neighboring cores, e.g., the neighboring core above (or north PE), the neighboring core below (or south PE), and the neighboring core to the right (or east PE), this core can then address, via the neighbor links, 4 memory cells, i.e., those of the three neighboring cores plus its own. Each core's memory cell is itself accessible by 4 processing cores, i.e., by the three immediate neighboring cores and its own core, with the exception of cores whose memory cell is located on the edges of the core matrix.

[0072] The [Fig.4] is a flowchart of the steps to define logical groups of computing cores according to an embodiment of the invention.

[0073] The 400 process is performed before the implementation of an inference. It initially consists of determining a pattern for grouping computing cores into groups of computing cores. A grouping pattern defines a number 'n' of computing cores per group, 'n' being at least 2 cores, and depending at most on the number of neighbors accessible by neighbor links.

[0074] The choice of a pattern will depend on the applications for which inferences are implemented. More specifically, the choice of pattern will be dictated by the memory requirements of the different layers of the neural network, and therefore by the need to group several local memories 'm'.

[0075] Step 402 also consists of defining the position of the core(s) of a group which are immediate neighbors.

[0076] In a subsequent step 404, the method consists of logically creating the core groups according to the predefined grouping pattern. In each core group, a single computing core is assigned to perform the calculations of a neural operation (during inference); that is, only the computing units of this dedicated core are active to execute calculations related to a neural operation, and the computing units of the other cores in the group are not used for this execution.

[0077] Furthermore, step 404 consists of defining a local group memory for each group of computing cores. The local group memory of a group of cores is defined by a capacity 'CM' which is equal to the sum of the capacities 'Cm' of the local memories of all the cores in that group.

[0078] Figures 6a to 6c illustrate examples of computing core grouping patterns according to the invention.

[0079] Figure 6a illustrates part of a multi-core system with a 602 rectangular grouping pattern. For such a pattern, a core group contains two direct neighboring computing cores, here designated by PE(2,2) and PE(2,3), having respectively a computing unit UC(2,2) and UC(2,3) and a local memory m(2,2) and m(2,3).

[0080] It is defined that the computing core PE(2,2) is the one dedicated to the computations of neural operations. Thus, the computing unit UC(2,2) of the computing core PE(2,2) is the only computing unit active for the group for neural operations, and the computing unit UC(2,3) of the computing core PE(2,3) remains inactive for these operations.

[0081] Advantageously, the computing unit UC(2,3) remains available during the execution of a neural operation on the computing unit UC(2,2), to perform ancillary processing. Parallel processing may consist of performing synchronization tasks between the computing cores, or simple neural operations or operations that do not require the use of coefficients from the ongoing neural operation (for example, the execution of activation or normalization layers).

[0082] The creation of a group with two computing cores allows the local memory capacity for the group to be increased, and to become equal to the sum of the local memory capacity m(2,2) of the dedicated computing core PE(2,2) and the local memory capacity m(2,3) of the computing core PE(2,3).

[0083] The multi-core system thus reconfigured into a plurality of groups of computing cores with one core dedicated to computing per group, makes it possible to increase the amount of data to be processed by the UC(2, 2) core.

[0084] Figure 6b illustrates a portion of a multi-core system with a 604 grouping pattern that allows the creation of core groups with four directly adjacent processing cores, here designated PE(1,1), PE(2,1), PE(2,2), and PE(3,1). Core PE(1,1) has a processing unit UC(1,1) and a local memory m(1,1). Core PE(2,1) has a processing unit UC(2,1) and a local memory m(2,1). Core PE(2,2) has a processing unit UC(2,2) and a local memory m(2,2). Core PE(3,1) has a processing unit UC(3,1) and a local memory m(3,1).

[0085] It is defined that the computing core PE(2,1) is the one dedicated to the computations of neural operations. Thus, the computing unit UC(2,1) of the computing core PE(2,1) is the only computing unit active for the group for neural operations, and the computing units UC(1,1), UC(2,2) and UC(3,1) of the directly neighboring computing cores remain inactive for these operations.

[0086] Similar to the example in [Fig.6a], the computing units of the direct neighboring cores remain available during the execution of a neural operation on the computing unit UC(2,1), to perform ancillary processing on data routed via the communication buses of the matrix.

[0087] The creation of a group with four computing cores allows the local memory capacity for the group to be increased, and to become equal to the sum of the local memory capacity m(2,l) of the dedicated computing core PE(2,1) and the capacity of each local memory m(l,l), m(2,2) and m(3,l) of the direct neighboring computing cores.

[0088] Figure 6c illustrates another example of a cross-shaped 606 grouping pattern that allows the creation of core groups with five directly adjacent computing cores. The computing cores are designated for a group by PEcenter, PEnorth, PEsouth, PEest, and PEwest.

[0089] The PEoentre core has a central processing unit (CPU) and a central local memory. The PEnord core has a central processing unit (CPU) and a central local memory (mnord). The PEsud core has a central processing unit (CPU) and a central local memory (msud). The PEest core has a central processing unit (CPU) and a central local memory (mest). The PEouest core has a central processing unit (CPU) and a central local memory (mest).

[0090] It is defined that the PEcentre computing core is the one dedicated to the computations of neural operations. Thus, the UCcentre computing unit is the only active computing unit for the group for neural operations, and the UCnord, UCsud, UCest and UCoue st computing units of the direct neighboring computing cores remain inactive for these operations.

[0091] Similar to the examples in Figures 6a and 6b, during the execution of a current neural operation on the dedicated computing unit UCcentre, the computing units of the direct neighboring cores of a dedicated core (i.e., core dedicated to neural calculations) remain available to perform ancillary processing.

[0092] The creation of a group with five computing cores allows the local memory capacity for the group to be increased, and to become equal to the sum of all the capacities of the computing cores of the group, i.e. the sum of the local memory capacity mcentre of the dedicated computing core PEcentre plus the capacity of each local memory mnord, msud, mest and moue st of the direct neighboring computing cores.

[0093] Some examples of patterns for grouping computing cores with the assignment of a dedicated computing core have been given for the sake of clarity and not for the sake of limitation. Those skilled in the art may be able to identify other variations of these patterns.

[0094] Figure 5 is a flowchart of the computational steps performed on a multi-core system for the inference of convolutional neural networks. The convolutional neural network is implemented according to an embodiment on a multi-core computing platform which contains a multi-core system where each computing core includes a local core memory of capacity 'Cm' and at least one computing unit, a communication bus network, and neighbor links between direct neighboring cores.

[0095] The multi-core system was prepared in a preparatory phase prior to inference according to a logic programming (i.e. the 400 method) where groups of computing cores were defined according to a chosen grouping pattern with assignment of a computing core dedicated to the calculations of a neural operation per group of cores.

[0096] The inference calculation method 500 begins with a step 502 when input data corresponding to a matrix to be processed (for example a complete image, or a partial image with a limited number of rows and columns) by a neural operation are available in a memory external to the multi-core system, for example a DDR type memory (memory 106 of [Fig.1]).

[0097] In a first step 504, the method allows the initialization of the local core group memory M, by preloading a subset of the operands required to perform the neuronal operation.

[0098] In one embodiment, the preloading step in the local group memories consists of storing operands in all or part of the memory M.

[0099] In one embodiment, for example where each group of computing cores processes different channels of a complete or partial image, the preloading step in the local group memories consists of transferring and storing different operands in the local group memories.

[0100] In one embodiment, for example where each group of computing cores processes a different partial image, the preloading step in the local group memories consists of transferring and storing the same coefficients for each group of cores and different pixels.

[0101] In one embodiment, the preloading step in the local group memory consists of preloading a subset of neuronal layer coefficients obtained during a neural network training phase.

[0102] When all the local group memories are initialized, the process proceeds with a step 506 of executing a neural operation. Advantageously, the execution is carried out within each group by the computing units of the dedicated computing cores that have been previously assigned.

[0103] The execution of a neural operation consists on the one hand of transmitting (508) to the dedicated computing cores, from a memory external to the multi-core computing platform, operands complementary to the operands preloaded in local group memories, and on the other hand of performing (510) the neural operation on the dedicated computing units with all the preloaded and complementary operands.

[0104] Advantageously, steps 508 and 510 can be carried out in parallel so as to process in 510 data for a pixel P(nl), i.e. whose data was transmitted in a previous cycle, and in parallel transmit data for a current pixel Pn which will be processed in the next cycle.

[0105] In another embodiment, steps 508 and 510 are performed sequentially to process the same pixel.

[0106] In one embodiment, the method may include, during step 506 of execution of a neural operation, data processing steps performed on computing units of computing cores directly adjacent to a dedicated computing core.

[0107] According to some embodiments, the step of transmitting complementary operands can consist of transmitting pixel-type data or "feature-map" values ​​according to the established anglicism.

[0108] The step of transmitting complementary operands may consist of transmitting data from an external memory to the multi-core computing platform to the groups of cores via the communication bus network.

[0109] In one embodiment, step 510 of execution of a neural operation consists of performing in a group of computing cores, multiplication-accumulation operations, using, via the neighbor links between immediate neighboring cores, operands preloaded in the different local memories of the different immediate neighboring cores.

[0110] When the neural execution operation is complete, the process iterates (loops back to step 502) to move on to the next operation.

[0111] The process continues with a step 512 consisting of transmitting the result of the execution 510 of a neural operation to a memory external to the multi-core computing platform.

[0112] In one embodiment, step 512 of result transmission, is performed during the execution of a neural operation in order to transmit the result of a neural operation performed to a previous cycle.

[0113] A solution for the inference of convolutional neural networks has thus been described, which allows only a portion of the computing cores of a multi-core system to be used to perform neural computations, and which allows the local memory capacity of each computing core dedicated to computations to be extended as the sum of the local memory capacities of direct neighboring computing cores.

[0114] The invention has been described for a multi-core platform architecture without a reconfigurable FPGA-type component. Other architectures with programming logic (PL) can nevertheless be used. Preparing a component for CNN inference would consist of defining a logic architecture in the PL portion of the platform and programming the core groups responsible for the dominant part of the neural processing, adapting the principles described.

Claims

Demands

1. A computing method for inferring convolutional neural networks, the convolutional neural network being implemented on a multi-core computing platform comprising a plurality of computing cores, each core comprising a local core memory of capacity 'Cm' and a computing unit, the computing cores being grouped into groups of computing cores, such that in a group, one core is assigned to compute the neural operations and is connected via neighbor links to at least one immediate neighboring computing core, the method comprising, during an inference (502), steps consisting of: - preloading (504) a subset of operands required to perform a neural operation, into the local core memories of the groups of cores,the local core memories of a group of cores defining a local group memory with a capacity 'CM' equal to the sum of the capacities 'Cm' of the local core memories of the cores belonging to the same group; - execute (506) the neural operation, said execution comprising steps consisting of transmitting (508) to the groups of cores, operands complementary to the operands preloaded in local group memory; and performing (510), by the processing unit of said assigned processing cores, the neural operation with all the preloaded and complementary operands.

2. A calculation method according to claim 1 wherein the preloading step in each local group memory consists of storing the operands of said subset of operands in all or part of the local core memories of said group.

3. A calculation method according to any one of claims 1 or 2 wherein the preloading step in each local group memory consists of preloading a subset of neural layer coefficients obtained during a neural network training phase.

4. A calculation method according to any one of the preceding claims, wherein the operand transmission step complementary methods consist of transmitting pixel-type data or transmitting "feature-map" values.

5. A computing method according to any one of the preceding claims, wherein the transmission step consists of transmitting data from an external memory to the multi-core computing platform via a communication bus network, said bus network being arranged on the multi-core platform to route data over all computing cores and between computing cores.

6. A computing method according to any one of the preceding claims, wherein the step of performing the neural operation includes a step of transferring data stored in the local memories of the groups, via the neighbor links between immediate neighboring cores.

7. A computing method according to any one of the preceding claims, wherein the step of executing a neural operation consists of performing multiplication-accumulation operations by the computing units of said dedicated cores.

8. A computing method according to any one of the preceding claims comprising a step (512) of transmitting a result of a neural operation to a memory external to the multi-core computing platform, said step being performed after the execution of a current neural operation to transmit a current result or performed during the execution of a neural operation to transmit a result of a previous neural operation.

9. A computing method according to any one of the preceding claims, wherein the operations of the execution of a neural operation are carried out in parallel or sequentially.

10. A computing method according to any one of the preceding claims further comprising, during the neural execution step, data processing steps operated on computing units of computing cores directly adjacent to a dedicated computing core.

11. A computing method according to any one of the preceding claims further comprising a step of putting the inactive computing units of the computing cores dedicated to storage to sleep or putting them into "power gating".

12. A computing method according to any one of the preceding claims comprising an initial step (400) of creating groups of cores according to a predefined pattern, the grouping defining for each group, a computing core dedicated to calculations such that only the computing units of said core are active to perform calculations, and such that the local group memory of a group of cores is defined by a capacity 'CM' equal to the sum of the capacities 'Cm' of the local core memories of the cores of said group.

13. A computing device for inference of convolutional neural networks, the convolutional neural network being implemented on a multi-core computing platform comprising a plurality of computing cores, each core comprising a local core memory of capacity 'Cm' and a computing unit, the computing cores being grouped into groups of computing cores, such that in a group, one core is assigned to the computations of neural operations and is connected via neighbor links to at least one immediate neighboring computing core, the device comprising means for implementing the steps of the method according to any one of claims 1 to 12.

Images