Method and computational tool for determining the transfer function between pairs of consecutive layers of a neural network

Abstract

Embodiments of the present disclosure relate to a method and computing tool for determining transfer functions between pairs of consecutive layers of a neural network. The method may be performed before a processing unit implements the neural network. The neural network includes a series of layers and at least one operator applied between at least one pair of consecutive layers. A computing tool generates executable code to be executed by the processing unit to implement the neural network. The computing tool generates at least one transfer function between at least one pair of layers, the at least one transfer function taking the form of a set of pre-computed values.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority from European Patent Application No. 19290051.2, filed on July 10, 2019, which is incorporated herein by reference. Technical Field

[0003] Implementation modes and embodiments of the present invention relate to a method for implementing a neural network. Background Art

[0004] Neural networks are widely used to solve various statistical problems, especially data classification problems.

[0005] After a machine learning phase that is usually supervised (i.e., performed using a reference database of already classified data), the neural network "learns" autonomously and becomes able to apply the same classification to unknown data.

[0006] One might mention convolutional neural networks (CNNs), a type of neural network in which the connection pattern between two layers is inspired by the visual cortex of animals. This type of neural network allows for efficient recognition of objects or people in images or videos.

[0007] The architecture of a neural network typically consists of a series of layers, with each layer receiving its input from the output of the previous layer.

[0008] Deep neural networks tend to become increasingly complex, and more and more users are looking to quantize the input and output data of each layer of the neural network in order to increase processing speed and reduce the memory space required to store this data.

[0009] For example, the output data and input data of each layer may be quantized to eight bits or sixteen bits, but have the same or different quantization formats.

[0010] In particular, all layers may not be quantized with the same precision and have the same range of values, these parameters depending inter alia on the dynamic range of the output data of the layer.

[0011] In a neural network, input data is multiplied by at least one weight of a given value for each layer.

[0012] “Weight” is a term whose meaning is well known to those skilled in the art in the field of neural networks. The weight refers to a neuron parameter that can be configured to obtain good output data.

[0013] These weights are initially defined in the neural network using a training database. More precisely, the neural network processes an image extracted from the database and produces a prediction as output, i.e., a determination of which class the image likely belongs to, even though the class it describes is already known.

[0014] Based on the accuracy of the result, all weights of the neural network are updated, for example using a back-propagation algorithm.

[0015] Once again, the weights of each layer (especially for memory optimization reasons) can be represented using sixteen or eight bits.

[0016] Examples of quantization schemes are well known to those skilled in the art.

[0017] Information about these quantization schemes can be found in particular in Benoit Jacob et al., “Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference”, arXiv:1712.05877v1 [cs.LG], 15 December 2017, Google Inc. ( https: / / arxiv.org / pdf / 1712.05877.pdf ) and in Raghuraman Krishnamoorthi's paper "Quantizing deep convolutional networks for efficient inference: A whitepaper", arXiv:1806.08342v1 [cs.LG], June 21, 2018, Google ( https: / / arxiv.org / pdf / 1806.08342.pdf ) in the article.

[0018] Furthermore, most commonly, one or more operators are specified that are applied to the output of the layer. Such operators can be of any type, and are most commonly non-linear. The operators can also be ONNX type operators.

[0019] It should be recalled that ONNX (Open Neural Network Exchange) is a community project related to AI that, in particular, provides definitions of extensible computational graph models as well as definitions of built-in operators and standard data types.

[0020] Such ONNX operators are well known to those skilled in the art, and reference may be made, for example, to the documentation available at the following Internet link:

[0021] https: / / github.com / onnx / onnx / blob / master / docs / Operators.md

[0022] Conventional methods of determining a transfer function between two layers to which one or more operators are applied do not work.

[0023] Specifically, these methods first need to convert each quantized data of the output layer of the first layer into data expressed in floating-point format, then approximate the operator in floating-point format, for example, using cascade, and apply this approximation to each output data of the first layer, and finally requantize the result in an appropriate quantization format to deliver the quantized input data of the next layer.

[0024] This is done in real time by the processor executing the neural network and is completely inefficient in terms of computation time and memory. Summary of the Invention

[0025] Implementation modes and embodiments of the present invention relate to neural networks, particularly but not exclusively to deep neural networks, and in particular to determining transfer functions between pairs of consecutive layers of a neural network to which one or more operators are applied.

[0026] The embodiments provide a satisfactory solution to the problems of the prior art.

[0027] In one embodiment, a method is provided that can be performed prior to implementing a neural network by a processing unit. The neural network includes a series of layers and at least one operator applied between at least one pair of consecutive layers. A computing tool generates executable code to be executed by the processing unit to implement the neural network. The computing tool generates at least one transfer function between at least one pair of layers, the at least one transfer function taking the form of a set of pre-computed values.

[0028] Additional embodiments and variations are discussed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Other advantages and characteristics of the invention will become apparent on reading the detailed description of a completely non-limiting mode of implementation and embodiment, together with the accompanying drawings, in which:

[0030] Figures 1 to 3 It relates to implementation modes and embodiments of the present invention. DETAILED DESCRIPTION

[0031] According to one mode of realisation and embodiment, a method and a computing tool are provided which enable determination of such a transfer function in a manner which is efficient in terms of memory and computing time and which reduces the performance of the neural network in question by a minimum amount.

[0032] According to one implementation mode and embodiment, it is proposed not to calculate one or more transfer functions in real time during the implementation of the neural network, but to pre-calculate the values of this or these transfer functions, for example to form one or more tables, i.e. it is proposed to calculate these values advantageously during or after the training phase of the neural network, before its implementation by a processing unit (which may be, for example, a microprocessor of a device such as a smartphone or even a tablet computer). These examples are non-limiting.

[0033] According to one aspect, a method is provided that is performed prior to implementing a neural network by a processing unit (e.g., a microprocessor of a device such as a mobile phone). The neural network comprises a series of layers and at least one operator applied between at least one pair of consecutive layers. The method comprises generating, using a computing tool (e.g., a personal laptop), executable code intended for execution by the processing unit to implement the neural network, and generating, using the computing tool, at least one transfer function between at least one pair of layers, the at least one transfer function taking the form of a set of pre-computed values.

[0034] These values are pre-calculated in the sense that they are calculated before the processing unit implements the neural network. Therefore, during the implementation of the neural network, i.e., during the execution of the executable code by the processing unit, there is no need to calculate one or more transfer functions in real time. Extracting the corresponding values from the pre-calculated values is straightforward.

[0035] Typically, output data of a first layer in a pair of layers is advantageously quantized using a first quantization scheme, and input data of a second layer in the pair is advantageously quantized using a second quantization scheme.

[0036] The two quantization schemes can be the same or different.

[0037] Likewise, the number of bits of the quantized input data and the number of bits of the quantized output data may be the same or different.

[0038] The transfer function takes into account the first and second quantization schemes and the at least one operator.

[0039] In general, an operator can be any function (or "activation" to use the term familiar to those skilled in the art of neural networks).

[0040] Thus, the operator may be, for example, an "identity" operator (OP(x)=x). In this case, the transfer function merely behaves like a convergence function, allowing data quantized using a first quantization scheme to be converted into data quantized using a second quantization scheme.

[0041] According to one mode of implementation, at least one transfer function is intended to receive, during implementation of the neural network, quantized output data from a first layer of the pair, to apply at least one operator and to deliver quantized input data to a second layer of the pair.

[0042] Generating the at least one transfer function includes, for each possible quantized value of the output data, applying an inverse first quantization function to convert the quantized value into a floating point value, applying at least one operator to the floating point value to obtain an intermediate floating point value, and applying a second quantization function to the intermediate value to deliver a corresponding quantized value of the input data. Generating further includes generating a lookup table that forms the set of values and contains all corresponding possible values of the quantized input data for all possible values of the quantized output data.

[0043] Any type of operator can be used, such as operators with multiple inputs and multiple outputs.

[0044] However, in this case, it is preferable to limit the number of bits of the quantized input and output data in order to limit the size of the lookup table and therefore the memory space required when subsequently storing it in a device intended to implement a neural network.

[0045] For this purpose, for at least one operator, a point-to-point operator is preferably used, ie an operator with one input and one output. Thus, the input and output data can be quantized to any number of bits, for example eight or sixteen bits.

[0046] According to one mode of implementation, in which the neural network comprises at least one operator applied between P pairs of consecutive layers, generating P respective transfer functions between the P pairs of layers comprises generating a maximum of P sets of values.

[0047] In particular, during the pre-computation of each set of values representing the P transfer functions, it may be possible for a given set of values to be used for two transfer functions between two different pairs of layers.

[0048] Typically, a phase of training the neural network is provided, and during or after the training phase, the generation of at least one transfer function is advantageously implemented, which allows the values of all precalculated values to be adjusted, in particular according to the values of the weights of the neural network, the values of these weights being optimized during the training phase.

[0049] According to another aspect, there is provided a computing tool configured to implement the method as defined above.

[0050] According to another aspect, a method for implementing a neural network using a processing unit is provided, the neural network comprising a series of layers and at least one operator applied between at least one pair of consecutive layers.

[0051] The implementation method includes storing in a memory executable code that enables implementation of a neural network and a set of values representing at least one transfer function between at least one pair of layers, and executing the executable code with a processing unit. The execution includes determining the transfer function by extracting from the set of stored values a value corresponding to a value of quantized output data of a first layer in the at least one pair of layers, the extracted value being the quantized input data of a second layer in the pair.

[0052] According to one mode of implementation, the set of values representing at least one transfer function between at least one pair of layers is stored in the form of a lookup table, the transfer function being intended, during implementation of the neural network, to receive quantized output data from a first layer of the pair, to apply at least one operator and to deliver quantized input data to a second layer of the pair, the at least one lookup table containing, for all possible values of the quantized output data, all corresponding possible values of the quantized input data.

[0053] At least one operator is, for example, a point-to-point operator.

[0054] When the neural network comprises at least one operator applied between P pairs of consecutive layers, a maximum of P sets of values representing P transfer functions between the P pairs of layers are, for example, stored in a memory.

[0055] According to another aspect, there is provided a device configured to implement the implementation method defined above.

[0056] According to another aspect, a computing tool is provided, comprising generating means configured before implementation by a processing unit of a neural network, the neural network comprising a sequence of layers and at least one operator applied between at least one pair of consecutive layers, to generate executable code intended to be executed by the processing unit in order to implement the neural network, and to generate at least one transfer function between at least one pair of layers, the at least one transfer function being in the form of a set of precomputed values.

[0057] According to one embodiment, at least one transfer function is intended to receive quantized output data from a first layer of the pair, apply at least one operator, and deliver quantized input data to a second layer of the pair during implementation of the neural network. For the purpose of generating the at least one transfer function, for each possible quantized value of the output data, the generating means are configured to apply an inverse first quantization function, thereby converting the quantized value into a floating-point value, apply at least one operator to the floating-point value to obtain an intermediate floating-point value, apply a second quantization function to the intermediate value to deliver a corresponding quantized value of the input data, and generate a lookup table containing all corresponding possible values of the quantized input data for all possible values of the quantized output data.

[0058] According to one embodiment, at least one operator is a point-to-point operator.

[0059] According to one embodiment, a computing tool is provided wherein the neural network comprises at least one operator applied between P pairs of consecutive layers, and the generating means is configured to generate a maximum of P lookup tables when generating P respective transfer functions between the P pairs of layers.

[0060] According to one embodiment, the computing means are configured to implement a phase of training the neural network, and the generating means are configured to generate at least one transfer function during or after the training phase.

[0061] According to one embodiment, the computing means may comprise a microprocessor or microcontroller incorporated into the generating device.

[0062] According to another aspect, a device is provided. The device includes a processing unit configured to implement a digital neural network. The neural network includes a series of layers and at least one operator applied between at least one pair of consecutive layers. A memory is configured to store executable code enabling implementation of the neural network and at least one set of values representing at least one transfer function between at least one pair of layers. The processing unit is configured to execute the executable code, the execution including determining the at least one transfer function by extracting, from the at least one set of stored values, a value corresponding to a value of quantized output data of a first layer in at least one pair of layers, the extracted value being quantized input data of a second layer in the pair.

[0063] According to one embodiment, at least one set of values representing at least one transfer function between at least one pair of layers, the transfer function being intended to receive quantized output data from a first layer of the pair, apply at least one operator and deliver quantized input data to a second layer of the pair during implementation of a neural network, is stored in the form of at least one lookup table, the at least one lookup table containing all corresponding possible values of the quantized input data for all possible values of the quantized output data.

[0064] According to one embodiment, at least one operator is a point-to-point operator.

[0065] According to an embodiment, wherein the neural network comprises at least one operator applied between P pairs of consecutive layers, the memory is configured to store at most P sets of values representing P transfer functions between the P pairs of layers.

[0066] According to one embodiment, the processing unit may comprise a microprocessor or comprise an actual hardware accelerator.

[0067] Reference will now be made to the drawings.

[0068] exist Figure 1, reference character ORD designates a computing tool including a microprocessor or a microcontroller, such as a personal laptop computer.

[0069] The microprocessor of the tool ORD incorporates software generation means GEN which will in particular generate (step 11 ) an executable code representing a neural network DNN, in particular a deep neural network.

[0070] like Figure 1 As schematically illustrated in , the neural network DNN comprises a series of layers, the current layer of which is called LYk.

[0071] Reference notation DSk designates quantized output data of layer LYk.

[0072] Reference notation DEk+1 designates quantized input data of the next layer LYk+1 of level k+1.

[0073] Reference sign Opk designates the operator which is applied here between a layer Lyk and the next layer LYk+1.

[0074] Reference notation Qk designates a quantization scheme of output data DSk, and reference notation Qk+1 designates a quantization scheme of input data DEk+1.

[0075] The quantization schemes Qk and Qk+1 may be the same or different depending, inter alia, on the required accuracy of the corresponding data.

[0076] In the case where there is an operator between two consecutive layers of a neural network, it is necessary to determine a transfer function that will receive the quantized output data from the first layer of the pair of consecutive layers (here layer LYk), to which the operator OPk will be applied and which will deliver the quantized input data to the second layer of the pair (here layer LYk+1).

[0077] According to the invention, the transfer function will be generated in the form of a table TF of pre-calculated values.

[0078] This generation 12 is also carried out by the generation means and it is advantageously carried out during or after the phase PHT of training the neural network in order to take into account the final values of the weights of the neural network.

[0079] Typically, there is a lookup table TF for each transfer function between two layers.

[0080] Thus, when, for example, P operators are applied between P pairs of layers of a neural network, multiple pre-computed lookup tables may be identical for certain pairs of layers.

[0081] In other words, if Figure 1As illustrated in , assuming that one or more operators are applied between P pairs of layers of a neural network, the generating device will generate T lookup tables, where T is less than or equal to P.

[0082] Thus, the calculation tool ORD will deliver on the one hand the executable code CX and on the other hand T look-up tables TFt, where t varies between 1 and T.

[0083] In order to implement a neural network within a device APP, such as (by way of non-limiting example) a smartphone or indeed a tablet computer equipped with a processing unit UT (such as a microprocessor or indeed a hardware accelerator), the executable code CX and the various look-up tables TFt are stored (step 13) in the memory MM.

[0084] The memory may include one or more physical memories. In the case of multiple physical memories, the executable code CX may be stored in a first memory, and the lookup table TFt may be stored in another memory.

[0085] The processing unit UT then implements the neural network DNN by executing the executable code CX on the one hand and by determining the various transfer functions FTk on the other hand (step 14 ), which is delivered to the processing unit together with any kind of input data.

[0086] As Figure 1 As schematically illustrated by the dotted arrows in , these various transfer functions FTk are not determined by real-time calculation, but are determined by extracting pre-calculated values from the corresponding lookup table TFt based on the value of the output data of the first layer in the corresponding paired layers.

[0087] This situation will be described in more detail below.

[0088] The implementation of a neural network DNN ultimately allows, for example, to obtain a decision DF, such as a classification decision.

[0089] More particularly, reference will now be made to Figure 2 and 3 , in order to describe an example of generating a lookup table TF in more detail.

[0090] For simplicity, it is assumed that only a single operator OPk is applied between two consecutive layers LYk and LYk+1.

[0091] Typically, for each possible quantized value of the output data DSk, the generation of the transfer function between these two layers involves the need to apply an inverse first quantization function Qk-1, thereby allowing the value quantized using a given quantization scheme to be converted into a floating-point value, the application of an operator OPk to the floating-point value in order to obtain an intermediate floating-point value, and the application of a second quantization function Qk+1 to the intermediate value in order to deliver the corresponding quantized value of the input data DEk+1 (quantized using the same quantization scheme or another quantization scheme).

[0092] A lookup table TFi is generated (step 23) and contains all possible values of the quantized input data for all possible values of the quantized output data.

[0093] Furthermore, these pre-calculated values take into account the quantization scheme of the output data of layer LYk and the quantization scheme of the input data of layer LYk+1.

[0094] like Figure 2 As illustrated in , the lookup table is generated by first defining an index j representing a quantized value of the output data DSk.

[0095] These quantized values are quantized to nb bits, and for example, to 8 bits.

[0096] like Figure 2 As shown in block 21 of FIG, for unsigned quantization (a), j is between 0 and 2 nb -1 changes.

[0097] For signed quantization (b), j is in -2 nb-1 and 2 nb-1 -1 changes.

[0098] For signed quantization with symmetric range (c), j is between -2 nb-1 +1 to 2 nb-1 -1 changes.

[0099] For example, when quantization to 8 bits (nb=8) is employed, in case (a), j can take all values between 0 and 255 inclusive in decimal notation.

[0100] Then look up the index i of table TF( Figure 2 Box 22 in is defined.

[0101] In case (a), i is equal to j.

[0102] In case (b), i equals j+2 nb-1 .

[0103] In case (c), i equals j+2 nb-1 -1.

[0104] Typically, i is equal to j - minR, where minR specifies the minimum value of the quantized range of values for j.

[0105] Furthermore, i varies between 0 and 2nb-1.

[0106] Thus, for quantization to 8 bits, 256 values are obtained for the lookup table TF. Each value TF(i) of this lookup table corresponds to a quantized value of the input data DEk+1 of the layer LYk+1, which in turn corresponds to a value j of the quantized output data DSk of the layer LYk.

[0107] Therefore, during the implementation of the neural network, for a given quantized value of the output data of a layer LYk, and therefore for a given value of j, the corresponding quantized value of the input data DEk+1 will be obtained directly from the corresponding index i in the table, this input data being quantized using the correct quantization scheme and produced by applying the operator OPk to the corresponding output data of the previous layer.

[0108] Furthermore, when there is a set of output data for a given layer, a set of input data for the next layer is obtained by extracting the corresponding value of the input data for each output data in the set from the lookup table TF.

[0109] Figure 3 Various possible quantization schemes and various possible operators are illustrated, which are advantageously point-to-point operators, ie, operators that do not modify the dimensions of the tensors to which they are applied.

[0110] Regarding the operator OPk, for example, the hyperbolic tangent function, the sigmoid function, the leaky-ReLu function, and the Erf function may be mentioned. All these operators are well known to those skilled in the art and are described, for example, in the aforementioned ONNX document, which can be accessed via the following Internet https: / / github.com / onnx / onnx / blob / master / docs / Operators.md.

[0111] Regarding the inverse quantization operation Q-1k, that is, the operation of converting the quantized value into a floating point value, for example, Figure 3 As illustrated in block 31 of , uniform affine quantization (1) may be used, where the "scale" parameter defines the quantization step size, and the "zero_point" parameter is the offset between real zero and zero of the quantized value.

[0112] Asymmetric quantization (2) or symmetric quantization (3) or fixed Qmn type quantization (4) can also be used.

[0113] All of these examples are non-limiting and well known to those skilled in the art, who are referred to the aforementioned articles by Krishnamoorthi and Jacob for more details, for all intents and purposes.

[0114] The function Qk+1 can be Figure 3 An inverse quantization scheme of one of the quantization schemes (1) to (4) mentioned in block 31 of .

[0115] Of course, the invention is not limited to the realization modes and embodiments that have just been described, and encompasses all variants of the invention.

[0116] Therefore, multiple operators can be applied between two consecutive layers of a neural network.

[0117] In this case, the lookup table of transfer functions will apply these various transfer operators to index j via successive applications of these operator cascades.

[0118] Furthermore, the present invention can be used advantageously in conjunction with the invention described in French patent application filed under number 1902855, for example.

Claims

1. A method for processing image data by implementing a neural network by a processing unit, the neural network comprising a series of layers and at least one operator applied between at least one pair of consecutive layers, the method comprising: generating, using a computing tool, executable code intended to be executed by the processing unit to implement the neural network; generating, using the calculation tool, at least one transfer function between the at least one pair of consecutive layers, the at least one transfer function taking the form of a set of pre-calculated values; as well as Receiving quantized output data from a first layer of the at least one pair of consecutive layers via at least one transfer function, wherein generating the at least one transfer function comprises: For each possible quantized value, applying an inverse first quantization function to convert the quantized value to a floating point value; applying the at least one operator to the floating-point value to obtain an intermediate floating-point value; applying a second quantization function to the intermediate floating point value to deliver a corresponding quantized value to a second layer of the at least one pair of consecutive layers; and A lookup table is generated, the lookup table including, for all possible values of the quantized output data, all corresponding possible values of the quantized input data.

2. The method of claim 1 , further comprising executing the executable code to implement the neural network. The method of claim 1 , wherein the at least one operator is a point-to-point operator.

4. The method of claim 1 , wherein the neural network comprises at least one operator applied between P pairs of consecutive layers, and wherein generating the at least one transfer function comprises: P transfer functions are generated between P pairs of the layers by generating at most P sets of values.

5. The method of claim 1 , further comprising performing a training phase for the neural network, wherein the at least one transfer function is generated during or after the training phase. The method of claim 1 , wherein the computing means comprises a microprocessor or a microcontroller.

7. A computing tool comprising: processor; as well as A non-transitory computer-readable medium storing software to be executed by the processor to implement the method according to claim 1.

8. A method for processing image data using a processing unit implementing a neural network, the neural network comprising a series of layers and at least one operator applied between at least one pair of consecutive layers, the method comprising: storing in a memory executable code and a set of values representing at least one transfer function between the at least one pair of consecutive layers; executing, with the processing unit, the executable code, the executing comprising determining the transfer function by extracting, from the stored set of values, values corresponding to values of quantized output data of a first layer of the at least one pair of consecutive layers, the extracted values being quantized input data of a second layer of the at least one pair; For each possible quantized value, applying an inverse first quantization function to convert the quantized value to a floating point value; applying the at least one operator to the floating-point value to obtain an intermediate floating-point value; applying a second quantization function to the intermediate floating-point value to deliver a corresponding quantized value to a second layer of the at least one pair of consecutive layers; as well as A lookup table is generated, the lookup table including all corresponding possible values of the quantized input data for all possible values of the quantized output data.

9. The method of claim 8 , wherein the set of values representing the at least one transfer function between at least one pair of the layers is stored in the form of a lookup table, wherein the function is intended to receive quantized output data from the first layer of the at least one pair, apply the at least one operator, and deliver the at least one lookup table, quantized input data to the second layer of the at least one pair during implementation of the neural network, the lookup table containing possible values of all corresponding quantized input data for all possible values of the quantized output data.

10. The method of claim 8, wherein the at least one operator is a point-to-point operator.

11. The method of claim 8, wherein the neural network comprises at least one operator applied between P pairs of consecutive layers, a maximum of P sets of values representing P of the transfer functions between P pairs of the layers being stored in the memory.

12. A computing device comprising: processor; as well as A non-transitory computer-readable medium storing software to be executed by the processor to implement the method according to claim 8.

13. A computational tool for implementing a neural network to process image data, comprising: processor; as well as A non-transitory computer-readable medium storing software to be executed by the processor prior to implementation of the neural network, the neural network comprising a series of layers and at least one operator applied between at least one pair of consecutive layers, the software, when executed by the processor, causing the processor to: generating executable code intended to be executed by a processing unit to implement said neural network, generating at least one transfer function between the at least one pair of consecutive layers, the at least one transfer function taking the form of a set of pre-computed values, For each possible quantized value, applying an inverse first quantization function to convert the quantized value to a floating point value; applying the at least one operator to the floating-point value to obtain an intermediate floating-point value; applying a second quantization function to the intermediate floating-point value to deliver a corresponding quantized value to a second layer of the at least one pair of consecutive layers; as well as A lookup table is generated that includes all corresponding possible values of the quantized input data for all possible values of the quantized output data.

14. The computing tool of claim 13, wherein the at least one operator is a point-to-point operator.

15. The computing tool of claim 13 , wherein the neural network comprises at least one operator applied between P pairs of consecutive layers, and wherein the software, when executed by the processor, causes the processor to generate at most P sets of values when generating P respective transfer functions between the P pairs of layers.

16. The computing tool of claim 13, wherein the software, when executed by the processor, causes the processor to perform a training phase of the neural network, wherein the at least one transfer function is generated during or after the training phase.

17. A computing device comprising: a processing unit configured to implement a digital neural network to process the image data, the neural network comprising a series of layers and at least one operator applied between at least one pair of consecutive layers; as well as a memory configured to store executable code that allows the neural network to be implemented and at least one set of values representing at least one transfer function between the at least one pair of consecutive layers; The processing unit is configured to: executing the executable code to determine the at least one transfer function by extracting a value from the at least one stored set of values, the extracted value corresponding to a value of quantized output data of a first layer of the at least one pair of consecutive layers, the extracted value being quantized input data of a second layer of the at least one pair, For each possible quantized value, applying an inverse first quantization function to convert the quantized value to a floating point value; applying the at least one operator to the floating-point value to obtain an intermediate floating-point value; applying a second quantization function to the intermediate floating-point value to deliver a corresponding quantized value to a second layer of the at least one pair of consecutive layers; as well as A lookup table is generated, the lookup table including all corresponding possible values of the quantized input data for all possible values of the quantized output data.

18. The apparatus of claim 17 , wherein the set of values representing the at least one transfer function between the at least one pair of consecutive layers is stored as at least one lookup table, wherein the at least one transfer function receives quantized output data from the first layer of the at least one pair of consecutive layers, applies the at least one operator, and delivers quantized input data to the second layer of the at least one pair of consecutive layers during implementation of the neural network, the at least one lookup table containing, for all possible values of the quantized output data, all corresponding possible values of the quantized input data.

19. The apparatus of claim 17, wherein the at least one operator is a point-to-point operator.

20. The apparatus of claim 17, wherein the neural network comprises at least one operator applied between P pairs of consecutive layers, and wherein the memory is configured to store at most P sets of values representing P of the transfer functions between P pairs of the layers.

21. The apparatus of claim 17, wherein the processing unit comprises a microprocessor.

22. The apparatus of claim 17, wherein the processing unit comprises a hardware accelerator.

Images