Computing device and audio encoding method employing a neural network

Abstract

The invention describes a computing device implementing a neural network comprising at least one neuron that implements a multiplication between an input vector X=(X1,...XN) and a vector of parameters comprising a term defined as follows: W t = 1 t M, where: - M is a matrix defined as a product M=MN-1...M1 of N-1 matrices Mi, where each matrix is an elementary rotation operator applied to a pair of input components of the matrix and parameterized by a respective rotation angle θi, and - 1 is a vector of dimension N, all terms of which are zero except one, equal to 1, positioned on a row corresponding to the second output component of the last rotation MN-1. The invention also describes advantageous applications of the neural network to signal encoding, image processing, and specific hardware implementation modalities.

Description

Description Title: Computer Device and Audio Encoding Process Using Neural Networks Technical Field

[0001] This disclosure relates to a computer device implementing an artificial neural network, a method for training the neural network, and applications of the neural network including audio encoding and image processing. Previous technique

[0002] In recent years, the field of artificial neural networks has seen significant advances, particularly in their application to complex tasks such as image processing and audio encoding. These networks often rely on computationally intensive operations, such as generalized matrix multiplication (GEMM).

[0003] The generalized matrix multiplication operation calculates the product of matrices as follows: Y = α × WX + β × Y where W, X, and Y are matrices, and a and p are real-valued parameters. By setting a = 1 and p = 0, Y becomes the result of the multiplication WX, where W represents the layer weighting coefficient matrix and X represents the layer input values. When p = 1, the matrix produces the result of a residual layer, allowing the establishment of a direct "skip connection." Computing generalized matrix multiplication can be resource-intensive and time-consuming. In particular, generalized matrix multiplication is widely used in convolutional layers and fully connected dense layers, which concentrate the majority of neural network execution time on the CPU or GPU.

[0004] Therefore, there is a need to perform matrix multiplication in a more efficient way in terms of computation time and computational resources.

[0005] In particular, in the field of audio encoding, a variational auto-encoder (VAE) type neural network is known, for example from Défossez et al., 2022, comprising an encoding block into a latent space and a decoding block, as well as an intercalated vector quantization block allowing the dimensionality of the data to be reduced in the latent space and thus to achieve compression of the audio signal.

[0006] One drawback of this type of neural network is the high complexity of the operations implemented at the level of vector quantization, implying high computational and memory requirements. Exposed

[0007] This disclosure improves the situation.

[0008] A computer system is proposed, comprising at least one computer configured to implement an artificial neural network, the artificial neural network comprising at least one layer comprising at least one neuron implementing a multiplication between an input vector X=(XI,... XN) and a parameter vector, to obtain an output component y, characterized in that the parameter vector comprises a term defined as follows: W t = 1 t M Or: M is a unitary square matrix of dimension N*N defined as a product M=MN 1 ... M1 of N-1 matrices Mi where each matrix is ​​an elementary rotation operator applied to a pair of input components of the matrix Mi and parameterized by a respective rotation angle 0i bounded between -TT / 2 and TT / 2, and 1 is an N-dimensional vector whose terms are all null except for one, equal to 1, positioned on a line corresponding to the second output component of the last MN-1 rotation of the matrix product.

[0009] In the following, such a layer of an artificial neural network is called a "spiral layer".

[0010] The parameter matrix of the proposed neural network layer replaces classical weighting coefficients with elementary rotation angles, without increasing the number of parameters to be trained, for the same neural network structure. Furthermore, the parameter vector is structured to retain only the second output component of the last elementary rotation operator, which simplifies calculations and reduces the computational resources and memory required to train and implement the neural network.

[0011] In some embodiments, the parameter vector is the product of a gain component G and the term W t The gain component allows the magnitude of the layer's output vector to be modified, since the W' term is unitary.

[0012] In some embodiments, each elementary rotation operator is an elementary Givens rotation matrix defined as follows: / 11— i 0 0 0 _ 0 -sin0j cos 0 j 1 I 0 cos sinfy 0 I 0 0 Ai-i-1 '

[0013] In some embodiments, each elementary rotation operator is defined as follows: f m k,k = 1 if kp, q = — sin0 ; = — m„„ = [m w ] fc =i,2. N 2, m„ n = cos 0 ; = m n " 0 otherwise This definition allows a parallel implementation of some of the rotations in two dimensions (i.e. on two components).

[0014] Alternatively, each elementary rotation operator is defined as follows: I i— i 0 0 0 0 —tan 1 0 0 1 tan 0 On-il,nl 0 0 and the gain is defined by G = G o HE cos G; where Go is a parameterizable gain value. The parameterizable gain value can either be a "trainable" parameter, i.e., adjusted during the training of the neural network, or fixed to a predetermined value, for example equal to 1 to maintain a unitary norm for the parameter vector, particularly in the case of an orthonormalization layer defined below.

[0015] In some embodiments, to allow parallel implementation of subsets of rotation operators, each elementary rotation operator is defined as follows: C m k,k = 1 if kp,q = [m k l ]k=i,2. N = ■ 1=1, 2,..., N MP ' W P lp = “ g T 1 AN 0I Wl = ~ qp MQ ' Q 0 otherwise

[0016] In some embodiments, the neural network comprising this structure of elementary rotation operators is implemented on rotation angles represented in binary format, that is, by being encoded on a specific integer number of bits. Where appropriate, the structure of each neuron implementing tangent-based elementary rotation operators allows for a naturally non-linear quantization of the rotation angles.

[0017] In some embodiments, the artificial neural network was trained on a training database, with the training of the artificial neural network comprising an iterative modification of the rotation angles of the elementary rotation operators and, where appropriate, the gain components. In particular, the cost functions classically used for training artificial networks are differentiable with respect to the rotation angles, so this network can be trained by implementing a stochastic gradient descent algorithm.

[0018] The non-linear quantization performed by the tangent function of elementary rotation operators is particularly advantageous after network training. Indeed, this quantization is more precise in the vicinity of zero due to the shape of the tangent function. Now, the weight statistics of a trained neural network are generally close to zero, which gives the trained network good accuracy despite the use of binary parameters.

[0019] In some embodiments, the device further includes memory storing a plurality of lookup tables, where each lookup table stores a plurality of pre-calculated values ​​of trigonometric functions, and the processor is configured to approximate the calculation of elementary rotations applied to the input vector by accessing the lookup tables. This computational method allows the neural network to be implemented with much more limited memory and computational resources than traditional neural networks with dense layers. For example, the computer implementing the trained neural network, or even the computer performing the neural network training, can include one or more FPGAs.

[0020] Alternatively, the computer can be configured to implement a CORDIC-type algorithm for calculating rotation angles applied to the input vector. A CORDIC-type algorithm uses a series of iterations to calculate a rotation angle, approximating this angle by a series of elementary rotations performed with multiplications by powers of 2, which is equivalent to performing decimal shifts in the binary representations of the angles being processed. This implementation is particularly efficient for computers without a hardware multiplication unit, such as microcontrollers, FPGAs, etc.

[0021] In some embodiments, the computing device further includes an interface for receiving an image to be processed, and the processor is configured to implement the artificial neural network for image processing. Image processing can be one of the following: Classification, Segmentation, Filtering, Edge detection, Pixel interpolation, Mathematical morphology (erosion, dilation...).

[0022] The spiral layer structure is particularly well-suited for efficient image processing due to the geometric nature of the images themselves, which often include patterns or features that can be described by rotations or angular orientations. For example, in object recognition and segmentation, the use of angular rotations in the layer can improve recognition regardless of the object's orientation in the image.

[0023] In some embodiments, the (so-called spiral) layer includes at least one additional neuron Nj, 1 < j < K where K < N — 1, configured to implement a multiplication between an input vector Xj and a parameter vector W of the additional neuron, the parameter vector W being defined by: W * = 1J Mj Or: M is a unitary square matrix of dimension (Nj)*(Nj) defined as a product M=MJ, N j-1 ... Mj,i of Nj-1 matrices Mj,i where each matrix is ​​an elementary rotation operator applied to a pair of input components of the matrix Mj,i and parameterized by a respective rotation angle 0i bounded between - TT / 2 and TT / 2, 1j is a vector of dimension Nj whose terms are all null except for one, equal to 1, positioned on a row corresponding to the second output component of the last matrix of the matrix product, and where the input vector of the additional neuron is composed of the first output component of each elementary rotation operator of the parameter vector of the previous neuron Nj-i of the layer.

[0024] This particular layer structure allows for the projection of the input vector X onto an orthonormal basis. Therefore, the neural network can inherently include a layer that performs this orthonormal projection, or orthonormalization. Depending on the requirements, this layer can be integrated at various points in the network, for example, to perform a compression stage by extracting the principal components of the data, or to remove noise from a signal. An orthonormalization layer can also improve the explainability of the neural network by decomposing a signal into independent components (for example, for images: brightness, contrast, intensity, sharpness, etc.).

[0025] In some embodiments, an application of the orthonormalization layer is to perform a Low-Rank Approximation (LoRA) technique. This technique decomposes each (m rows x n columns) layer W of a standard lattice according to its single-value decomposition (SVD): W = USV' where U and V are orthogonal matrices containing the right and left singular vectors, and S is a diagonal matrix of (r x r) dimensions containing the singular values, where r < min(m,ri) denotes the rank of W. In this case, the orthonormalization layer forms the matrix U of the SVD decomposition.

[0026] The learning of this matrix U can be carried out during training, then the projected components are truncated to r' « r, reducing the dimension of the matrix S to r' and finally the matrix V corresponds to a standard layer (e.g. a dense layer) learned in such a way as to minimize the cost function of the network.

[0027] After applying the LoRA technique, the number of parameters is r' x (m+n) whereas initially it was mxn for W. A gain in complexity is obtained when r' < (mxn) / (m+n).

[0028] According to another application, in some embodiments, the rotation angle values ​​of the layer's elementary rotation operators are adapted to perform a Karhunen-Loeve transform of the input vector x on the output vector y.

[0029] This Karhunen-Loeve transform can be obtained during neural network training by training the so-called spiral layer to adjust the rotation angle values ​​of the rotation operators of each neuron in the layer to maximize the variance of the output vector components. In some embodiments, training the spiral layer network can involve iteratively training the rotation angles of the derotation operators of each neuron in the layer to maximize the variance of the neuron's output component. In other words, this training involves training the first neuron to maximize the variance of its output component, then the second neuron, and so on.

[0030] The layer thus obtained makes it possible to reduce the dimensionality of the data, for example in audio or video encoding applications, while maintaining good fidelity of the input signal, by removing the last output components of the Karhunen-Loeve transform.

[0031] In some embodiments, the computer is configured to implement the artificial neural network in a building energy disaggregation algorithm, also known by the English acronym NILM for Non-Intrusive Load Monitoring. This type of algorithm makes it possible to decompose a signal, in this case an electrical signal, into different components corresponding to equipment in a building (for example, household appliances, electrical appliances) in order to analyze the individual electrical energy consumption of each piece of equipment.

[0032] In some embodiments, the neural network is of the VQ-VAE (Vector Quantization - Variational Auto Encoder) type, comprising: An encoding module configured to encode input data into a latent space; The layer performing a Karhunen-Loeve transform of the encoded data z' derived from the mean vector (p) of the encoded data in the latent space after network training, A truncation module configured to remove a set of components, starting from a given component, from the data resulting from the Karhunen-Loeve transform; a vector quantization module configured to implement quantization of the truncated data; and A decoding module configured to reconstruct the input data from the quantized data, plus the average vector.

[0033] This neural network can typically be used for encoding audio or video signals by applying a VQ-VAE neural network to such a signal, thus forming the input data. The so-called spiral layer, configured to perform a Karhunen-Loeve transform on the output data of the encoding module, identifies the principal components of this input data, and the truncation module removes secondary components to efficiently compress the data. For a given number of retained principal components, this approach, which includes truncation of the KLT components, minimizes the root mean square error between the initial data vector and the one reconstructed from the truncated components.This makes it possible to significantly reduce the cost of the vector quantization operation, while maintaining very good fidelity of the encoded signal, by keeping its principal components in the latent space.

[0034] Thus, according to another object, a method for encoding an audio or video signal is proposed, comprising: Receiving an audio or video signal to be encoded, The implementation, on the received signal, of an artificial neural network previously trained on a training database, comprising: o an encoding module configured to encode the signal in a latent space, o a module performing a Karhunen-Loeve transform of the encoded signal z', from which an average vector (p) of the training data encoded in the latent space has been deduced, o A truncation module configured to remove a set of components, starting from a given component, from the output signal of the Karhunen-Loeve transform, o A vector quantization module configured to implement quantization on the truncated signal, and o A decoding module configured to reconstruct the input audio signal from the quantized signal plus the average vector, in which the module performing a Karhunen-Loeve transform comprises a layer including a set of neurons Nj where j=0.. K-1, K < N - l where each neuron implements a multiplication between an input vector Xj and a parameter vector W } , to obtain a component yj of an output vector y=(yi,...yK), in which the input vector Xj is formed: for the first neuron No, by the encoded signal z', for the following neurons Nj, j=1.. K-1, by the first output component of each elementary rotation operator of the parameter vector of the preceding neuron of the layer, in which each parameter vector W includes a term defined as follows: Wf = 1J Mj Or: Mj is a unitary square matrix of dimension (Nj)*(Nj) defined as a product Mj=Mj, Nji...Mj,i of Nj-1 matrices Mj,i where each matrix is ​​an elementary rotation operator applied to a pair of input components of the matrix Mj,i and parameterized by a respective rotation angle 0i bounded between -TT / 2 and TT / 2,1j is a vector of dimension Nj whose terms are all null except for one, equal to 1, positioned on a row corresponding to the second output component of the last matrix of the matrix product, and where the rotation angles of the layer have been trained to maximize the variance of the output vector components.

[0035] According to another object, a method for training an artificial neural network is proposed, implemented by at least one computer, in which the artificial neural network includes a layer comprising at least one neuron implementing a multiplication between an input vector X=(XI,... XN) and a parameter matrix, to obtain an output component y, characterized in that the parameter matrix includes a term defined as follows: W t = 1 t M Or: M is a unitary square matrix of dimension n*n defined as a product M=M n1... M1 of N-1 matrices Mi where each matrix is ​​an elementary rotation operator applied to a pair of input components of the matrix Mi and parameterized by a respective rotation angle 0i bounded between - TT / 2 and TT / 2, 1 is a vector of dimension n whose terms are all null except for one, equal to 1, positioned on a row corresponding to the second output component of the last matrix MN 1 of the matrix product, the training includes adjusting the rotation angles of elementary rotation operators to minimize a network loss function on a training database.

[0036] In some embodiments, the parameter vector is the product of a gain component G and the term and the training also includes an iterative adjustment of the gain component.

[0037] In some embodiments, the training process includes: at least one forward pass, including the evaluation of the artificial neural network for a training dataset, the recording in memory, during each forward pass, of all the calculated parameters of the product Mx, the calculation of the loss function of the network at the end of the forward pass, and at least one backward pass including an update of the gain components and rotation angles from the gradients of the loss function with respect to the gain component and rotation angles of the elementary rotation operators, in which the values ​​of the gradients of the loss function with respect to the rotation angles of the elementary rotation operators are determined from the parameters of the product Mx stored in memory.

[0038] Training the neural network therefore requires less computational resources than training an equivalent dense layer (i.e., for identical sizes of the input and output vectors of the layer) since it is possible to call into memory, during the backpass, certain values ​​used for the calculation of the gradients of the loss function without having to recalculate them.

[0039] In some embodiments, the layer includes at least one additional neuron Nj, 1 < j < K where K < N — 1, configured to implement a multiplication between an input vector Xj and a parameter vector Wj of the additional neuron, the parameter vector Wj being defined by: Wj = Ij Mj Or: Mj is a unitary square matrix of dimension (Nj)*(Nj) defined as a product M=MJ, N j-1 ... Mj,i of Nj-1 matrices Mj,i where each matrix is ​​an elementary rotation operator applied to a pair of input components of the matrix Mj,i and parameterized by a respective rotation angle 0i bounded between - TT / 2 and TT / 2, 1j is a vector of dimension nj whose terms are all null except for one, equal to 1, positioned on a row corresponding to the second output component of the last matrix Mj, Nji of the matrix product, and where the input vector of the additional neuron is composed of the first output component of each elementary rotation operator of the parameter vector of the previous neuron Nj-i in the layer, the training method includes iterative training of the rotation angles of the elementary rotation operators of each neuron in the layer to maximize the variance of the neuron's output component. Thus, by construction, the neural network layer is trained to perform a Karhunen-Loeve transform of the input vector x onto the output vector y.

[0040] In some embodiments, the training method includes accessing a memory storing a plurality of lookup tables, where each lookup table stores a plurality of pre-calculated values ​​of trigonometric functions, to approximate the calculation of elementary rotations applied to the input vector of the neural network layer. In another training method, it is also possible to use lookup tables to simplify the calculation of elementary rotation operators (in particular, the sine, cosine, and tangent functions).

[0041] In some embodiments, the training method includes the implementation of a CORDIC-type algorithm for calculating the elementary rotations applied to the input vector of the neural network layer. In this training method as well, the use of a CORDIC algorithm enables fast calculations requiring few computational resources.

[0042] According to another object, a computer program product is described, comprising code instructions for implementing the training method according to the preceding description, when this program is executed by a processor.

[0043] According to another object, a device for training an artificial neural network is described, comprising at least one processor and one memory, configured to implement the training process according to the preceding description. Brief description of the drawings

[0044] Other features, details, and advantages will become apparent upon reading the detailed description below and analyzing the attached drawings, on which: Fig. 1

[0045] [Fig. 1] represents the equivalence between a neuron of a classical dense layer involving weighting coefficients and a neuron of a layer according to the present disclosure, implementing a multiplication from angular parameters, where each rectangle represents an angular rotation by a specific angle. Fig. 2

[0046] [Fig. 2] represents an alternative equivalent diagram to that in Figure 1 of a single-layer neuron according to the present disclosure. Fig. 3

[0047] [Fig. 3] represents an equivalent diagram of another neuron structure implementing multiplication from angular parameters. Fig. 4

[0048] [Fig. 4] schematically represents a so-called spiral layer, comprising a set of neurons implementing multiplications from angular parameters. Fig. 5

[0049] [Fig. 5] represents parameters of a neuron that can be stored during a forward pass in a neural network training process. Fig. 6

[0050] [Fig.6] represents the projection of an angle with value in a set of finite cardinality, between 0 and TT / 2, operated by the tangent function. Fig. 7

[0051] [Fig. 7] schematically represents a spiral layer structure configured to perform an orthonormalization of the input vector on the output vector of the layer. Fig. 8

[0052] [Fig. 8] schematically represents a training model of an orthonormalization layer allowing this layer to also perform a Karhunen-Loeve transform of the input vector on the output vector. Fig. 9

[0053] [Fig. 9] schematically represents a VQ-VAE type neural network integrating a layer configured to perform a Karhunen-Loeve transform of a signal encoded in a latent space. Fig. 10

[0054] [Fig. 10] represents different examples of serial or parallel calculation of rotation operators of a neuron in a so-called spiral layer. Fig. 11

[0055] [Fig. 11] schematically represents a computer system, including a computer and a memory, configured to implement or to train a neural network according to embodiments. Description of the implementation methods Spiral layer

[0056] Reference is now made to Figure 1, which, in its upper part, shows the structure of a neuron in a classical dense layer implementing the GEMM generalized matrix multiplication described in the introduction. Like any matrix multiplication, the product WX of the components of the input vector and the weighting coefficient matrix can be interpreted as a set of elementary cross products between each row vector w of W and each column vector x of X. The elementary operation implemented in this matrix product can therefore be written as the following scalar product: N y = (iv, x) = w l x = Wj Xj (1) 7 = 1 ( w \ ( X \ where the two vectors w = I ■ ■ ■ j and x = ■ ■ ■ are elements of B' v Assuming that the vector W NJ \ N J weighting coefficients wa a unitary norm (i.e., ||w||2= w l w = 1), then this vector can be parameterized as follows: = 1 t × M N-1 × ... × M1(2) where the vector 1 = (0, ..., 0, 1) t and the matrices Mi for i=1,, N-1 are elementary rotation operators applied to pairs of input components, and parameterized by a respective rotation angle 0i. For example, the matrices Mi can be square, antisymmetric rotation matrices of size NxN corresponding to two-dimensional rotation matrices known as Givens matrices: / I i— 1 0 0 0j_ l,n— i— 1\ = 0 - Sin0j cos 0j 0 \ 1 [I 0 cos 0, sinôj 0 ]

[0057] In this example, each matrix Mi implements a rotation of an angle between the i ème and (i+1) èmecomponents of the input vector to which the matrix applies. As MM' = I N and M = M w _| x... x M t The matrix M represents a linear transformation implementing a rotation in IR space W .

[0058] Each matrix Mi is orthogonal (i.e., MjMf = I N Therefore, the weight vector w is constructed so as to have a unit norm: w'w = l t x M x M t xl = l

[0059] Furthermore, it can be shown that by imposing the condition cos > 0, that is, 0i between -TT / 2 and TT / 2, there exists a unique correspondence between the angles 0 L and the weights w f : Vi e {1,.... N — 1}, Qi e [— TC / 2, +7T / 2] / w \ 0, = tan” 1 t+1

[0060] To establish a strict equivalence between such a neuron implementing elementary rotation operators and a classical neuron of a dense layer, the constraint of a unit norm on the weighting coefficients whv = 1 must be relaxed. To do this, an additional gain GeIR can be inserted at the output of the dot product calculation, leading to a strict equivalence between the classical structure based on a direct dot product calculation and the layer described herein based on rotation operators (equation (3)) and a gain.

[0061] These two structures have the same number (N+1) of parameters including the bias, as shown in Figure 1, the upper part of which represents a neuron of a dense layer and the / w i\ N+1 parameters formed by means of b and N weighting coefficients w = I ■" I, and whose part \w N / The lower layer represents a neuron in a layer hereafter referred to as the "spiral" (this term stems from the fact that the layer induces both a rotation by angles 0 L and a translation by the gain G), where the neuron implements a matrix formed by a product of elementary rotation operators Mi where each rotation operator is parameterized by N-1 angles 0 it optionally a gain G and a bias value b.

[0062] In the lower part of Figure 1, each rectangle represents an angular rotation of an angle 0 L implementation on two successive components of the input vector: Si = —ex sin 0; + e2X cos 0; s 2 = e i x cos 9; + e 2 x s i n

[0063] Advantageously, in some embodiments, only the second component S2, corresponding to the right-hand branch of a rectangle in Figure 1, is calculated, to be injected into the next rotation. This therefore corresponds to the implementation, for each neuron, of a parameter matrix defined as follows: Where G is an optional gain component, and vector 1 is an N-dimensional vector whose terms are all null except for the last one, which is equal to 1. Thus, this vector 1 allows us to consider only the second output component of the last elementary rotation operator.

[0064] Thus, in a neural network, a layer performing GEMM matrix multiplication can be replaced by a so-called spiral layer comprising at least one neuron, implementing a multiplication between an input vector X=(XI,... XN) and a parameter vector, to obtain an output component y, where the parameter vector includes a term defined as follows: W t = 1 t M Or: M is a unitary square matrix of dimension N*N defined as a product M=MN 1 ... M1 of N-1 matrices Mj where each matrix is ​​an elementary rotation operator applied to a pair of input components of the matrix Mj and parameterized by a respective rotation angle 0i bounded between -TT / 2 and TT / 2, and 1 is an N-dimensional vector whose terms are all nuisance except for one, equal to 1, applied to the second output component of the last MN-I matrix.

[0065] It is understood that in the case represented in Figure 1, the elementary rotation operators are applied to successive pairs of input components, and the vector 1 is zero everywhere except for the term corresponding to the last row, which is equal to 1. This corresponds to a sequential, or serial, implementation of the neuron. Referring to Figure 10, a neuron in a spiral layer can advantageously be implemented so that at least some matrices of the matrix product M are calculated in parallel, particularly when the neural network is executed by a multi-core processor (e.g., a GPU), in order to significantly reduce execution time.

[0066] The upper part of Figure 10 schematically represents the chronology of the application of elementary rotation operators to two successive components of the input vector, represented by the squares 0i, in a neuron of the so-called spiral layer. The components Xi of the input vector correspond to the lines on the ordinate, and the abscissas represent elementary execution times by the computer.

[0067] The upper part of Figure 10 therefore represents a case where a neuron in the so-called spiral layer is calculated serially. In this case, this neuron performs a multiplication between the input vector X=(XI,... XN) and the parameter vector = 1? M, where M is the product of the matrices MN 1 ... M1 applied one after the other, to successive pairs of components of the input vector, and the vector 1 is zero except in the last row, applied to the second component of the last matrix MN 1. We observe in this case that the calculation of a neuron for an input vector of dimension N requires N-1 successive calculations (for each rotation operator) by the processor.

[0068] The lower part of Figure 10 shows computational variants for a spiral-layer neuron, allowing parallel computation of several rotation operators, notably by a multi-core processor. In this case, the neuron always performs a multiplication between the input vector x=(xi,... XN) and a parameter vector W t= 1? M, where M is the product of matrices MN 1...M1 applied to pairs of components of the input vector, but the matrix product is configured such that a first subset of matrices applies to distinct pairs of components of the input vector (i.e., each matrix applies to a respective pair of components of the input vector), all these matrices being calculable in parallel. In the example in the figure, an elementary rotation of angle 01 is applied to components xi and X2, then a rotation of angle 03 is applied to components X3 and X4, and so on.

[0069] A second subset of matrices is then applied to pairs of output components from the first subset, and so on (where each matrix in the second subset is applied to a pair of output components from a respective pair of matrices in the preceding subset). The vector 1, meanwhile, is also zero everywhere except in one row where the component of the vector is equal to 1; this row corresponds to the second output component of the last matrix MN 1 in the matrix product.

[0070] We can therefore define the elementary rotation operators Mj of the matrix M as follows, this definition being equivalent to the neuron structure represented in figure 1 but also allowing the possibility of parallel calculation of subsets of rotation operators: pm k k = 1 if kp,q = [m k l ]k=i 2 ' 1=1,2, m p , q = cos 0, = m qv 0 otherwise

[0071] Alternatively, the elementary rotation operators can also be defined as follows: C m k,k = 1 if kp,q = [m kil ]k=i,2,..., 1=1,2,..., N ™p,q — — ™q,p 0 otherwise

[0072] This definition is equivalent to the neuron structure shown in figures 2 and 3 (applying the same definition of gain respectively) but allows parallel calculation of subsets of rotation operators.

[0073] The representation on the left shows an example in which a matrix from the second subset performs a rotation of angle 02 on pairs of output components of two consecutive matrices from the first subset, while the representation on the right provides another example where a matrix from the second subset is applied to pairs of output components of non-consecutive matrices from the first subset.

[0074] The set of matrices in the first subset can be calculated in parallel during a first elementary execution time of the calculator, then the set of matrices in the second subset can be calculated in parallel during a second elementary execution time, and so on. Thus, theoretically, if N is even (N=2M=2 kThen the first subset contains M 2D matrix rotations; the second set contains M / 2 2D matrix rotations; the third set contains M / 4 rotations, and so on... therefore, we will have k = log2(N) subsets in total. If N is odd (i.e., N = 2M + 1, a very improbable case in neural networks), then there will be N / 2 = M 2D matrix rotations in the first subset; M / 2 2D matrix rotations in the second subset, and so on... and we will have 1 + k = 1 + log2(N) subsets in total. In practice, the cardinality of the first subset can correspond to the maximum number of calculations that can be executed in parallel by the given computer.

[0075] This method of parallel execution of certain spiral layer rotation operators is applicable to all the applications or implementations described below (in particular implementation methods, orthonormalization, KLT transform, applications to audio encoding, video, etc.)

[0076] Each elementary rotation operator Mj can be defined by: I i— i 0 0 0 — sin0j cos0j 0 0 cos 6i sin 0 As in the case of the neuron shown in Figure 1, or, in the case of the neurons shown in Figures 2 and 3, by: fi-i 0 0 0 —tan 6i 1 0 0 1 tan 0 (4) On-il,nl 0 0 I -n — î —

[0077] In the case where the layer includes a gain G, the parameter vector is defined as follows: = Gl t M

[0078] Referring to Figure 4, the layer can of course include several neurons calculated in parallel. Each neuron Nj, j=O..M performs a multiplication between the input vector X and a parameter vector Wj to obtain a component yj of an output vector y=(yi,...yM). The gain then becomes a vector G=(Gi, GM).

[0079] When the elementary rotation operators Mi have the structure described in equation (3), the implementation of a neuron represents a computational cost of (2N-1) MULT + N ADD, where "MULT" and "ADD" are respectively multiplication and addition operations between scalars.

[0080] However, considering an elementary rotation operator: ) = M f s2 ei l = f-si n 3i cos s 3i} ( J J l l e2J co sdi sind e e i ' The input / output relationship can be reformulated as follows: VS2' 1 \ e t + e2tan 0, )

[0081] In each elementary rotation operator, the multiplicative term cos s &i can therefore be factored, leading to the equivalent neuron structure shown in Figure 2. From this structure, all multiplicative terms cos s; for i=1,..., N-1, in the gain to obtain an equivalent structure represented in figure 3 with a gain: G' = GJ” J cos 0, i where G is the neuron gain in the equivalent structure of Figures 1 and 2, ta n 0 j = tan01, tan0 m ta n 0' = -, Vm = 2,..., N — 1 nr=i c ° s 0 k

[0082] Each elementary operator (each rectangle in the figure) implements an operation resulting from the combination of an elementary rotation and a homothety: G ) = ^ ) < T'-

[0083] As before, only the second output component of each elementary operator is used as the input of the next elementary operator. By adding a bias value to each neuron, the total number of parameters in a layer comprising N input components and M output components, each of which includes: a bias parameter, a gain parameter, N-1 angular rotation parameters, This results in a total of Mx(N+1) parameters, which is exactly the same number of parameters as a dense layer with the same number of input and output components. Regarding the computational complexity of such a layer in terms of the number of additions (ADD) and multiplications (MULT) required, the structural variant shown in Figure 3 and corresponding to equation (4) has the following complexity: M x ( 1 ADD + (Af - 1) x (1 ADD + 1 MULT) + 1 MULT) = M x ( / VADD + N MULT), that is to say the same arithmetic complexity as a dense layer with the same number of input and output components (including a bias parameter).

[0084] Referring to Figure 6, we have shown the projection of an angle θ between 0 and TT / 2 onto a straight line using the tangent function. This figure reveals that for angles θ equally distributed between 0 and TT / 2, the tan θ values ​​are not equally distributed between 0 and infinity. On the contrary, the closer the θ values ​​are to 0, the closer the tan θ values ​​are for angles separated by a constant distance. Thus, when a neural network includes a so-called spiral layer, whose neurons have the structure described with reference to Figures 2 and 3—that is, whose elementary rotation operators include the application of the tangent function—and this neural network is applied to parameters θ in binary format, i.e.coded on a predetermined number of bits and only able to take a finite number of values ​​between a minimum value and a maximum value, the application of the tangent function to each angle of rotation 0 achieves a non-linear quantization of the angles of rotation.

[0085] When the neural network is trained, since the trained parameters (in this case the 0 angles) are generally close to zero, this non-linear quantization by construction of the layer structure allows to give good accuracy to the network. Implementation methods

[0086] With reference to Figure 11, a computer system 1 is schematically represented, configured to implement an artificial neural network comprising a so-called spiral layer as described above, i.e., comprising at least one neuron that performs a multiplication between an input vector X=(XI,... XN) and a parameter vector, to obtain an output component y, where the parameter vector includes a term = 1? M, where M is a product of elementary rotation operators each applied to a pair of input components and each parameterized by a respective rotation angle 0i bounded between - TT / 2 and TT / 2, and 1 is a vector of dimension N whose components are all zero except one, equal to 1, positioned so as to apply to the second output component of the last elementary rotation operator of the product.

[0087] The computing device 1 comprises at least one computer 10 and a memory 11. The same computing device, or a separate device 1' also comprising at least one computer 10' and a memory 11', is configured to train such a neural network. The computer 10, 10' may comprise one or more of the following types of computers: CPU, GPU, ASIC, FPGA.

[0088] In some embodiments, the layer described above is implemented using a CORDIC (Coordinate Rotation Digital Computer) type algorithm. The CORDIC algorithm is particularly well-suited to trigonometric operations because it uses iterative shifting and addition techniques to efficiently compute rotations, sines, cosines, and tangents without multipliers. This results in significantly reduced hardware complexity compared to the matrix multiplications required in dense layers, which are computationally intensive and require numerous multipliers and adders. Particularly in the case of hardware implementations on FPGA or ASIC computers, the use of CORDIC operators can reduce the need for expensive multiplication units, thereby minimizing resource usage, power consumption, and overall circuit size.

[0089] Alternatively or in addition, the computer's memory 11, 11' can also store a plurality of look-up tables, where each look-up table stores a plurality of pre-calculated values ​​of trigonometric functions (e.g., tangents in the case of a layer containing neurons from Figures 2 or 3), to approximate the calculation of elementary rotations applied to the input vector of the neural network layer. If necessary, the computer 10, 10' can access the look-up tables during training or after training the neural network to easily and quickly calculate the trigonometric functions of the rotation operators. In the case of a combination of implementing a CORDIC algorithm with look-up tables, a pre-recorded look-up table can also store the incremental angle shifts used in a CORDIC algorithm.This allows for the rapid retrieval of necessary trigonometric values ​​without requiring real-time computation, a significant advantage over traditional dense layers where nonlinear activations and weight calculations cannot be pre-computed as efficiently due to their variability. In dense layers, LUTs would become impractically large or inaccurate due to the wide variability of magnitudes and weight distributions. Thus, CORDIC-based implementations not only achieve superior hardware efficiency but also facilitate more predictable and faster operations, making them particularly advantageous for low-power or real-time applications in edge computing.

[0090] In some embodiments, the computing device T is configured to train a neural network comprising a spiral layer as described above on a training database. This training relies on an iterative process aimed at adjusting the trainable parameters of the network, in this case the rotation angles of the parameter vector W and, where applicable, the gain components, to minimize an error function, also called a cost function or loss function.

[0091] This iterative process includes: At least one forward pass, where a set of training data is propagated through the network layer by layer, and the network is evaluated on this training data. The result of this propagation is a prediction of the network. The network's loss function is calculated at the end of the forward pass. The network loss function can be defined in various ways, but it quantifies the difference between the network's prediction and the expected output (ground truth). The goal of training the neural network is to minimize this loss function by adjusting the trainable parameters. At least one backpass, which uses gradient backpropagation to calculate the gradients of the loss function with respect to the trainable parameters of the network. Backpropagation begins at the output layer and propagates in the opposite direction to the input layer. Once the loss function gradients are determined, the trainable parameters are updated. Parameter adjustment is performed in the opposite direction to the gradient, in order to reduce the error function.

[0092] Referring to Figure 5, in some embodiments, the spiral layer structure can be used to reduce the computations performed during training. Denoting £ the error function of the network and denoting z a target value that the network must predict for a given input during training, and z the value predicted by the network for the same input, the gradient of the loss function with respect to the rotation angles is given as follows: d£(z, z) d£(z,z) dz p dy d0 k dz p X dyj X d0 k dy j Where z p is the output of the network layers located after the so-called spiral layer. The magnitude-^ 1is the partial derivative of an output component of a neuron in the spiral layer with respect to one of the rotation angles involved in that neuron. Now, as can be seen in the figure, the first output components, denoted a t of each rotation operator applied to two successive components of the input vector, represented in the figure by each block "0 t " correspond to the local gradients - dv duî■ of the neuron's output with respect to each rotation angle 0 L These initial output components are calculated during the Mx product calculation during a forward pass: / «i \ = M x t ®nl / \ has n /

[0093] Therefore, in advantageous embodiments of the training process, the components a, of the product Mx which correspond to the local derivatives of each output component with respect to each angle of rotation are stored in memory 11', and these components are retrieved during the next backpass to allow the determination of the gradients of the loss function of the network with respect to these angles. Orthonormalization layer

[0094] With reference to Figure 7, we have shown an example of a spiral layer with a particular structure, configured to project the input vector x onto an orthonormal basis y, also called an orthonormal projection. This layer is referred to herein as the orthonormalization layer.

[0095] The first layer of this network, which allows the calculation of the first component yi from the components of the input vector x, is as described previously, that is: y1 = l1 t Max Where Mi is a product of elementary rotation operators, each applied to a pair of components of the input vector x and parameterized by a respective rotation angle 0i bounded between -TT / 2 and TT / 2, and 11 is a vector of dimension N whose components are all zero except one, equal to 1, applied to the second output component of the last elementary rotation operator of the product Mi.

[0096] In addition, the layer includes at least one additional neuron Nj, 1 < j < K where K < N - 1, configured to implement a multiplication between a layer-specific input vector Xj and a parameter vector Wj of this additional neuron, the parameter vector Wj being defined by: Wj = Ij Mj Or: Mj is a unitary square matrix of dimension (Nj)*(Nj) defined as a product Mj = Mj n _j_.... Mj of Nj-1 matrices Mj,i where each matrix is ​​an elementary rotation operator applied to a pair of input components and parameterized by a respective rotation angle 0i bounded between - TT / 2 and TT / 2, 1j is a vector of dimension Nj whose terms are all null except for one, equal to 1, positioned on a row corresponding to the second output component of the last matrix Mj, Nji of the matrix product, and where the input vector Xj of the additional neuron Nj is composed of the first output component of each elementary rotation operator of the parameter vector Xj-i of the previous neuron Nj-i of the layer.

[0097] In other words, and as shown in Figure 7, the orthonormalization layer is built recursively from neuron to neuron, with the number of parameters decreasing from 1 at each neuron. By construction, the number of neurons in this layer, corresponding to the number of output components, is at most equal to the number of components in the input vector.

[0098] By decomposing x = x s + x s± where S is the vector line S = vect(Wj)' such that x s = y * Wj with y = {x, Wj), the rotation matrix Mj defines a vector isometry and therefore preserves angles, thus the vector x = Mj x = Mj x s + Mj x s± with orthogonality between the vectors MjX s and M y x s± .

[0099] Or, Mj x s = Mj (y * W j ) = y M j M j t 1 j ( —x1sin 01+ x2cos 01\ -( -xi cos 0, + x2sin 0J sin 02+ x3cos 02j car The global vector after rotation is: / —x1sin 01+ x2cos 01 % > M X > I —(—Xi cos 01+ x2sin 0,)' sin 02+ x3cos 02 y

[0101] Thus, by construction, each output component yj of a neuron Nj is orthogonal with the input vector of the next neuron Nj+i. Consequently, all the components of the vector y are orthogonal to each other (in other words, a Gram-Schmidt decomposition has been performed on an orthonormal basis).

[0102] With reference to Figure 8, we have represented by the JvC block gthe orthonormalization layer described above. In some embodiments, the layer can be configured, by a particular training of the rotation angles, to perform a Karhunen-Loeve transform (KLT) of the input vector x on the output vector y; that is to say, the components of the output vector are uncorrelated and ordered according to their variance.

[0103] To achieve this, the layer training involves iteratively training the rotation angles of the elementary rotation operators of each neuron in the layer to maximize the variance of each neuron's output component. This training, which can be described as local since the rotation angles are not trained to minimize the overall loss function of the network, but to maximize the variance of each output component, is represented by the "local angle optimization" block in Figure 8.

[0104] This results in a layer that structurally enables the implementation of a KLT transform in the network. This decomposition offers numerous advantages, as it allows data to be represented in the network in a way where its components are uncorrelated. This has advantageous applications, for example, in principal component analysis, signal compression, or encoding.

[0105] In particular, with reference to Figure 9, we have shown an example of the integration of a layer performing a KLT type transform in a neural network of the variational autoencoder type, and in particular of the VQ-VAE type (for the English Vector Quantized Variational Autoencoder).

[0106] In embodiments of this disclosure, a VQ-VAE type neural network includes: an encoding module denoted ε configured to encode an input data x of the network into a latent space, i.e. a representation space of the encoded data, which is a space of reduced dimension compared to the input data space, the encoded data being denoted z. a module labeled U T performing a Karhunen-Loeve transform on the encoded data, from which an average vector (p) of the encoded training data in the latent space was deduced, denoted z', where the modulus corresponds to the spiral layer configured to perform a KLT-type transform as described above, the result of the KLT transform being denoted y A truncation module, denoted Trunc, configured to remove a set of components, starting from a given component, from the output signal y of the KLT transform, i.e., to retain only the first components of the signal y, An RVQ vector quantization module configured to implement quantization on the truncated signal, and A so-called padding module at the output of the vector quantization module that adds zeros to the output vector of the vector quantization module to restore it to the initial dimension of y before truncation, the output of the padding module being denoted y. A module performing an inverse transform U of the KLT transform to obtain a data z', to which is added the average vector (p) of the training data encoded in the latent space, and A decoding module D, configured to reconstruct the input data x from z.

[0107] This type of neural network is advantageously used for encoding audio or video signals, which allows signal compression by removing part of the information contained in the signals (by truncation of the output data of the KLT transform), while maintaining good fidelity to the original signal since the first output components of the KLT transform are preserved. Experimental results

[0108] To evaluate the effectiveness of the spiral layer described above, experiments were carried out on two reference datasets: MNIST and CIFAR-10. These datasets were chosen because of their widespread use in the literature, allowing a direct comparison with existing methods.

[0109] The MNIST dataset consists of 70,000 images of handwritten digits, divided into 60,000 training images and 10,000 test images. Each image is grayscale and has a resolution of 28x28 pixels. The task is to classify each image into one of 10 digit classes (0-9). This dataset serves as a benchmark for evaluating the performance of different neural network architectures on simple image classification tasks.

[0110] The CIFAR-10 dataset is a more complex testbed containing 60,000 32x32 color images divided into 10 classes, with 50,000 training images and 10,000 test images. The dataset covers various object classes, such as airplanes, cars, and animals, making it suitable for evaluating the model's generalizability to more complex visual data.

[0111] For both datasets, a multi-layered neural network architecture was implemented. In one variant, the network consisted of traditional dense (fully connected) layers. In the second variant, the dense layers were replaced by the proposed layer implementing elementary rotation operators. The evaluation aimed to compare the performance of these two architectures in terms of accuracy and computational cost.

[0112] The experiments were conducted using the following setup: Architecture: The two network architectures (one with dense layers and one with the new layer) were kept similar in terms of the number of layers and overall model complexity. The models were trained with a softmax output layer for classification. Optimizer: Using an Adam optimizer with a learning rate of 0.001 to train both models. Loss function: Cross-entropy loss was used for the MNIST and CIFAR-10 experiments, as it is the standard loss function for classification tasks. Training epochs: The models were trained for 100 epochs for MNIST and CIFAR-10. Batch size: A batch size of 128 was used for both datasets. Metrics: The models were evaluated using two key metrics: o Accuracy: The overall classification accuracy across the test set. o Computational efficiency: The number of parameters and the training time required for both models. Table 1 presents the results of the experiments on the MNIST dataset. As indicated, the dense layer and the layer described above exhibit virtually identical performance for the same number of parameters. Table 1 - Results on MNIST Model Accuracy (%) Parameters Time Dense layer 97.30% 50.890 79.49s "Spiral" layer 96.97% 50.890 102.65s The results on CIFAR-10 are presented in Table 2. Since CIFAR-10 is a more complex dataset than MNIST, these results allow us to observe more clearly the generalization capacity of the proposed layer compared to the dense layer. Table 2 presents a performance comparison between a dense layer (reference) and the proposed layer on the CIFAR-10 dataset, focusing on accuracy and the number of parameters. The dense layer achieves an accuracy of 45.03%, while the proposed layer ("spiral") surpasses it with an accuracy of 47.85%, showing an improvement of 2.82% for the same number of parameters (197,322), indicating that the proposed layer offers performance equivalent to, or even better than, a traditional dense layer with the same number of parameters. Table 2 - Results on Cl FAR - 10 Model Accuracy (%) Parameters Time Dense layer 45.03 197 322 73.64s "Spiral" layer 47.85 197 322 97.45s

Claims

Demands

1. A computer device (1), comprising at least one computer (10) configured to implement an artificial neural network, the artificial neural network comprising at least one layer comprising at least one neuron implementing a multiplication between an input vector X=(XI,... XN) and a parameter vector, to obtain an output component y, characterized in that the parameter vector comprises a term defined as follows: W t = 1 t M Or: M is a unitary square matrix of dimension N*N defined as a product M=MN 1 ... M1 of N-1 matrices Mj where each matrix is ​​an elementary rotation operator applied to a pair of input components of the matrix Mi and parameterized by a respective rotation angle 0i bounded between -TT / 2 and TT / 2, and 1 is an N-dimensional vector whose terms are all nuisance except for one, equal to 1, positioned on a line corresponding to the second output component of the last MN-I rotation.

2. A computer device according to claim 1, wherein the parameter vector is the product of a gain component G and the term P Ù .

3. A computer device according to claim 1 or 2, wherein each elementary rotation operator is an elementary Givens rotation matrix defined as follows: / I i— 1 ù ù l,n— i— 1\ _ / 0 — sin0j cos 0 ] 1 — I 0 cos 0 t sin0j 0 I

4. A computer device according to claim 1 or 2, wherein each elementary rotation operator is defined as follows: / ft—i 0 0 i,n— i— 1\ ,. I 0 —tan 0, 1 0 | 1 I 0 1 tan 0i ° and the gain is defined by G = G o HE cos G; where Go is a configurable gain value.

5. A computer device according to the preceding claim, wherein the rotation angles are represented in binary format, and the neural network layer performs a non-linear quantification of the rotation angles.

6. A computer device according to any one of the preceding claims, wherein the artificial neural network has been trained on a training database, the training of the artificial neural network comprising an iterative modification of the rotation angles of the elementary rotation operators.

7. A computer device according to any one of the preceding claims, further comprising a memory (11) storing a plurality of lookup tables where each lookup table stores a plurality of pre-calculated values ​​of trigonometric functions, and the processor is configured to approximate the calculation of elementary rotations applied to the input vector by calling the lookup tables.

8. A computer device according to any one of the preceding claims, wherein the computer is configured to implement a CORDIC-type algorithm for calculating elementary rotations applied to the input vector.

9. A computer device according to any one of the preceding claims, wherein the computer comprises one or more microcontrollers or FPGAs.

10. A computer device according to any one of the preceding claims, wherein the computer comprises several computing cores capable of performing operations in parallel, and the computer is configured to implement at least part of the calculation of the product Mx in parallel.

11. A computer device according to the preceding claim, wherein the computer is configured to, when calculating the product Mx, calculate in parallel the product of a first subset of matrices with the vector x, where each matrix of the subset is applied to a respective pair of components of the input vector x.

12. A computer device according to the preceding claim, wherein the computer is configured to, when calculating the product Mx, calculate in parallel the product of at least a second subset of matrices with a pair of respective output components of two matrices from the preceding subset of matrices.

13. A computer device according to any one of claims 10 to 12, wherein each rotation operator is defined as follows: / m k i = 1 if kp,q = [m kl ]k=i,2. N = K = = ' 1=1.

2. N m p q = cos = m q p 0 otherwise Or C m k,i = 1 if kp,q = [m k Ak=i,2. N = ■ MP ' P = ~ TAN 0I = ~ MQ ' Q 1=1, 2,..., N ™p,q — — ^q,p 0 otherwise

14. A computer device according to any one of the preceding claims, wherein the layer comprises at least one additional neuron Nj, 1 < j < K or K < N — 1, configured to implement a multiplication between an input vector Xj and a parameter vector Wj of the additional neuron, the parameter vector Wj being defined by: W = Ij Mj Or: Mj is a unitary square matrix of dimension (nj)*(nj) defined as a product M y =Mj, n - j-1 of nj-1 matrices Mj,i where each matrix is ​​an elementary rotation operator applied to a pair of input components of the matrix Mj,i and parameterized by a respective rotation angle 0i bounded between - TT / 2 and TT / 2, 1j is a vector of dimension nj whose terms are all null except for one, equal to 1, positioned on a row corresponding to the second output component of the last matrix Mj, n-ji of the matrix product, and where the input vector of the additional neuron is composed of the first output component of each elementary rotation operator of the parameter vector of the previous neuron Nj-i of the layer.

15. A computer device according to the preceding claim, wherein the rotation angle values ​​of the elementary rotation operators of the layer are adapted to perform a Karhunen-Loeve transform of the input vector x on the output vector y.

16. A computer device according to the preceding claim, wherein the layer training includes an optimization of the rotation angles of the rotation operators by maximizing the variance of the output vector components.

17. A computer device according to the preceding claim, wherein the layer training comprises an iterative training of the rotation angles of the elementary rotation operators of each neuron in the layer to maximize the variance of the neuron's output component.

18. Computer device according to any one of claims 15 or 16, wherein the computer is configured to implement the artificial neural network in a building energy disaggregation algorithm (NILM).

19. A computer device according to claim 15 or 16, wherein the neural network is of the VQ-VAE (Vector Quantization - Variational Auto Encoder) type, comprising: An encoding module configured to encode input data into a latent space; a layer performing a Karhunen-Loeve transform of the encoded data z' derived from the mean vector (p) of the encoded data in the latent space after network training; a truncation module configured to remove a set of components, starting from a determined component, from the data obtained as output from the Karhunen-Loeve transform. A vector quantization module configured to implement quantization of truncated data, and A decoding module configured to reconstruct the input data from the quantized data, plus the average vector.

20. Computer device according to the preceding claim, configured to implement an encoding of an audio or video signal by applying the artificial neural network to said signal.

21. A computer device according to any one of the preceding claims, further comprising an interface for receiving an image to be processed, and the processor is configured to implement the artificial neural network for image processing.

22. A computer device according to the preceding claim, wherein image processing is one of: Classification, Segmentation, Filtering, Edge detection, Pixel interpolation, Mathematical morphology.

23. A method for encoding an audio or video signal, comprising: Receiving an audio or video signal to be encoded, The implementation, on the received signal, of an artificial neural network previously trained on a training database, comprising: o an encoding module configured to encode the signal in a latent space, o a module performing a Karhunen-Loeve transform of the encoded signal z', from which an average vector (p) of the training data encoded in the latent space has been deduced, o A truncation module configured to remove a set of components, starting from a given component, from the output signal of the Karhunen-Loeve transform, o A vector quantization module configured to implement quantization on the truncated signal, and o A decoding module configured to reconstruct the input audio signal from the quantized signal plus the average vector, in which the module performing a Karhunen-Loeve transform comprises a layer including a set of neurons Nj where j=0.. K-1, K < N - l where each neuron implements a multiplication between an input vector Xj and a parameter vector W }, to obtain a component yj of an output vector y=(yi,...yK), in which the input vector Xj is formed: for the first neuron No, by the encoded signal z', for the following neurons Nj, j=1.. K-1, by the first output component of each elementary rotation operator of the parameter vector of the preceding neuron of the layer, in which each parameter matrix Wj includes a term defined as follows: Wj* = 1J Mj Or: Mj is a unitary square matrix of dimension (Nj)*(Nj) defined as a product Mj=Mj, Nji... Mj,i of nj-1 matrices Mj,i where each matrix is ​​an elementary rotation operator applied to a pair of input components of the matrix Mj,i and parameterized by a respective rotation angle 0i bounded between - TT / 2 and TT / 2, 1j is a vector of dimension nj whose terms are all null except for one, equal to 1, positioned on a row corresponding to the second output component of the last matrix Mj, Nji of the matrix product, and where the rotation angles of the layer have been trained to maximize the variance of the output vector components.

24. A method for training an artificial neural network, implemented by at least one processor, in which the artificial neural network comprises a layer including at least one neuron implementing a multiplication between an input vector X=(XI,... XN) and a parameter matrix, to obtain an output component y, characterized in that the parameter matrix includes a term defined as follows: W* = 1'M Or: M is a unitary square matrix of dimension N*N defined as a product M=MN 1 ... M1 of N-1 matrices Mi where each matrix is ​​an elementary rotation operator applied to a pair of input components of the matrix Mi and parameterized by a respective rotation angle 0i bounded between -TT / 2 and TT / 2,1 is a vector of dimension N whose terms are all null except for one, equal to 1, positioned on a row corresponding to the second output component of the last matrix MN 1 of the matrix product, the training includes adjusting the rotation angles of elementary rotation operators to minimize a network loss function on a training database.

25. A training method according to the preceding claim, wherein the parameter vector is the product of a gain component G and the term and the training also includes an iterative adjustment of the gain component.

26. A training method according to any one of claims 24 or 25, comprising: at least one forward pass, including the evaluation of the artificial neural network for a training dataset, the recording in memory, during each forward pass, of all the calculated parameters of the product Mx, the calculation of the network loss function at the end of the forward pass, and at least one backward pass comprising an update of the gain components and rotation angles from the gradients of the loss function with respect to the gain component and rotation angles of the elementary rotation operators, wherein the values ​​of the gradients of the loss function with respect to the rotation angles of the elementary rotation operators are determined from the parameters of the product Mx stored in memory.

27. ​​A training method according to any one of claims 24 to 26, wherein the layer comprises at least one additional neuron Nj, 1 < j < K where K < N — 1, configured to implement a multiplication between an input vector Xj and a parameter vector Wj of the additional neuron, the parameter vector Wj being defined by: Wj = 1J Mj Or: Mj is a unitary square matrix of dimension (Nj)*(Nj) defined as a product Mj =Mj, Nji... Mj,i of Nj-1 matrices Mj,i where each matrix is ​​an elementary rotation operator applied to a pair of input components of the matrix Mj,i and parameterized by a respective rotation angle 0i bounded between -TT / 2 and TT / 2, 1j is a vector of dimension Nj whose terms are all null except for one, equal to 1, positioned on a row corresponding to the second output component of the last matrix of the matrix product, and where the input vector of the additional neuron is composed of the first output component of each elementary rotation operator of the parameter vector of the previous neuron Nj-i of the layer, the training process includes iterative training of the rotation angles of the elementary rotation operators of each neuron in the layer to maximize the variance of the neuron's output component.

28. A training method according to any one of claims 24 to 27 comprising accessing a memory storing a plurality of lookup tables, where each lookup table stores a plurality of pre-calculated values ​​of trigonometric functions, to approximate the calculation of elementary rotations applied to the input vector of the neural network layer.

29. A training method according to any one of claims 24 to 28, comprising the implementation of a CORDIC-type algorithm for calculating the elementary rotations applied to the input vector of the neural network layer.

30. Product computer program, comprising code instructions for implementing the method according to any one of claims 23 to 29 when this program is executed by a processor.

31. Training device (1') for an artificial neural network, comprising at least one processor (10') and a memory (11'), configured to implement the training method according to any one of claims 24 to 29.

Images