Concept for coding parameter of neural network

Abstract

To provide a coding and decoding device, capable of effectively compressing a neural network parameter.SOLUTION: A decoding device contains steps of: decoding a neural network parameter by selecting a reconstruction level set in accordance with a quantization index that is decoded from a data stream of a past neural network parameter or a past neural network parameter that is before the past neural network parameter, or is coded by each data stream; and combining the neural network parameter in a reconstruction layer.SELECTED DRAWING: Figure 20

Description

[Technical field]

[0001] The present invention relates to an encoding concept for neural network parameters. [Background technology]

[0002] In its most basic form, the neural network is a chain of affine transformations followed by element-wise nonlinear functions. It can be represented as a directed acyclic graph, as shown in FIG. 1. FIG. 1 is a schematic diagram showing an example of a neural network, in which a two-layer feedforward neural network is shown as an example. In other words, FIG. 1 is a graph representation of a feedforward neural network. Specifically, this two-layer neural network is a nonlinear function that maps a four-dimensional input vector to a solid line. This neural network includes four neurons 10c corresponding to the four-dimensional input vector in the input layer, which is the input of the neural network, five neurons 10c in the hidden layer, and one neuron 10c in the output layer, which forms the output of the neural network. The neural network further includes neuronal interconnects 11 that connect neurons of different or subsequent layers. The neuronal interconnects 11 may have associated weights, which are associated with the relationship between the mutually connected neurons 10c. In particular, the weights weight the activation of the neurons of a layer when they are transferred to the subsequent layer, and in turn, a sum of the inbound weighted activations is formed at each neuron of the subsequent layer - corresponding to a linear function - followed by a non-linear scalar function being applied to the weighted sum formed at each neuron / node of the subsequent layer - corresponding to a non-linear function. Thus, each node, e.g. neuron 10c, is accompanied by a certain value, which is propagated forward to the next node by multiplication with the respective weight value of the edge, e.g. neuronal interconnect 11. Afterwards, all received values ​​are simply summed up.

[0003] TIFF2025016517000002.tif66151 Summary of the Invention [Problem to be solved by the invention]

[0004] Typically, neural networks have millions of parameters and their representation can require hundreds of MB (e.g., megabytes). As a result, their inference procedures involve the computation of many dot-product operations between large matrices, making them computationally expensive to execute. Therefore, reducing the complexity of dot-product operations is of great importance.

[0005] In addition to the above problems, the parameters of the neural network may need to be stored and may need to be transmitted, for example, from the server to the client. Furthermore, it may be advantageous to be able to provide information about the parameterization of the neural network to an entity gradually, for example in a federated learning environment, or when providing the parameterization of the neural network at different stages of quality that can be addressed when a particular recipient pays for or uses the neural network for inference.

[0006] It is therefore desirable to provide a concept for efficient coding of neural network parameters, which is more efficient, for example from a compression point of view. Additionally or alternatively, it is desirable to reduce the bitstream for neural network parameters and thus the signaling cost.

[0007] This object is achieved by the subject matter of the independent claims of the present application.

[0008] Further embodiments according to the invention are defined by the subject matter of the dependent claims of the present application. [Means for solving the problem]

[0009] An embodiment according to a first aspect of the present invention is an apparatus for decoding neural network parameters defining a neural network from a data stream, the apparatus being configured to sequentially decode the neural network parameters by selecting, for a current neural network parameter, a reconstruction level set from a plurality of reconstruction level sets in response to a quantization index decoded from the data stream for a previous neural network parameter, the apparatus further being configured to sequentially decode the neural network parameters by decoding, from the data stream, a quantization index for the current neural network parameter, where the quantization index indicates one reconstruction level of the selected reconstruction level set for the current neural network parameter, and by dequantizing the current neural network parameter onto one reconstruction level of the selected reconstruction level set indicated by the quantization index for the current neural network parameter.

[0010] A further embodiment according to the first aspect of the present invention is an apparatus for encoding neural network parameters defining a neural network into a data stream, the apparatus being configured to sequentially encode the neural network parameters by selecting, for a current neural network parameter, a reconstruction level set from among a plurality of reconstruction level sets in response to a quantization index encoded into the data stream for a previously encoded neural network parameter, the apparatus being further configured to sequentially encode the neural network parameters by quantizing the current neural network parameter into one reconstruction level of the selected reconstruction level set, and by encoding into the data stream a quantization index for the current neural network parameter indicating one reconstruction level to which the quantization index for the current neural network parameter is quantized.

[0011] A further embodiment according to the first aspect of the present invention comprises a method for decoding neural network parameters defining a neural network from a data stream, the method comprising the step of sequentially decoding the neural network parameters by selecting, for a current neural network parameter, a reconstruction level set from among a plurality of reconstruction level sets in response to a quantization index decoded from the data stream for a previous neural network parameter, the method further comprising the step of sequentially encoding the neural network parameters by decoding a quantization index for the current neural network parameter from the data stream, where the quantization index indicates one reconstruction level of the selected reconstruction level set for the current neural network parameter, and by dequantizing the current neural network parameter onto the one reconstruction level of the selected reconstruction level set indicated by the quantization index for the current neural network parameter.

[0012] A further embodiment according to the first aspect of the present invention comprises a method for encoding neural network parameters defining a neural network into a data stream, the method comprising sequentially encoding the neural network parameters by selecting, for a current neural network parameter, a reconstruction level set from among a plurality of reconstruction level sets in response to a quantization index encoded into the data stream for a previously encoded neural network parameter, the method further comprising the step of sequentially encoding the neural network parameters by quantizing the current neural network parameter into one reconstruction level of the selected reconstruction levels and by encoding, into the data stream, a quantization index for the current neural network parameter that indicates one reconstruction level to which the quantization index for the current neural network parameter is quantized.

[0013] The embodiment according to the first aspect of the present invention is based on the idea that the neural network parameters can be compressed more efficiently by using a non-constant quantizer but varying the same in the coding of the neural network parameters, i.e. by selecting a set of reconstruction levels depending on a quantization index decoded or coded from a data stream of the previous or, respectively, previously coded neural network parameters. Thus, the reconstruction vectors that can refer to an ordered set of neural network parameters can be packed more tightly in an N-dimensional signal space, where N denotes the number of neural network parameters in the set of samples to be processed. Such dependent quantization can be used for decoding and inverse quantization by a device for decoding, or for quantization and coding by a device for coding, respectively.

[0014] The embodiment according to the second aspect of the present invention is based on the idea that a more efficient neural network coding may be achieved if the parameterizations provided in the stages - called reconstruction layers to distinguish them from the layer configuration of the neural network in the neural layers - are then combined on a neural network parameter basis to result in an improved neural network parameterization compared to any of the stages. Thus, an apparatus for reconstructing neural network parameters defining a neural network may derive a first neural network parameter, for example a first reconstruction layer neural network parameter, for the first reconstruction layer to obtain, for each neural network parameter, a neural network parameter value of the first reconstruction layer. The first neural network parameter may have been previously transmitted, for example during an associative learning process. Furthermore, the first neural network parameter may be a neural network parameter value of the first reconstruction layer. Further, the device is configured to decode from the data stream for the second reconstruction layer the second neural network parameters, e.g., neural network parameters of the second reconstruction layer, to distinguish them from, e.g., final neural network parameters, and to result, for each neural network parameter, in a second reconstruction layer neural network parameter value. The second neural network parameters may not have a self-contained meaning in terms of the neural network representation, but may simply result in the neural network representation, i.e., e.g., final neural network parameters, when combined with the parameters of the first representation layer. Further, the device is configured to reconstruct the neural network parameters by combining, for each neural network parameter, the neural network parameter value of the first reconstruction layer and the neural network parameter value of the second reconstruction layer.

[0015] A further embodiment according to the second aspect of the present invention comprises an apparatus for encoding neural network parameters defining a neural network with first neural network parameters for a first reconstruction layer including, for each neural network parameter, a neural network parameter value of the first reconstruction layer, and further configured to encode second neural network parameters for a second reconstruction layer into the data stream, the second reconstruction layer including, for each neural network parameter, a neural network parameter value of the second reconstruction layer, the neural network parameters being reconfigurable by combining, for each neural network parameter, the neural network parameter value of the first reconstruction layer and the neural network parameter value of the second reconstruction layer.

[0016] A further embodiment according to the second aspect of the present invention comprises a method for reconstructing neural network parameters defining a neural network, comprising deriving, for a first reconstruction layer, first neural network parameters, which may have been previously transmitted, for example during an associative learning process, and which may for example be referred to as neural network parameters of the first reconstruction layer, to yield, for each neural network parameter, neural network parameter values ​​of the first reconstruction layer.

[0017] Further, the method includes a step of decoding second neural network parameters, which may be referred to as neural network parameters of a second reconstruction layer to distinguish them from final, e.g., reconstructed, neural network parameters, from the data stream for the second reconstruction layer to generate, for each neural network parameter, a neural network parameter value of the second reconstruction layer. Then, the method includes a step of reconstructing the neural network parameter by combining, for each neural network parameter, the neural network parameter value of the first reconstruction layer and the neural network parameter value of the second reconstruction layer. The second neural network parameters may not have a self-contained meaning with respect to the neural representation, but may simply lead to the neural representation, i.e., for example, the final neural network parameters, when combined with the parameters of the first representation layer.

[0018] A further embodiment according to the second aspect of the present invention includes a method for encoding neural network parameters defining a neural network using first neural network parameters for a first reconstruction layer including, for each neural network parameter, a neural network parameter value of the first reconstruction layer, the method including a step of encoding second neural network parameters for a second reconstruction layer into a data stream, the second reconstruction layer including, for each neural network parameter, a second reconstruction layer neural network parameter value, the neural network parameters being reconfigurable by combining, for each neural network parameter, the neural network parameter value of the first reconstruction layer and the neural network parameter value of the second reconstruction layer.

[0019] The embodiment according to the second aspect of the present invention is based on the idea that a neural network, e.g. defined by neural network parameters, can be efficiently compressed and / or transmitted, e.g. with a small amount of data in the bitstream, using sub-layers such as reconstruction layers, e.g. base layer and enhancement layer. The reconstruction layers can be defined such that the neural network parameters are reconfigurable by combining, for each neural network parameter, the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer. This distribution allows efficient coding, e.g. encoding and / or decoding, and / or transmission, of the neural network parameters. Thus, the second neural network parameters for the second reconstruction layer can be coded and / or transmitted separately in the data stream. [Brief description of the drawings]

[0020] The drawings are not necessarily to scale, emphasis instead being placed generally on illustrating the principles of the present invention. In the following description, various embodiments of the present invention are described with reference to the following drawings, in which embodiments of the present invention are illustrated: [Figure 1] FIG. 1 is an exemplary schematic diagram of a two-layer feedforward neural network that may be used with embodiments of the present invention. [Diagram 2] FIG. 2 is a schematic diagram illustrating a concept for inverse quantization performed in an apparatus for decoding neural network parameters defining a neural network from a data stream according to an embodiment. [Diagram 3] FIG. 3 is a schematic diagram illustrating a concept for quantization performed in an apparatus for encoding neural network parameters into a data stream according to an embodiment. [Figure 4]FIG. 4 is a schematic diagram illustrating a concept for decoding performed in an apparatus for reconstructing neural network parameters defining a neural network according to an embodiment. [Diagram 5] FIG. 5 is a schematic diagram illustrating a concept for encoding performed in an apparatus for reconstructing neural network parameters defining a neural network according to an embodiment. [Figure 6] FIG. 6 is a schematic diagram of the concept of using a reconstruction layer for neural network parameters for use in an embodiment according to the present invention. [Figure 7] FIG. 7 is a schematic diagram illustrating a uniform reconstruction quantizer according to an embodiment of the present invention. [Figure 8] FIG. 8 shows examples of allowable reconstruction vector positions for the simple case of two weight parameters according to an embodiment of the present invention. [Figure 9a] FIG. 9a illustrates an example for dependent quantization with two sets of reconstruction levels determined entirely by a single quantization step size Δ according to an embodiment of the present invention. [Figure 9b] FIG. 9b illustrates an example for dependent quantization with two sets of reconstruction levels determined entirely by a single quantization step size Δ according to an embodiment of the present invention. [Figure 9c] FIG. 9c illustrates an example for dependent quantization with two sets of reconstruction levels completely determined by a single quantization step size Δ according to an embodiment of the present invention. [Figure 10] FIG. 10 is a diagram illustrating an example of pseudocode illustrating a preferred embodiment of a neural network parameter reconfiguration process according to an embodiment of the present invention. [Figure 11] FIG. 11 is a diagram illustrating an example for dividing a reconstructed level set into two subsets according to an embodiment of the present invention. [Figure 12]FIG. 12 illustrates an example of pseudocode illustrating a preferred example of a reconfiguration process for neural network parameters for a layer according to an embodiment. [Figure 13] FIG. 13 is a diagram showing a preferred example of a state transition table sttab and table setId according to an embodiment of the present invention, which specifies the quantization set associated with a state. [Figure 14] FIG. 14 is a diagram showing a preferred example of a state transition table sttab and table setId according to an embodiment of the present invention, which specifies the quantization set associated with a state. [Figure 15] FIG. 15 illustrates pseudocode illustrating an alternative neural network parameter level reconstruction process in which quantization indices equal to 0 are excluded from state transitions and dependent scalar quantization, in accordance with an embodiment of the present invention. [Figure 16] FIG. 16 is a diagram illustrating an example of state transitions in dependent scalar quantization as a trellis structure according to an embodiment of the present invention. [Figure 17] FIG. 17 is a diagram showing an example of a basic trellis cell according to an embodiment of the present invention. [Figure 18] FIG. 18 illustrates an example trellis for dependent scalar quantization of eight neural network parameters according to an embodiment of the present invention. [Figure 19] FIG. 19 illustrates an example trellis structure that can be utilized to determine a sequence (or block) of quantization indexes that minimizes a cost measure (e.g., a Lagrangian cost measure D+λ·R) in accordance with an embodiment of the present invention. [Figure 20] FIG. 20 is a block diagram of a method for decoding neural network parameters defining a neural network from a data stream according to an embodiment of the present invention. [Figure 21] FIG. 21 is a block diagram of a method for encoding neural network parameters defining a neural network into a data stream according to an embodiment of the present invention. [Figure 22] FIG. 22 is a block diagram of a method for reconfiguring neural network parameters defining a neural network according to an embodiment of the present invention. [Diagram 23] FIG. 23 is a block diagram of a method for encoding neural network parameters that define a neural network according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Elements or components having equal or equivalent functions, even if depicted in different figures, are indicated in the following description with equal or equivalent reference numerals.

[0022] In the following description, a number of details are set forth to provide a more detailed description of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail in order not to obscure the embodiments of the present invention. In addition, features of different embodiments described below in this specification may be combined with each other unless otherwise specified.

[0023] The description begins with a presentation of some embodiments of the present application. This description is fairly general, but provides the reader with an overview of the functionality on which the embodiments of the present application are based. A more detailed description of these functionalities is then presented, along with the motivation for the embodiments and how they achieve the efficiency improvements mentioned above. The details can be combined individually and in combination with the currently described embodiments.

[0024] Fig. 2 is a schematic diagram illustrating the concept of inverse quantization performed in an apparatus for decoding neural network parameters defining a neural network from a data stream according to an embodiment. The neural network may include, for example, multiple interconnected neural network layers with neuronal interconnections between the neurons of the interconnected layers. Fig. 2 illustrates quantization indices 56 of, for example, encoded neural network parameters 13 in a data stream 14. The neural network parameters 13 may thus define or parameterize the neural network, such as in terms of weights between its neurons.

[0025] The device is configured to sequentially decode the neural network parameters 13. During this sequential process, the quantizer (reconstruction level set) is changed. This change allows to use a quantizer with fewer (or better, less dense) levels and therefore to code smaller quantization indexes, the quality of the neural network representation resulting from this quantization being improved compared to using a constant quantizer, compared to the required coding bit rate. More details will be given later. In particular, the device sequentially decodes the neural network parameters 13 by selecting 54 (reconstruction level selection) for the current neural network parameters 13' a reconstructed level set 48 (selected set) out of a plurality (50) of reconstructed level sets 52 (set 0, set 1) depending on the quantization indexes 58 decoded from the data stream 14 for the previous neural network parameters.

[0026] Further, the apparatus is configured to continuously decode the neural network panel 13 by decoding a quantization index 56 for the current neural network parameters 13' from the data stream 14, where the quantization index 56 indicates one reconstruction level of the selected set 48 of reconstruction levels for the current neural network parameters, and by inverse quantizing 62 the current neural network parameters 13' to one reconstruction level of the selected set 48 of reconstruction levels indicated by the quantization index 56 for the current neural network parameters.

[0027] The decoded neural network parameters 13 are represented, by way of example, by a matrix 15a, which may include deserialized neural network parameters 13 20b, which may relate to the weights of the neuron interconnections of the neural network.

[0028] Optionally, the number of reconstructed level sets 52, also sometimes referred to herein as quantizers, of the plurality (50) of reconstructed level sets 52 may be two, for example set 0 and set 1, as shown in FIG. 2 .

[0029] Furthermore, the apparatus may be configured to parameterize 60 the plurality (50) of reconstruction level sets 52 (e.g., set 0, set 1) by a predefined quantization step size (QP), e.g., denoted Δ or Δk, and derive the predefined quantization step size information from the data stream 14. Thus, a decoder according to an embodiment may be adaptive to a variable step size (QP).

[0030] Further, according to an embodiment, the neural network may include one or more NN layers, and the apparatus may be configured to derive, for each NN layer, information of a predefined quantization step size (QP) for the respective NN layer from the data stream 14, and parameterize, for each NN layer, a plurality 50 of reconstruction level sets 52 with the predefined quantization step size derived for the respective NN layer, to be used to dequantize neural network parameters belonging to the respective NN layer. Adapting the step size for the NN layer, and thus adapting the reconstruction level sets 52, may improve coding efficiency.

[0031] According to a further embodiment, the device may be configured to select 54 for the current neural network parameter 13' a reconstructed level set 48 of the plurality 50 reconstructed level sets 52 depending on a LSB (e.g. least significant bit) portion of a binarized quantization index 58 decoded from the data stream 14 for the previously decoded neural network parameter or a previously decoded bin (e.g. binary decision). The LSB comparison can be performed with low computational cost. In particular, a state transition can be used. The selection 54 can be performed by a state transition process by determining for the current neural network parameter 13' a reconstructed level set 48 of the plurality 50 reconstructed level sets 52 a reconstructed level set 48 of the plurality 50 reconstructed level sets 52 depending on a state associated with the current neural network parameter 13' and by updating a state for the subsequent neural network parameter depending on a quantization index 58 decoded from the data stream for the immediately preceding neural network parameter. Also alternative approaches other than state transition, for example by using a transition table, can be used as well and are defined below.

[0032] Additionally or alternatively, the device may be configured to select 54, for example, for the current neural network parameters 13', a reconstructed level set 48 from the plurality 50 of reconstructed level sets 52 depending on the result of a binary function of the quantization indexes 58 decoded from the data stream 14 for the previously decoded neural network parameters. The binary function may be, for example, a parity check using a bitwise "and" operation, signaling whether the quantization index 58 represents an even or odd number. This may provide information about the reconstructed level set 48 used to code the quantization index 58, and thus, for example, for a given order of the reconstructed level sets used in the corresponding encoder, for the set of reconstructed levels used to code the current neural network parameters 13'. The parity may be used for state transitions as described above.

[0033] Further, according to an embodiment, the apparatus may be configured to, for example, select 54 for a current neural network parameter 13' a set 48 of reconstruction levels from the plurality 50 of reconstruction level sets 52 depending on the parity of quantization indexes 58 decoded from the data stream 14 for previously decoded neural network parameters. The parity check may be performed at low computational cost, for example using a bitwise "and" operation.

[0034] Optionally, the device may be configured to decode the quantization indices 56 for the neural network parameters 13 and perform inverse quantization of the neural network parameters 13 along a common sequential order 14' among the neural network parameters 13. In other words, the same order may be used for both tasks.

[0035] FIG. 3 is a diagram that illustrates a schematic of the quantization concept performed in an apparatus for encoding neural network parameters into a data stream according to an embodiment. FIG. 3 illustrates a neural network (NN) 10 including neural network layers 10a, 10b, where the layers include neurons 10c, and the neurons of the interconnected layers are interconnected via neuronal interconnections 11. As an example, an NN layer (p-1) 10a and an NN layer (p) 10b are illustrated, where p is an index of the NN layer and 1≦p≦the number of layers of the NN. The neural network is defined or parameterized by neural network parameters 13, which may optionally be related to the weights of the neuronal interconnections 11 of the neural network 10. The neurons 10c of the hidden layer of FIG. 1 may represent the neurons of layer p (A, B, C, ...) of FIG. 3, and the neurons of the input layer of FIG. 1 may represent the neurons of layer p-1 (a, b, c, ...) illustrated in FIG. 3. The neural network parameters 13 may be related to the weights of the neuronal interconnections 11 of FIG. 1.

[0036] The relationships between neurons 10c in different layers are represented in Fig. 1 by a matrix 15a of neural network parameters 13. For example, if the network parameters 13 relate to weights of neuronal interconnections 11, the matrix 15a can be configured, for example, such that the matrix elements represent the weights between neurons 10c in different layers (e.g., a, b,... for layer p-1 and A, B,... for layer p).

[0037] The device is configured to sequentially code the neural network parameters 13, for example in serialization 20a. During this sequential process, the quantizer (reconstructed level set) is changed. This change allows the use of a quantizer with fewer (or better, less dense) levels, and therefore allows the encoding of smaller quantization indices, which improves the quality of the neural network representation resulting from this quantization, compared to the required encoding bit rate, compared to the use of a constant quantizer. More details will be given below. In particular, the device sequentially codes the neural network parameters 13 by selecting 54 for the current neural network parameters 13' a reconstructed level set 48 out of a plurality 50 of reconstructed level sets 52 depending on the quantization indices 58 coded in the data stream 14 for the previously coded neural network parameters.

[0038] The apparatus further quantizes 64 (Q) the current neural network parameters 13' onto one reconstruction level of the selected reconstruction level set 48, and The neural network parameters 13 are configured to be sequentially encoded by encoding a quantization index 56 for the current neural network parameters 13', the quantization index 56 for the current neural network parameters indicating one reconstruction level to which the current neural network parameters are quantized in the data stream 14. Optionally, the number of reconstruction level sets 52, also sometimes referred to herein as quantizers, out of the plurality 50 of reconstruction level sets 52 may be two, e.g., as indicated with set 0 and set 1.

[0039] According to an embodiment, as shown in Fig. 3, the device may be configured to, for example, parameterize 60 the plurality 50 of reconstruction level sets 52 by a predefined quantization step size (QP) and insert information about the predefined quantization step size in the data stream 14. This allows, for example, adaptive quantization to improve quantization efficiency, and changes in the coding method of the neural network parameters 13 can be communicated to the decoder together with information about the predefined quantization step size. Using a predefined quantization step size (QP) can reduce the amount of data for the transmission of information.

[0040] Further, according to an embodiment, the neural network 10 may include one or more neural network layers 10a, 10b. The device may then be configured to, for each neural network layer (p; p-1), insert information of a predefined quantization step size (QP) for the respective neural network layer into the data stream 14 and parameterize a plurality of reconstruction level sets 50 with the predefined quantization step size derived for the respective neural network layer for use in quantizing neural network parameters belonging to the respective neural network layer. As explained above, adapting the quantization depending on, for example, the neural network layer or the characteristics of the neural network layer may improve the quantization efficiency.

[0041] Optionally, the apparatus may be configured to select 54 for the current neural network parameters 13′ a set 48 of reconstruction levels from the plurality 50 of reconstruction level sets 52 in response to a previously coded bin or an LSB portion of a binarized quantization index 58 coded in the data stream 14 for the previously coded neural network parameters. The LSB comparison may be performed at low computational cost.

[0042] A state transition can be used similarly to the device for decoding described in Fig. 2. A state transition process can perform a selection 54 for the current neural network parameters 13' among the quantization level sets 48 among the plurality of 50 reconstructed level sets 52 by determining for the current neural network parameters 13' a set of reconstructed levels 48 among the plurality of 50 reconstructed level sets 52 depending on the state associated with the current neural network parameters 13' and by updating the state for the subsequent neural network parameters depending on the quantization index 58 coded in the data stream for the previous neural network parameters. Approaches other than state transition, for example by using a transition table, can be used as well and are defined below.

[0043] Additionally or alternatively, the device may be configured to select 54 a reconstruction level set 48 from the plurality 50 of reconstruction level sets 52 for the current neural network parameters 13' depending on the result of a binary function of the quantization indexes 58 coded in the data stream 14 for the previously coded neural network parameters. The binary function may be, for example, a parity check using a bitwise "and" operation, signaling whether the quantization index 58 represents an even or odd number. This may provide information about the reconstruction level set 48 used to code the quantization index 58, and thus, for example, for a given order of the reconstruction levels, a set of reconstruction levels 48 for the current neural network parameters 13' may be determined such that for the given order, a corresponding decoder may select a corresponding reconstruction level set 48. The parity may be used for state transitions as described above.

[0044] Furthermore, according to an embodiment, the apparatus may be configured to, for example, select 54, for the current neural network parameters 13', a quantization level set 48 from the plurality 50 of reconstruction level sets 52 depending on the parity of the quantization indexes 56 coded in the data stream 14 for the previously coded neural network parameters. The parity check may be performed at low computational cost, for example using a bitwise "and" operation.

[0045] Optionally, the device may be configured to encode quantization indices (56) for the neural network parameters (13) and to perform the quantization of the neural network parameters (13) along a common sequential order (14') between the neural network parameters (13), i.e. the same order may be used for both tasks.

[0046] Fig. 4 is a schematic diagram of a concept for arithmetically decoding quantized neural network parameters according to an embodiment. It can be used in the device of Fig. 2. Fig. 4 can therefore be seen as a possible extension of Fig. 2. It shows a data stream 14 in which quantization indexes 56 for the current neural network parameters 13' are decoded by the device of Fig. 4, as shown by way of an arbitrary example, by using arithmetic coding, for example binary arithmetic coding. For example, a probability model defined by a certain context is used, which depends on the reconstruction level set 48 selected for the current neural network parameters 13', as indicated by the arrow 123. Details are set out herein.

[0047] As explained with respect to Fig. 2, a selection 54 of a set of quantization levels 48 from the plurality of 50 reconstruction level sets 52 can be performed for the current neural network parameters 13' by determining a set of reconstruction levels 48 from the plurality of 50 reconstruction level sets 52 for the current neural network parameters 13' depending on the state associated with the current neural network parameters 13', and by updating the state for the subsequent neural network parameters depending on the quantization index 58 decoded from the data stream for the previous neural network parameters. Thus, the state is akin to a pointer to the reconstruction level set 48 to be used to code / decode the current neural network parameters 13', but it is updated with finer precision as it only distinguishes a number of states corresponding to the number of reconstruction sets, so that the state acts quasi-memory of past neural network parameters or past quantization indexes. Thus, the state specifies the order of the sets of reconstruction levels used to code / decode the neural network parameters 13. 4, for example, the quantization indexes (56) for the current neural network parameters (13') are decoded from the data stream (14) using arithmetic coding, using a probability model that corresponds (122) to the state for the current neural network parameters (13'). Adapting the probability model depending on the state may result in a better estimation of the probability model and improved coding efficiency. Furthermore, adapting depending on the state may allow for computationally efficient adaptation with less additional data transmission.

[0048] According to a further embodiment, the apparatus may be configured to decode the quantization index 56 for the current neural network parameter 13′ from the data stream 14 using binary arithmetic coding, e.g. using a probability model in which the binarization 82 of the quantization index 56 depends 122 on the state for the current neural network parameter 13′ for at least one bin 84.

[0049] Additionally or alternatively, the device may be configured such that the dependency of the probability model includes the selection 103 (derivation) of a context 87 from a set of contexts for the neural network parameters using the dependency, each context being configured to have a predefined probability model associated with it. The better the probability estimates used, the more efficient the compression. The probability model can be updated, for example, using context-adaptive (binary) arithmetic coding.

[0050] Optionally, the device may be configured to update a predefined probability model associated with each context based on the quantization indexes arithmetically coded with the respective context, in this way the probability model of the context is adapted to the actual statistics.

[0051] Further, the apparatus may be configured to decode the quantization index 56 for the current neural network parameters 13′ from the data stream 14 using binary arithmetic coding, e.g., for at least one bin of the binarized quantization index, using a probability model corresponding to the reconstruction level set 48 selected for the current neural network parameters 13′.

[0052] Optionally, the at least one bin may include a significance bin indicating whether the current neural network parameter quantization index 56 is equal to zero. Additionally or alternatively, the at least one bin may include a sign bin indicating whether the current neural network parameter quantization index 56 is greater than or less than zero. Furthermore, the at least one bin may include a greater-than-X bin indicating whether the absolute value of the current neural network parameter quantization index 56 is greater than X, where X is an integer greater than zero.

[0053] In the following, in Fig. 5, the opposite of the concept for decoding described with reference to Fig. 4 can be described. Therefore, all the explanations and advantages can be applied to the aspect of the concept for encoding below as appropriate.

[0054] Fig. 5 is a schematic diagram illustrating a concept for arithmetically coding neural network parameters according to an embodiment. It can be used in the device of Fig. 3. Fig. 5 can therefore be seen as a possible extension of Fig. 3. It shows a data stream 14 in which quantization indexes 56 for the current neural network parameters 13' are coded by the device of Fig. 3 as indicated by the use of arithmetic coding, for example binary arithmetic coding as an arbitrary example. A probability model defined for example by a certain context is used, which depends on the reconstruction level set 48 selected for the current neural network parameters 13', as indicated by the arrow 123. The details are set out herein.

[0055] A selection 54 is performed for the current neural network parameters 13' as described with respect to Figure 3. The selection 54 selects a quantization level set 48 from the plurality 50 of reconstructed level sets 52 for the current neural network parameters 13' in response to a state associated with the current neural network parameters 13', and by updating the state for the subsequent neural network parameters in response to a quantization index 58 encoded in the data stream for the immediately preceding neural network parameters, through a state transition process.

[0056] Thus, the states are quasi-pointers to the reconstruction level sets 48 to be used to encode / decode the current neural network parameters 13′, but are updated with finer precision as they only distinguish a number of states corresponding to the number of reconstruction sets, so that the states essentially act as a memory of past neural network parameters or past quantization indexes. Thus, the states define the order of the reconstruction level sets to be used to encode / decode the neural network parameters 13.

[0057] Additionally, the quantization indices 56 for the current neural network parameters 13' may be encoded into the data stream 14 using arithmetic coding that employs a probability model that corresponds to the states 122 for the current neural network parameters 13'.

[0058] 3, for example, the quantization index 56 is coded into the data stream 14 using a binary arithmetic coding for the current neural network parameters 13' using a probability model corresponding to a state 122 for the current neural network parameters 13' for at least one bin 84 of the binarized 82 version of the quantization index 56. Adapting the probability model depending on the state may improve coding efficiency, since the probability model may be suitable for estimating the probability model. Furthermore, the state-dependent adaptation may allow for a computationally efficient adaptation with a small amount of additional data transmitted.

[0059] Additionally or alternatively, the apparatus may be configured such that the dependency of the probabilistic model includes selecting 103 (deriving) a context 87 from a set of contexts for the neural network parameters using the dependency, each context having a predefined probabilistic model associated therewith.

[0060] Optionally, the apparatus may be configured to update a predefined probability model associated with each context based on quantization indexes arithmetically coded using the respective context.

[0061] Furthermore, the apparatus may be configured to encode the quantization indexes 56 of the current neural network parameters 13' in the data stream 14 using binary arithmetic coding, for example by using a probability model corresponding to the selected reconstruction level set 48 for the current neural network parameters 13' for at least one bin of the binarization of the quantization indexes. To use binary arithmetic coding, the quantization indexes 56 may be binarized.

[0062] Optionally, the at least one bin may include a significance bin indicating whether the current neural network parameter quantization index 56 is equal to zero. Additionally or alternatively, the at least one bin may include a sign bin indicating whether the current neural network parameter quantization index 56 is greater than or less than zero. Furthermore, the at least one bin may include a greater-than-X bin indicating whether the absolute value of the current neural network parameter quantization index 56 is greater than X, where X is an integer greater than zero.

[0063] The next described embodiment focuses on another aspect of the present application, which is that the parameterization of the neural network is coded in stages or reconstruction layers, and for each NN parameter, one value from each stage needs to be combined to obtain an improved / enhanced representation of the neural network, and at least one may itself represent a reasonable representation of the neural network, but is enhanced in any of the contributing stages of low quality, although the latter possibility is not essential to this aspect.

[0064] Fig. 6 is a schematic diagram of the concept of using reconstruction layers for neural network parameters for use in an embodiment according to the present invention. Fig. 6 shows that the reconstruction layer i, e.g. the second reconstruction layer, the reconstruction layer i-1, e.g. the first reconstruction layer, and the neural network (NN) layer p, e.g. layer 10b from Fig. 3, are layers represented in the form of an array or matrix, e.g. matrix 15a from Fig. 3.

[0065] 6 shows a concept of an apparatus 310 for reconstructing neural network parameters 13 defining a neural network. The apparatus is thus configured to derive first neural network parameters 13a, which may have been previously transmitted, for example during an associative learning process, for example for a first reconstruction layer, for example reconstruction layer i-1, and to result in neural network parameter values ​​of the first reconstruction layer for each neural network parameter, for example for each weight or each inter-neuron connection. This derivation may otherwise include decoding or receiving the first neural network parameters 13a. Furthermore, the apparatus is configured to decode 312 second neural network parameters 13b, which may for example be referred to as neural network parameters of the second reconstruction layer to distinguish them from, for example, final neural network parameters, for example parameters 13, for the second reconstruction layer, from the data stream 14 for generating neural network parameter values ​​of the second reconstruction layer for each neural network parameter 13. Thus, two contribution values ​​of the first and second reconstruction layers may be obtained for each NN parameter, and the encoding / decoding of the first and / or second NN parameter values ​​may use dependent quantization according to Figures 2 and 3 and / or arithmetic encoding / decoding of quantization indexes as described in Figures 4 and 5. The second neural network parameters 13b may not have a self-contained meaning in terms of neural representation, but may simply lead to the neural network representation, i.e. the final neural network parameters, when combined with the parameters of the first representation layer.

[0066] Further, the apparatus is configured to reconstruct 314 the neural network parameters 13 by combining (CB) for each neural network parameter the neural network parameter values ​​of the first reconstruction layer and the second reconstruction layer neural network parameter values, for example using element-wise addition and / or multiplication.

[0067] Further, FIG. 6 shows the concept of an apparatus 320 for encoding neural network parameters 13 defining a neural network using a first neural network parameter 13a for a first reconstruction layer, e.g., reconstruction layer i-1. The first reconstruction layer comprises, for each neural network parameter 13, a neural network parameter value of the first reconstruction layer. The apparatus is thus configured to encode 322 a second neural network parameter 13b for a second reconstruction layer, e.g., reconstruction layer i, into the data stream. The second reconstruction layer comprises, for each neural network parameter 13, a neural network parameter value of the second reconstruction layer. The neural network parameters 13 are reconfigurable by combining (CB) the neural network parameter values ​​of the first reconstruction layer with the neural network parameter values ​​of the second reconstruction layer, e.g., using element-wise addition and / or multiplication for each neural network parameter.

[0068] Optionally, the apparatus 310 may be configured to decode 316 first neural network parameters for the first reconstruction layer from the data stream 14 or from a separate data stream.

[0069] Simply put, decomposition of the neural network parameters 13 may allow for more efficient encoding and / or decoding and transmission of the parameters.

[0070] Further embodiments are disclosed below, in particular involving neural network coding concepts. The following description provides further details that can be used both individually and in combination with the above-described embodiments.

[0071] First, a method for entropy coding of parameters of a neural network with Dependent Scalar Quantization according to an embodiment of the present invention is presented.

[0072] A method for parameter coding of a set of neural network parameters 13 (also called weights, weight parameters or parameters) using dependent scalar quantization is described. The parameter coding presented herein consists of dependent scalar quantization of the parameters 13 (e.g., as described in the context of FIG. 3) and entropy coding of the obtained quantization indexes 56 (e.g., as described in the context of FIG. 5). On the decoder side, a set of reconstructed neural network parameters 13 can be obtained by entropy decoding of the quantization indexes 56 (e.g., as described in the context of FIG. 4) and dependent reconstruction of the neural network parameters 13 (e.g., as described in the context of FIG. 2). In contrast to parameter coding with independent scalar quantization and entropy coding, the set of allowable reconstruction levels for the neural network parameters 13 depends on the transmitted quantization index 56 that precedes the current neural network parameters 13' in the reconstruction order. The following presentation additionally describes a method for entropy coding of the quantization indexes that specify the reconstruction levels used in the dependent scalar quantization.

[0073] This description is primarily directed to lossy coding of neural network parameter layers in neural network compression, but is also applicable to lossy coding in other fields.

[0074] The methodology of the device is divided into different main parts:

[0075] 1. Quantization 2. Lossless Coding 3. Lossless Decoding

[0076] In order to appreciate the main advantages of the embodiments described below, a brief introduction to the topic of neural networks and related methods for parameter encoding is first provided. Nevertheless, all of the disclosed aspects, features, and concepts can be used separately or in combination with the embodiments described herein.

[0077] 2. Related Methods for Quantization and Entropy Coding Working Draft 2 of the MPEG-7 Part 17 standard for Compression of Neural Networks for Multimedia Content Description and Analysis [2] applies independent scalar quantization and entropy coding to neural network parameter coding.

[0078] 2.1 Scalar quantizer The neural network parameters are quantized using a scalar quantizer. As a result of the quantization, the set of allowable values ​​of the parameters 13 is reduced. In other words, the neural network parameters are mapped to a countable set (actually a finite set) of so-called reconstruction levels. The reconstruction level set represents a proper subset of the set of possible neural network parameter values. To simplify the entropy coding below, the allowable reconstruction levels are represented by quantization indices 56, which are transmitted as part of the bitstream 14. On the decoder side, the quantization indices 56 are mapped to the reconstructed neural network parameters 13. The possible values ​​of the reconstructed neural network parameters 13 correspond to the reconstruction level set 52. On the encoder side, the result of the scalar quantization is a set of (integer) quantization indices 56.

[0079] TIFF2025016517000003.tif55153

[0080] In this context, the term “independent scalar quantization” refers to the property that given a quantization index q56 for any weighting parameter 13, the associated reconstructed weighting parameter t′13′ can be determined independently from all quantization indices for other weighting parameters.

[0081] 2.1.1 Encoder Operation: Quantization Standards for neural network compression only specify the bitstream syntax and the reconstruction process. When considering parameter coding for a given set of original neural network parameters 13 and a given quantization step size (QP), the encoder has many degrees of freedom. The quantization indices q of layers 10a, 10b are k Given 56, the entropy coding must follow a uniquely defined algorithm for writing the data into the bitstream 14 (i.e., constructing an arithmetic codeword). However, given the original set of weight parameters (e.g., layers), the quantization index q k The encoding algorithm to obtain 56 is outside the scope of any standard for neural network compression. In the following description, we assume that the quantization step size (QP) of each neural network parameter 13 is known. Nevertheless, the encoder calculates the quantization step size (QP) of each neural network (weight) parameter t k Quantizer index q for 13 k 56. Since the choice of quantization index determines both the distortion (or reconstruction / approximation quality) and the bitrate, the quantization algorithm used has a substantial impact on the rate-distortion performance of the generated bitstream 14.

[0082] TIFF2025016517000004.tif123152

[0083] The quantization process performs well in terms of rate-distortion if it minimizes the Lagrangian function D+λ·R, where D is the distortion of the neural network parameter set (e.g., the MSE distortion or the MAE distortion), R is the number of bits required to transmit the quantization index 56, and λ is the Lagrangian multiplier.

[0084] TIFF2025016517000005.tif75150

[0085] TIFF2025016517000006.tif64151

[0086] In fact, nearest neighbor quantization is the trivial case with λ=0, which is applied in Working Draft 2 of the MPEG-7 part 17 standard on Compression by Neural Networks for Multimedia Content Description and Analysis.

[0087] 2.2 Entropy Coding As a result of the uniform quantization applied in the previous step, the weight parameters are mapped to a finite set of so-called reconstruction levels. These can be represented by an (integer) quantizer index 56 (also called parameter level or weight level) and a quantization step size (QP), which may be fixed for all layers, for example. To recover all quantized weight parameters of a layer, the step size (QP) and the dimension of the layer may be known by the decoder. These may be transmitted separately, for example.

[0088] 2.2.1 Quantization index coding using context-adaptive binary arithmetic coding (CABAC) The quantization indexes 56 (integer representation) are then transmitted using an entropy coding technique. Thus, the weight layers are mapped to a sequence of quantized weight levels using a scan. For example, a row first scan order can be used, starting from the top row of the matrix and encoding the contained values ​​from left to right. In this way, all rows are encoded from top to bottom. The scan may be performed as shown in FIG. 3, for example along a common sequential order 14', for a matrix 15a containing neural network parameters 13 that may be related to the weights of the neuronal interconnects 11. The matrix may represent a weight layer, for example the weights between layers p-1 10a and p 10b of the neuronal interconnects 11, or between the hidden layer and the input layer, as shown in FIG. 3 and FIG. 1, respectively. It should be noted that any other scan may be applied. For example, a matrix (e.g., matrix 15a in FIG. 2 or FIG. 3) may be transposed, or flipped horizontally and / or vertically, and / or rotated by 90 / 180 / 270 degrees left or right before applying row-first scanning.

[0089] 3 and 5, an apparatus according to an embodiment may be configured to encode the quantization index 56 for the current neural network parameters 13′ into the data stream 14 using binary arithmetic coding, using a probability model corresponding to a state 122 for the current neural network parameters 13′ for at least one bin 84 of the binarized 82 quantization index 56. The binary arithmetic coding using the probability model may be context-adaptive binary arithmetic coding (CABAC).

[0090] In other words, according to the embodiment, CABAC is used for level encoding. For details, refer to [3]. Thus, the quantized weight level q56 may be decomposed into a series of binary symbols or syntax elements, such as bins (binary decisions), and then passed to a binary arithmetic coder (CABAC). In the first step, a binary syntax element sig_flag is derived for the quantized weight level, which specifies whether the corresponding level is equal to zero. In other words, at least one bin of the binarized 82 quantization index 56 shown in FIG. 4 may include a significance bin indicating whether the quantization index 56 of the current neural network parameter is equal to zero.

[0091] If sig_flag is equal to 1, a further binary syntax element sign_flag is derived. This bin indicates whether the current weight level is positive (e.g., bin = 0) or negative (e.g., bin = 1). In other words, at least one bin of the binarized 82 quantization index 56 shown in FIG. 4 may include a sign bin 86 indicating whether the quantization index 56 of the current neural network parameter is greater than or less than zero.

[0092] Next, the unary sequence of bins is encoded, followed by encoding of the fixed-length sequence as follows.

[0093] The variable k is initialized with a non-negative integer, and X is initialized with 1 << k.

[0094] One or more syntax elements called abs_level_greater_X, which indicates that the absolute value of the quantized weight level is greater than X, are encoded. When abs_level_greater_X is equal to 1, the variable k is updated (e.g., incremented by 1), then 1<<k is added to X, and then abs_level_greater_X is encoded again. This procedure continues until abs_level_greater_X is equal to 0. After that, the quantization index can be encoded with a fixed-length code of length k. For example, the variable rem = X - |q| can be encoded using k bits. Alternatively, the variable rem' can be defined as rem'=(1<<k)-rem-1, which is encoded using k bits. Other mappings of the variable rem to a fixed-length code of k bits can also be used.

[0095] In other words, at least one bin of the binarized 82 quantization index 56 shown in FIG. 4 can include a greater-than-X bin indicating whether the absolute value of the quantization index 56 of the current neural network parameters is greater than X, where X is an integer greater than zero.

[0096] When incrementing k by 1 after each abs_level_greater_X, this approach is identical to applying exponential Golomb coding (when the sign_flag is not considered).

[0097] Also, when the maximum absolute value abs_max is known on both the encoder side and the decoder side, for the next abs_Level_greater_X to be transmitted, when X >= abs_max holds, the encoding of the abs_Level_greater_X syntax element can be terminated.

[0098] TIFF2025016517000007.tif91152

[0099] In other words, as described with respect to Figures 2 and 4, an apparatus according to an embodiment can be configured to decode the quantization index 56 for the current neural network parameter 13' from the data stream 14 using binary arithmetic coding by using a probability model corresponding to the state 122 for the current neural network parameter 13' for at least one bin 84 of the binarized 82 quantization index 56.

[0100] At least one bin of the binarized 82 quantization index 56 shown in FIG. 5 may include a significance bin indicating whether the current neural network parameter quantization index 56 is equal to zero. Additionally or alternatively, the at least one bin may include a sign bin 86 indicating whether the current neural network parameter quantization index 56 is greater than or less than zero. Furthermore, the at least one bin may include a greater-than-X bin indicating whether the absolute value of the current neural network parameter quantization index 56 is greater than X, where X is an integer greater than zero.

[0101] In the preferred embodiment, k is initialized to 0 and updated as follows: After each abs_level_greater_X equals 1, any necessary updates to k are made according to the following rules: if X>X', then k is incremented by 1, where X' is a constant corresponding to the application. For example, X' is a number (e.g., between 0 and 100) that the encoder derives and communicates to the decoder.

[0102] 2.2.3 Context Modeling In CABAC entropy coding, most syntax elements with quantized weight levels 56 are coded using binary probability modeling. Each binary decision (bin) is associated with a context. The context represents a probability model for the class of coded bins. The probability for one of the two possible bin values ​​is estimated for each context based on the values ​​of the bin already coded in the corresponding context. Depending on the application, different context modeling approaches can be applied. Typically, for some bins related to quantized weight coding, the context used for coding is selected based on the syntax elements already transmitted. Depending on the actual application, different probability estimators can be selected, such as those of SBMP 0, or HEVC 0 or VTM-4.0 0. This choice affects, for example, compression efficiency and complexity.

[0103] In other words, a probability model such as that described with respect to FIG. 5, eg, context 87, additionally relies on the quantization indexes of previously coded neural network parameters.

[0104] Respectively, the probability models as described with respect to FIG. 4, for example context 87, further depend on the quantization indices of previously decoded neural network parameters.

[0105] A context modeling scheme that is suitable for a wide range of neural networks is described as follows: To decode the quantized weight level q56 at a particular position (x,y) of the weight matrix (layer), a local template is applied to the current position. This template contains a number of other (ordered) positions, e.g. (x-1,y), (x,y-1), (x-1,y-1), etc. For each position, a status identifier is derived.

[0106] TIFF2025016517000008.tif34152

[0107] For a given template, we derive a sequence of status identifiers and map each possible constellation of status identifier values ​​to a context index that identifies the context in which it is used. The templates and mappings may be different for different syntax elements. For example, we derive a sequence of status identifiers s from a template that contains (ordinal) positions (x-1,y), (x,y-1), (x-1,y-1), x-1,y , s x,y-1 , s x-1,y-1 For example, this sequence can be expressed as follows: x-1,y +3*s x,y-1 +9*s x-1,y-1 For example, a context index C may be used to identify multiple contexts for a sig_flag.

[0108] In a preferred embodiment (call it Approach 1), the quantized weight level q at position (x,y) x,y The local template for a sig_flag or for a sign_flag of consists of only one position (x-1,y) (i.e., the left neighbor). The associated status identifier s x-1,y is derived according to the preferred embodiment Si1.

[0109] For sig_flag, x-1,y Depending on the value of , one of three contexts is selected, or for sign_flag, s x-1,y Depending on the value of , one of the other three contexts is selected.

[0110] In another preferred embodiment (call it approach 2), the local template for the sig flag contains three ordered positions (x-1,y), (x-2,y), (x-3,y). x-1,y ,s x-2,y ,s x-3,y The associated sequence of is derived according to the preferred embodiment Si2.

[0111] For sig_flag, the context index C is derived as follows:

[0112] s x-1,y ≠0 then C=0, otherwise s x-2,y ≠0 then C=1, otherwise s x-3,y ≠0 then C=2, otherwise s x-3,y If ≠ 0, then C = 2. Otherwise, C = 3.

[0113] TIFF2025016517000009.tif24132

[0114] Similarly, one may increase or decrease the number of left neighbors such that the context index C is equal to the distance to the next non-zero weight on the left (not exceeding the template size).

[0115] Each abs_level_greater_X flag can, for example, apply its own set of two contexts, one of which is selected depending on the value of sign_flag.

[0116] In a preferred embodiment, for abs_level_greater_X flags where X is less than a predefined number X', different contexts are distinguished depending on the values ​​of X and / or sign_flag.

[0117] In a preferred embodiment, for abs_level_greater_X flags where X is greater than or equal to a predefined number X', different contexts are differentiated only in response to X.

[0118] In another preferred embodiment, abs_level_greater_X flags with X greater than or equal to a predefined number X' are coded using a fixed code length of 1 (eg, using a bypass mode of the arithmetic coder).

[0119] Furthermore, some or all of the syntax elements may be encoded without using the context. Instead, they are encoded with a fixed length of 1 bit, for example, using the so-called bypass bins of CABAC.

[0120] In another preferred embodiment, the fixed-length remainder rem is encoded using the bypass mode.

[0121] In another preferred embodiment, the encoder determines a predefined number X', distinguishes two contexts according to the sign for each syntax element abs_level_greater_X where X < X', and uses one context for each abs_level_greater_X where X >= X'.

[0122] In other words, a probability model such as that described with respect to FIG. 5, for example context 87, can be selected 103 for the current neural network parameters from among a subset of the probability models according to the quantization index of the previously encoded neural network parameters related to the part of the neural network adjacent to the part to which the current neural network parameters are related.

[0123] This part can be defined, for example, by the template described above, a template that includes the (ordered) positions (x - 1, y), (x, y - 1), (x - 1, y - 1).

[0124] Respectively, a probability model such as that described with respect to FIG. 5 can be selected for the current neural network parameters from among a subset of the probability models according to the quantization index of the previously decoded neural network parameters related to the part of the neural network adjacent to the part to which the current neural network parameters are related.

[0125] 3 Additional Methods In the following, we describe an additional, and therefore optional, method for compression / transmission of neural network 10 in which a reconstructed layer, e.g., neural network layer p from FIG. 6, is a composition of different sub-layers, such as reconstructed layer i-1 and reconstructed layer i from FIG. 6, which may be transmitted separately.

[0126] 3.1 Base and Enhancement Layer Concepts This concept introduces two types of sub-layers, called base layer and enhancement layer. Then, the reconstruction process (e.g. adding all sub-layers) defines how the reconstructed layer can be obtained from the sub-layers. The base layer contains base values, which can be chosen, for example, to be efficiently represented or compressed / transmitted in a first step. The enhancement layer contains enhancement information, for example difference values ​​that can be added to the (base) layer values ​​to reduce distortion measures (e.g. with respect to the original layer). In another example, the base layer contains coarse values ​​(from training with a small training set) and the enhancement layer contains refinement values ​​(based on the full training set or, more generally, on another training set). The sub-layers may be stored / transmitted separately.

[0127] In a preferred embodiment, the layer L to be compressed R , for example, neural network parameters, such as weights of the neural network, which may be represented by matrix 15a in FIG. 2 and FIG. 3, are included in the base layer L B and one or more enhancement layers L E,1 ,L E,2 ,···,L E,N Then, in the first step, the base layer is compressed / transmitted, and in the subsequent step, the enhancement layer L E,1 ,L E,2 ,...,L E,N is compressed / transmitted (separately).

[0128] TIFF2025016517000010.tif38152

[0129] TIFF2025016517000011.tif36152

[0130] In other words, the embodiment of the present invention is R or for example in the form of a reconstructed layer L R and configured to reconstruct the neural network parameters 13 by a parameter-wise sum or a parameter-wise product of the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer, for each neural network parameter, using

[0131] Respectively, in the case of the device for encoding neural network parameters 13 according to the embodiment, the neural network parameters 13 are reconfigurable for each neural network parameter by a parameter-wise sum or a parameter-wise product of the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer.

[0132] In a further preferred embodiment, the methods of 2.1 and / or 2.2 are applied to a subset or all of the sublayers.

[0133] In a particularly preferred embodiment, an entropy coding scheme using context modeling (e.g., the same as or similar to 2.2.3) is applied, but with the addition of one or more sets of context models according to one or more of the following rules:

[0134] a) each sub-layer applies its own set of contexts. In other words, an embodiment according to the present invention includes an apparatus configured to encode / decode first neural network parameters 13a for a first reconstruction layer to / from a data stream or another data stream and to encode / decode second neural network parameters 13b for a second reconstruction layer to / from a data stream by context-adaptive entropy coding using different probability contexts for the first and second reconstruction layers.

[0135] b) The context set selected for the enhancement layer parameters to be coded depends on the value of the parameter at the same position in the preceding layer (e.g. base layer) in the coding order. The first set of context models is selected whenever the parameter at the same position is equal to zero, otherwise the second set is selected. In other words, an embodiment according to the present invention comprises an apparatus configured to code the neural network parameter values ​​of a second reconstruction layer, e.g. the parameters of an enhancement layer, into a data stream by context-adaptive entropy coding using a probability model corresponding to the neural network parameter values ​​of the first reconstruction layer, e.g. the parameter at the same position in a preceding layer (e.g. base layer) in the coding order. A further embodiment comprises an apparatus configured to code the second reconstruction layer neural network parameter values ​​into a data stream by context-adaptive entropy coding by selecting a probability context set from a set of probability context sets depending on the first reconstruction layer neural network parameter values ​​and by selecting a probability context to be used from the selected probability context set depending on the first reconstruction layer neural network parameter values. Respectively, for an apparatus for decoding neural network parameters 13 according to an embodiment, said apparatus may be configured to decode a second reconstruction layer neural network parameter value from a data stream by context-adaptive entropy decoding using a probability model corresponding to the first reconstruction layer neural network parameter value. Respectively, further embodiments include an apparatus configured to decode a second reconstruction layer neural network parameter value from a data stream by context-adaptive entropy decoding by selecting a probability context set from a set of probability context sets corresponding to the first reconstruction layer neural network parameter value and by selecting a probability context to be used from the selected probability context set corresponding to the first reconstruction layer neural network parameter value.

[0136] c) The context set selected for the parameter of the enhancement layer to be coded corresponds to the value of the parameter at the same position of the preceding layer (e.g., base layer) in the coding order. The first set of context models is selected whenever the parameter at the same position is less than zero (negative), the second set is selected when the parameter at the same position is greater than zero (positive), and the third set is selected otherwise. In other words, an embodiment according to the present invention includes, for example, an apparatus for coding. The set of probability context sets includes three probability context sets. The apparatus is configured to select the first probability context set from the set of probability context sets as the selected probability context set when the neural network parameter value of the first reconstruction layer is negative, to select the second probability context set from the set of probability context sets as the selected probability context set when the neural network parameter value of the first reconstruction layer is positive, and to select the third probability context set from the set of probability context sets as the selected probability context set when the neural network parameter value of the first reconstruction layer is zero. Respectively, for an apparatus for decoding neural network parameters 13 according to an embodiment, the set of probability context sets may include three probability context sets, and the apparatus may be configured to select a first probability context set from the set of probability context sets as the selected probability context set when the neural network parameter value of the first reconstruction layer is negative, to select a second probability context set from the set of probability context sets as the selected probability context set when the neural network parameter value of the first reconstruction layer is positive, and to select a third probability context set from the set of probability context sets as the selected probability context set when the neural network parameter value of the first reconstruction layer is zero.

[0137] d) The context set selected for an enhancement layer parameter to be coded corresponds to the value of the parameter at the same position in the preceding layer (e.g., base layer) in coding order. The first set of context models is selected whenever the (absolute) value of the parameter at the same position is greater than X (where X is the parameter), otherwise the second set is selected. In other words, an embodiment according to the present invention includes an apparatus, wherein the set of probability context sets includes two probability context sets, and the apparatus is configured to select a first probability context set from the set of probability context sets as a selected probability context set if a parameter value of a neural network of a first reconstruction layer, e.g., a value of a parameter at the same position in a preceding layer (e.g., a base layer) in the coding order, is greater than a predetermined value, e.g., X, and to select a second probability context set from the set of probability context sets as the selected probability context set if the neural network parameter value of the first reconstruction layer is not greater than the predetermined value; or to select the first probability context set from the set of probability context sets as the selected probability context set if the absolute value of the neural network parameter value of the first reconstruction layer is greater than the predetermined value and to select the second probability context set from the set of probability context sets as the selected probability context set if the absolute value of the neural network parameter value of the first reconstruction layer is not greater than the predetermined value.Respectively, for an apparatus for decoding neural network parameters 13 according to an embodiment, the set of probability contexts may include two probability context sets, and the apparatus may be configured to select a first probability context set from the set of probability context sets as a selected probability context set if the neural network parameter value of the first reconstruction layer is greater than a predetermined value, for example X, and to select a second probability context set from the set of probability context sets as a selected probability context set if the neural network parameter value of the first reconstruction layer is not greater than the predetermined value, or to select a first probability context set from the set of probability context sets as a selected probability context set if the absolute value of the neural network parameter value of the first reconstruction layer is greater than a predetermined value, and to select a second probability context set from the set of probability context sets as a selected probability context set if the absolute value of the neural network parameter value of the first reconstruction layer is not greater than the predetermined value.

[0138] 4 Neural network parameter coding with dependent scalar quantization In this section, further optional aspects and features to the concepts and embodiments according to the present invention as described in the context of FIGS. 2-4 are disclosed.

[0139] A modified concept of neural network parameter coding is described. The main change to the previously described neural network parameter coding is that the neural network parameters 13 are not quantized and reconstructed independently. Instead, the allowed reconstruction levels of the neural network parameters 13 correspond to the selected quantization indexes 56 of the preceding neural network parameters in the reconstruction order. The concept of dependent scalar quantization is combined with a modified entropy coding in which the probability model selection (or alternatively the codeword table selection) for the neural network parameters depends on the allowed reconstruction level set. However, it should be noted that the previously described embodiment can use and / or incorporate and / or extend any of the features described below, separately or in combination.

[0140] 4.1 Advantages over related neural network parameter coding The advantage of dependent quantization of neural network parameters is that the allowed reconstruction vectors are densely packed in an N-dimensional signal space (where N represents the set of samples to be processed, e.g., the number of samples or neural network parameters 13 in layers 10a, 10b). The reconstruction vector of a neural network parameter set refers to the ordered reconstructed neural network parameters (or, alternatively, the ordered reconstructed samples) of the neural network parameter set. The effect of dependent scalar quantization is illustrated in FIG. 8 for the simplest case of two neural network parameters. FIG. 8 shows an example of the positions of allowed reconstruction vectors for the simple case of two weight parameters. FIG. 8(a) is an example of independent scalar quantization, and FIG. 8(b) is an example of dependent scalar quantization. FIG. 8a shows the allowed reconstruction vectors 201 (representing points on a two-dimensional plane) for independent scalar quantization. As can be seen, the set of allowed values ​​for the second neural network parameter t1′13 does not depend on the selected value for the first reconstructed neural network parameter t0′13. FIG. 8(b) shows an example of dependent scalar quantization. Note that in contrast to independent scalar quantization, the selectable reconstruction values ​​for the second neural network parameter t1'13 depend on the reconstruction level selected for the first neural network parameter t0'13. In the example of FIG. 8b, there are two different sets 52 of available reconstruction levels for the second neural network parameter t1'13 (illustrated in different colors). If the quantization index 56 for the first neural network parameter t0'13 is even (..., -2, 0, 2,...), any reconstruction level 201a of the first set (blue dots) can be selected for the second neural network parameter t1'13.Then, if the quantization index 56 for the first neural network parameter t0' is an odd number (..., -3, -1, 1, 3, ...), any reconstruction level 201b of the second set (red points) can be selected for the second neural network parameter t1' 13. In this example, the reconstruction levels of the first and second sets are shifted by half the quantization step size (any reconstruction level of the second set is located between two reconstruction levels of the first set).

[0141] Dependent scalar quantization of neural network parameters 13 has the effect that for a given average number of reconstructed vectors 201 per N-dimensional unit volume, the expected value of the distance between a given input vector of neural network parameters 13 and the closest available reconstructed vector is reduced. As a result, the average distortion between the input vector of neural network parameters and the vector reconstructed neural network parameters can be reduced for a given average number of bits. In vector quantization, this effect is called space-filling gain. Using dependent scalar quantization for neural network parameter sets 13, most of the potential space-filling gain of high-dimensional vector quantization can be exploited. And, in contrast to vector quantization, the implementation complexity of the reconstruction process (or decoding process) is comparable to that of the related neural network parameter encoding with independent scalar quantizers.

[0142] 4.2 Overview The main change is the dependency quantization, as mentioned before. The reconstructed neural network parameters t k '13 is the associated quantization index q k 56, as well as the quantization indices q0, q1, . . . , q k-1Note that in dependent quantization, the order of reconstruction of the neural network parameters 13 must be uniquely determined. The quantization index q k The performance of the overall neural network codec can typically be improved if knowledge of the reconstruction level set associated with 56 is also utilized in the entropy coding. That is, it is typically preferable to switch contexts (probability models) or codeword books based on the reconstruction level set applied to the neural network parameters.

[0143] Entropy coding is usually uniquely specified given the entropy decoding process, but as with the related neural network parameter coding, there is a lot of freedom in the choice of quantization indexes given the original neural network parameters.

[0144] The embodiments described herein are not limited to layer-wise neural network coding, but are also applicable to neural network parameter coding of any finite set of neural network parameters 13.

[0145] In particular, the method can also be applied to sub-layers as described in sec. 3.1.

[0146] 4.3 Dependent quantization of neural network parameters Dependent quantization of neural network parameters 13 refers to the notion that the available reconstruction level sets for a neural network parameter 13 depend on the quantization index chosen for the preceding neural network parameter in the reconstruction order (e.g., within the same set of neural network parameters, such as a layer or sublayer).

[0147] In a preferred embodiment, a plurality of sets of reconstruction levels are predefined, and one of the predefined sets is selected for reconstructing the current neural network parameters based on the quantization index of the preceding neural network parameters in the coding order. In other words, the device according to the embodiment may be configured to select 54 a reconstruction level set 48 out of a plurality 50 of reconstruction level sets 52 for the current neural network parameters 13 depending on the quantization index (58) for the previous, e.g. preceding, neural network parameters.

[0148] A preferred embodiment for defining the set of reconstruction levels is described in Section 4.3.1. Identification and signaling of the selected reconstruction level is described in Section 4.3.2. Section 4.3.3 describes a preferred embodiment for selecting one of a predefined set of reconstruction levels for the current neural network parameter (based on the selected quantization index of the preceding neural network parameter in the reconstruction order).

[0149] 4.3.1 Reconstruction level set In a preferred embodiment, the set of allowed reconstruction levels for the current neural network parameters is selected (based on the quantization index of the preceding neural network parameter in the coding order) from among a collection of predefined sets 52 of reconstruction levels (two or more sets, e.g., Set 0 and Set 1 from Figures 2 and 3).

[0150] In a preferred embodiment, the parameter determines the quantization step size Δ(QP), and all (in all sets of reconstruction levels) reconstruction levels represent integer multiples of the quantization step size Δ. However, it should be noted that each set of reconstruction levels contains only a subset of integer multiples of the quantization step size Δ(QP). Such a configuration for dependent quantization, where all possible reconstruction levels for all sets of reconstruction levels represent integer multiples of the quantization step size (QP), can be considered as an extension of uniform reconstruction quantizers (URQs). Its basic advantage is that the reconstructed neural network parameters 13 can be calculated by an algorithm with very low computational complexity (described in more detail below).

[0151] The reconstructed level sets can be completely discontinuous, but it is also possible for one or more reconstructed levels to be included in multiple sets (although the sets may still differ at other reconstructed levels).

[0152] In a preferred embodiment, the dependent scalar quantization for the neural network parameters uses just two different sets of reconstruction levels, e.g., set 0 and set 1. And in a particularly preferred embodiment, the dependent scalar quantization for the neural network parameters t k All the reconstruction levels of the two sets for 13 are calculated by the quantization step size Δ k (QP). Note that the quantization step size Δ k Note that (QP) simply represents a scaling factor for both sets of acceptable reconstruction values. The same two sets of reconstruction levels are used for all neural network parameters 13.

[0153] In FIG. 9, three preferred configurations ((a)-(c)) for two sets of reconstruction levels (set 0 and set 1) are illustrated. FIG. 9 shows an example of dependent quantization with two sets of reconstruction levels completely determined by a single quantization step size Δ(QP). The two available sets of reconstruction levels are highlighted in different colors (set 0 in blue and set 1 in red). Examples of quantization indices indicating the reconstruction levels in the set are shown by numbers below the circles. The hollow and filled circles indicate two different subsets in the reconstruction level set, which can be used to determine the reconstruction level set of the next neural network parameter in the reconstruction order. The figure shows three preferred configurations with two reconstruction level sets: (a) the two sets are disjoint and symmetric about zero, (b) both sets contain a reconstruction level equal to zero but are otherwise disjoint, and the sets are asymmetric around zero, and (c) both sets contain a reconstruction level equal to zero but are otherwise disjoint, and both sets are symmetric around zero. Note that all reconstruction levels lie on a grid given by an integer multiple (IV) of the quantization step size Δ. Furthermore, it should be noted that a particular reconstruction level may be included in both sets.

[0154] The two sets depicted in FIG. 9(a) are discontinuous. Each integer multiple of the quantization step size Δ(QP) is included in only one of the sets. The first set (set 0) includes all even integer multiples (IV) of the quantization step size, while the second set (set 1) includes all odd integer multiples of the quantization step size. In both sets, the distance between two adjacent reconstruction levels is twice the quantization step size. These two sets are typically suitable for high-rate quantization, i.e., settings in which the variance of the neural network parameters is significantly larger than the quantization step size (QP). However, in parameter coding of neural networks, the quantizer usually operates in the low-rate domain. Typically, the absolute values ​​of many original neural network parameters 13 are closer to zero than any non-zero multiple of the quantization step size (QP). In that case, it is typically preferable for zero to be included in both quantization sets (reconstruction level sets).

[0155] The two quantization sets shown in FIG. 9(b) both contain zero. In set 0, the distance between the reconstructed level equal to zero and the first reconstructed level greater than zero is equal to the quantization step size (QP), while all other distances between two adjacent reconstructed levels are equal to twice the quantization step size. Similarly, in set 1, the distance between the reconstructed level equal to zero and the first reconstructed level less than zero is equal to the quantization step size, while all other distances between two adjacent reconstructed levels are equal to twice the quantization step size. Note that both reconstructed sets are asymmetric around zero. This can lead to inefficiencies, as it becomes difficult to accurately estimate the probability of the code.

[0156] A preferred arrangement of two sets of reconstruction levels is shown in Fig. 9(c). The reconstruction levels in the first quantization set (labeled as set 0 in the figure) represent even integer multiples of the quantization step size (note that this set is actually the same as set 0 in Fig. 9(a)). The second quantization set (labeled as set 1 in the figure) contains all odd integer multiples of the quantization step size and also has reconstruction levels equal to zero. Note that both reconstruction sets are symmetric about zero. Reconstruction levels equal to zero are included in both reconstruction sets, otherwise the reconstruction sets are discontinuous. The combination of both reconstruction sets contains all integer multiples of the quantization step size.

[0157] In other words, according to an embodiment, for example, an apparatus for encoding / decoding neural network parameters 13 is provided, in which the number of the plurality 50 of reconstruction level sets 52 is two (e.g., set 0, set 1), and the plurality of reconstruction level sets includes a first reconstruction level set (set 0) that includes zero and an even multiple of a predetermined quantization step size, and a second reconstruction level set (set 1) that includes zero and an odd multiple of the predetermined quantization step size.

[0158] Furthermore, all reconstruction levels of all reconstruction level sets may represent integer multiples (IV) of a predetermined quantization step size (QP), and an apparatus for decoding, for example, neural network parameters 13 according to an embodiment may be configured to inverse quantize the neural network parameters 13 by deriving, for each neural network parameter, an intermediate integer value, for example an integer multiple (IV) depending on the reconstruction level set selected for the respective neural network parameter and the entropy decoding quantization index 58 for the respective neural network parameter 13', and by multiplying, for each neural network parameter 13, the intermediate value of the respective neural network parameter by a predetermined quantization step size for the respective neural network parameter 13.

[0159] Respectively, all reconstruction levels of all reconstruction level sets may represent integer multiples (IV) of a predefined quantization step size (QP), and the apparatus, for example according to an embodiment, may be configured to quantize the neural network parameters 13 such that the same can be inverse quantized by deriving, for each neural network parameter, an intermediate integer value depending on the selected reconstruction level set of the respective neural network parameter and the entropy coding quantization index of the respective neural network parameter, and by multiplying, for each neural network parameter, the intermediate value of the respective neural network parameter by the predefined quantization step size of the respective neural network parameter.

[0160] The embodiments defined herein are not limited to the configuration shown in FIG. 9. Any other two different sets of reconstruction levels can be used. Multiple reconstruction levels may be included in both sets. Alternatively, the combination of both quantization sets may not include all possible integer multiples of the quantization step size. Furthermore, it is possible to use more than two sets of reconstruction levels for dependent scalar quantization of neural network parameters.

[0161] 4.3.2 Signaling the selected reconfiguration level The reconstruction level that the encoder selects from among the allowed reconstruction levels must be indicated inside the bitstream 14. As with conventional independent scalar quantization, this can be achieved by using so-called quantization indexes 56, also called weighting levels. The quantization index 56 (or weighting level) is an integer value that uniquely identifies an available reconstruction level inside the quantization set 52 (i.e., inside the set of reconstruction levels). The quantization index 56 is sent to the decoder as part of the bitstream 14 (using any entropy coding technique). At the decoder side, the reconstructed neural network parameters 13 can be uniquely calculated based on the current set of reconstruction levels 48 (which is determined by the preceding quantization index in the coding / reconstruction order) and the transmitted quantization index 56 for the current neural network parameters 13'.

[0162] In a preferred embodiment, the assignment of quantization indexes 56 to the reconstruction levels within a set of reconstruction levels (or quantization set) follows the following rules: For illustrative purposes, the reconstruction levels in FIG. 9 are labeled with associated quantization indexes 56 (the quantization indexes are given by the numbers under the circles representing the reconstruction levels). If the reconstruction level set includes a reconstruction level equal to 0, then a quantization index equal to 0 is assigned to the reconstruction level equal to 0. A quantization index equal to 1 is assigned to the smallest reconstruction level greater than 0, a quantization index equal to 2 is assigned to the next reconstruction level greater than 0 (i.e., the second smallest reconstruction level greater than 0), and so on. Or, in other words, the reconstruction levels greater than 0 are labeled with integer values ​​greater than 0 (i.e., 1, 2, 3, etc.) in ascending order of their value. Similarly, a quantization index −1 is assigned to the largest reconstruction level less than 0, a quantization index −2 is assigned to the next (i.e., the second largest) reconstruction level less than 0, and so on. Or, in other words, reconstruction levels less than 0 are labeled with integer values ​​less than 0 in descending order of value (i.e., −1, −2, −3, etc.). The example in Figure 9 illustrates the described quantization index assignments for all quantization sets except for set 1 in Figure 9(a) (which does not contain a reconstruction level equal to 0).

[0163] For quantization sets that do not contain a reconstruction level equal to 0, one way of assigning quantization indexes 56 to reconstruction levels is as follows: all reconstruction levels greater than 0 are assigned a quantization index greater than 0 (in ascending order of value), and all reconstruction levels less than 0 are assigned a quantization index less than 0 (in descending order of value). Thus, the assignment of quantization indexes 56 basically follows the same concept as for quantization sets that contain a reconstruction level equal to 0, except that there are no quantization indexes equal to 0 (see the label of quantization set 1 in FIG. 9(a)). The entropy coding of the quantization indexes 56 must take that into account. For example, the quantization index 56 is often transmitted by coding its absolute value (ranging from 0 to the maximum supported value) and, for absolute values ​​that are not equal to 0, additionally coding the sign of the quantization index 56. In the case that there is no quantization index 56 equal to 0, the entropy coding can be modified so that the absolute level minus 1 is transmitted (the value of the corresponding syntax element ranges from 0 to the maximum supported value) and a sign is always transmitted. Alternatively, the assignment rule for assigning the quantization indexes 56 to the reconstruction levels can be modified. For example, one of the reconstruction levels close to 0 can be labeled with a quantization index equal to 0. The remaining reconstruction levels are then labeled according to the following rule: Reconstructed levels greater than the reconstruction level with a quantization index equal to 0 are assigned a quantization index greater than 0 (the quantization index increases with the value of the reconstruction level); and quantization indexes less than 0 are assigned to reconstruction levels less than the reconstruction level with a quantization index equal to 0 (the quantization index decreases with the value of the reconstruction level). One possibility for such an assignment is shown in FIG. 9(a) by the numbers in parentheses (if no numbers in parentheses are given, the other numbers apply).

[0164] As mentioned above, in a preferred embodiment, two different sets of reconstruction levels (also called quantization sets) are used, and the reconstruction levels within both sets represent integer multiples of the quantization step size (QP), including cases where the quantization step size is changed on a layer basis (e.g., by sending layer quantization parameters within the bitstream 14) or for another finite set (e.g., block) of neural network parameters 13 (e.g., by sending block quantization parameters within the bitstream 14).

[0165] The use of reconstruction levels representing integer multiples of the quantization step size (QP) allows a computationally less complex algorithm for the reconstruction of the neural network parameters 13 at the decoder side, which is explained below based on the preferred example of FIG. 9(c) (similar simple algorithms exist for other configurations, in particular for the settings shown in FIG. 9(a) and FIG. 9(b)). In the configuration shown in FIG. 9(c), the first quantization set includes all even integer multiples of the quantization step size (QP) and the second quantization set includes all odd integer multiples of the quantization step size plus a reconstruction level equal to 0, which is included in both quantization sets. The reconstruction process of the neural network parameters can be implemented similarly to the algorithm specified in the pseudocode of FIG. 10. FIG. 10 shows an example of pseudocode illustrating a preferred example of the reconstruction process of the neural network parameters 13. k represents an index that specifies the order of reconstruction of the current neural network parameters 13′, the quantization index 56 of the current neural network parameters is represented by level[k] 210, and the quantization step size Δ k (QP) is represented by quant_step_size[k], and trec[k]220 is the parameter t of the reconstructed neural network. k' represents the value of setId[k]. The variable setId[k] 240 specifies the set of reconstruction levels to be applied to the current neural network parameters 13'. It is determined based on the neural network parameters preceding it in the reconstruction order; possible values ​​of setId[k] are 0 and 1. The variable n specifies the integer factor of the quantization step size (QP), e.g., the intermediate value IV; it is given by the set of reconstruction levels selected (i.e., the value of setId[k]) and the transmitted quantization index level[k].

[0166] TIFF2025016517000012.tif115154

[0167] When the second quantization set is used, the variable n is equal to twice the quantization index level[k] minus the sign function sign(level[k]) of the quantization index. In this case, it can be expressed by the reconstruction level of the second quantization set Set1 in FIG. 9(c), where Set1 includes all odd integer multiples of the quantization step size (QP).

[0168] Once the variable n (which specifies an integer multiple of the quantization step size) is determined, n is set to the quantization step size Δ k By multiplying by , the reconstructed neural network parameters t k ' is required.

[0169] In other words, the number of reconstruction level sets 52 among the plurality 50 of reconstruction level sets 52 may be two, and an apparatus for decoding and / or encoding, for example, neural network parameters 13 according to an embodiment of the present invention may be configured to derive intermediate values ​​of each neural network parameter as follows: If the selected reconstruction level set for each neural network parameter is the first set, then double the quantization index for each neural network parameter to obtain an intermediate value for each neural network parameter; and If the selected reconstruction level set for each neural network parameter is a second set and the quantization index of each neural network parameter is equal to zero, set the intermediate value of each sample to zero; and If the selected reconstruction level set for each neural network parameter is a second set and the quantization index for each neural network parameter is greater than zero, multiplying the quantization index for each neural network parameter by two and subtracting one from the multiplication result to obtain an intermediate value for each neural network parameter; and If the selected reconstruction level set for the current neural network parameters is the second set, and the quantization index for each neural network parameter is less than zero, the quantization index of each neural network parameter is doubled and one is added to the multiplication result to obtain an intermediate value of each neural network parameter.

[0170] 4.3.3 Dependent Reconfiguration of Neural Network Parameters In addition to the choice of the set of reconstruction levels discussed in Sections 4.3.1 and 4.3.2, another important design aspect of dependent scalar quantization in neural network parameter coding is the algorithm used to switch between the defined quantization sets (sets of reconstruction levels). The algorithm used determines the "packing density" that can be achieved in the N-dimensional space of neural network parameters13 (and therefore in the N-dimensional space of reconstructed samples as well). The higher the packing density, the better the coding efficiency will ultimately be.

[0171] A preferred method for determining the set of reconstruction levels for the next neural network parameters is based on the division of the quantization set, as shown in FIG. 11. FIG. 11 is a diagram illustrating an example for dividing a set of reconstruction levels into two subsets according to an embodiment of the present invention. The two quantization sets shown are the quantization sets of the preferred example of FIG. 9(c). The two subsets of quantization set 0 are labeled with "A" and "B", and the two subsets of quantization set 1 are labeled with "C" and "D". Note that the quantization set shown in FIG. 11 is the same quantization set as the quantization set of FIG. 9(c). Each of the two (or more) quantization sets is divided into two subsets. In the preferred example of FIG. 11, the first quantization set (denoted as set 0) is divided into two subsets (denoted as A and B), and the second quantization set (denoted as set 1) is also divided into two subsets (denoted as C and D). Although not the only possibility, the division for each quantization set is preferably made so that directly adjacent reconstruction levels (and therefore adjacent quantization indexes) are associated with different subsets. In a preferred embodiment, each quantization set is divided into two subsets. In Fig. 9, the division of the quantization set into subsets is indicated by hollow and filled circles.

[0172] In a particularly preferred embodiment shown in FIG. 11 and FIG. 9(c), the following splitting rules are applied: Subset A consists of all even quantization indices of quantization set 0. Subset B consists of all odd quantization indices of quantization set 0. Subset C consists of all even quantization indices of quantization set 1. Subset D consists of all odd quantization indices of quantization set 1.

[0173] Note that the subset used is typically not explicitly indicated inside the bitstream 14. Instead, it can be derived based on the quantization set used (e.g., set 0 or set 1) and the quantization indexes 56 actually transmitted. For the preferred partitioning shown in FIG. 11, the subsets can be derived by a bitwise "and" operation of the transmitted quantization index levels and 1. Subset A consists of all quantization indexes in set 0 with (level&1) equal to 0, subset B consists of all quantization indexes in set 0 with (level&1) equal to 1, subset C consists of all quantization indexes in set 1 with (level&1) equal to 0, and subset D consists of all quantization indexes in set 1 with (level&1) equal to 1.

[0174] In a preferred embodiment, the quantization set (set of allowed reconstruction levels) used to reconstruct the current neural network parameters 13' is determined based on the subset associated with the last two or more quantization indexes 56. An example in which the last two subsets (which are given by the last two quantization indexes) are used is shown in Table 1. The determination of the quantization set specified by this table represents a preferred embodiment. In other embodiments, the quantization set for the current neural network parameters 13' is determined by the subset associated with the last three or more quantization indexes 56. For the first neural network parameter (or subset of neural network parameters) of a layer, there is no data on the subset of the preceding neural network parameters (because there is no preceding neural network parameter). In a preferred embodiment, a predefined value is used in such cases. In a particularly preferred embodiment, a subset A is inferred for all neural network parameters that are not available. That is, when reconstructing the first neural network parameter, we infer the two preceding subsets as "AA" (or "AAA" if we consider the three preceding neural network parameters), and therefore use quantization set 0 according to table 1. And for the second neural network parameter, we determine the subset of the previous quantization index by its value (for the first neural network parameter, set 0 is used, so the subset is either A or B), but for the second last quantization index (which does not exist) we infer the subset to be equal to A. Of course, any other rule can be used for inferring the default value for the non-existent quantization index. It is also possible to use other syntax elements to derive the default subset for the non-existent quantization index. As a further alternative, it is also possible to use the last quantization index 56 of the preceding set of neural network parameters 13 for initialization.

[0175] Table 1: Example for determining the quantization set (set of available reconstruction levels) to be used for the next neural network parameters based on the subsets associated with the two last quantization indexes according to an embodiment of the present invention. The subsets are shown in the left table column and they are uniquely determined by the quantization set to be used (for the two last quantization indexes) and the so-called path (which may be determined by the parity of the quantization indexes). The quantization set and, in brackets, the path for the subset are listed in the second column from the left. The third column specifies the associated quantization set. The last column shows the values ​​of the so-called state variables, which can be used to simplify the process for determining the quantization set.

[0176] [Table 1]

[0177] It should be noted that the subset (A, B, C or D) of the quantization index 56 is determined by the quantization set to be used (set 0 or set 1) and the subset to be used within the quantization set (e.g. A or B for set 0, C or D for set 1). The selected subset within the quantization set is also called a path (to specify a path when the dependent quantization process is represented by a trellis structure, as will be described later). In our convention, a path is equal to either 0 or 1. Then, subset A corresponds to path 0 in set 0, subset B corresponds to path 1 in set 0, subset C corresponds to path 0 in set 1, and subset D corresponds to path 1 in set 1. Thus, the quantization set of the next neural network parameter is also uniquely determined by the quantization set (set 0 or set 1) and path (path 0 or path 1) associated with the last two (or more) quantization indexes. In Table 1, the relevant quantization sets and paths are specified in the second column.

[0178] TIFF2025016517000014.tif50158

[0179] In other words, the number of reconstruction level sets 52 among the plurality 50 reconstruction level sets 52 may be two, for example set 0 and set 1, and an apparatus for decoding, for example, neural network parameters 13 according to an embodiment of the present invention may be configured to derive a subset index for each neural network parameter based on the selected set of reconstruction levels for each neural network parameter and a binary function of the quantization index for each neural network parameter, resulting in four possible values, for example A, B, C, or D for the subset index, and to select a set of reconstruction levels 48 among the plurality 50 reconstruction level sets 52 for a current neural network parameter 13' depending on the subset index for a previously decoded neural network parameter.

[0180] A further embodiment according to the present invention comprises an apparatus configured to select 54 a reconstructed level set 48 from a plurality of 50 reconstructed level sets 5) using a selection rule corresponding to a subset index of a number of previously decoded neural network parameters for a current neural network parameter 13', e.g., as shown in the first column of Table 1, and to use the selection rule for all or a portion of the neural network parameters.

[0181] According to a further embodiment, the number of previously decoded neural network parameters to which the selection rule corresponds is two, eg, a subset of the two last quantization indexes, as shown in Table 1.

[0182] According to an additional embodiment, the subset index for each neural network parameter is derived based on the parity of the selected set of reconstruction levels for the respective neural network parameter and the quantization index for the respective neural network parameter, e.g., with path=(level[k]&1).

[0183] Respectively, for an apparatus for encoding neural network parameters 13 according to an embodiment, the number of reconstruction level sets 52 of the plurality 50 reconstruction level sets 52 may be two, for example set 0 and set 1, and the apparatus may be configured to derive a subset index for each neural network parameter based on the selected set of reconstruction levels for each neural network parameter and a binary function of the quantization index for each neural network parameter, resulting in four possible values ​​for the subset index, for example A, B, C, D, and to select 54 a reconstruction level set 48 out of the plurality 50 reconstruction level sets 52 for a current neural network parameter 13' depending on the subset index for a previously encoded neural network parameter.

[0184] A further embodiment according to the present invention comprises an apparatus configured to select 54 a set 48 of reconstruction levels from a plurality 50 of reconstruction level sets 52 using a selection rule corresponding to a subset index of a number of previously encoded neural network parameters for a current neural network parameter 13′, e.g., as shown in the first column of Table 1, and to use the selection rule for all or a portion of the neural network parameters.

[0185] According to a further embodiment, the number of previously coded neural network parameters to which the selection rule corresponds is two, eg, a subset of the two last quantization indexes, as shown in Table 1.

[0186] According to an additional embodiment, the subset index for each neural network parameter is derived based on the selected set of reconstruction levels for the respective neural network parameter and the parity of the quantization index of the respective neural network parameter, e.g., path=(level[k]&1).

[0187] The transitions between the quantization sets 52 (sets 0 and 1) can also be elegantly represented by state variables. An example of such a state variable is shown in the last column of Table 1. In this example, the state variable has four possible values ​​(0, 1, 2, 3). The state variable, on the other hand, specifies the quantization set to be used for the current neural network parameters 13'. In the preferred example of Table 1, quantization set 0 is used only if the state variable is equal to 0 or 2, and quantization set 1 is used only if the state variable is equal to 1 or 3. On the other hand, the state variable also specifies the possible transitions between the quantization sets. By using state variables, the rules of Table 1 can be described by smaller state transition tables. As an example, Table 2 specifies a state transition table for the rules given in Table 1. This represents a preferred embodiment. Given the current state, we specified a quantization set for the parameters of the current neural network (second column). It further specifies the state transitions based on the path associated with the selected quantization index 56 (the path specifies the subset A, B, C, or D to be used, given a quantization set). Note that by using the concept of state variables, it is not necessary to actually keep track of the selected subset: when reconstructing the parameters of a layer of the neural network, it is enough to update the state variables and determine the path of quantization indices to be used.

[0188] Table 2: A preferred example of a state transition table for a configuration having four states according to an embodiment of the present invention. [Table 2]

[0189] That is, for example, an apparatus for decoding neural network parameters according to an embodiment may be configured to select 54 a quantization level set 48 from the plurality of 50 reconstructed level sets 52 for a current neural network parameter 13' by a state transition process, by determining a set of quantization levels 48 from the plurality of 50 reconstructed level sets 52 depending on a state associated with the current neural network parameter 13', and by updating the state of the subsequent neural network parameter depending on a quantization index 58 decoded from a data stream of the immediately preceding neural network parameter.

[0190] Respectively, for an apparatus for encoding neural network parameters 13 according to an embodiment, said apparatus may be configured to select 54 a set 48 of reconstruction levels 48 of the plurality 50 of reconstruction level sets 52 for a current neural network parameter 13' by a state transition process, by determining a set 48 of reconstruction levels 48 of the plurality 50 of reconstruction level sets 52 depending on a state associated with the current neural network parameter 13', and by updating a state for a subsequent neural network parameter depending on a quantization index 58 encoded in the data stream of the immediately preceding neural network parameter.

[0191] TIFF2025016517000016.tif46160

[0192] In other words, an apparatus according to an embodiment, for example for decoding neural network parameters, can be configured to update the state of a subsequent neural network parameter using a binary function of quantization indexes 58 decoded from a data stream for a previous neural network parameter, for example according to Table 2.

[0193] Further, the apparatus according to the embodiment may be configured to update the state for a subsequent neural network parameter using the parity of the quantization index 58 decoded from the data stream 14 for the previous neural network parameter, e.g., path=(level[k]&1).

[0194] Respectively, for an apparatus for encoding neural network parameters 13 according to an embodiment, the apparatus may be configured to update the state for a subsequent neural network parameter using a binary function of the quantization indexes 58 encoded in the data stream for the immediately preceding neural network parameter.

[0195] Further, an apparatus for encoding, for example, neural network parameters 13 according to an embodiment may be configured to use the parity of the quantization indexes 58 encoded in the data stream for the immediately preceding neural network parameter to update the state for the subsequent neural network parameter, for example according to Table 2.

[0196] In a preferred embodiment, state variables with four possible values ​​are used. In other embodiments, state variables with a different number of possible values ​​are used. Of particular interest are state variables whose number of possible values ​​of a state variable represents an integer power of two, i.e., 4, 8, 16, 32, 64, etc. It should be noted that in the preferred configuration (as given in Tables 1 and 2), state variables with four possible values ​​are equivalent to an approach in which the current quantization set is determined by a subset of the two last quantization indexes. State variables with eight possible values ​​correspond to a similar approach in which the current quantization set is determined by a subset of the three last quantization indexes. State variables with 16 possible values ​​correspond to an approach in which the current quantization set is determined by a subset of the last four quantization indexes. Although it is generally preferred to use state variables with a number of possible values ​​equal to an integer power of two, the embodiments are not limited to this setting.

[0197] In a particularly preferred embodiment, a state variable is used that has eight possible values: 0, 1, 2, 3, 4, 5, 6, 7. In the preferred example Table 3, quantization set 0 is used only if the state variable is equal to 0, 2, 4 or 6, and quantization set 1 is used only if the state variable is equal to 1, 3, 5 or 7.

[0198] 1 is a preferred example of a state transition table for a configuration having eight states, according to an embodiment. [Table 3]

[0199] That is, in embodiments of the present invention, the state transition process is configured to transition between either four or eight possible states.

[0200] Further, the apparatus for decoding / encoding neural network parameters 13 according to the embodiment can be configured to transition between an even number of possible states in the state transition process, and the number of the plurality of 50 reconstruction level sets 52 is two, where for the current neural network parameters 13', a set of quantization levels 48 from the quantization set 52 is determined according to the state associated with the current neural network parameters 13', such that if the state belongs to the first half of the even number of possible states, a first reconstruction level set from the plurality of 50 reconstruction level sets 52 is determined, and if the state belongs to the second half of the even number of possible states, a second reconstruction level set from the plurality of 50 reconstruction level sets 52 is determined.

[0201] According to a further embodiment, for example an apparatus for decoding neural network parameters 13 may be configured to perform state updates by means of a transition table that maps a combination of state and parity of a quantization index 58 decoded from the data stream of a previous neural network parameter onto another state associated with a subsequent neural network parameter.

[0202] Accordingly, an apparatus for encoding neural network parameters 13 according to an embodiment may be configured to perform state updates via a transition table that maps a combination of state and parity of the quantization index 58 encoded in the data stream for a previous neural network parameter to another state associated with a subsequent neural network parameter.

[0203] Using the state transition concept, the current state, and therefore the current quantization set, is uniquely determined by the previous state and previous quantization index 56 (in the reconstruction order). However, for the first neural network parameter 13 of a finite set (e.g. a layer), there is no previous state and no previous quantization index. It is therefore required that the state for the first neural network parameter of a layer is uniquely defined. There are various possibilities. The preferred options are: The first state for a layer is always set equal to a fixed predefined value. In the preferred embodiment, the first state is set equal to 0. The value of the first state is explicitly transmitted as part of the bitstream 14. This includes an approach in which only a subset of the possible state values ​​may be indicated by the corresponding syntax element. The value of the first state is derived based on other syntax elements of the layer, meaning that even if the corresponding syntax element (or syntax elements) are used to signal other aspects to the decoder, they are additionally used to derive the first state for the dependent scalar quantization.

[0204] The concept of state transition of dependent scalar quantization allows for a low-complexity implementation for the reconstruction of neural network parameters 13 in the decoder. A preferred example of a single-layer neural network parameter reconstruction process is shown in FIG. 12 using pseudocode in C language format. FIG. 12 shows an example of pseudocode showing a preferred example for the reconstruction process of neural network parameters 13 of a layer according to an embodiment of the present invention. Note that the derivation of the quantization index and the derivation of the reconstructed values, for example using the quantization step size or alternatively using a codebook, may be performed in separate loops, one after the other. That is, in other words, the derivation of "n" and state update can be performed in a first loop, and the derivation of "trec" can be performed in another second loop. The array level 210 represents the transmitted neural network parameter levels (quantization indexes 56) of the layer, and the array trec 220 represents the corresponding reconstructed neural network parameters 13. The quantization step size Δk (QP) applied to the current neural network parameters 13′ is denoted by quant_step_size[k]. 2d table sttab 230 specifies a state transition table, e.g., according to any of Table 1, Table 2 and / or Table 3, and table setId 240 specifies a quantization set associated with state 250.

[0205] In the pseudocode of FIG. 12, the index k specifies the reconstruction order of the neural network parameters. The last index layerSize specifies the reconstruction index of the last reconstructed neural network parameter. The variable layerSize may be set equal to the number of neural network parameters in the layer. The reconstruction process of each single neural network parameter is the same as in the example of FIG. 10. For the example of FIG. 10, the quantization index is represented by level[k] 210, and the associated reconstructed neural network parameter is represented by trec[k] 220. Also, the state variable is represented by state 210. Note that in the example of FIG. 12, the state is set equal to 0 at the beginning of the layer. However, as mentioned above, other initializations (e.g., based on the values ​​of some syntax elements) are also possible. The 1d table setId[] 240 specifies the quantization sets associated with different values ​​of the state variables, and the 2d table sttab[][] 230 specifies the state transitions given the current state (first argument) and a path (second argument). In this example, the path is given by the parity of the quantization index (bitwise and using the & operator), but other notions are possible. In a C-style syntax, example tables are shown in Figures 13 and 14 (these tables are identical to Tables 2 and 3, in other words, they can provide representations of Tables 2 and 3).

[0206] 13 is a diagram showing a preferred example of a state transition table sttab 230 and a table setId 240 that specifies the quantization set associated with a state 250 according to an embodiment of the present invention. The table, given in C-style syntax, represents the table defined in Table 2.

[0207] 14 is a diagram showing a preferred example of a state transition table sttab 230 and a table setId 240 that specifies the quantization set associated with a state 250 according to an embodiment of the present invention. The table, given in C-style syntax, represents the table defined in Table 3.

[0208] In another embodiment, all quantization indices 56 equal to zero are excluded from the state transitions and dependent reconstruction process. The information of whether a quantization index 56 is equal to zero or not is only used to split the neural network parameters 13 into zero and non-zero neural network parameters. The reconstruction process for dependent scalar quantization is only applied to the ordered set of non-zero quantization indices 56. All neural network parameters associated with quantization indices equal to zero are simply set equal to zero. The corresponding pseudocode is shown in FIG. 15. FIG. 15 shows pseudocode illustrating an alternative reconstruction process of neural network parameter levels in which quantization indices equal to zero are excluded from state transitions and dependent scalar quantization according to an embodiment of the present invention.

[0209] State transitions in dependent quantization can also be represented using a trellis structure, as shown in FIG. 16. FIG. 16 is a diagram illustrating an example of state transitions in dependent scalar quantization as a trellis structure according to an embodiment of the present invention. The horizontal axis represents different neural network parameters 13 in the reconstruction order. The vertical axis represents different possible states 250 in the dependent quantization and reconstruction process. The connections shown specify the available paths between states for different neural network parameters. The trellis shown in this figure corresponds to the state transitions specified in Table 2. For each state 250, there are two paths that connect the state for the current neural network parameter 13' with the two possible states for the next neural network parameter 13 in the reconstruction order. The paths are labeled Path 0 and Path 1, and the numbers correspond to the path variables introduced above (in the preferred embodiment, the path variables are equal to the parity of the quantization index). Note that each path uniquely specifies a subset (A, B, C, or D) for the quantization index. The subsets are specified in brackets in Figure 16. Given an initial state (eg state 0), the path through the trellis is uniquely specified by the transmitted quantization index 56.

[0210] In the example of Figure 16, state (0,1,2,3) has the following properties: State 0: The previous quantization index level[k-1] specifies a reconstruction level of set 0, and the current quantization index level[k] specifies a reconstruction level of set 0. State 1: The previous quantization index level[k-1] specifies the reconstruction levels of set 0, and the current quantization index level[k] specifies the reconstruction levels of set 1. State 2: The previous quantization index level[k-1] specifies the reconstruction levels of set 1, and the current quantization index level[k] specifies the reconstruction levels of set 0. · State 3: The previous quantization index level[k-1] specifies the reconstruction level of set 1, and the current quantization index level[k] specifies the reconstruction level of set 1.

[0211] A trellis consists of a concatenation of so-called basic trellis cells. An example of such a basic trellis cell is shown in FIG. 17. FIG. 17 is a diagram showing an example of a basic trellis cell according to an embodiment of the present invention. It should be noted that the present invention is not limited to a trellis having four states 250. In other embodiments, the trellis can have more states 250. In particular, any number of states representing an integer power of two is suitable. In a particularly preferred embodiment, the number of states 250 is equal to eight, for example as in Table 3. Even if the trellis has more than two states 250, each node for the current neural network parameters 13' is typically connected with two states for the previous neural network parameters 13 and two states for the next neural network parameters 13. However, it is also possible that a node is connected with more than two states of the previous neural network parameters or more than two states of the next neural network parameters. Note that a fully connected trellis (where each state 250 is connected to every state 250 of the previous neural network parameter 13 and every state 250 of the next neural network parameter 13) would correspond to an independent scalar quantization.

[0212] In a preferred embodiment, the initial state cannot be freely chosen (as some side information rate is required to transmit this decision to the decoder). Instead, the initial state is either set to a predefined value or its value is derived based on other syntax elements. In this case, not all paths and states 250 are available for the first neural network parameter. As an example of a four-state trellis, FIG. 18 shows a trellis structure when the initial state is equal to 0. FIG. 18 shows an example trellis for dependent scalar quantization of eight neural network parameters according to an embodiment of the present invention. The first state (left side) represents the initial state and is set equal to 0 in this example.

[0213] 4.4 Entropy Coding The quantization indexes obtained by the dependent quantization are coded using an entropy coding method. Any entropy coding method can be applied for this. In a preferred embodiment of the present invention, the entropy coding method according to section 2.2 (see section 2.2.1 for the coding method and section 2.2.2 for the decoding method) using Context-Adaptive Binary Arithmetic Coding (CABAC) is applied. For this purpose, the non-binary values ​​are first mapped onto a series of binary decisions (so-called bins) (binarization) in order to transmit the quantization indexes as absolute values, as shown for example in Fig. 5.

[0214] It should be noted that any of the concepts discussed here can be combined with the methodological and related concepts in Section 3, especially those related to context modelling.

[0215] 4.4.1 Context modeling for dependent scalar quantization The main aspect of dependent scalar quantization is that there are different sets of permissible reconstruction levels (also called quantization sets) for the neural network parameters 13. The quantization set for the current neural network parameters 13' is determined based on the value of the quantization index 56 for the preceding neural network parameters. Considering the preferred example of FIG. 11 and comparing the two quantization sets, it is clear that the distance between a reconstruction level equal to zero and an adjacent reconstruction level is larger in set 0 than in set 1. Thus, the probability that the quantization index 56 is equal to zero is larger when set 0 is used and smaller when set 1 is used. In the preferred embodiment, this effect is exploited in the entropy coding by switching the codeword table or probability model based on the quantization set (or state) used for the current quantization index.

[0216] It should be noted that for a suitable switching of the codeword table or probability model, the paths (associations with the subsets of the quantization set used) of all preceding quantization indexes must be known when entropy decoding the current quantization index (or the corresponding binary decision of the current quantization index). It is therefore necessary that the neural network parameters 13 are coded in a reconstruction order. Thus, in a preferred embodiment, the coding order of the neural network parameters 13 is equal to their reconstruction order. Besides that aspect, any coding / reconstruction order of the quantization indexes 56 is possible, e.g. the orders specified in section 2.2.1 are any other uniquely defined order.

[0217] In other words, embodiments according to the present invention include an apparatus for encoding neural network parameters using a probability model that additionally depends on the quantization indexes of previously encoded neural network parameters, for example.

[0218] Respectively, embodiments according to the present invention include, for example, an apparatus for decoding neural network parameters, using a probability model that additionally depends on the quantization indexes of previously decoded neural network parameters.

[0219] At least some of the bins for the absolute levels are typically coded using adaptive probability models (also called contexts). In a preferred embodiment of the present invention, the probability model for one or more bins is selected based on the quantization set of the corresponding neural network parameters (or, more generally, the corresponding state variables, e.g., a relationship according to any of Tables 1-3). The selected probability model may depend on multiple parameters or characteristics of the already transmitted quantization indexes 56, one of the parameters being the quantization set or state applied to the quantization indexes being coded.

[0220] In other words, an apparatus according to an embodiment, for example for encoding neural network parameters 13, can be configured to pre-select a subset of probability models from among a plurality of probability models depending on a state or set of reconstruction levels 48 selected for the current neural network parameters 13' and to select a probability model for the current neural network parameters from among the subset of probability models 121 depending on the quantization indexes of previously encoded neural network parameters.

[0221] An apparatus for decoding e.g. neural network parameters 13 according to an embodiment may be configured to pre-select a subset of probability models from among a plurality of probability models depending on a state or set of reconstruction levels 48 selected for the current neural network parameters 13′ and to select a probability model for the current neural network parameters from among the subset of probability models depending on a quantization index 121 of previously decoded neural network parameters.

[0222] In combination with the inventive concept as e.g. described in the context of FIG. 9, an embodiment according to the present invention, e.g. for encoding and / or decoding of neural network parameters 13, includes an apparatus configured to pre-select a subset of probability models from among a plurality of probability models depending on a state or set of reconstruction levels 48 selected for a current neural network parameters 13′, such that the pre-selected subset for a first state or reconstruction level set is disjoint from the pre-selected subsets for any other state or reconstruction level set.

[0223] In a particularly preferred embodiment, the syntax for transmitting the quantization index of a layer includes a bin, e.g., the above-mentioned sig_flag, that specifies whether the quantization index is equal to 0 or not. The probability model used to code this bin is selected from a set of two or more probability models. The choice of the probability model used depends on the quantization set (i.e., the set of reconstruction levels) applied to the corresponding quantization index 56. In another embodiment of the present invention, the probability model used depends on the current state variable (which means the quantization set used).

[0224] In a further embodiment, the syntax for transmitting the quantization index of a layer includes a bin, e.g., the aforementioned sign_flag, that specifies whether the quantization index is greater than or less than zero. That is, the bin indicates the sign of the quantization index. The choice of the probability model used depends on the quantization set (i.e., the set of reconstruction levels) that is applied to the corresponding quantization index. In another embodiment, the probability model used depends on the current state variable (which means the quantization set used).

[0225] In a further embodiment, the syntax for transmitting the quantization index includes a bin, e.g., abs_level_greater_X as mentioned above (see section 0 for details), that specifies whether the absolute value of the quantization index (parameter level of the neural network) is greater than X. The probability model used to code this bin is selected from a set of two or more probability models. The choice of the probability model used depends on the quantization set (i.e., the reconstruction level set) applied to the corresponding quantization index 56. In another embodiment, the probability model used depends on the current state variables (which state variables refer to the quantization set used).

[0226] One advantageous aspect of the embodiments discussed herein is that dependent quantization of the neural network parameters 13 is combined with entropy coding, such that the selection of a probability model for one or more bins (also called quantization levels) of the binary representation of the quantization indexes depends on a quantization set (a set of allowed reconstruction levels) or corresponding state variable for the current quantization index. The quantization set 52 (or state variable) is given by the quantization index 56 (or a subset of bins representing the quantization index) of the preceding neural network parameter in the coding and reconstruction order.

[0227] In a preferred embodiment, the described choice of probability model is combined with one or more of the following entropy coding aspects: The absolute values ​​of the quantization indexes are transmitted using a binary scheme consisting of a number of bins that are coded using an adaptive probability model, and a suffix part (non-adaptive probability model with pmf (e.g. probability mass function) (0.5, 0.5) for all bins) that is coded in bypass mode of the arithmetic coding engine if the adaptively coded bins do not already fully specify the absolute values. In a preferred embodiment, the binarization used for the suffix part depends on the values ​​of the quantization indexes already transmitted. · The binarization for the absolute value of the quantization index includes an adaptively coded bin that specifies if the quantization index is not equal to 0. The probability model (called context) used to code this bin is selected from a set of candidate probability models. The selected candidate probability model is determined not only by the quantization set (set of allowed reconstruction levels) or state variables for the current quantization index 56, but also in addition by the already transmitted quantization indexes for that layer. In a preferred embodiment, the quantization set (or state variables) determines a subset of the available probability models (also called context set), and the already coded values ​​of the quantization index determine the probability model used within this subset (context set).

[0228] In an embodiment, the probability model to be used within a context set is determined based on the values ​​of already coded quantization indexes in a local neighborhood of the current neural network parameters, e.g., a template as described in 2.2.3. In the following, we list some exemplary measures that can be derived based on the values ​​of quantization indexes in a local neighborhood and then used to select a probability model for a pre-determined context set. The sign of the quantization index that is not equal to 0 within a local neighborhood. The number of quantization indices in the local neighborhood that are not equal to 0. This number may be clipped (truncated) to a maximum value. The sum of the absolute values ​​of the quantization indices in a local neighborhood. This number may be clipped to a maximum value. The difference between the sum of the absolute values ​​of the quantization indices in a local neighborhood and the number of quantization indices in the local neighborhood that are not equal to 0. This number may be clipped to a maximum value.

[0229] In other words, embodiments according to the present invention include an apparatus for encoding neural network parameters configured to select a probability model for a current neural network parameter from a subset of probability models depending on characteristics of quantization indices of previously encoded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameter relates, the characteristics comprising one or more of the following: The signs of the non-zero quantization indices of previously coded neural network parameters that relate to portions of the neural network adjacent to the portion to which the current neural network parameters relate. the number of quantization indices of previously coded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameters are associated, the number being nonzero the sum of the absolute values ​​of the quantization indices of the previously coded neural network parameters that relate to the parts of the neural network adjacent to the part to which the current neural network parameters relate; the sum of the absolute values ​​of the quantization indices of previously coded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameter is associated; and a non-zero number of quantization indices of previously coded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameters are associated; and The difference between.

[0230] Respectively, embodiments according to the present invention include, for example, an apparatus for decoding neural network parameters, the apparatus being configured to select a probability model for a current neural network parameter from a subset of probability models depending on properties of quantization indices of previously decoded neural network parameters associated with portions of the neural network neighboring the portion to which the current neural network parameter is associated, the properties including one or more of the following: The signs of the non-zero quantization indices of previously decoded neural network parameters that relate to portions of the neural network adjacent to the portion to which the current neural network parameters relate. The number of quantization indices of previously decoded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameters are associated, the number being non-zero. the sum of the absolute values ​​of the quantization indices of previously decoded neural network parameters that relate to the parts of the neural network adjacent to the part to which the current neural network parameters relate; the sum of the absolute values ​​of the quantization indices of previously decoded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameter is associated; and a non-zero number of quantization indices of previously decoded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameters are associated; and The difference.

[0231] · The binarization for the absolute value of the quantization index includes adaptively coded bins, e.g. abs_level_greater_X, that specify whether the absolute value of the quantization index is greater than X. The probability model (called context) used to code these bins is selected from among a set of candidate probability models. The selected probability model is determined not only by the quantization set (set of allowed reconstruction levels) or state variables for the current quantization index, but also by the already transmitted quantization index for the layer, e.g. by means of a template as described above. In a preferred embodiment, the quantization set (or state variables) determines a subset of the available probability models (also called context set), and the data of the already coded quantization indexes can be used to determine, e.g. in other words, the probability model to be used within this subset (context set). The selection of the probability model can use any of the methods described above (for bins specifying whether the quantization index is not equal to 0).

[0232] Additionally, the apparatus according to the present invention may be configured to position the previously coded neural network parameters 13 such that they are associated with the same neural network layer as the current neural network parameters 13'.

[0233] Furthermore, an apparatus for encoding e.g. neural network parameters according to the present invention may be configured to position one or more of the previously encoded neural network parameters such that they are associated with a neuronal interconnection emerging from or leading to the neuron 10c to which the neuronal interconnection 11 to which the current neural network parameter refers is associated or another neuron adjacent to said neuron.

[0234] In a further embodiment, the apparatus can be configured to encode the quantization indexes 56 for the current neural network parameters 13′ into the data stream 14 using binary arithmetic coding by using a probability model corresponding to previously encoded neural network parameters for one or more leading bins of the binarized quantization indexes and by using equi-probable bypass mode suffix bins of the binarized quantization indexes following the one or more leading bins.

[0235] The suffix bins of the binarized quantization index represent the bins of the binarized code of the suffix binarization for binarizing the value of the quantization index, and may have an absolute value exceeding the maximum absolute value that can be represented by one or more leading bins. Thus, an apparatus according to an embodiment of the present invention may be configured to select the suffix binarization depending on the quantization index 56 of the previously coded neural network parameters 13.

[0236] Respectively, an apparatus according to the present invention, for example for decoding neural network parameters, may be configured to position previously decoded neural network parameters 13 such that they are associated with the same neural network layer as the current neural network parameters 13'.

[0237] According to a further embodiment, for example an apparatus for decoding neural network parameters according to the present invention can be configured to position one or more of the previously decoded neural network parameters 13 such that they are associated with a neuronal interconnection 11 emerging from, or leading to, the neuron 10c to which the neuronal interconnection referred to by the current neural network parameter is associated or another neuron adjacent to said neuron.

[0238] In a further embodiment, the apparatus can be configured to decode the quantization indexes 56 for the current neural network parameters 13' from the data stream 14 using binary arithmetic coding by using a probability model corresponding to previously decoded neural network parameters for one or more leading bins of the binarized quantization indexes, and by using equal probability bypass mode suffix bins of the binarized quantization indexes following the one or more leading bins.

[0239] The suffix bin of the binarization of the quantization index represents a bin of the binarization code of the suffix binarization for binarizing the value of the quantization index, and its absolute value exceeds the maximum absolute value representable by one or more leading bins. Thus, the device according to the embodiment can be configured to select the suffix binarization depending on the quantization index of the previously decoded neural network parameters.

[0240] TIFF2025016517000018.tif94156

[0241] TIFF2025016517000019.tif143153

[0242] In other words, embodiments according to the present invention include an apparatus configured to use the Viterbi algorithm and a rate-distortion cost metric to perform the selection and / or quantization.

[0243] An exemplary encoding algorithm for selecting appropriate quantization indexes for a layer can be composed of the following major steps: 1. Set the rate-distortion cost to 0 in the initial state. 2. For all neural network parameters 13 in the encoding order, do the following: For each subset A, B, C, D, determine the quantization index 56 that minimizes the distortion given the original neural network parameters 13. b. For all trellis nodes (0, 1, 2, 3) for the current neural network parameters 13', do the following: i Calculate the rate-distortion cost of the two paths connecting the previous state of the neural network parameters 13 and the current state. The cost is calculated by multiplying the cost for the previous state by the cost for D k +λ R k where D k and R k represents the distortion and rate for selecting the quantization indexes of the subset (A, B, C, D) associated with the considered connection. ii. Assign the minimum of the calculated cost to the current node and remove any connections to previous neural network parameter 13 states that do not represent the minimum cost path. NOTE: After this step, all nodes for the current neural network parameters 13' have a single connection to any node for the preceding neural network parameters 13. 3. Compare the costs of the four final nodes (for the last parameter in the coding order) and select the node with the smallest cost. Note that this node is associated with a unique path through the trellis (all other connections were pruned in the previous step). 4. Follow the selected path (specified by the end node) in reverse order and collect the quantization indices 56 associated with the connections between the trellis nodes.

[0244] It should be noted that the determination of the quantization indexes 56 based on the Viterbi algorithm is substantially less complex than rate-distortion optimized quantization (RDOQ) for independent scalar quantization. Nevertheless, simpler coding algorithms for dependent quantization also exist. For example, starting from a predefined initial state (or quantization set), the quantization indexes 56 can be determined in the coding / reconstruction order by minimizing an arbitrary cost amount that only considers the influence of the current quantization index. Given the determined quantization index (and all preceding quantization indexes) for the current parameter, the quantization set for the next neural network parameter 13 is known. And, therefore, this algorithm can be applied to all neural network parameters in coding order.

[0245] The method according to the following embodiment is illustrated in FIGS.

[0246] 20 is a block diagram of a method 400 for decoding neural network parameters defining a neural network from a data stream. The method 400 includes sequentially decoding the neural network parameters by selecting 54 for a current neural network parameter a reconstruction level set from a plurality of reconstruction level sets in response to a quantization index decoded from the data stream for a previous neural network parameter, by decoding a quantization index 420 of the current neural network parameter from the data stream, where the quantization index indicates one reconstruction level of the selected set of reconstruction levels for the current neural network parameter, and by dequantizing 62 the current neural network parameter onto one reconstruction level of the selected set of reconstruction levels indicated by the quantization index for the current neural network parameter.

[0247] 21 is a block diagram of a method 500 for encoding neural network parameters defining a neural network from a data stream. The method 500 includes successively encoding neural network parameters by selecting 54 for a current neural network parameter a reconstruction level set from among a plurality of reconstruction level sets in response to a quantization index encoded in the data stream for a previously encoded neural network parameter, by quantizing 64 the current neural network parameter to one reconstruction level from the selected set of reconstruction levels, and by encoding 530 a quantization index of the current neural network parameter into the data stream indicating one reconstruction level to which the quantization index of the current neural network parameter is quantized.

[0248] 22 is a block diagram of a method for reconstructing neural network parameters defining a neural network according to an embodiment of the present invention. The method 600 includes deriving first neural network parameters 610 for a first reconstruction layer to provide, for each neural network parameter, a neural network parameter value of the first reconstruction layer. The method 600 further includes decoding 620 (e.g., as shown by arrow 312 in FIG. 6) second neural network parameters of a second reconstruction layer from the data stream to provide, for each neural network parameter, a neural network parameter value of the second reconstruction layer, and reconstructing 630 (e.g., as shown by arrow 314 in FIG. 6) the neural network parameters by combining, for each neural network parameter, the neural network parameter value of the first reconstruction layer and the neural network parameter value of the second reconstruction layer.

[0249] 23 is a block diagram of a method for encoding neural network parameters defining a neural network according to an embodiment of the present invention. The method 700 includes encoding 710 a second neural network parameter of a second reconstruction layer into a data stream (e.g., as shown by arrow 322 in the figure) using a first neural network parameter of a first reconstruction layer including a neural network parameter value of a first reconstruction layer for each neural network parameter, the second reconstruction layer including a second reconstruction layer neural network parameter value for each neural network parameter, and the neural network parameter is reconstructable by combining the first reconstruction layer neural network parameter value and the second reconstruction layer neural network parameter value for each neural network parameter.

[0250] TIFF2025016517000020.tif236169

[0251] For example, the two-dimensional integer array StateTransTab[][] shown on line 1014 specifies the state transition table for dependent scalar quantization, and is as follows:

[0252] TIFF2025016517000021.tif225170

[0253] The inputs to this process are: The variable tensorDims that specifies the dimensions of the tensor to be decoded. · A variable entryPointOffset that indicates whether an entry point exists for decoding and, if so, the entry point offset. · A variable codebookId that indicates whether a codebook is applied and, if so, which codebook to use. The output of this process is a variable recParam of type TENSOR_FLOAT with dimensions equal to tensorDims.

[0254] TIFF2025016517000022.tif88146

[0255] Regarding the derivation process of ctxInc, which indicates the context or probability estimation to be used for the syntax element sig_flag,

[0256] The input to this process is the sig_flag decoded before the current sig_flag, the state value stateId, and the associated sign_flag (if any). If there is no sig_flag decoded before the current sig_flag, it is assumed to be 0. Also, if the sign_flag associated with a previously decoded sig_flag has not been decoded, it is assumed to be 0.

[0257] The output of this process is the variable ctxInc. The variable ctxInc is derived as follows. If sig_flag is 0, stateId*3 is set in ctxInc. Otherwise, if sign_flag is equal to 0, then ctxInc is set to stateId*3+1. Otherwise, ctxInc is set to stateId*3+2.

[0258] The above example illustrates the concept of encoding / decoding neural network parameters 13 to / from a data stream 14, where the neural network parameters 13 may relate to the weights of the neuronal interconnections 11 of the neural network 10, e.g. the weights of a weight tensor. The decoding / encoding of the neural network parameters 13 is performed sequentially. See the for-next loop 1000 which cycles through the weights of the tensor with as many weights as the product of the number of weights per dimension of the tensor. The weights are scanned in a predefined order TensorIndex(dimensions, i, scan_order). For the current neural network parameters idx13', one of the two reconstruction level sets 52 is selected according to the quantization state stateId which is continuously updated based on the quantization index 58 decoded from the data stream of the previous neural network parameters at 1018 and 1020. In particular, a quantization index for the current neural network parameters idx is decoded from the data stream in 1012, the quantization index indicating one reconstruction level of the selected reconstruction level set for the current neural network parameters 13′. The two reconstruction level sets are defined by duplication in 1016 followed by addition of one or minus one depending on the quantization state index in 1018 and 1020. Now, in 1018 and 1020, the current neural network parameters 13′ are actually dequantized onto one reconstruction level of the selected reconstruction level set indicated by the quantization index QuantParam[idx] for the current neural network parameters 13′. A step size stepSize is used to parameterize the reconstruction level set in 3001 to 3003. Information about this predefined quantization step size stepSize is derived from the data stream via the syntax element qp_value. The latter may be coded in the data stream for the entire tensor or the entire NN layer, respectively, or for the entire NN.That is, the neural network 10 may include one or more neural network layers 10a, 10b, and for each neural network layer, information of a predetermined quantization step size (QP) for the respective neural network layer may be derived from the data stream 14, and then, for each neural network layer, multiple reconstruction level sets may be parameterized using the derived predetermined quantization step size for the respective neural network layer to be used to dequantize the neural network parameters 13 belonging to the respective neural network layer.

[0259] The first reconstruction level set for stateId=0 here includes zero and even multiples of the predefined quantization step size, and the second reconstruction level set for stateId=1 includes zero and odd multiples of the predefined quantization step size (QP), as can be seen at 1018 and 1020. For each neural network parameter 13, at 1015-1021 an intermediate integer value QuantParam[idx] (IV) is derived as a function of the reconstruction level set selected for the respective neural network parameter 13 and the entropy-decoded quantization index QuantParam[idx] for the respective neural network parameter, and then, for each neural network parameter, the intermediate value for the respective neural network parameter is multiplied by the predefined quantization step size for the respective neural network parameter at 4001.

[0260] The selection of one of the two reconstructed level sets (e.g., set 0, set 1) for the current neural network parameters 13' is made according to the LSB portion of the quantization indexes decoded from the data stream of previously decoded neural network parameters, as shown at 1014, and the transition table transitions from stateId to the next quantization state nextSt according to the LSB of QuantParam[idx] such that statId depends on the past sequence of already decoded quantization indexes 56. Thus, the state transition depends on the result of a binary function of the quantization indexes 56 decoded from the data stream of previously decoded neural network parameters, i.e., their parity. In other words, for a current neural network parameter, the selection of a reconstruction level set among the multiple reconstruction level sets is performed by the state transition process by determining, for the current neural network parameter, a reconstruction level set among the multiple reconstruction level sets depending on the state statId associated with the current neural network parameter in 1018 and 1020, and by updating the stateId in 1014 for the subsequent neural network parameter for which the stateId is to be determined next, but not necessarily the NN parameter to be next coded / decoded, depending on the quantization index decoded from the data stream for the previous neural network parameter, i.e. the parameter for which the stateId was determined up to now. For example, here the current neural network parameter is used for the update to bring about the stateId of the NN parameter to be next coded / decoded. The update in 1014 is performed using a binary function of the quantization index decoded from the data stream of the previous (current) neural network parameter, i.e. its parity. The state transition process is configured to transition between eight possible states. The transitions are performed via a table StateTransTab[][].In the state transition process, transitions are made between these eight possible states, and for the current neural network parameters, a first of the two reconstruction level sets is determined by determining a set of reconstruction levels from the quantization set according to the state stateId associated with the current neural network parameters in 1018 and 1020 if the state belongs to the first half of the even possible states, i.e., the odd states, and a second of the two reconstruction level sets is determined if the state belongs to the second half of the even possible states, i.e., the even states. The update of the state statId is performed by a transition table StateTransTab[][]. The transition table maps the combination of the state statID decoded from the data stream of the previous (current) neural network parameters and the parity of the quantization index (58), QuantParam[idx]&1, to another state associated with the subsequent neural network parameters.

[0261] The quantization index of the current neural network parameter is encoded into the data stream and decoded from the data stream using arithmetic coding with a probability model that depends on the reconstruction level set selected for the current neural network parameter, or more precisely, the quantization state stateId, i.e., the state for the current neural network parameter 13'. See the third parameter in the call to function int_param at 1012. In particular, the quantization index of the current neural network parameter can be encoded into the data stream and decoded from the data stream using binary arithmetic coding / decoding with a probability model that corresponds to the state for the current neural network parameter for at least one bin of the binarization of the quantization index. Here, the bin sig_flag of the binarization sig_flag, sign_flag (optional), abs_level_greater_x[j], abs_level_greater_x2[j], abs_remainder is a significance bin that indicates whether the quantization index of the current neural network parameter (56) is equal to zero or not. The dependency of the probability model includes selecting a context from a set of contexts of the neural network parameters using the dependency, where each context is configured to be associated with a predefined probability model, where the context of sig_flag is selected using ctxInc as an index incrementer for indexing the context from the list of contexts, each of which is associated with a binary probability model. The model can be updated using the bins associated with the context, i.e., the predefined probability model associated with each context can be updated based on the quantization index arithmetically coded with the respective context.

[0262] (Note) It should be noted that the probability model for sig_flag further depends on the quantization index of the previously decoded neural network parameters, i.e., the sig_flag of the previously decoded neural network parameters, and its sign_flag - indicating its sign. More precisely, depending on the state stateId, among the plurality of probability models, i.e., among the context incrementer states 0...23, a subset of probability models is preselected, i.e., among the eight of {0...23} including three consecutive contexts, among the subset of probability models for sig_flag, the probability model for the current neural network parameters is selected depending on the quantization index of the previously decoded neural network parameters (121), i.e., based on the sig_flag and sign_flag of the previous NN parameters. The subset preselected for the first value of stateID is inconsistent with the subset preselected for the other values ​​of stateID. The previous NN parameters for which sig_flag and sign_flag are used relate to the parts of the neural network adjacent to the part to which the current neural network parameters relate.

[0263] A number of embodiments have been described above. It should be noted that aspects and features of the embodiments may be used individually or in combination. Furthermore, aspects and features of the embodiments according to the first and second aspects of the present invention may be used in combination.

[0264] A further embodiment configures the device, where the neural network parameters are associated with one of the reconstruction layers, e.g., an enhancement layer, of the reconstruction layers in which the neural network 10 is represented. The device may configure the neural network such that it is reconfigurable by combining, per neural network parameter, with the neural network parameters of one or more other reconstruction layers, which correspond, e.g., associated with a common neuronal interconnection, or frankly, which are collocated in a matrix representation of the NN layer in a different representation layer.

[0265] For example, as described in this embodiment, features and aspects of the first and second aspects of the present invention may be combined. Any features of the dependent claims from the second aspect are intended to be transferable here too to yield further embodiments.

[0266] Additionally, an apparatus according to an aspect of the present invention may be configured to encode the quantization indexes 56 for the current neural network parameters 13′ into the data stream 14 using arithmetic coding with a probability model that depends on the neural network parameters that correspond to the current neural network parameters.

[0267] Respectively, further embodiments configure an apparatus, where the neural network parameters are associated with one of the reconstruction layers in which the neural network 10 is represented, e.g., an enhancement layer. The apparatus may be configured to reconstruct the neural network by combining, on a neural network parameter basis, the neural network parameters with corresponding, e.g., associated with common neuronal interconnections, or frankly, collocated in a matrix representation of the NN layers in different representation layers, neural network parameters of one or more other reconstruction layers.

[0268] For example, as described in this embodiment, features and aspects of the first and second aspects of the present invention may be combined. Any features of the dependent claims from the second aspect are intended to be transferable here too to yield further embodiments.

[0269] Additionally, an apparatus according to an aspect of the present invention may be configured to decode quantization indices 56 of the current neural network parameters 13' from the data stream 14 using arithmetic coding, using a probability model that depends on corresponding neural network parameters that correspond to the current neural network parameters.

[0270] In other words, the neural network parameters of the reconstruction layer, e.g. the second neural network parameters as described, can be encoded / decoded and / or quantized / dequantized according to the concepts described with respect to Figures 3 and 5 and Figures 2 and 4, respectively.

[0271] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0272] The data stream of the present invention can be stored on a digital storage medium or can be transmitted over a transmission medium, such as a wireless transmission medium or a wired transmission medium such as the Internet.

[0273] Depending on particular implementation requirements, embodiments of the present invention can be implemented in hardware or in software. Implementation can be performed using a digital storage medium, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM or flash memory, having electronically readable control signals stored thereon, which cooperates (or can cooperate) with a programmable computer system such that the respective methods are performed.

[0274] Some embodiments according to the present invention comprise a data carrier having an electronically readable control signal, which is capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0275] Generally, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product is run on a computer. The program code can, for example, be stored on a machine readable carrier.

[0276] Another embodiment comprises the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0277] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0278] A further embodiment of the inventive method is, therefore, a data carrier (or digital storage medium, or computer readable medium) having recorded thereon the computer program for performing one of the methods described herein.

[0279] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, The data stream or the sequence of signals may for example be arranged to be transmitted via a data communication connection, for example the Internet.

[0280] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0281] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0282] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0283] The above-described embodiments merely exemplify the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. It is therefore intended to be limited only by the scope of the appended claims, and not by the specific details presented in the description and explanation of the embodiments herein.

[0284] TIFF2025016517000023.tif187152

Claims

1. 1. An apparatus for decoding neural network parameters (13) defining a neural network (10) from a data stream (14), comprising: decoding quantization indices (56) for the current neural network parameters (13') from said data stream (14); and Dequantizing (62) the current neural network parameters (13'). configured to sequentially decode the neural network parameters (13) by The device comprises: performing the inverse quantization using dependent quantization based on a state transition process; decoding the quantization indexes (56) for the current neural network parameters (13') from the data stream (14) using arithmetic coding using a probability model that depends (122) on the state of the state transition process of the current neural network parameters (13') and on the quantization indexes of previously decoded neural network parameters; preselecting a subset of probability models from among a plurality of probability models depending on the state of the state transition process of the current neural network parameters (13'); and selecting the probability model for the current neural network parameters from among the subset of probability models depending on the quantization indexes (121) for previously decoded neural network parameters; selecting a probability model for the current neural network parameter from among a subset comprising first, second and third probability models depending on values ​​of the quantization indexes of previously decoded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameter relates; It is configured as follows: wherein if the previously decoded neural network parameters are less than zero, a first probability model is selected; a second probability model is selected if the previously decoded neural network parameters are greater than zero; If the previously decoded neural network parameters are equal to zero, the third probability model is selected. Device.

2. The device described in claim 1, configured to decode the quantization indexes (56) for the current neural network parameters (13') from the data stream (14) using arithmetic coding with a probability model that depends (123) on a set of reconstruction levels (48) for inverse quantizing the current neural network parameters (13').

3. An apparatus for encoding neural network parameters defining a neural network into a data stream, comprising: quantizing (64) the current neural network parameters (13'); and encoding quantization indices (56) for the current neural network parameters into the data stream (14); configured to sequentially encode the neural network parameters (13') by The device comprises: performing the quantization using dependent quantization based on a state transition process; encoding the quantization indexes (56) for the current neural network parameters (13') into the data stream (14) using arithmetic coding using a probability model that depends (122) on the state of the state transition process of the current neural network parameters (13') and on the quantization indexes of previously coded neural network parameters; preselecting a subset of probability models from among a plurality of probability models depending on the state of the state transition process of the current neural network parameters (13'); and selecting the probability model for the current neural network parameters from among the subset of probability models depending on the quantization indexes (121) for previously coded neural network parameters; selecting a probability model for the current neural network parameter from among a subset comprising first, second and third probability models depending on values ​​of the quantization indexes of previously coded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameter relates; It is configured as follows: wherein the first probability model is selected if the previously encoded neural network parameters are less than zero; if the previously encoded neural network parameters are greater than zero, the second probability model is selected; If the previously encoded neural network parameters are equal to zero, the third probability model is selected. Device.

4. A method (400) for decoding neural network parameters (13) defining a neural network (10) from a data stream (14), comprising: decoding (420) quantization indices (56) for the current neural network parameters (13') from said data stream (14); and Dequantizing (62) the current neural network parameters (13'). and sequentially decoding the neural network parameters (13) by The method further comprises: performing the inverse quantization using dependent quantization based on a state transition process; decoding the quantization indices (56) for the current neural network parameters (13') from the data stream (14) using arithmetic coding that uses a probability model that depends (122) on the state of the state transition process of the current neural network parameters (13') and on the quantization indices of previously decoded neural network parameters; preselecting a subset of probability models from among a plurality of probability models depending on the state of the state transition process of the current neural network parameters (13'), and selecting the probability model for the current neural network parameters from among the subset of probability models depending on the quantization indexes (121) for previously decoded neural network parameters; selecting a probability model for the current neural network parameter from a subset comprising first, second and third probability models depending on values ​​of the quantization indices of previously decoded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameter relates; further comprising wherein the first probability model is selected if the previously decoded neural network parameters are less than zero; if the previously decoded neural network parameters are greater than zero, the second probability model is selected; If the previously decoded neural network parameters are equal to zero, the third probability model is selected. method.

5. A method (500) for encoding neural network parameters defining a neural network into a data stream, comprising: quantizing (64) the current neural network parameters (13'); and encoding (530) quantization indices (56) for the current neural network parameters into the data stream (14); sequentially encoding the neural network parameters (13) by The method further comprises: performing the quantization using dependent quantization based on a state transition process; encoding the quantization indices (56) for the current neural network parameters (13') into the data stream (14) using arithmetic coding using a probability model that depends (122) on the state of the state transition process of the current neural network parameters (13') and on the quantization indices of previously coded neural network parameters; preselecting a subset of probability models from among a plurality of probability models depending on the state of the state transition process of the current neural network parameters (13'), and selecting the probability model for the current neural network parameters from among the subset of probability models depending on the quantization indexes (121) for previously coded neural network parameters; selecting a probability model for the current neural network parameters from a subset comprising first, second and third probability models in dependence on values ​​of the quantization indices of previously coded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameters relate; Including, wherein the first probability model is selected if the previously encoded neural network parameters are less than zero; if the previously encoded neural network parameters are greater than zero, the second probability model is selected; If the previously encoded neural network parameters are equal to zero, the third probability model is selected. method.

6. A non-transitory digital storage medium storing a computer program for performing the method of claim 4 or 5 when the program is run on one or more computers.

Images