Concepts for encoding the parameters of a neural network

Abstract

To compress neural network parameters more efficiently. [Solution] A non-constant quantizer is used, but the neural network parameters are changed during the encoding of the neural network parameters 13, i.e., a set of reconstruction levels 52 is selected according to the quantization index 58 that has been decoded from or encoded in the data stream 14 of the previous or each previously encoded neural network parameters. This is done stepwise—called reconstruction layers to distinguish them from the layer configuration of the neural network in the neural layers—and the parameterizations provided in these stages are then combined on a neural network parameter basis.

Description

[Technical Field]

[0001] Embodiments of the present invention relate to the concept of encoding neural network parameters. [Background technology]

[0002] 1. Application Areas The most basic form of a neural network is one in which a neural network constitutes a chain of affine transformations followed by element-wise nonlinear functions. As shown in Figure 1, it can be represented as a directed acyclic graph. Figure 1 is a schematic diagram showing an example of a neural network, specifically a two-layer feedforward neural network. In other words, Figure 1 is a graphical representation of a feedforward neural network. Specifically, this two-layer neural network is a nonlinear function that maps a four-dimensional input vector to a solid line. This neural network includes four neurons 10c in the input layer, corresponding to the four-dimensional input vector, five neurons 10c in the hidden layer, and one neuron 10c in the output layer, which forms the output of the neural network. The neural network further includes neuron interconnections 11 that connect neurons in different—or subsequent—layers. Neuron interconnects 11 may associate weights, which are associated with the relationships between interconnected neurons 10c. In particular, when weights are transferred to subsequent layers, they weight the activation of neurons in a given layer, and sequentially, a sum of inbound weighted activations is formed at each neuron in the subsequent layers—corresponding to a linear function—then a nonlinear scalar function is applied to the weighted sum formed at each neuron / node in the subsequent layers—corresponding to a nonlinear function. Thus, each node, e.g., neuron 10c, is associated with a specific value, which is propagated forward to the next node by multiplication with the respective weight values ​​of the edges, e.g., neuron interconnects 11. All received values ​​are then simply aggregated.

[0003] JPEG2026067854000002.jpg73169 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] Typically, neural networks have millions of parameters, and their representation can require hundreds of megabytes (MB). As a result, their inference procedures involve numerous dot product operations between large matrices, requiring significant computational resources to execute. Therefore, reducing the complexity of dot product operations is crucial.

[0005] In addition to the problems mentioned above, there is also the need to store a vast number of neural network parameters, which may need to be transmitted, for example, from a server to a client. Furthermore, it can be advantageous to gradually provide information about the neural network's parameterization to entities, for example, in a federated learning environment, or when providing neural network parameterization at different stages of quality that can be addressed when a particular recipient pays for or uses the neural network for inference.

[0006] Therefore, it is desirable to provide a concept for efficient encoding of neural network parameters, for example, from the standpoint of compression. Furthermore, or alternatively, it is desirable to reduce the bitstream and, consequently, the signaling cost for the neural network parameters.

[0007] This objective is achieved by the subject matter of the independent claim of the present application.

[0008] Further embodiments of the present invention are defined by the subject matter of the dependent claims of this application. [Means for solving the problem]

[0009] An embodiment according to a first aspect of the present invention is an apparatus for decoding neural network parameters defining a neural network from a data stream, configured to sequentially decode the neural network parameters by selecting a set of reconstruction levels from a plurality of reconstruction level sets for the current neural network parameter in accordance with the quantization index decoded from the data stream for previous neural network parameters. Furthermore, the apparatus is configured to sequentially decode the neural network parameters by decoding a quantization index for the current neural network parameter from the data stream, where the quantization index indicates one of the selected reconstruction level sets for the current neural network parameter, and by dequantizing the current neural network parameter onto one of the selected reconstruction level sets indicated by the quantization index for the current neural network parameter.

[0010] A further embodiment according to the first aspect of the present invention is a device for encoding neural network parameters defining a neural network into a data stream, configured to sequentially encode the neural network parameters by selecting a set of reconstruction levels from a plurality of reconstruction level sets in accordance with the quantization index encoded in the data stream for previously encoded neural network parameters for the current neural network parameter. Furthermore, the device is configured to sequentially encode the neural network parameters by quantizing the current neural network parameter to one of the selected reconstruction level sets, and by encoding in the data stream a quantization index for the current neural network parameter indicating one of the reconstruction levels to which the quantization index for the current neural network parameter is quantized.

[0011] A further embodiment according to a first aspect of the present invention comprises a method for encoding neural network parameters defining a neural network into a data stream. The method comprises sequentially decoding neural network parameters by selecting a reconstruction level set from a plurality of reconstruction level sets for the current neural network parameters according to quantization indexes decoded from a data stream for previous neural network parameters. Further, the method comprises sequentially encoding neural network parameters by decoding a quantization index for the current neural network parameters from the data stream, where the quantization index indicates one of the reconstruction levels of a selected reconstruction level set for the current neural network parameters, and inverse quantizing the current neural network parameters onto one of the reconstruction levels of the selected reconstruction level set indicated by the quantization index for the current neural network parameters.

[0012] A further embodiment according to the first aspect of the present invention comprises a method for encoding neural network parameters defining a neural network into a data stream. The method comprises sequentially encoding the neural network parameters by selecting a reconstruction level set from a plurality of reconstruction level sets for the current neural network parameters according to the quantization indexes encoded in the data stream for the previously encoded neural network parameters. Further, the method comprises sequentially encoding the neural network parameters by quantizing the current neural network parameters to one of the selected reconstruction levels and by encoding into the data stream a quantization index for the current neural network parameters indicating the one reconstruction level to which the quantization index for the current neural network parameters is quantized.

[0013] Embodiments according to the first aspect of the present invention are based on the idea that although using a non-uniform quantizer, by varying the same among the encoded neural network parameters, i.e., by selecting a set of reconstruction levels according to the quantization indexes decoded from the data stream of the previous or each previously encoded neural network parameters, or each encoded quantization index, the neural network parameters can be compressed more efficiently. Therefore, a reconstruction vector that can refer to an ordered set of neural network parameters can be packed more densely in an N-dimensional signal space, where N indicates the number of neural network parameters in the set of samples to be processed. Such dependent quantization can be used respectively for decoding and inverse quantization by a device for decoding or for quantization and encoding by a device for encoding.

[0014] Embodiments according to a second aspect of the present invention are based on the idea that if the parameterization provided in the stages—called reconstruction layers to distinguish them from the layer configuration of the neural network in the neural layers—is then combined on a neural network parameter basis to result in improved neural network parameterization compared to any of the stages, then more efficient neural network coding may be achieved. Accordingly, an apparatus for reconstructing neural network parameters that define a neural network can derive the neural network parameters of the first reconstruction layer for the first neural network parameters, for example, for the first reconstruction layer, and obtain the neural network parameter values ​​of the first reconstruction layer for each neural network parameter. The first neural network parameters may have been previously transmitted, for example, during an associative learning process. Furthermore, the first neural network parameters may be the neural network parameter values ​​of the first reconstruction layer. Furthermore, the device is configured to decode the data stream for the second reconstructed layer in order to distinguish the second neural network parameters, for example, the neural network parameters of the second reconstruction layer, from, for example, the final neural network parameters, thereby obtaining the second reconstructed layer neural network parameter values ​​for each neural network parameter. The second neural network parameters may not have a self-contained meaning from the perspective of neural network representation, and may simply yield a neural network representation, i.e., the final neural network parameters, when combined with the parameters of the first representation layer. Moreover, the device is configured to reconstruct the neural network parameters by combining the neural network parameter values ​​of the first reconstructed layer and the neural network parameter values ​​of the second reconstructed layer for each neural network parameter.

[0015] A further embodiment according to a second aspect of the present invention comprises a device for encoding neural network parameters that define a neural network, for each neural network parameter, using a first neural network parameter for a first reconstruction layer, which includes the neural network parameter values ​​of the first reconstruction layer. Furthermore, the device is configured to encode a second neural network parameter for a second reconstruction layer into a data stream, the second reconstruction layer including the neural network parameter values ​​of the second reconstruction layer for each neural network parameter, and the neural network parameter is reconstructible for each neural network parameter by combining the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer.

[0016] A further embodiment according to a second aspect of the present invention comprises a method for reconstructing neural network parameters that define a neural network. The method includes, for each neural network parameter, deriving a first neural network parameter for the first reconstruction layer, which may have been previously transmitted, for example during an associative learning process, and which may be called the neural network parameter of the first reconstruction layer, in order to obtain the neural network parameter value of the first reconstruction layer.

[0017] Furthermore, the method includes the step of decoding a second neural network parameter from the data stream for the second reconstruction layer, which may be called the neural network parameter of the second reconstruction layer to distinguish it from, for example, the final, reconstructed neural network parameter, and generating a neural network parameter value for the second reconstruction layer for each neural network parameter. The method then includes the step of reconstructing the neural network parameter for each neural network parameter by combining the neural network parameter value of the first reconstruction layer and the neural network parameter value of the second reconstruction layer. The second neural network parameter does not have a self-contained meaning with respect to the neural representation, but may simply derive a neural representation, i.e., the final neural network parameter, when combined with the parameters of the first representation layer.

[0018] A further embodiment according to a second aspect of the present invention includes a method for encoding neural network parameters that define a neural network, using a first neural network parameter for a first reconstruction layer, which includes the neural network parameter value of the first reconstruction layer for each neural network parameter. The method includes the step of encoding a second neural network parameter for a second reconstruction layer into a data stream, the second reconstruction layer including a second reconstruction layer neural network parameter value for each neural network parameter, and the neural network parameter is reconfigurable for each neural network parameter by combining the neural network parameter value of the first reconstruction layer and the neural network parameter value of the second reconstruction layer.

[0019] Embodiments according to a second aspect of the present invention are based on the idea that a neural network, defined, for example, by neural network parameters, can be efficiently compressed and / or transmitted, for example, when the amount of data in a bitstream is small, using sublayers such as reconstruction layers, for example, a base layer and an enhancement layer. The reconstruction layers can be defined such that the neural network parameters can be reconstructed for each neural network parameter by combining the neural network parameter values ​​of a first reconstruction layer and the neural network parameter values ​​of a second reconstruction layer. This distribution enables efficient coding, e.g., coding and / or decoding and / or transmission of the neural network parameters. Thus, the second neural network parameters for the second reconstruction layer can be coded and / or transmitted separately to the data stream. [Brief explanation of the drawing]

[0020] The drawings are not necessarily to scale, and instead the focus is on illustrating the general principles of the present invention. In the following description, various embodiments of the present invention will be described with reference to the following drawings, in which embodiments of the present invention will be described. [Figure 1] Figure 1 is a schematic diagram illustrating an example of a two-layer feedforward neural network that may be used in conjunction with embodiments of the present invention. [Figure 2] Figure 2 is a schematic diagram illustrating a concept for inverse quantization performed within a device for decoding neural network parameters that define a neural network from a data stream, according to an embodiment. [Figure 3] Figure 3 is a schematic diagram illustrating a concept for quantization performed within a device for encoding neural network parameters into a data stream, according to an embodiment. [Figure 4]Figure 4 is a schematic diagram illustrating a concept for decoding performed within a device for reconstructing neural network parameters that define a neural network, according to an embodiment. [Figure 5] Figure 5 is a schematic diagram illustrating a concept for coding performed within a device for reconstructing neural network parameters that define a neural network, according to an embodiment. [Figure 6] Figure 6 is a schematic diagram of a concept using a reconstruction layer for neural network parameters, for use in embodiments of the present invention. [Figure 7] Figure 7 is a schematic diagram showing an illustration of a uniform reconstruction quantizer according to an embodiment of the present invention. [Figure 8] Figure 8 shows an example of the acceptable reconstruction vector positions for a simple case of two weight parameters according to an embodiment of the present invention. [Figure 9a] Figure 9a shows an example for dependent quantization having two sets of reconstruction levels completely determined by a single quantization step size Δ according to an embodiment of the present invention. [Figure 9b] Figure 9b shows an example for dependent quantization having two sets of reconstruction levels, which are entirely determined by a single quantization step size Δ, according to an embodiment of the present invention. [Figure 9c] Figure 9c shows an example for dependent quantization having two sets of reconstruction levels, which are entirely determined by a single quantization step size Δ, according to an embodiment of the present invention. [Figure 10] Figure 10 shows an example of pseudocode illustrating a preferred example of a neural network parameter reconstruction process according to an embodiment of the present invention. [Figure 11] Figure 11 shows an example of dividing a reconfiguration level set into two subsets according to an embodiment of the present invention. [Figure 12]Figure 12 shows an example of pseudocode illustrating a preferred example of the neural network parameter reconstruction process for a layer according to an embodiment. [Figure 13] Figure 13 shows a preferred example of a state transition table sttab and a table setId according to an embodiment of the present invention, which specify the quantization set associated with the state. [Figure 14] Figure 14 shows a preferred example of a state transition table sttab and a table setId according to an embodiment of the present invention, which specify the quantization set associated with the state. [Figure 15] Figure 15 shows pseudocode illustrating an alternative reconstruction process at the neural network parameter level according to an embodiment of the present invention, in which quantization indices equal to zero are excluded from state transitions and dependent scalar quantization. [Figure 16] Figure 16 shows an example of a state transition in dependent scalar quantization as a trellis structure according to an embodiment of the present invention. [Figure 17] Figure 17 shows an example of a basic trellis cell according to an embodiment of the present invention. [Figure 18] Figure 18 shows an example of a trellis for dependent scalar quantization of eight neural network parameters according to an embodiment of the present invention. [Figure 19] Figure 19 shows an example of a trellis structure that can be used to determine a sequence (or block) of quantization indices that minimizes a cost amount (e.g., Lagrangian cost amount D + λ·R) according to an embodiment of the present invention. [Figure 20] Figure 20 is a block diagram of a method for decoding neural network parameters that define a neural network from a data stream, according to an embodiment of the present invention. [Figure 21] Figure 21 is a block diagram of a method for encoding neural network parameters that define a neural network into a data stream, according to an embodiment of the present invention. [Figure 22] Figure 22 is a block diagram of a method for reconstructing neural network parameters that define a neural network, according to an embodiment of the present invention. [Figure 23] Figure 23 is a block diagram of a method for encoding neural network parameters that defines a neural network, according to an embodiment of the present invention. [Modes for carrying out the invention]

[0021] Elements or components having equal or equivalent functions will be indicated by the same or equivalent reference numbers in the following description, even if they are represented in different diagrams.

[0022] The following description includes several details to provide a more detailed description of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention can be carried out without these specific details. In other examples, well-known structures and devices are shown in block diagram form rather than in detail, so as not to obscure embodiments of the present invention. In addition, features of different embodiments described herein can be combined with each other unless otherwise specified.

[0023] This description begins with a presentation of several embodiments of the present application. This description is fairly general, but it provides the reader with an overview of the functionality on which the embodiments of the present application are based. Subsequently, a more detailed description of these functionalities is presented, along with the motivations for the embodiments and how they achieve the efficiency improvements described above. The details can be combined with the embodiments described, individually and in combination.

[0024] Figure 2 is a schematic diagram illustrating the concept of inverse quantization performed within a device for decoding neural network parameters that define a neural network from a data stream according to an embodiment. The neural network may include multiple interconnected neural network layers, for example, having neuronal interconnections between neurons in interconnected layers. Figure 2 shows the quantization index 56 of, for example, encoded neural network parameters 13 in the data stream 14. The neural network parameters 13 can therefore define or parameterize the neural network, such as in terms of the weights between its neurons.

[0025] This device is configured to sequentially decode neural network parameters 13. During this sequential process, the quantizer (reconstruction level set) is changed. This change allows the use of a quantizer with fewer (or better, less dense) levels, and therefore allows encoding of smaller quantization indices, resulting in a higher quality neural network representation compared to using a constant quantizer, relative to the required encoding bitrate. Further details will be provided later. Specifically, for the current neural network parameter 13', the device sequentially decodes the neural network parameter 13 by selecting a reconstruction level set 48 (selected set) from among multiple (50) reconstruction level sets 52 (set 0, set 1) according to the quantization index 58 decoded from the data stream 14 for the previous neural network parameter 13'.

[0026] Furthermore, the device is configured to sequentially decode the neural network panel 13 by decoding a quantization index 56 for the current neural network parameter 13' from the data stream 14, where the quantization index 56 indicates one of a selected set 48 of reconstruction levels for the current neural network parameter, and by dequantizing the current neural network parameter 13' to one of the selected set 48 of reconstruction levels indicated by the quantization index 56 for the current neural network parameter 62.

[0027] The decoded neural network parameters 13 are represented, for example, by matrix 15a. The matrix may also include the deserialized neural network parameters 13, which may relate to the weights of the neuronal interconnections in the neural network.

[0028] Optionally, the number of multiple (50) reconstruction level sets 52, sometimes referred to herein as quantizers, may be two, for example, set 0 and set 1, as shown in Figure 2.

[0029] Furthermore, the device can be configured to parameterize a plurality (50) set of reconstruction levels 52 (e.g., set 0, set 1) by a predetermined quantization step size (QP), for example, indicated by △ or △k, and to derive information of the predetermined quantization step size from the data stream 14. Thus, the decoder according to the embodiment can be adapted to a variable step size (QP).

[0030] Furthermore, according to the embodiment, the neural network may include one or more neural network (NN) layers, and the device may be configured to parameterize a plurality of reconstruction level sets 52 for each NN layer using the predetermined quantization step size derived for each NN layer, so that for each NN layer, information on a predetermined quantization step size (QP) for each NN layer is derived from the data stream 14, and the neural network parameters belonging to each NN layer are used to dequantize each NN layer. Adaptation of the step size for the NN layers, and therefore the adaptation of the reconstruction level sets 52, can improve coding efficiency.

[0031] In a further embodiment, the device may be configured to select 54 a reconstruction level set 48 from a plurality of 50 reconstruction level sets 52 for the current neural network parameter 13', depending on the LSB (e.g., least significant bit) portion of the binarized quantization index 58 decoded from a data stream 14 for a previously decoded neural network parameter, or the previously decoded bin (e.g., a binary decision). LSB comparison can be performed at a low computational cost. In particular, state transitions can be used. Selection 54 can be performed by a state transition process by determining a reconstruction level set 48 from a plurality of 50 reconstruction level sets 52 for the current neural network parameter 13', depending on the state associated with the current neural network parameter 13', and by updating the state for the subsequent neural network parameter according to the quantization index 58 decoded from the data stream for the immediately preceding neural network parameter. Alternatively approaches other than state transitions, such as using a transition table, can also be used and are defined below.

[0032] In addition, or alternatively, the device may be configured to select, for example, a reconstruction level set 48 from a plurality of reconstruction level sets 52 for the current neural network parameter 13', based on the result of a binary function of the quantization index 58 decoded from a data stream 14 for previously decoded neural network parameters. The binary function may be, for example, a parity check using a bitwise "and" operation that signals whether the quantization index 58 represents an even or odd number. This can provide information about the reconstruction level set 48 used to encode the quantization index 58, and therefore, for example, for a predetermined order of reconstruction level sets used in the corresponding encoder, for the set of reconstruction levels used to encode the current neural network parameter 13'. Parity can be used for the state transitions described above.

[0033] Furthermore, according to the embodiment, the device can be configured to select, for example, a set of reconstruction levels 48 from a set of 50 reconstruction levels 52 for the current neural network parameter 13', according to the parity 54 of the quantization index 58 decoded from the data stream 14 for the previously decoded neural network parameter. The parity check can be performed at a low computational cost, for example, using a bitwise "and" operation.

[0034] Optionally, the device can be configured to decode the quantization index 56 for the neural network parameters 13 and perform inverse quantization of the neural network parameters 13 along a common continuous order 14' among the neural network parameters 13. In other words, the same order can be used for both tasks.

[0035] Figure 3 is a schematic diagram illustrating the concept of quantization performed within a device for encoding neural network parameters into a data stream, according to an embodiment. Figure 3 shows a neural network (NN) 10 including neural network layers 10a and 10b, where each layer contains neurons 10c, and the neurons in interconnected layers are interconnected via neuron interconnects 11. As an example, NN layer (p-1) 10a and NN layer (p) 10b are shown, where p is the index of the NN layer, and 1 ≤ p ≤ the number of layers in NN. The neural network is defined or parameterized by neural network parameters 13, which may optionally be related to the weights of the neuron interconnects 11 of the neural network 10. The neurons 10c in the hidden layer of Figure 1 may represent neurons in layer p(A, B, C, ...) in Figure 3, and the neurons in the input layer of Figure 1 may represent neurons in layer p-1(a, b, c, ...) shown in Figure 3. The neural network parameters 13 can be related to the weights of the neuron interconnects 11 in Figure 1.

[0036] The relationships between neurons 10c in different layers are represented in Figure 1 by a matrix 15a of neural network parameters 13. For example, if the network parameters 13 are related to the weights of neuron interconnections 11, the matrix 15a can be constructed such that, for example, the matrix elements represent the weights between neurons 10c in different layers (e.g., a, b, ... for layer p-1, A, B, ... for layer p).

[0037] This device is configured to sequentially encode neural network parameters 13, for example, in serial 20a (serialization). During this sequential processing, the quantizer (reconstruction level set) is changed. This change allows the use of a quantizer with fewer (or better, less dense) levels, and therefore allows encoding of smaller quantization indices, resulting in an improved quality of the neural network representation obtained from this quantization compared to using a constant quantizer, relative to the required encoding bitrate. Further details will be described later. Specifically, for the current neural network parameter 13', the device sequentially encodes the neural network parameter 13 by selecting a reconstruction level set 48 from a plurality of reconstruction level sets 52 52 according to the quantization indices 58 encoded in the data stream 14 for previously encoded neural network parameters 54.

[0038] Furthermore, the device quantizes the current neural network parameters 13' onto one of the selected reconstruction levels 48, and, The neural network parameters 13 are configured to sequentially encode the neural network parameters 13 by encoding the quantization index 56 for the current neural network parameters 13', which indicates one reconstruction level in which the quantization index 56 for the current neural network parameters is quantized in the data stream 14. Optionally, the number of reconstruction level sets 52, sometimes referred to herein as quantizers, can be two, for example, as shown using sets 0 and 1.

[0039] According to one embodiment, as shown in Figure 3, the device can be configured to parameterize a plurality of 50 reconstruction level sets 52 by a predetermined quantization step size (QP), for example, and to insert information about the predetermined quantization step size into the data stream 14. This enables, for example, adaptive quantization to improve quantization efficiency, and changes in the encoding method of neural network parameters 13 can be transmitted to the decoder along with information about the predetermined quantization step size. By using a predetermined quantization step size (QP), the amount of data required for information transmission can be reduced.

[0040] Furthermore, according to the embodiment, the neural network 10 may include one or more NN layers 10a, 10b. The device can be configured to insert information of a predetermined quantization step size (QP) for each NN layer (p; p-1) into the data stream 14, and to parameterize a plurality of 50 reconstruction level sets 52 using the predetermined quantization step size derived for each NN layer, so as to be used for quantizing the neural network parameters belonging to each NN layer. As explained earlier, for example, adapting quantization according to the characteristics of the NN layer can improve quantization efficiency.

[0041] Optionally, the device can be configured to select a set of reconstruction levels 48 from a set of 50 reconstruction levels 52 for the current neural network parameters 13', depending on the LSB portion or previously encoded bins of the binarized quantization index 58 encoded in the data stream 14 for the previously encoded neural network parameters. LSB comparison can be performed at a low computational cost.

[0042] A state transition can be used, similar to the decoding apparatus described in Figure 2. The state transition process allows selection 54 for the current neural network parameter 13' from the quantization level set 48 of the quantization level set 48 of the quantization level set 52 of the quantization level set 52, by determining a set of reconstruction levels 48 of the quantization level set 58 of the quantization level set 58 of the quantization level set 52 of the quantization level set 52 for the current neural network parameter 13', depending on the state associated with the current neural network parameter 13', and by updating the state for the subsequent neural network parameter depending on the quantization index 58 encoded in the data stream for the immediately preceding neural network parameter. Other approaches besides state transitions, such as using a transition table, can also be used and are defined below.

[0043] Furthermore, or alternatively, the device may be configured to select a reconstruction level set 48 from a plurality of reconstruction level sets 52 for the current neural network parameter 13', depending on the result of a binary function of the quantization index 58 encoded in a data stream 14 for previously encoded neural network parameters. The binary function may be, for example, a parity check using a bitwise "and" operation that signals whether the quantization index 58 represents an even or odd number. This may also provide information about the reconstruction level set 48 used to encode the quantization index 58, and thus, for example, a set of reconstruction levels 48 for the current neural network parameter 13' can be determined for a predetermined order of reconstruction levels, so that the corresponding decoder selects the corresponding reconstruction level set 48 for a predetermined order. Parity can be used for the state transitions described above.

[0044] Furthermore, according to the embodiment, the device can be configured to select, for example, a quantization level set 48 from a plurality of reconstruction level sets 52 for the current neural network parameter 13', depending on the parity of the quantization index 56 encoded in the data stream 14 for the previously encoded neural network parameter. The parity check can be performed at a low computational cost, for example, using a bitwise "and" operation.

[0045] Optionally, the device can be configured to encode quantization indices (56) for the neural network parameters (13) and to perform the quantization of the neural network parameters (13) along a common continuous order (14') between the neural network parameters (13). That is, the same order can be used for both tasks.

[0046] Figure 4 is a schematic diagram of the concept for arithmetic decoding of quantized neural network parameters according to an embodiment. This can be used within the apparatus of Figure 2. Thus, Figure 4 can be seen as a possible extension of Figure 2. It shows a data stream 14 to be decoded by the apparatus of Figure 4, where the quantization index 56 for the current neural network parameters 13' is arithmetic coding, e.g., binary arithmetic coding, as shown as an arbitrary example. For example, a probabilistic model defined by some context is used, which depends on a selected set of reconstruction levels 48 for the current neural network parameters 13', as indicated by arrows 123. Further details are set forth herein.

[0047] As explained with respect to Figure 2, the state transition process allows a selection 54 to select a set of quantization levels 48 from a set of 50 reconstruction level sets 52 for the current neural network parameter 13' by determining a set of reconstruction levels 48 from a set of 50 reconstruction level sets 52 for the current neural network parameter 13', depending on the state associated with the current neural network parameter 13', and by updating the state for subsequent neural network parameters depending on the quantization index 58 decoded from the data stream for the previous neural network parameter. Thus, the state is equivalent to a pointer to the set of reconstruction level sets 48 to be used to encode / decode the current neural network parameter 13', but this is updated with finer precision, however, by distinguishing only a number of states corresponding to the number of reconstruction sets, so that the state functions pseudo-as a memory of past neural network parameters or past quantization indices. Thus, the state defines the order of the sets of reconstruction levels used to encode / decode the neural network parameter 13'. As shown in Figure 4, for example, the quantization index (56) for the current neural network parameter (13') is decoded from the data stream (14) using arithmetic coding with a probabilistic model (122) corresponding to the state for the current neural network parameter (13'). By adapting the probabilistic model according to the state, the estimation of the probabilistic model can be improved, and coding efficiency can be increased. Furthermore, by adapting according to the state, computationally efficient adaptation may be possible with the transmission of less additional data.

[0048] In a further embodiment, the apparatus may be configured to decode the quantization index 56 for the current neural network parameter 13' from the data stream 14 using binary arithmetic coding, for example, using a probabilistic model that depends on the state of the current neural network parameter 13' for at least one bin 84 of the binarized quantization index 56.

[0049] Furthermore, or alternatively, the device may be configured such that the dependency of the probabilistic model includes a selection 103 (derivative) of context 87 from a set of contexts for the neural network parameters that use the dependency, each context being configured to associate with a given probabilistic model. The better the probabilistic estimation used, the more efficient the compression. The probabilistic model can be updated, for example, using context-adaptive (binary) arithmetic coding.

[0050] Optionally, the device can be configured to update a predetermined probability model associated with each context based on arithmetic-encoded quantized indices using each context. In this way, the contextual probability model is adapted to actual statistics.

[0051] Furthermore, the device can be configured to decode the quantization index 56 for the current neural network parameters 13' from the data stream 14 using binary arithmetic coding, for example, for at least one bin of the binarized quantization index, using a probabilistic model corresponding to a set of reconstruction levels 48 selected for the current neural network parameters 13'.

[0052] Optionally, at least one bin may include a significance bin indicating whether the quantization index 56 of the current neural network parameter is equal to zero. Furthermore, or alternatively, at least one bin may include a sign bin indicating whether the quantization index 56 of the current neural network parameter is greater than zero or less than zero. Furthermore, at least one bin may include a greater-than-X bin indicating whether the absolute value of the quantization index 56 of the current neural network parameter is greater than X, where X is an integer greater than zero.

[0053] Figure 5 illustrates the opposite of the decoding concept described using Figure 4. Therefore, all explanations and advantages can be appropriately applied to the following aspects of the coding concept.

[0054] Figure 5 is a schematic diagram illustrating a concept for arithmetic coding of neural network parameters according to an embodiment. This can be used within the apparatus of Figure 3. Thus, Figure 5 can be seen as a possible extension of Figure 3. It shows a data stream 14 that is coded by the apparatus of Figure 3, such that the quantization index 56 for the current neural network parameters 13' is coded by arithmetic coding, for example, binary arithmetic coding as an arbitrary example. For example, a probabilistic model defined by some context is used, which depends on a set of reconstruction levels 48 selected for the current neural network parameters 13', as indicated by arrows 123. Further details are set forth herein.

[0055] As explained with respect to Figure 3, selection 54 is performed on the current neural network parameter 13'. Selection 54 selects a quantization level set 48 from among multiple 50 reconstruction level sets 52 by a state transition process, by determining a quantization level set 48 from among multiple 50 reconstruction level sets 52 for the current neural network parameter 13', depending on the state associated with the current neural network parameter 13', and by updating the state for the subsequent neural network parameter according to the quantization index 58 encoded in the data stream for the previous neural network parameter.

[0056] Therefore, the state is equivalent to a pointer to the reconstruction level set 48 that should be used to encode / decode the current neural network parameter 13', but is updated with finer precision, simply distinguishing a number of states corresponding to the number of reconstruction sets, so that the state pseudo functions as memory for past neural network parameters or past quantization indices. Thus, the state defines the order of the reconstruction level sets used to encode / decode the neural network parameter 13'.

[0057] Furthermore, the quantization index 56 for the current neural network parameter 13' can be encoded into the data stream 14 using arithmetic coding that employs a probabilistic model corresponding to the state 122 for the current neural network parameter 13'.

[0058] For example, as shown in Figure 3, the quantization index 56 is encoded into the data stream 14 using binary arithmetic coding for the current neural network parameters 13', using a probabilistic model corresponding to the state 122 for the current neural network parameters 13' for at least one bin 84 of the binarized quantization index 56 82. Since the probabilistic model may be suitable for estimating the probabilistic model, adapting the probabilistic model according to the state may improve coding efficiency. Furthermore, state-dependent adaptation may enable computationally efficient adaptation with less additional data transmitted.

[0059] Furthermore, or alternatively, the device can be configured such that the dependency of the probabilistic model includes a selection 103 (derivation) of context 87 from a set of contexts of neural network parameters using the dependency, and each context is associated with a predetermined probabilistic model.

[0060] Optionally, the device can be configured to update a predetermined probabilistic model associated with each context based on arithmetic-encoded quantization indices using each context.

[0061] Furthermore, the device can be configured to encode the quantization index 56 of the current neural network parameter 13' into the data stream 14 using binary arithmetic coding, for example, by using a probabilistic model corresponding to a selected set of reconstruction levels 48 for the current neural network parameter 13' for at least one bin of the binarized quantization index. In order to use binary arithmetic coding, the quantization index 56 can be binarized.

[0062] Optionally, at least one bin may include a significance bin indicating whether the quantization index 56 of the current neural network parameter is equal to zero. Furthermore, or alternatively, at least one bin may include a sign bin indicating whether the quantization index 56 of the current neural network parameter is greater than zero or less than zero. Furthermore, at least one bin may include a greater-than-X bin indicating whether the absolute value of the quantization index 56 of the current neural network parameter is greater than X, where X is an integer greater than zero.

[0063] The embodiments described below focus on another aspect of the present invention, where the parameterization of the neural network is encoded in a stage or reconstruction layer, and for each NN parameter, one value from each stage must be combined to obtain an improved / enhanced representation of the neural network, at least one of which may itself represent a reasonable representation of the neural network but is enhanced to one of the stages that contribute less quality, although the latter possibility is not essential to this embodiment.

[0064] Figure 6 is a schematic diagram of the concept of using a reconstruction layer for neural network parameters for use in embodiments of the present invention. Figure 6 shows that reconstruction layer i, which is, for example, a second reconstruction layer, reconstruction layer i-1, which is, for example, a first reconstruction layer, and neural network (NN) layer p, which is, for example, layer 10b from Figure 3, are layers represented in the form of arrays or matrices, such as matrix 15a from Figure 3.

[0065] Figure 6 shows a conceptual device 310 for reconstructing neural network parameters 13 that define a neural network. Thus, the device is configured to derive first neural network parameters 13a, which may be called the neural network parameters of the first reconstruction layer, for example for a first reconstruction layer, e.g., reconstruction layer i-1, that may have been previously transmitted, for example during the associative learning process, and to yield neural network parameter values ​​for the first reconstruction layer for each neural network parameter, e.g., for each weight or for each interneuron connection. This derivation may otherwise include decoding or receiving the first neural network parameters 13a. Furthermore, the device is configured to decode 312 second neural network parameters 13b, which may be called the neural network parameters of the second reconstruction layer, for example, to distinguish them from, for example, the final neural network parameters, e.g., parameter 13, for the second reconstruction layer from a data stream 14 for generating neural network parameter values ​​for the second reconstruction layer for each neural network parameter 13. Therefore, two contribution values ​​for the first and second reconstruction layers may be obtained for each NN parameter, and the encoding / decoding of the first and / or second NN parameter values ​​can be performed using dependent quantization as shown in Figures 2 and 3, and / or arithmetic encoding / decoding of quantization indices as described in Figures 4 and 5. The second neural network parameter 13b may not have a self-contained meaning from the perspective of neural representation, but may simply derive the neural network representation, i.e., the final neural network parameters, when combined with the parameters of the first representation layer.

[0066] Furthermore, the device is configured to reconstruct the neural network parameters 13 314 for each neural network parameter by combining the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer, for example, using element-wise addition and / or multiplication (CB).

[0067] Furthermore, Figure 6 illustrates the concept of a device 320 for encoding neural network parameters 13 that define a neural network, using a first neural network parameter 13a for a first reconstruction layer, e.g., reconstruction layer i-1. The first reconstruction layer contains the neural network parameter value of the first reconstruction layer for each neural network parameter 13. Thus, the device is configured to encode a second neural network parameter 13b for a second reconstruction layer, e.g., reconstruction layer i, into a data stream 322. The second reconstruction layer contains the neural network parameter value of the second reconstruction layer for each neural network parameter 13. The neural network parameters 13 can be reconstructed by combining (CB) the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer, for example, using element-wise addition and / or multiplication for each neural network parameter.

[0068] Optionally, the device 310 can be configured to decode the first neural network parameters for the first reconstruction layer from the data stream 14 or from a separate data stream to the device 316.

[0069] Simply put, the decomposition of neural network parameters 13 may allow for more efficient encoding and / or decoding and transmission of the parameters.

[0070] Further embodiments, particularly those involving the neural network coding concept, are disclosed below. The following description provides further details that can be used individually and in combination with the embodiments described above.

[0071] First, we present a method for entropy coding of neural network parameters with dependent scalar quantization according to an embodiment of the present invention.

[0072] A method for parameter coding a set of neural network parameters 13 (also called weights, weight parameters, or parameters) using dependent scalar quantization is described. The parameter coding presented herein consists of dependent scalar quantization of the parameters 13 (e.g., as described in the context of Figure 3) and entropy coding of the resulting quantization index 56 (e.g., as described in the context of Figure 5). On the decoder side, the set of reconstructed neural network parameters 13 can be obtained by entropy decoding of the quantization index 56 (e.g., as described in the context of Figure 4) and dependent reconstruction of the neural network parameters 13 (e.g., as described in the context of Figure 2). In contrast to parameter coding with independent scalar quantization and entropy coding, the acceptable set of reconstruction levels for the neural network parameters 13 depends on the transmitted quantization index 56 that precedes the current neural network parameter 13' in the reconstruction order. The following presentation further describes a method for entropy coding of the quantization index that specifies the reconstruction level used in dependent scalar quantization.

[0073] This explanation primarily focuses on lossy coding of the neural network parameter layer in neural network compression, but it can also be applied to lossy coding in other fields.

[0074] The methodology of this device can be divided into the following distinct main parts:

[0075] 1. Quantization 2. Lossless encoding 3. Lossless Decryption

[0076] To understand the main advantages of the embodiments described below, we will first briefly introduce the topic of neural networks and the relevant methods for parameter coding. Nevertheless, all aspects, features, and concepts disclosed can be used separately or in combination with the embodiments described herein.

[0077] 2. Related methods for quantization and entropy coding. Working Draft 2 of the MPEG-7 Part 17 standard for neural network compression for describing and analyzing multimedia content[2] applies independent scalar quantization and entropy coding to neural network parameter coding.

[0078] 2.1 Scalar Quantizer The parameters of the neural network are quantized using a scalar quantizer. As a result of quantization, the set of acceptable values ​​for parameter 13 is reduced. In other words, the parameters of the neural network are mapped to a countable (actually finite) set of so-called reconstruction levels. The set of reconstruction levels represents an appropriate subset of the set of possible neural network parameter values. To simplify the entropy coding below, the acceptable reconstruction levels are represented by quantization index 56, which is transmitted as part of the bitstream 14. On the decoder side, quantization index 56 is mapped to the reconstructed neural network parameter 13. The possible values ​​of the reconstructed neural network parameter 13 correspond to the set of reconstruction levels 52. On the encoder side, the result of scalar quantization is a set of (integer) quantization indices 56.

[0079] JPEG2026067854000003.jpg51170

[0080] In this context, the term "independent scalar quantization" refers to the property that, given a quantization index q56 for any weight parameter 13, the associated reconstructed weight parameter t'13' can be determined independently of all quantization indices for other weight parameters.

[0081] 2.1.1 Encoder Operation: Quantization Standards for neural network compression only specify the bitstream syntax and reconstruction process. Given a given set of original neural network parameters 13 and a given quantization step size (QP), the encoder has many degrees of freedom. The quantization index q of layers 10a and 10b... k Given 56, entropy coding must follow a uniquely defined algorithm for writing data to bitstream 14 (i.e., constructing an arithmetic codeword). However, given the original set of weight parameters (e.g., layers), the quantization index q k The encoding algorithm to obtain 56 is outside the scope of neural network compression standards. In the following explanation, we assume that the quantization step size (QP) of each neural network parameter 13 is known. Nevertheless, the encoder takes each neural network (weight) parameter t k Quantizer index q for 13 k You have the freedom to choose 56. The choice of quantization index determines both the distortion (or reconstruction / approximation quality) and the bitrate, so the quantization algorithm used has a substantial impact on the rate-distortion performance of the resulting bitstream 14.

[0082] JPEG2026067854000004.jpg142169

[0083] The quantization process yields rate-distortion-efficient results if the Lagrangian function D + λ·R is minimized. Here, D is the distortion of the neural network parameter set (e.g., MSE distortion or MAE distortion), R is the number of bits required to transmit quantization index 56, and λ is the Lagrangian multiplier.

[0084] JPEG2026067854000005.jpg85170

[0085] JPEG2026067854000006.jpg75169

[0086] In fact, nearest neighbor quantization is a trivial case where λ=0 and is applied in Working Draft 2 of the MPEG-7 Part 17 standard for neural network compression for describing and analyzing multimedia content.

[0087] 2.2 Entropy Coding As a result of the uniform quantization applied in the previous step, the weight parameters are mapped to a finite set of so-called reconstruction levels. These can be represented by (integer) quantizer index 56 (also called parameter levels or weight levels) and the quantization step size (QP), which may be fixed for all layers, for example. To reconstruct all quantized weight parameters of a layer, the step size (QP) and dimension of the layer may be known by the decoder. These may be transmitted separately, for example.

[0088] 2.2.1 Encoding of quantized indices using Context-adaptive binary arithmetic coding (CABAC) The quantization index 56 (integer representation) is then transmitted using entropy coding techniques. Thus, the weight layers are mapped to a sequence of weight levels quantized using scans. For example, a row-first scan order can be used, where the values ​​contained are coded from left to right, starting from the top row of the matrix. In this method, all rows are coded from top to bottom. The scan may be performed on a matrix 15a containing neural network parameters 13 that may be related to the weights of the neuron interconnects 11, for example, along a common continuous order 14', as shown in Figure 3. The matrix may represent the weights between layers p-1 10a and p 10b of the neuron interconnects 11, or between the hidden layer and the input layer, as shown in Figures 3 and 1, respectively. Note that any other arbitrary scan can be applied. For example, a matrix (e.g., matrix 15a in Figure 2 or Figure 3) can be transposed, or flipped horizontally and / or vertically, and / or rotated 90 / 180 / 270 degrees to the left or right before applying a row-first scan.

[0089] As described with respect to Figures 3 and 5, the apparatus according to the embodiment can be configured to encode the quantization index 56 for the current neural network parameter 13' into the data stream 14 using binary arithmetic coding, using a probabilistic model corresponding to the state 122 for the current neural network parameter 13' for at least one bin 84 of the binarized quantization index 56 82. The binary arithmetic coding using the probabilistic model can be context-adaptive binary arithmetic coding (CABAC).

[0090] In other words, according to the embodiment, CABAC is used for level encoding. For details, refer to [3]. Thus, the quantized weight level q56 may be decomposed into a series of binary symbols or syntax elements, such as bins (binary decisions), and then passed to a binary arithmetic coder (CABAC). In the first step, a binary syntax element sig_flag is derived for the quantized weight level, which specifies whether the corresponding level is equal to zero. In other words, at least one bin of the binarized 82 quantization index 56 shown in FIG. 4 may include a significance bin indicating whether the quantization index 56 of the current neural network parameter is equal to zero.

[0091] If sig_flag is equal to 1, a further binary syntax element sign_flag is derived. This bin indicates whether the current weight level is positive (e.g., bin = 0) or negative (e.g., bin = 1). In other words, at least one bin of the binarized 82 quantization index 56 shown in FIG. 4 may include a sign bin 86 indicating whether the quantization index 56 of the current neural network parameter is greater than or less than zero.

[0092] Next, the unary sequence of bins is encoded, followed by encoding the fixed-length sequence as follows.

[0093] The variable k is initialized with a non-negative integer, and X is initialized with 1 << k.

[0094] One or more syntax elements, called abs_level_greater_X, indicating that the absolute value of the quantized weight level is greater than X, are encoded. When abs_level_greater_X is equal to 1, the variable k is updated (e.g., incremented by 1), then 1<<k is added to X, and then abs_level_greater_X is encoded again. This procedure continues until abs_level_greater_X is equal to 0. After that, the quantization index can be encoded with a fixed-length code of length k. For example, the variable rem = X - |q| can be encoded using k bits. Alternatively, the variable rem' can be defined as rem'=(1<<k)-rem-1, which is encoded using k bits. Other mappings of the variable rem to a fixed-length code of k bits can also be used instead.

[0095] In other words, at least one bin of the binarized 82 quantization index 56 shown in FIG. 4 can include a greater-than-X bin indicating whether the absolute value of the quantization index 56 of the current neural network parameters is greater than X, where X is an integer greater than zero.

[0096] When k is incremented by 1 after each abs_level_greater_X, this approach is identical to applying exponential Golomb coding (when the sign_flag is not considered).

[0097] Also, if the maximum absolute value abs_max is known on both the encoder side and the decoder side, for the next abs_Level_greater_X to be transmitted, when X >= abs_max holds, the encoding of the abs_Level_greater_X syntax element can be terminated.

[0098] JPEG2026067854000007.jpg103169

[0099] In other words, as described with respect to Figures 2 and 4, the apparatus according to the embodiment can be configured to decode the quantization index 56 for the current neural network parameters 13' from the data stream 14 using binary arithmetic coding by using a probabilistic model corresponding to the state 122 for the current neural network parameters 13' for at least one bin 84 of the binarized quantization index 56 82.

[0100] At least one bin of the binarized quantization index 56 shown in Figure 5 may include a significance bin indicating whether the quantization index 56 of the current neural network parameter is equal to zero. Alternatively, at least one bin may include a sign bin 86 indicating whether the quantization index 56 of the current neural network parameter is greater than zero or less than zero. Furthermore, at least one bin may include a greater-than-X bin indicating whether the absolute value of the quantization index 56 of the current neural network parameter is greater than X, where X is an integer greater than zero.

[0101] In a preferred embodiment, k is initialized to 0 and updated as follows: After each abs_level_greater_X equals 1, any necessary updates to k are made according to the following rules: If X > X', k is incremented by 1, where X' is a constant corresponding to the application. For example, X' is a number derived by the encoder and communicated to the decoder (e.g., between 0 and 100).

[0102] 2.2.3 Context Modeling In CABAC entropy coding, most syntax elements with quantized weight levels 56 are coded using binary probabilistic modeling. Each binary decision (bin) is associated with a context, which represents a probabilistic model for the class of coded bins. The probability for one of two possible bin values ​​is estimated for each context based on the values ​​of bins already coded in the corresponding context. Depending on the application, different context modeling approaches can be applied. Typically, for several bins related to quantized weight coding, the context used for coding is selected based on syntax elements already transmitted. Depending on the actual application, different probabilistic estimators may be selected, such as SBMP 0, or HEVC 0 or VTM-4.0 0. This selection affects, for example, compression efficiency and complexity.

[0103] In other words, the probabilistic model described in Figure 5, for example with respect to context 87, additionally depends on the quantization index of the previously encoded neural network parameters.

[0104] Each of these probabilistic models, such as the one described in Figure 4 for context 87, further depends on the quantization index of the previously decoded neural network parameters.

[0105] A contextual modeling scheme that fits a wide range of neural networks is described as follows: A local template is applied to the current position to decode the weight level q56 quantized at a specific position (x,y) in the weight matrix (layer). This template contains many other (ordered) positions, such as (x-1,y), (x,y-1), (x-1,y-1), etc. For each position, a status identifier is derived.

[0106] JPEG2026067854000008.jpg37170

[0107] For a specific template, derive a sequence of status identifiers and map each possible constellation of the values of the status identifiers to a context index that identifies the context in which it is used. The template and mapping may be different for different syntactic elements. For example, from a template that includes the (ordered) positions (x-1,y), (x,y-1), (x-1,y-1), a sequence of status identifiers s x-1,y , s x,y-1 , s x-1,y-1 is derived. For example, this sequence may be mapped to a context index C = s x-1,y + 3*s x,y-1 + 9*s x-1,y-1 . For example, the context index C can be used to identify a number of contexts for sig_flag.

[0108] In a preferred embodiment (approach 1), for sign_flag, or for the quantized weight level q x,y at position (x,y) for sig_flag, the local template consists of only one position (x-1,y) (i.e., the left neighbor). The associated status identifier s x-1,y is derived according to the preferred embodiment Si1.

[0109] For sig_flag, one of three contexts is selected according to the value of s x-1,y , or for sign_flag, one of another three contexts is selected according to the value of s x-1,y .

[0110] In another preferred embodiment (approach 2), the local template for sig flag includes three ordered positions (x-1,y), (x-2,y), (x-3,y). The associated sequence of status identifiers s x-1,y , s x-2,y , s x-3,y is derived according to the preferred embodiment Si2.

[0111] For sig_flag, the context index C is derived as follows.

[0112] s x-1,y If ≠0, then C=0; otherwise, s x-2,y If ≠0, then C=1; otherwise, s x-3,y If ≠0, then C=2; otherwise, s x-3,y If ≠ 0, then C=2. Otherwise, C=3.

[0113] JPEG2026067854000009.jpg25170

[0114] Similarly, the number of left-side neighbors may be increased or decreased so that the context index C is equal to the distance to the next non-zero weight on the left (not exceeding the template size).

[0115] Each abs_level_greater_X flag can apply its own set of two contexts, for example. Then, one of the two contexts is selected depending on the value of sign_flag.

[0116] In a preferred embodiment, for abs_level_greater_X flags where X is less than a predefined number X', different contexts are distinguished depending on the value of X and / or sign_flag.

[0117] In a preferred embodiment, for the abs_level_greater_X flag where X is greater than or equal to a predefined number X', different contexts are distinguished only in relation to X.

[0118] In another preferred embodiment, an abs_level_greater_X flag having X greater than or equal to a predefined number X' is encoded using a fixed code length of 1 (for example, using the bypass mode of an arithmetic encoder).

[0119] Furthermore, some or all of the syntax elements may be encoded without using context. Instead, they are encoded with a fixed length of 1 bit, for example, using the so-called bypass bins of CABAC.

[0120] In another preferred embodiment, the fixed-length remainder rem is encoded using the bypass mode.

[0121] In another preferred embodiment, the encoder determines a predefined number X', distinguishes two contexts according to the sign for each syntax element abs_level_greater_X where X < X', and uses one context for each abs_level_greater_X where X >= X'.

[0122] In other words, a probability model as described with respect to FIG. 5, for example context 87, can be selected 103 for the current neural network parameters from among a subset of the probability models according to the quantization index of the previously encoded neural network parameters related to the part of the neural network adjacent to the part to which the current neural network parameters are related.

[0123] This part can be defined, for example, by a template as described above, a template that includes the (ordered) positions (x - 1, y), (x, y - 1), (x - 1, y - 1).

[0124] Respectively, a probability model as described with respect to FIG. 5 can be selected for the current neural network parameters from among a subset of the probability models according to the quantization index of the previously decoded neural network parameters related to the part of the neural network adjacent to the part to which the current neural network parameters are related.

[0125] 3 Additional Methods Below, we will describe additional, and therefore optional, methods for compressing / transmitting the neural network 10, where the reconstructed layers, for example, neural network layer p from Figure 6, are composed of different sublayers such as reconstructed layer i-1 and reconstructed layer i from Figure 6, which may be transmitted separately, for example.

[0126] 3.1 Concepts of Base Layer and Enhancement Layer This concept introduces two types of sublayers called base layers and enhancement layers. The reconstruction process (e.g., adding all sublayers) then defines how the reconstructed layers can be obtained from the sublayers. The base layer contains base values ​​and can be selected, for example, to be efficiently represented or compressed / transmitted in the first step. The enhancement layer contains enhancement information, which can be added to the values ​​of the (base) layer, for example, to reduce distortion metrics (e.g., relative to the original layer). In another example, the base layer contains coarse values ​​(from training with a small training set), and the enhancement layer contains refinement values ​​(based on the full training set or, more generally, another training set). Sublayers may be stored / transmitted separately.

[0127] In a preferred embodiment, the layer L is compressed. R For example, the neural network parameters, such as the weights that can be represented by matrix 15a in Figures 2 and 3, are the base layer L B and one or more enhancement layers L E,1 ,L E,2 ,···,L E,N It is then broken down into two parts. In the first step, the base layer is compressed / transmitted, and in the following step, the enhancement layer L E,1 ,L E,2 ,..., L E,N It is compressed / sent (separately).

[0128] JPEG2026067854000010.jpg40164

[0129] JPEG2026067854000011.jpg34166

[0130] In other words, the embodiment according to the present invention is a reconstructed layer L R In the form of, or for example, a reconstructed layer L R The apparatus includes a device configured to reconstruct the neural network parameters 13 for each neural network parameter by the sum or product of the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer, in parameter units.

[0131] In each embodiment, for the apparatus for encoding neural network parameters 13, the neural network parameters 13 can be reconstructed for each neural network parameter by the sum or product of the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer in parameter units.

[0132] In a further preferred embodiment, the methods of 2.1 and / or 2.2 are applied to a subset or all sublayers.

[0133] In a particularly preferred embodiment, an entropy coding scheme using context modeling (e.g., homogeneous or similar to 2.2.3) is applied, but one or more sets of context models are added according to one or more of the following rules:

[0134] a) Each sublayer applies its own set of contexts. In other words, embodiments of the present invention include a device configured to encode / decode a first neural network parameter 13a for a first reconstruction layer into / from a data stream or another data stream, and to encode / decode a second neural network parameter 13b for a second reconstruction layer into / from a data stream by context-adaptive entropy coding using a different probabilistic context for the first and second reconstruction layers.

[0135] b) The set of contexts selected for the parameters of the enhancement layer to be encoded depends on the values ​​of the corresponding parameters in the preceding layer in the encoding order (e.g., the base layer). A first set of context models is always selected when the corresponding parameters are equal to zero; otherwise, a second set is selected. In other words, embodiments of the present invention include a device configured to encode the neural network parameter values ​​of a second reconstructed layer, e.g., the parameters of an enhancement layer, into a data stream by context-adaptive entropy coding using a probabilistic model corresponding to the neural network parameter values ​​of a first reconstructed layer, e.g., the values ​​of the corresponding parameters in the preceding layer in the encoding order (e.g., the base layer). Further embodiments include a device configured to encode the second reconstructed layer neural network parameter values ​​into a data stream by context-adaptive entropy coding by selecting a probabilistic context set from a set of probabilistic context sets in response to the first reconstructed layer neural network parameter values, and by selecting a probabilistic context to use from the probabilistic context set selected in response to the first reconstructed layer neural network parameter values. Each embodiment includes a device for decoding the neural network parameters 13 according to the embodiment, wherein the device can be configured to decode the second reconstructed layer neural network parameter values ​​from a data stream by context-adaptive entropy decoding using a probabilistic model corresponding to the first reconstructed layer neural network parameter values. Further embodiments include a device configured to decode the second reconstructed layer neural network parameter values ​​from a data stream by context-adaptive entropy decoding by selecting a probabilistic context set from a set of probabilistic context sets corresponding to the first reconstructed layer neural network parameter values, and by selecting a probabilistic context to be used from the selected probabilistic context set corresponding to the first reconstructed layer neural network parameter values.

[0136] c) The selected context set for the parameters of the enhancement layer to be encoded corresponds to the parameter values ​​at the same position in the preceding layer (e.g., the base layer) in the encoding order. A first set of context models is always selected if the parameter at the same position is less than zero (negative), a second set is selected if the parameter at the same position is greater than zero (positive), and a third set is selected otherwise. In other words, embodiments of the present invention include, for example, an apparatus for encoding. The set of probabilistic context sets includes three probabilistic context sets. The apparatus is configured to select a first probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set if the neural network parameter value of the first reconstruction layer is negative, to select a second probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set if the neural network parameter value of the first reconstruction layer is positive, and to select a third probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set if the neural network parameter value of the first reconstruction layer is zero. In each embodiment, for the apparatus for decoding the neural network parameters 13, the set of probability context sets may include three probability context sets, and the apparatus may be configured to select a first probability context set from the set of probability context sets as the selected probability context set if the neural network parameter value of the first reconstruction layer is negative, to select a second probability context set from the set of probability context sets as the selected probability context set if the neural network parameter value of the first reconstruction layer is positive, and to select a third probability context set from the set of probability context sets as the selected probability context set if the neural network parameter value of the first reconstruction layer is zero.

[0137] d) The set of contexts selected for the parameters of the enhancement layer to be encoded corresponds to the values ​​of the parameters at the same position in the preceding layer (e.g., the base layer) in the encoding order. The first set of context models is always selected when the (absolute) value of the parameter at the same position is greater than X (where X is the parameter); otherwise, the second set is selected. In other words, an embodiment of the present invention includes an apparatus, the set of probabilistic context sets includes two probabilistic context sets, and the apparatus is configured to select a first probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set if the parameter value of the neural network in the first reconstruction layer, for example, the value of the parameter at the same position in a preceding layer in the coding order (e.g., the base layer), is greater than a predetermined value, e.g., X; to select a second probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set if the neural network parameter value of the first reconstruction layer is not greater than the predetermined value; or to select a first probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set if the absolute value of the neural network parameter value of the first reconstruction layer is greater than the predetermined value; and to select a second probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set if the absolute value of the neural network parameter value of the first reconstruction layer is not greater than the predetermined value.Each embodiment of the apparatus for decoding the neural network parameters 13 can include two sets of probability contexts, and the apparatus can be configured to select a first probability context set from the set of probability context sets as the selected probability context set if the neural network parameter value of the first reconstruction layer is greater than a predetermined value, e.g., X, and to select a second probability context set from the set of probability context sets as the selected probability context set if the neural network parameter value of the first reconstruction layer is not greater than the predetermined value, or to select a first probability context set from the set of probability context sets as the selected probability context set if the absolute value of the neural network parameter value of the first reconstruction layer is greater than a predetermined value, and to select a second probability context set from the set of probability context sets as the selected probability context set if the absolute value of the neural network parameter value of the first reconstruction layer is not greater than the predetermined value.

[0138] 4. Neural network parameter coding with dependent scalar quantization This section discloses further optional aspects and features of the concept and embodiments of the present invention, as described in the context of Figures 2 to 4.

[0139] The following describes a modified concept of neural network parameter coding. The main change from the previously described neural network parameter coding is that the neural network parameters 13 are not quantized and reconstructed independently. Instead, the permissible reconstruction levels of the neural network parameters 13 correspond to the selected quantization index 56 of the preceding neural network parameters in the reconstruction order. The concept of dependent scalar quantization is combined with modified entropy coding, where probabilistic model selection (or alternatively, codeword table selection) for the neural network parameters depends on a set of permissible reconstruction levels. However, it should be noted that the previously described embodiments can be used and / or incorporated and / or extended using any of the features described below individually or in combination.

[0140] 4.1 Advantages compared to parameter coding of related neural networks The advantage of dependent quantization of neural network parameters is that the allowed reconstruction vector is densely packed in an N-dimensional signal space (where N represents the set of samples being processed, e.g., the number of samples in layers 10a and 10b, or the number of neural network parameters 13). The reconstruction vector of the neural network parameter set refers to the ordered reconstructed neural network parameters (or, alternatively, the ordered reconstructed samples) of the neural network parameter set. The effect of dependent scalar quantization is illustrated in Figure 8 for the simplest case of two neural network parameters. Figure 8 shows an example of the location of the allowed reconstruction vector for the simple case of two weight parameters. Figure 8(a) is an example of independent scalar quantization, and Figure 8(b) is an example of dependent scalar quantization. Figure 8a shows the allowed reconstruction vector 201 (representing a point on a two-dimensional plane) in the case of independent scalar quantization. As can be seen, the set of allowed values ​​for the second neural network parameter t1'13 does not depend on the selected values ​​for the first reconstructed neural network parameter t0'13. Figure 8(b) shows an example of dependent scalar quantization. Note that, in contrast to independent scalar quantization, the selectable reconstruction values ​​for the second neural network parameter t1'13 depend on the reconstruction level selected for the first neural network parameter t0'13. In the example in Figure 8b, there are two different sets 52 available reconstruction levels 52 for the second neural network parameter t1'13 (illustrated in different colors). If the quantization index 56 for the first neural network parameter t0'13 is even (...,-2,0,2,...), then any reconstruction level 201a from the first set (blue dots) can be selected for the second neural network parameter t1'13.Then, if the quantization index 56 for the first neural network parameter t0' is odd (..., -3, -1, 1, 3, ...), it is possible to select any reconstruction level 201b from the second set (red dots) for the second neural network parameter t1'13. In this example, the reconstruction levels of the first and second sets are shifted by half the quantization step size (any reconstruction level in the second set lies between two reconstruction levels in the first set).

[0141] Dependent scalar quantization of neural network parameters 13 has the effect of reducing the expected value of the distance between a given input vector of neural network parameters 13 and the nearest available reconstructed vector, with respect to a given average number of reconstructed vectors 201 per N-dimensional unit volume. As a result, the mean distortion between the input vector of the neural network parameter and the vector-reconstructed neural network parameter can be reduced with respect to a given average number of bits. In vector quantization, this effect is called the space-filling gain. By using dependent scalar quantization for the neural network parameter set 13, a large portion of the potential space-filling gain of high-dimensional vector quantization can be utilized. Furthermore, in contrast to vector quantization, the implementation complexity of the reconstruction (or decoding) process is comparable to the implementation complexity of the related neural network parameter coding using independent scalar quantizers.

[0142] 4.2 Overview The main change, as mentioned above, is dependency quantization. The reconstructed neural network parameters t are determined by the reconstruction order index k>0. k '13 is the associated quantization index q k Not only 56, but also the quantization indices q0, q1, ..., q of the preceding neural network parameters in the reconstruction order. k-1It will also depend on this. Note that in dependent quantization, the reconstruction order of the neural network parameters 13 must be uniquely determined. Quantization index q k When knowledge of the reconstruction level set related to 56 is also utilized in entropy coding, the overall performance of the neural network codec can typically be improved. In other words, it is typically preferable to switch the context (probability model) or codeword table based on the reconstruction level set applied to the neural network parameters.

[0143] Entropy coding is usually uniquely defined given the entropy decoding process. However, as with related neural network parameter coding, there are many degrees of freedom in the selection of quantization indices given the original neural network parameters.

[0144] The embodiments described herein are not limited to layer-based neural network coding. They are also applicable to neural network parameter coding of any finite set of neural network parameters 13.

[0145] In particular, this method can also be applied to sublayers as described in sec.3.1.

[0146] 4.3 Dependent Quantization of Neural Network Parameters Dependent quantization of neural network parameter 13 refers to the concept that the set of reconstruction levels available for neural network parameter 13 depends on the selected quantization index for preceding neural network parameters in the reconstruction order (for example, within the same set of neural network parameters such as a layer or sublayer).

[0147] In a preferred embodiment, a plurality of sets of reconstruction levels are predefined, and one of the predefined sets is selected to reconstruct the current neural network parameters based on the quantization index of the preceding neural network parameters in the coding order. In other words, the apparatus according to the embodiment can be configured to select a reconstruction level set 48 from a plurality of reconstruction level sets 52 for the current neural network parameters 13, depending on the quantization index (58) of a previous, for example, preceding neural network parameter.

[0148] A preferred embodiment for defining a set of reconstruction levels is described in Section 4.3.1. The identification and signaling of selected reconstruction levels are described in Section 4.3.2. Section 4.3.3 describes a preferred embodiment for selecting one of a predefined set of reconstruction levels for the current neural network parameters (based on the selected quantization index of the preceding neural network parameters in the reconstruction order).

[0149] 4.3.1 Reconfiguration Level Set In a preferred embodiment, the acceptable set of reconstruction levels for the current neural network parameters is selected from a predefined set of 52 reconstruction levels (two or more sets, e.g., set 0 and set 1 from Figures 2 and 3) (based on the quantization index of the preceding neural network parameters in the coding order).

[0150] In a preferred embodiment, the parameters determine the quantization step size Δ(QP), and all reconstruction levels (in all sets of reconstruction levels) represent integer multiples of the quantization step size Δ. However, it should be noted that each set of reconstruction levels contains only a subset of integer multiples of the quantization step size Δ(QP). Such a configuration for dependent quantization can be considered an extension of the Uniform Reconstruction Quantizer (URQs), where all possible reconstruction levels for all sets of reconstruction levels represent integer multiples of the quantization step size (QP). Its fundamental advantage is that the reconstructed neural network parameters 13 can be computed by algorithms with very low computational complexity (described in more detail below).

[0151] Reconstruction level sets can be completely discontinuous, but one or more reconstruction levels can be included in multiple sets (although the sets still differ in other reconstruction levels).

[0152] In a preferred embodiment, the dependent scalar quantization for the neural network parameters uses just two different sets of reconstruction levels, e.g., set 0 and set 1. And in a particularly preferred embodiment, the neural network parameter t k All reconstruction levels of the two sets for 13 are equal to the quantization step size Δ for this neural network parameter 13. k This represents an integer multiple of (QP). Note that the quantization step size is Δ. k Note that (QP) merely represents the scaling factor for the acceptable reconstruction values ​​of both sets. The same two sets of reconstruction levels are used for all neural network parameters 13.

[0153] Figure 9 illustrates three preferred configurations ((a)-(c)) for two sets of reconstruction levels (set 0 and set 1). Figure 9 shows an example of dependent quantization with two sets of reconstruction levels that are entirely determined by a single quantization step size Δ(QP). The two available sets of reconstruction levels are highlighted in different colors (set 0 is blue, set 1 is red). An example of a quantization index indicating the reconstruction level within a set is shown by the numbers below the circles. Hollow and filled circles represent two distinct subsets within the set of reconstruction levels, which can be used to determine the set of reconstruction levels for the next neural network parameter in the reconstruction order. The figure shows three preferred configurations with two sets of reconstruction levels: (a) the two sets are mismatched and symmetric with respect to zero, (b) both sets contain a reconstruction level equal to zero but are otherwise mismatched, and the sets are asymmetric around zero, and (c) both sets contain a reconstruction level equal to zero but are otherwise mismatched, and both sets are symmetric around zero. Note that all reconstruction levels reside on a grid given by integer multiples (IV) of the quantization step size Δ. Furthermore, it should be noted that certain reconstruction levels may be included in both sets.

[0154] The two sets depicted in Figure 9(a) are discontinuous. Each integer multiple of the quantization step size Δ(QP) is contained in only one of the sets. The first set (set 0) contains all even integer multiples (IV) of the quantization step size, while the second set (set 1) contains all odd integer multiples of the quantization step size. In both sets, the distance between two adjacent reconstruction levels is twice the quantization step size. These two sets are typically suitable for high-rate quantization, i.e., settings where the variance (square deviation) of the neural network parameters is significantly larger than the quantization step size (QP). However, in neural network parameter coding, the quantizer typically operates in the low-rate region. Typically, the absolute values ​​of many original neural network parameters 13 are closer to zero than any non-zero multiple of the quantization step size (QP). In that case, it is typically preferable that zero is included in both quantization sets (reconstruction level sets).

[0155] The two quantization sets shown in Figure 9(b) both contain zero. In set 0, the distance between a reconstruction level equal to zero and a first reconstruction level greater than zero is equal to the quantization step size (QP), while all other distances between two adjacent reconstruction levels are equal to twice the quantization step size. Similarly, in set 1, the distance between a reconstruction level equal to zero and a first reconstruction level less than zero is equal to the quantization step size, while all other distances between two adjacent reconstruction levels are equal to twice the quantization step size. It should be noted that both reconstruction sets are asymmetric around zero. This can be inefficient because it becomes difficult to accurately estimate the probability of the sign.

[0156] A preferred configuration for two sets of reconstruction levels is shown in Figure 9(c). The reconstruction levels included in the first quantization set (labeled Set 0 in the figure) represent even integer multiples of the quantization step size (note that this set is actually the same as Set 0 in Figure 9(a)). The second quantization set (labeled Set 1 in the figure) includes all odd integer multiples of the quantization step size, and also includes reconstruction levels equal to 0. Note that both reconstruction sets are symmetric with respect to zero. Reconstruction levels equal to zero are included in both reconstruction sets; otherwise, the reconstruction sets are discontinuous. The combination of both reconstruction sets includes all integer multiples of the quantization step size.

[0157] In other words, according to the embodiment, the apparatus comprises, for example, a device for encoding / decoding neural network parameters 13, wherein the number of multiple 50 reconstruction level sets 52 is 2 (e.g., set 0, set 1), and the multiple reconstruction level sets include a first reconstruction level set (set 0) which includes zero and even multiples of a predetermined quantization step size, and a second reconstruction level set (set 1) which includes zero and odd multiples of a predetermined quantization step size.

[0158] Furthermore, all reconstruction levels in all reconstruction level sets may represent integer multiples (IV) of a predetermined quantization step size (QP), and according to the embodiment, a device for decoding neural network parameters 13 may be configured to dequantize the neural network parameters 13 by deriving an intermediate integer value for each neural network parameter, for example, an integer multiple (IV) corresponding to the selected reconstruction level set for each neural network parameter and the entropy decoding quantization index 58 for each neural network parameter 13', and by multiplying the intermediate value of each neural network parameter by a predetermined quantization step size for each neural network parameter 13.

[0159] Each of the reconstruction levels in each reconstruction level set may represent an integer multiple (IV) of a predetermined quantization step size (QP), and the device can be configured, for example according to the embodiment, to encode the neural network parameters 13, by deriving an intermediate integer value for each neural network parameter corresponding to the selected reconstruction level set of each neural network parameter and the entropy coding quantization index of each neural network parameter, and by multiplying the intermediate value of each neural network parameter by a predetermined quantization step size of each neural network parameter, so that the same can be dequantized.

[0160] The embodiments specified herein are not limited to the configuration shown in Figure 9. Any two other different sets of reconstruction levels can be used. Multiple reconstruction levels may be included in both sets. Alternatively, the combination of both quantization sets may not include all possible integer multiples of the quantization step size. Furthermore, it is possible to use two or more sets of reconstruction levels for dependent scalar quantization of neural network parameters.

[0161] 4.3.2 Signaling of the Selected Reconstruction Level The reconstruction level that the encoder selects from the allowed reconstruction levels must be indicated within the bitstream 14. As with conventional independent scalar quantization, this can be achieved using a so-called quantization index 56, also called a weight level. The quantization index 56 (or weight level) is an integer value that uniquely identifies the reconstruction level available within the quantization set 52 (i.e., within the set of reconstruction levels). The quantization index 56 is sent to the decoder as part of the bitstream 14 (using any entropy coding technique). On the decoder side, the reconstructed neural network parameters 13 can be uniquely calculated based on the current set of reconstruction levels 48 (which is determined by the preceding quantization index in the coding / reconstruction order) and the sent quantization index 56 for the current neural network parameters 13'.

[0162] In a preferred embodiment, the assignment of quantization index 56 to the reconstruction levels within a set of reconstruction levels (or quantization set) follows the following rules: For illustrative purposes, the reconstruction levels in Figure 9 are assigned their associated quantization index 56 (the quantization index is given by the number below the circle representing the reconstruction level). If the set of reconstruction levels includes a reconstruction level equal to 0, then a quantization index equal to 0 is assigned to the reconstruction level equal to 0. A quantization index equal to 1 is assigned to the smallest reconstruction level greater than 0, a quantization index equal to 2 is assigned to the next reconstruction level greater than 0 (i.e., the second smallest reconstruction level greater than 0), and so on. Alternatively, in other words, reconstruction levels greater than 0 are labeled with integer values ​​greater than 0 in ascending order of their value (i.e., 1, 2, 3, etc.). Similarly, quantization index -1 is assigned to the largest reconstruction level less than 0, quantization index -2 is assigned to the next (i.e., the second largest) reconstruction level less than 0, and so on. Alternatively, to put it another way, reconstruction levels less than 0 are labeled with integer values ​​less than 0 in descending order of value (i.e., -1, -2, -3, etc.). The example in Figure 9 illustrates the quantization index assignments described for all quantization sets except set 1 in Figure 9(a) (which does not contain any reconstruction levels equal to 0).

[0163] One way to assign quantization index 56 to reconstruction levels for a quantization set that does not contain any reconstruction levels equal to 0 is as follows: assign a quantization index greater than 0 to all reconstruction levels greater than 0 (in ascending order of value), and assign a quantization index less than 0 to all reconstruction levels less than 0 (in descending order of value). Thus, the assignment of quantization index 56 follows essentially the same concept as a quantization set that contains reconstruction levels equal to 0, but differs in that there are no quantization indices equal to 0 (see the label for quantization set 1 in Figure 9(a)). This point must be taken into consideration in entropy coding of quantization index 56. For example, quantization index 56 is often transmitted by encoding its absolute value (ranging from 0 to the maximum supported value), and for absolute values ​​that are not equal to 0, by additionally encoding the sign of quantization index 56. If there are no quantization index 56 equal to 0, the entropy coding may be modified so that the value of the absolute level minus 1 is transmitted (the value of the corresponding syntax element ranges from 0 to the maximum supported value), and the sign is always transmitted. As an alternative, the assignment rules for assigning quantization index 56 to reconstruction levels can be modified. For example, one reconstruction level close to 0 may be labeled with a quantization index equal to 0. The remaining reconstruction levels are then labeled according to the following rules: Reconstruction levels greater than those with a quantization index equal to 0 are assigned a quantization index greater than 0 (quantization index increases with the reconstruction level value). And quantization indices less than 0 are assigned to reconstruction levels smaller than those with a quantization index equal to 0 (quantization index decreases with the reconstruction level value). One possible such assignment is shown in the numbers in parentheses in Figure 9(a) (other numbers apply if no numbers are given in parentheses).

[0164] As described above, in a preferred embodiment, two different sets of reconstruction levels (also called quantization sets) are used, where the reconstruction levels within both sets represent integer multiples of the quantization step size (QP). This includes cases where the quantization step size is modified on a layer basis (e.g., by sending layer quantization parameters into the bitstream 14) or by another finite set of neural network parameters 13 (e.g., blocks) (e.g., by sending block quantization parameters into the bitstream 14).

[0165] The use of reconstruction levels representing integer multiples of the quantization step size (QP) enables computationally intensive and complex algorithms for reconstructing the neural network parameters 13 on the decoder side. This is described below based on the preferred example in Figure 9(c) (similar simple algorithms exist for other configurations, particularly the settings shown in Figures 9(a) and 9(b)). In the configuration shown in Figure 9(c), the first quantization set includes all even integer multiples of the quantization step size (QP), and the second quantization set includes all odd integer multiples of the quantization step size plus a reconstruction level equal to 0 (which is included in both quantization sets). The neural network parameter reconstruction process can be implemented similarly to the algorithm defined in the pseudocode in Figure 10. Figure 10 shows an example of pseudocode illustrating a preferred example of the neural network parameter reconstruction process 13. k represents an index specifying the reconstruction order of the current neural network parameters 13', the quantization index 56 of the current neural network parameters is represented by level[k]210, and Δ is the quantization step size applied to the current neural network parameters 13'. k (QP) is represented by quant_step_size[k], and trec[k]220 is the parameter t of the reconstructed neural network. kThis represents the value of '. The variable setId[k]240 specifies the set of reconstruction levels to be applied to the current neural network parameter 13'. It is determined based on the preceding neural network parameters in the reconstruction order; the possible values ​​of setId[k] are 0 and 1. The variable n defines an integer coefficient of the quantization step size (QP), e.g., the intermediate value IV; it is given by the selected set of reconstruction levels (i.e., the value of setId[k]) and the transmitted quantization index level[k].

[0166] JPEG2026067854000012.jpg122167

[0167] When using the second quantization set, the variable n is equal to twice the quantization index level[k] minus the sign function sign(level[k]) of the quantization index. In this case, it can be represented by the reconstruction level of the second quantization set Set1 in Figure 9(c), where Set1 includes all odd integer multiples of the quantization step size (QP).

[0168] Once the variable n (which specifies an integer multiple of the quantization step size) is determined, the quantization step size Δ is assigned to n. k By multiplying by t, the reconstructed neural network parameters t k ' is required.'

[0169] In other words, the number of reconstruction level sets 52 among the multiple 50 reconstruction level sets 52 may be 2, and the apparatus for decoding and / or encoding, for example, neural network parameters 13 according to an embodiment of the present invention may be configured to derive intermediate values ​​for each neural network parameter as follows. If the selected set of reconstruction levels for each neural network parameter is the first set, then the quantization index for each neural network parameter is doubled to obtain the intermediate value for each neural network parameter; and, The second set is the selected set of reconstruction levels for each neural network parameter, and if the quantization index of each neural network parameter is equal to zero, the midpoint of each sample is set to zero; and, The second set is the selected set of reconstruction levels for each neural network parameter, and if the quantization index for each neural network parameter is greater than zero, the quantization index for each neural network parameter is doubled, and 1 is subtracted from the result of that multiplication to obtain the intermediate value for each neural network parameter, and The second set is the selected set of reconstruction levels for the current neural network parameters, and if the quantization index for each neural network parameter is less than zero, the quantization index for each neural network parameter is doubled, and 1 is added to the result of that multiplication to obtain the midpoint for each neural network parameter.

[0170] 4.3.3 Dependent Reconstruction of Neural Network Parameters In addition to the selection of the set of reconstruction levels described in sections 4.3.1 and 4.3.2, another important design aspect of dependent scalar quantization in neural network parameter coding is the algorithm used to switch between defined sets of quantization (sets of reconstruction levels). The algorithm used determines the "packing density" that can be achieved in the N-dimensional space of the neural network parameters (and therefore the N-dimensional space of the reconstructed samples). The higher the packing density, the better the coding efficiency in the end.

[0171] A preferred method for determining the set of reconstruction levels for the next set of neural network parameters is based on the division of quantization sets, as shown in Figure 11. Figure 11 shows an example of dividing a set of reconstruction levels into two subsets according to an embodiment of the present invention. The two quantization sets shown are the quantization sets in the preferred example in Figure 9(c). The two subsets of quantization set 0 are labeled with "A" and "B", and the two subsets of quantization set 1 are labeled with "C" and "D". Note that the quantization sets shown in Figure 11 are the same quantization sets as those in Figure 9(c). Each of the two (or more) quantization sets is divided into two subsets. In the preferred example in Figure 11, the first quantization set (labeled set 0) is divided into two subsets (labeled A and B), and the second quantization set (labeled set 1) is also divided into two subsets (labeled C and D). While not the only possibility, the partitioning of each quantization set is preferably carried out such that directly adjacent reconstruction levels (and therefore adjacent quantization indices) are associated with different subsets. In a preferred embodiment, each quantization set is partitioned into two subsets. In Figure 9, the partitioning of the quantization set into subsets is indicated by hollow and filled circles.

[0172] In the particularly preferred embodiments shown in Figures 11 and 9(c), the following division rules apply. - Subset A consists of all even quantization indices from quantization set 0. - Subset B consists of all odd quantization indices from quantization set 0. - Subset C consists of all even quantization indices from quantization set 1. - Subset D consists of all odd quantization indices of quantization set 1.

[0173] It should be noted that the subsets used are typically not explicitly shown within bitstream 14. Instead, they can be derived based on the quantization set used (e.g., set 0 or set 1) and the quantization indices actually transmitted. In the preferred partition shown in Figure 11, the subsets can be derived by a bitwise "and" operation of the transmitted quantization index level and 1. Subset A consists of all quantization indices of set 0 where (level&1) is equal to 0, subset B consists of all quantization indices of set 0 where (level&1) is equal to 1, subset C consists of all quantization indices of set 1 where (level&1) is equal to 0, and subset D consists of all quantization indices of set 1 where (level&1) is equal to 1.

[0174] In a preferred embodiment, the quantization set (set of acceptable reconstruction levels) used to reconstruct the current neural network parameter 13' is determined based on a subset associated with the last two or more quantization indices 56. An example in which the last two subsets (given by the last two quantization indices) are used is shown in Table 1. The determination of the quantization set specified by this table represents a preferred embodiment. In other embodiments, the quantization set for the current neural network parameter 13' is determined by a subset associated with the last three or more quantization indices 56. For the first neural network parameter of a layer (or a subset of neural network parameters), there is no data on a subset of preceding neural network parameters (because there are no preceding neural network parameters). In a preferred embodiment, a predefined value is used in such cases. In a particularly preferred embodiment, a subset A is inferred for all unavailable neural network parameters. In other words, when reconstructing the first neural network parameters, we infer that the preceding two subsets are "AA" (or "AAA" if we consider the preceding three neural network parameters), and therefore, according to Table 1, we use quantization set 0. For the second neural network parameter, we determine the subset of the preceding quantization index by its value (since set 0 is used for the first neural network parameter, the subset is either A or B), but we infer that the subset for the second and final quantization index (which does not exist) is equal to A. Of course, it is possible to use any other arbitrary rule to infer a default value for a non-existent quantization index. It is also possible to use other syntactic elements to derive a default subset for a non-existent quantization index. As a further alternative, it is also possible to use the last quantization index 56 of the preceding set of neural network parameter 13 for initialization.

[0175] Table 1: An example for determining the quantization set (set of available reconstruction levels) to be used for the next neural network parameters based on subsets related to the two final quantization indices according to embodiments of the present invention. The subsets are shown in the left column of the table, and they are uniquely determined by the quantization set to be used (for the two final quantization indices) and the so-called path (which may be determined by the parity of the quantization indices). The quantization set and the path for the subset in parentheses are listed in the second column from the left. The third column specifies the relevant quantization set. The last column shows the values ​​of the so-called state variables, which can be used to simplify the process of determining the quantization set.

[0176] [Table 1]

[0177] It should be noted that the subset (A, B, C, or D) of quantization index 56 is determined by the quantization set used (set 0 or set 1) and the subset used within that quantization set (for example, A or B for set 0, or C or D for set 1). The subset selected within a quantization set is also called a path (as it specifies a path when representing the dependent quantization process in a trellis structure, as will be discussed later). By convention, a path is equal to either 0 or 1. Then, subset A corresponds to path 0 in set 0, subset B corresponds to path 1 in set 0, subset C corresponds to path 0 in set 1, and subset D corresponds to path 1 in set 1. Thus, the quantization set of the next neural network parameter is also uniquely determined by the quantization set (set 0 or set 1) and path (path 0 or path 1) associated with the last two (or more) quantization indices. In Table 1, the relevant quantization sets and paths are specified in the second column.

[0178] JPEG2026067854000014.jpg50166

[0179] In other words, the number of reconstruction level sets 52 among the multiple 50 reconstruction level sets 52 may be, for example, two, such as set 0 and set 1. An apparatus for decoding, for example, a neural network parameter 13 according to an embodiment of the present invention can derive a subset index for each neural network parameter based on a selected set of reconstruction levels for each neural network parameter and a binary function of the quantization index for each neural network parameter, thereby obtaining four possible values, for example, A, B, C, or D for the subset index. The apparatus can be configured to select a set of reconstruction levels 48 from the multiple 50 reconstruction level sets 52 for the current neural network parameter 13', depending on the subset index for a previously decoded neural network parameter.

[0180] A further embodiment of the present invention includes a device configured to select a reconstruction level set 48 from a plurality of reconstruction level sets 5) for the current neural network parameters 13' using a selection rule corresponding to a subset index of a number of previously decoded neural network parameters, for example, as shown in the first column of Table 1, and to use the selection rule for all or some of the neural network parameters.

[0181] In a further embodiment, the number of previously decoded neural network parameters to which the selection rule corresponds is two, which are a subset of the two last quantization indices, as shown in Table 1, for example.

[0182] According to an additional embodiment, a subset index of each neural network parameter is derived based on a selected set of reconstruction levels for each neural network parameter and the parity of the quantization index for each neural network parameter, for example, using path=(level[k]&1).

[0183] Each embodiment of the apparatus for encoding neural network parameters 13 can be configured such that the number of multiple 50 reconstruction level sets 52 is two, for example, set 0 and set 1, and the apparatus can be configured to derive a subset index for each neural network parameter based on a selected set of reconstruction levels for each neural network parameter and a binary function of the quantization index for each neural network parameter, resulting in four possible values ​​for the subset index, for example, A, B, C, and D, and to select a reconstruction level set 48 from the multiple 50 reconstruction level sets 52 for the current neural network parameter 13' according to the subset index for the previously encoded neural network parameter.

[0184] A further embodiment of the present invention includes a device configured to select a set of reconstruction levels 48 from a plurality of reconstruction level sets 52 for the current neural network parameters 13' using a selection rule corresponding to a subset index of a large number of previously encoded neural network parameters, for example as shown in the first column of Table 1, and to apply the selection rule to all or some of the neural network parameters.

[0185] In a further embodiment, the number of previously encoded neural network parameters to which the selection rule corresponds is 2, which is a subset of the two last quantization indices, as shown in Table 1, for example.

[0186] According to an additional embodiment, a subset index for each neural network parameter is derived based on a selected set of reconstruction levels for each neural network parameter and the parity of the quantization index for each neural network parameter, e.g., using path=(level[k]&1).

[0187] Transitions between quantization sets 52 (set 0 and set 1) can also be elegantly represented by a state variable. An example of such a state variable is shown in the last column of Table 1. In this example, the state variable has four possible values ​​(0, 1, 2, 3). The state variable also specifies the quantization set to be used for the current neural network parameters 13'. In the preferred example in Table 1, quantization set 0 is used only if the state variable is equal to 0 or 2, and quantization set 1 is used only if the state variable is equal to 1 or 3. The state variable also specifies the possible transitions between quantization sets. Using a state variable, the rules in Table 1 can be described in a smaller state transition table. As an example, Table 2 specifies a state transition table for the rules given in Table 1. This represents a preferred embodiment. The current state is specified, and the quantization set is specified for the current neural network parameters (second column). It further specifies state transitions based on paths associated with the selected quantization index 56 (the paths specify which subset A, B, C, or D is used, given the quantization sets). It should be noted that by using the concept of state variables, there is no need to keep track of the actual selected subset. When reconstructing the parameters of a layer of a neural network, it is sufficient to update the state variables and determine the path of the quantization indices to be used.

[0188] Table 2: A preferred example of a state transition table for a configuration having four states according to an embodiment of the present invention. [Table 2]

[0189] In other words, the embodiment provides an apparatus for decoding neural network parameters, which can be configured to select a quantization level set 48 from a plurality of 50 reconstruction level sets 52 for the current neural network parameter 13' by a state transition process, by determining a set of quantization levels 48 from a plurality of 50 reconstruction level sets 52 according to the state associated with the current neural network parameter 13', and by updating the state of the subsequent neural network parameter according to the quantization index 58 decoded from the data stream of the immediately preceding neural network parameter.

[0190] Each embodiment of the apparatus for encoding the neural network parameter 13 may be configured to select a set of reconstruction levels 48 from a plurality of 50 reconstruction level sets 52 for the current neural network parameter 13' by a state transition process, by determining a set of reconstruction levels 48 from a plurality of 50 reconstruction level sets 52 in accordance with the state associated with the current neural network parameter 13', and by updating the state for the subsequent neural network parameter in accordance with the quantization index 58 encoded in the data stream of the immediately preceding neural network parameter.

[0191] JPEG2026067854000016.jpg44166

[0192] In other words, according to the embodiment, a device for decoding neural network parameters, for example, can be configured to update the state of subsequent neural network parameters, for example, according to Table 2, using a binary function of quantization index 58 decoded from a data stream for the immediately preceding neural network parameter.

[0193] Furthermore, the apparatus according to the embodiment can be configured to update the state for subsequent neural network parameters using the parity of the quantization index 58 decoded from the data stream 14 for the immediately preceding neural network parameter, for example, path=(level[k]&1).

[0194] Each embodiment of the device for encoding the neural network parameters 13 can be configured to update the state of subsequent neural network parameters using a binary function of the quantization index 58 encoded in the data stream for the immediately preceding neural network parameter.

[0195] Furthermore, the apparatus for encoding, for example, a neural network parameter 13 according to the embodiment can be configured to update the state of a subsequent neural network parameter, for example, according to Table 2, using the parity of the quantization index 58 encoded in the data stream for the immediately preceding neural network parameter.

[0196] In a preferred embodiment, a state variable with four possible values ​​is used. In other embodiments, a state variable with a different number of possible values ​​is used. Of particular interest are state variables where the number of possible values ​​of the state variable is a power of 2, i.e., 4, 8, 16, 32, 64, etc. Note that in the preferred configuration (as given in Tables 1 and 2), a state variable with four possible values ​​is equivalent to an approach where the current quantization set is determined by a subset of the two last quantization indices. A state variable with eight possible values ​​corresponds to a similar approach where the current quantization set is determined by a subset of the three last quantization indices. A state variable with sixteen possible values ​​corresponds to an approach where the current quantization set is determined by a subset of the last four quantization indices. Although it is generally preferred to use a state variable with a number of possible values ​​equal to a power of 2, embodiments are not limited to this setting.

[0197] In a particularly preferred embodiment, a state variable having eight possible values ​​(0, 1, 2, 3, 4, 5, 6, 7) is used. In Table 3 of the preferred example, quantization set 0 is used only when the state variable is equal to 0, 2, 4, or 6, and quantization set 1 is used only when the state variable is equal to 1, 3, 5, or 7.

[0198] This is a preferred example of a state transition table for a configuration having eight states, according to the embodiment. [Table 3]

[0199] In other words, in embodiments of the present invention, the state transition process is configured to transition between four or eight possible states.

[0200] Furthermore, the apparatus for decoding / encoding neural network parameters 13 according to the embodiment can be configured such that, in the state transition process, it transitions between an even number of possible states and the number of multiple 50 reconstruction level sets 52 is 2, where, for the current neural network parameter 13', a set of quantization levels 48 from the quantization set 52 is determined according to the state associated with the current neural network parameter 13', so that if the state belongs to the first half of an even number of possible states, a first reconstruction level set from the multiple 50 reconstruction level sets 52 is determined, and if the state belongs to the second half of an even number of possible states, a second reconstruction level set from the multiple 50 reconstruction level sets 52 is determined.

[0201] In a further embodiment, for example, a device for decoding a neural network parameter 13 may be configured to perform state updates by a transition table that maps the combination of state and parity of the quantization index 58 decoded from the data stream of the immediately preceding neural network parameter onto another state associated with a subsequent neural network parameter.

[0202] Accordingly, the apparatus for encoding the neural network parameters 13 according to the embodiment can be configured to update the state by a transition table that maps the combination of the state and parity of the quantization index 58 encoded in the data stream for the immediately preceding neural network parameter to another state associated with the subsequent neural network parameter.

[0203] Using the concept of state transitions, the current state, and therefore the current quantization set, is uniquely determined by the previous state and the previous quantization index 56 (in the reconstruction order). However, for the first neural network parameter 13 of a finite set (e.g., a layer), there is no previous state or previous quantization index. Therefore, it is required that the state for the first neural network parameter of a given layer be uniquely defined. There are several possibilities. The preferred options are as follows: The first state for the layer is always set to a fixed, predefined value. In a preferred embodiment, the first state is set to 0. • The value of the first state is explicitly sent as part of bitstream 14. This includes an approach where only a subset of possible state values ​​can be indicated by the corresponding syntax element. The value of the first state is derived based on other syntax elements in that layer. That is, even if the corresponding syntax elements (or syntax elements) are used for other forms of signaling to the decoder, they are additionally used to derive the first state for the dependent scalar quantization.

[0204] The concept of dependent scalar quantization state transitions enables a less complex implementation for reconstructing the neural network parameters 13 in the decoder. A preferred example of the single-layer neural network parameter reconstruction process is shown in Figure 12 using pseudocode in C language format. Figure 12 shows an example of pseudocode illustrating a preferred example of the layer neural network parameter reconstruction process according to an embodiment of the present invention. Note that the derivation of the quantization index and the derivation of the reconstructed value, for example using the quantization step size, or alternatively using a codebook, may be performed in separate loops. In other words, the derivation of "n" and state updates can be performed in a first loop, and the derivation of "trec" can be performed in another second loop. The array level 210 represents the transmitted neural network parameter level (quantization index 56) of that layer, and the array trec 220 represents the corresponding reconstructed neural network parameter 13. The quantization step size Δk(QP) applied to the current neural network parameter 13' is denoted by quant_step_size[k]. The 2d table sttab (2dtablesttab) 230 specifies the state transition table according to, for example, Table 1, Table 2, and / or Table 3, and the table setId (tablesetId) 240 specifies the quantization set associated with state 250.

[0205] In the pseudocode of Figure 12, index k specifies the reconstruction order of the neural network parameters. The last index, layerSize, specifies the reconstruction index of the last reconstructed neural network parameter. The variable layerSize may be set to be equal to the number of neural network parameters in the layer. The reconstruction process for each single neural network parameter is the same as in the example of Figure 10. In the example of Figure 10, the quantization index is represented by level[k]210, and the associated reconstructed neural network parameter is represented by trec[k]220. Also, the state variable is represented by state210. Note that in the example of Figure 12, state is set to equal to 0 at the beginning of the layer. However, as mentioned above, other initializations (e.g., based on the values ​​of some syntax elements) are also possible. The 1d table setId[](1dtablesetId[])240 specifies a quantization set associated with different values ​​of a state variable, and the 2d table sttab[][](2dtablesttab[][])230 specifies a state transition given the current state (first argument) and the path (second argument). In this example, the path is given by the parity of the quantization index (using bits and the operator &), but other concepts are also possible. Examples of tables in C-style syntax are shown in Figures 13 and 14 (these tables are identical to Tables 2 and 3, or in other words, they can provide representations of Tables 2 and 3).

[0206] Figure 13 shows a preferred example of a state transition table sttab230 and a table setId240 specifying the quantization set associated with state 250 according to an embodiment of the present invention. A table given in C-style syntax represents a table defined in Table 2.

[0207] Figure 14 shows a preferred example of a state transition table sttab230 and a table setId240 specifying a quantization set associated with state 250 according to an embodiment of the present invention. The table given in C-style syntax represents the table defined in Table 3.

[0208] In another embodiment, all quantization indices 56 equal to 0 are excluded from the state transition and dependency reconstruction process. Information on whether a quantization index 56 is equal to or not is used only to divide the neural network parameters 13 into 0 and non-0 neural network parameters. The reconstruction process for dependency scalar quantization applies only to an ordered set of non-zero quantization indices 56. All neural network parameters associated with quantization indices equal to 0 are simply set to equal to 0. The corresponding pseudocode is shown in Figure 15. Figure 15 shows pseudocode illustrating an alternative neural network parameter-level reconstruction process according to an embodiment of the present invention, in which quantization indices equal to 0 are excluded from state transition and dependency scalar quantization.

[0209] Furthermore, state transitions in dependent quantization can also be represented using a trellis structure, as shown in Figure 16. Figure 16 is a diagram showing an example of state transitions in dependent scalar quantization as a trellis structure according to an embodiment of the present invention. The horizontal axis represents different neural network parameters 13 in the reconstruction order. The vertical axis represents different possible states 250 in the dependent quantization and reconstruction process. The connections shown specify the available paths between states for different neural network parameters. The trellis shown in this figure corresponds to the state transitions specified in Table 2. For each state 250, there are two paths connecting the state for the current neural network parameter 13' to two possible states for the next neural network parameter 13 in the reconstruction order. The paths are labeled as path 0 and path 1, where this number corresponds to the path variable described above (in a preferred embodiment, the path variable is equal to the parity of the quantization index). Note that each path uniquely specifies a subset (A, B, C, or D) of the quantization index. In Figure 16, subsets are specified in parentheses. Given an initial state (e.g., state 0), the path through the trellis is uniquely determined by the transmitted quantization index 56.

[0210] In the example in Figure 16, the state (0,1,2,3) has the following properties: State 0: The previous quantization index level[k-1] specifies the reconstruction level of set 0, and the current quantization index level[k] specifies the reconstruction level of set 0. State 1: The previous quantization index level[k-1] specifies the reconstruction level for set 0, and the current quantization index level[k] specifies the reconstruction level for set 1. State 2: The previous quantization index level[k-1] specifies the reconstruction level for set 1, and the current quantization index level[k] specifies the reconstruction level for set 0. State 3: The previous quantization index level[k-1] specifies the reconstruction level of set 1, and the current quantization index level[k] specifies the reconstruction level of set 1.

[0211] A trellis consists of a series of so-called basic trellis cells. An example of such basic trellis cells is shown in Figure 17. Figure 17 is a diagram showing an example of a basic trellis cell according to an embodiment of the present invention. It should be noted that the present invention is not limited to a trellis having four states 250. In other embodiments, a trellis can have more states 250. In particular, any number of states representing an integer power of 2 is preferred. In a particularly preferred embodiment, the number of states 250 is equal to 8, for example, as in Table 3. Even when a trellis has two or more states 250, each node for the current neural network parameter 13' is typically connected to two states for the previous neural network parameter 13 and two states for the next neural network parameter 13. However, it is also possible for a node to be connected to two or more states for the previous neural network parameter or to two or more states for the next neural network parameter. It should be noted that a fully connected trellis (where each state 250 is connected to all states 250 of the previous neural network parameter 13 and all states 250 of the next neural network parameter 13) would correspond to independent scalar quantization.

[0212] In a preferred embodiment, the initial state cannot be freely selected (because some side information rate is required to send this decision to the decoder). Instead, the initial state is either set to a predefined value or its value is derived based on other syntax elements. In this case, not all paths and states are available for the first neural network parameter. As an example of a four-state trellis, Figure 18 shows a trellis structure when the initial state is equal to 0. Figure 18 shows an example trellis for dependent scalar quantization of eight neural network parameters according to an embodiment of the present invention. The first state (left) represents the initial state, which in this example is set to equal to 0.

[0213] 4.4 Entropy Coding The quantization index obtained by dependent quantization is encoded using an entropy coding scheme. Any entropy coding scheme can be applied. In a preferred embodiment of the present invention, an entropy coding scheme according to Section 2.2 (see Section 2.2.1 for the coding method and Section 2.2.2 for the decoding method) using context-adaptive binary arithmetic coding (CABAC) is applied. For this purpose, as shown in Figure 5, for example, non-binary values ​​are first mapped onto a series of binary decisions (so-called bins) (binarization) in order to transmit the quantization index as an absolute value.

[0214] It should be noted that all the concepts described here can be combined with the methods and related concepts in Section 3 (particularly those concerning context modeling).

[0215] 4.4.1 Contextual Modeling of Dependent Scalar Quantization A key aspect of dependent scalar quantization is the existence of different sets of permissible reconstruction levels (also called quantization sets) for the neural network parameter 13. The quantization set for the current neural network parameter 13' is determined based on the value of the quantization index 56 for the preceding neural network parameter. Considering the preferred example in Figure 11, and comparing the two quantization sets, it is clear that the distance between a reconstruction level equal to zero and an adjacent reconstruction level is greater in set 0 than in set 1. Thus, the probability of quantization index 56 being equal to zero is greater when set 0 is used and smaller when set 1 is used. In preferred embodiments, this effect is exploited in entropy coding by switching a codeword table or probabilistic model based on the quantization set (or state) used for the current quantization index.

[0216] Note that, for a suitable switching of codeword tables or probabilistic models, when entropy decoding the current quantization index (or the corresponding binary decision of the current quantization index), the paths of all preceding quantization indices (in relation to a subset of the quantization set to be used) must be known. Therefore, it is necessary that the neural network parameters 13 be encoded in reconstruction order. Thus, in a preferred embodiment, the encoding order of the neural network parameters 13 is equal to their reconstruction order. In addition to that embodiment, any encoding / reconstruction order of the quantization indices 56 is possible, for example, the order specified in Section 2.2.1 is any other arbitrary uniquely defined order.

[0217] In other words, embodiments of the present invention include, for example, an apparatus for encoding neural network parameters using a probabilistic model that additionally depends on the quantization index of previously encoded neural network parameters.

[0218] Each embodiment of the present invention includes, for example, an apparatus for decoding neural network parameters, which uses a probabilistic model that additionally depends on the quantization index of previously decoded neural network parameters.

[0219] At least some of the bins for the absolute level are typically encoded using an adaptive probabilistic model (also called a context). In a preferred embodiment of the present invention, the probabilistic model for one or more bins is selected based on a quantization set of the corresponding neural network parameters (or, more generally, a relationship by any of the corresponding state variables, e.g., Tables 1-3). The selected probabilistic model may depend on several parameters or characteristics of the quantization index 56 already transmitted, one of which is the quantization set or state applied to the quantization index being encoded.

[0220] In other words, according to the embodiment, the apparatus for encoding, for example, neural network parameters 13 can be configured to pre-select a subset of probabilistic models from among several probabilistic models, depending on the state or set 48 of the reconstruction level selected for the current neural network parameters 13', and to select a probabilistic model for the current neural network parameters from among a subset of 121 probabilistic models, depending on the quantization index of the previously encoded neural network parameters.

[0221] An embodiment of the apparatus for decoding, for example, a neural network parameter 13 may be configured to pre-select a subset of probabilistic models from among a plurality of probabilistic models, depending on the state or set 48 of the reconstruction level selected for the current neural network parameter 13', and to select a probabilistic model of the current neural network parameter from among the subset of probabilistic models, depending on the quantization index 121 of the previously decoded neural network parameter.

[0222] For example, in combination with an inventive concept as described in the context of Figure 9, an embodiment of the present invention for, for example, encoding and / or decoding of neural network parameters 13 includes a device configured to pre-select a subset of probabilistic models from among a plurality of probabilistic models, depending on the state or set 48 of the reconstruction level selected for the current neural network parameter 13', such that a pre-selected subset for a first state or reconstruction level set is uncoupled with any other state or reconstruction level set.

[0223] In a particularly preferred embodiment, the syntax for transmitting the quantization index of a layer includes a bin, e.g., the aforementioned sig_flag, that specifies whether the quantization index is equal to 0 or not. The probabilistic model used to encode this bin is selected from a set of two or more probabilistic models. The selection of the probabilistic model used depends on the quantization set (i.e., the set of reconstruction levels) applied to the corresponding quantization index 56. In another embodiment of the present invention, the probabilistic model used depends on the current state variable (wherein the state variable is the quantization set used).

[0224] In a further embodiment, the syntax for transmitting the quantization index of a layer includes a bin, e.g., the aforementioned sign_flag, that specifies whether the quantization index is greater than or less than zero. That is, the bin indicates the sign of the quantization index. The choice of probabilistic model used depends on the set of quantizations (i.e., the set of reconstruction levels) applied to the corresponding quantization index. In another embodiment, the probabilistic model used depends on the current state variable (the state variable signifies the set of quantizations used).

[0225] In a further embodiment, the syntax for transmitting the quantization index includes a bin that specifies whether the absolute value of the quantization index (the parameter level of the neural network) is greater than X, for example, the aforementioned abs_level_greater_X (see Section 0 for details). The probabilistic model used to encode this bin is selected from a set of two or more probabilistic models. The selection of the probabilistic model used depends on the quantization set (i.e., the reconfiguration level set) applied to the corresponding quantization index 56. In another embodiment, the probabilistic model used depends on the current state variable (wherein the state variable is the quantization set used).

[0226] One advantageous aspect of the embodiments discussed herein is that the dependent quantization of the neural network parameter 13 is combined with entropy coding, and the selection of a probabilistic model for one or more bins (also called quantization levels) of the binary representation of the quantization index depends on a quantization set (a set of allowed reconstruction levels) or a corresponding state variable for the current quantization index. The quantization set 52 (or state variable) is given by the quantization index 56 (or subset of bins representing the quantization index) of the preceding neural network parameter in the coding and reconstruction order.

[0227] In a preferred embodiment, the described selection of the probabilistic model is combined with one or more of the following aspects of entropy coding: The absolute value of the quantization index is transmitted using a binary scheme consisting of a number of bins encoded using an adaptive probabilistic model, and a suffix portion encoded in bypass mode of the arithmetic coding engine (a non-adaptive probabilistic model with a pmf (e.g., probability mass function) (0.5, 0.5) for all bins) if the adaptively encoded bins do not already specify the absolute value. In a preferred embodiment, the binarization used for the suffix portion depends on the value of the quantization index already transmitted. The binarization of the absolute value of the quantization index includes an adaptively encoded bin that specifies whether the quantization index is not equal to zero. The probabilistic model (called the context) used to encode this bin is selected from a set of candidate probabilistic models. The selected candidate probabilistic model is determined not only by the quantization set (set of allowed reconstruction levels) or state variable for the current quantization index 56, but also by the quantization index already transmitted for that layer. In a preferred embodiment, the quantization set (or state variable) determines a subset of available probabilistic models (also called the context set), and the value of the already encoded quantization index determines the probabilistic model to be used within this subset (context set).

[0228] In the embodiment, the probabilistic model used within the context set is determined based on the already encoded quantization index values ​​in the local neighborhood of the current neural network parameters, for example, a template as described in 2.2.3. Below, we list some exemplary measures that can be derived based on the quantization index values ​​in the local neighborhood and then used to select a probabilistic model for a predetermined context set. The sign of a quantization index that is not equal to zero within its local neighborhood. • The number of non-zero quantization indices within the local neighborhood. This number may be clipped (shortened) to the maximum value. • The sum of the absolute values ​​of the quantization indices in the local neighborhood. This value may be clipped to the maximum value. The difference between the sum of the absolute values ​​of quantization indices in the local neighborhood and the number of non-zero quantization indices in the local neighborhood. This number may be clipped to the maximum value.

[0229] In other words, embodiments of the present invention include, for example, an apparatus for encoding neural network parameters configured to select a probabilistic model of the current neural network parameters from a subset of probabilistic models depending on the characteristics of the quantization index of previously encoded neural network parameters related to a portion of the neural network adjacent to the portion to which the current neural network parameters relate, the characteristics of which include one or more of the following: The sign of the non-zero quantization index of a previously encoded neural network parameter, relating to the portion of the neural network adjacent to the portion to which the current neural network parameter relates. The number of non-zero quantization indices of previously encoded neural network parameters related to the portion of the neural network adjacent to the portion to which the current neural network parameters relate. The sum of the absolute values ​​of the quantization indices of previously encoded neural network parameters, related to the portion of the neural network adjacent to the portion of the neural network to which the current neural network parameters are related. The sum of the absolute values ​​of the quantization indices of previously encoded neural network parameters related to the portion of the neural network adjacent to the portion related to the current neural network parameters, The number of quantization indices of previously encoded neural network parameters related to the portion of the neural network adjacent to the portion of the neural network to which the current neural network parameters relate, and a non-zero number, The difference between them.

[0230] Each embodiment of the present invention includes, for example, an apparatus for decoding neural network parameters. The apparatus is configured to select a probabilistic model for the current neural network parameters from a subset of probabilistic models, according to the characteristics of the quantization index of a previously decoded neural network parameter associated with a portion of the neural network adjacent to the portion to which the current neural network parameter is related. The characteristics include one or more of the following: The sign of the non-zero quantization index of a previously decoded neural network parameter, relating to the portion of the neural network adjacent to the portion to which the current neural network parameter relates. The number of non-zero quantization indices of previously decoded neural network parameters related to the portion of the neural network adjacent to the portion to which the current neural network parameters relate. The sum of the absolute values ​​of the quantization indices of previously decoded neural network parameters, related to the portion of the neural network adjacent to the portion to which the current neural network parameters are related. The sum of the absolute values ​​of the quantization indices of previously decoded neural network parameters related to the portion of the neural network adjacent to the portion to which the current neural network parameters are relevant, The number of quantization indices of previously decoded neural network parameters, which are related to the portion of the neural network adjacent to the portion of the neural network to which the current neural network parameters are related, and which are non-zero numbers. The difference.

[0231] The binarization of the absolute value of the quantization index includes an adaptively encoded bin, e.g., abs_level_greater_X, that specifies whether the absolute value of the quantization index is greater than X. The probabilistic model (called context) used to encode these bins is selected from a set of candidate probabilistic models. The selected probabilistic model is determined not only by the quantization set (set of acceptable reconstruction levels) or state variable for the current quantization index, but also by the quantization index already transmitted to the layer, for example using the template described above. In a preferred embodiment, the quantization set (or state variable) determines a subset of available probabilistic models (also called the context set), and the data of the already encoded quantization index can be used to determine, for example, the probabilistic model used within this subset (context set). The selection of the probabilistic model can be done using any of the methods described above (for the bin that specifies whether the quantization index is not equal to 0).

[0232] Furthermore, the apparatus according to the present invention can be configured to position the previously encoded neural network parameters 13 such that the previously encoded neural network parameters 13 are associated with the same neural network layer as the current neural network parameters 13'.

[0233] Furthermore, the apparatus for encoding, for example, neural network parameters according to the present invention can be configured to position one or more previously encoded neural network parameters such that one or more previously encoded neural network parameters relate to a neuron interconnect emerging from a neuron 10c or another neuron adjacent to the neuron referenced by the current neural network parameter, or to neuron interconnects leading to these neurons.

[0234] A device according to a further embodiment can be configured to encode the quantization index 56 for the current neural network parameters 13' into a data stream 14 using binary arithmetic coding by using a probabilistic model corresponding to previously encoded neural network parameters for one or more leading bins of the binarized quantization index, and by using equi-probable bypass mode suffix bins of the binarized quantization index following one or more leading bins.

[0235] The suffix bin of the binarized quantization index represents the bin of the suffix binarization binarization code for binarizing the value of the quantization index, and its absolute value may exceed the maximum absolute value that can be represented by one or more leading bins. Therefore, the apparatus according to the embodiment of the present invention can be configured to select suffix binarization in accordance with the quantization index 56 of the previously encoded neural network parameter 13.

[0236] Each of the present inventions, for example, an apparatus for decoding neural network parameters, may be configured to position previously decoded neural network parameters 13 such that previously decoded neural network parameters relate to the same neural network layer as the current neural network parameters 13'.

[0237] According to a further embodiment, for example, an apparatus for decrypting neural network parameters according to the present invention is configured to locate one or more previously decrypted neural network parameters such that the one or more previously decrypted neural network parameters are related to a neuron interconnect associated with neuron 10c to which the current neural network parameters refer or a neuron interconnect 11 emerging from another neuron adjacent to the neuron, or a neuron interconnect 11 directed towards these neurons.

[0238] The apparatus according to a further embodiment is configured to decrypt the quantization index 56 for the current neural network parameters 13' from the data stream 14 using binary arithmetic coding by using a probability model corresponding to the previously decrypted neural network parameters for one or more leading bins of the binarized quantization index and by using an equiprobable bypass mode suffix bin of the binarized quantization index following the one or more leading bins.

[0239] The suffix bin of the binarization of the quantization index represents the bin of the binarization code of the suffix binarization for binarizing the value of the quantization index, and its absolute value exceeds the maximum absolute value that can be represented by one or more leading bins. Thus, the apparatus according to an embodiment can be configured to select the suffix binarization according to the quantization index of the previously decrypted neural network parameters.

[0240] JPEG2026067854000018.jpg103167

[0241] JPEG2026067854000019.jpg118170

[0242] In other words, embodiments according to the present invention include an apparatus configured to use the Viterbi algorithm and the rate distortion cost amount to perform selection and / or quantization.

[0243] An exemplary coding algorithm for selecting an appropriate quantization index for a layer can consist of the following main steps. 1. Set the rate distortion cost in the initial state to 0. 2. For all neural network parameters 13 in the coding order, do the following. a. For each subset A, B, C, D, determine a quantization index 56 that minimizes the distortion with respect to the given original neural network parameter 13. b. For all trellis nodes (0, 1, 2, 3) with respect to the current neural network parameter 13', do the following. i Calculate the rate distortion costs of two paths connecting the state of the previous neural network parameter 13 and the current state. The cost is given as the sum of the cost for the previous state and D k +λ·R k where D k and R k represent the distortion and rate for selecting the quantization index of the subset (A, B, C, D) associated with the connection being considered. ii Assign the minimum value of the calculated cost to the current node and remove the connection to the state of the previous neural network parameter 13 that does not represent the path with the minimum cost. Note: After this step, all nodes with respect to the current neural network parameter 13' have a single connection to any node with respect to the previous neural network parameter 13. 3. Compare the costs of the four final nodes (for the last parameter in the coding order) and select the node with the minimum cost. Note that this node is associated with a unique path through the trellis (all other connections were removed in the previous step). 4. Follow the selected path (specified at the final node) in reverse order and collect the quantization index 56 associated with the connections between the trellis nodes.

[0244] It should be noted that determining the quantization index 56 based on the Viterbi algorithm is not substantially more complex than rate-distortion-optimized quantization (RDOQ) for independent scalar quantization. Nevertheless, simpler coding algorithms also exist for dependent quantization. For example, starting from a predefined initial state (or quantization set), the quantization index 56 can be determined in coding / reconstruction order by minimizing an arbitrary cost amount that considers only the effect of the current quantization index. Given the determined quantization index for the current parameter (and all preceding quantization indices), the quantization set for the next neural network parameter 13 is known. And therefore, this algorithm can be applied to all neural network parameters in coding order.

[0245] The method according to the following embodiment is shown in Figures 20, 21, 22, and 23.

[0246] Figure 20 is a block diagram of method 400 for decoding neural network parameters that define a neural network from a data stream. Method 400 sequentially decodes neural network parameters by the steps of: selecting a set of reconstruction levels from a set of reconstruction levels 54 for the current neural network parameter, depending on the quantization index decoded from the data stream for previous neural network parameters; decoding the quantization index 420 of the current neural network parameter from the data stream, where the quantization index indicates one reconstruction level from the selected set of reconstruction levels for the current neural network parameter; and dequantizing the current neural network parameter 62 onto one reconstruction level from the selected set of reconstruction levels indicated by the quantization index for the current neural network parameter.

[0247] Figure 21 is a block diagram of method 500 for encoding neural network parameters that define a neural network from a data stream. Method 500 includes sequentially encoding neural network parameters by the steps of: selecting a set of reconstruction levels from a plurality of reconstruction level sets 54 for the current neural network parameter according to the quantization index encoded in the data stream for the previously encoded neural network parameter; quantizing the current neural network parameter to one reconstruction level from the selected set of reconstruction levels 64; and encoding the quantization index of the current neural network parameter into the data stream 530, indicating one reconstruction level to which the quantization index of the current neural network parameter is quantized.

[0248] Figure 22 is a block diagram of a method for reconstructing neural network parameters that define a neural network according to an embodiment of the present invention. Method 600 includes the step of deriving a first neural network parameter 610 for a first reconstruction layer to obtain a first neural network parameter value for a first reconstruction layer for each neural network parameter. Method 600 further includes the step of decoding a second neural network parameter for a second reconstruction layer from a data stream 620 (as shown by arrow 312 in Figure 6, for example) to obtain a neural network parameter value for a second reconstruction layer for each neural network parameter, and the step of reconstructing a neural network parameter 630 (as shown by arrow 314 in Figure 6, for example) by combining the neural network parameter value for a first reconstruction layer and the neural network parameter value for a second reconstruction layer for each neural network parameter.

[0249] Figure 23 is a block diagram of a method for encoding neural network parameters that define a neural network, according to an embodiment of the present invention. The method 700 includes the step of encoding a second neural network parameter of a second reconstructed layer into a data stream (for example, as shown by arrow 322 in Figure 6) for each neural network parameter, using a first neural network parameter of a first reconstructed layer which includes the neural network parameter value of the first reconstructed layer, and the second reconstructed layer which includes the second reconstructed layer neural network parameter value for each neural network parameter, and the neural network parameter is reconstructible for each neural network parameter by combining the first reconstructed layer neural network parameter value and the second reconstructed layer neural network parameter value.

[0250] JPEG2026067854000020.jpg235168

[0251] For example, the two-dimensional integer array StateTransTab[][] shown on line 1014 specifies a state transition table for dependent scalar quantization, and is as follows:

[0252] JPEG2026067854000021.jpg224170

[0253] The inputs to this process are as follows: The variable `tensorDims` specifies the dimension of the tensor to be decoded. • A variable, entryPointOffset, that indicates whether an entry point for decoding exists, and if so, the entry point offset. • The `codebookId` variable indicates whether a codebook is applicable, and if so, which codebook to use. The output of this process is a variable recParam of type TENSOR_FLOAT, with dimensions equal to tensorDims.

[0254] JPEG2026067854000022.jpg76162

[0255] Regarding the derivation process of ctxInc, which indicates the context or probability estimation to be used for the syntax element sig_flag,

[0256] The input to this process is the sig_flag that was decoded before the current sig_flag, the state value stateId, and the associated sign_flag (if any). If there is no sig_flag that was decoded before the current sig_flag, it is considered 0. Also, if the sign_flag associated with a previously decoded sig_flag has not been decoded, it is considered 0.

[0257] The output of this process is the variable ctxInc. The variable ctxInc is derived as follows: If sig_flag is 0, stateId*3 will be set to ctxInc. Otherwise, if sign_flag is equal to 0, ctxInc is set to stateId*3+1. Otherwise, ctxInc will be set to stateId*3+2.

[0258] The above example illustrates the concept of encoding / decoding neural network parameters 13 to / from a data stream 14, where the neural network parameters 13 may relate to the weights of the neuron interconnects 11 of the neural network 10, for example, the weights of a weight tensor. The decoding / encoding of the neural network parameters 13 is performed sequentially. See the for-next loop 1000 which cycles through the weights of the tensor with the same number of weights as the product of the number of weights per dimension of the tensor. The weights are scanned in a predetermined order TensorIndex(dimensions,i,scan_order). For the current neural network parameter idx13', one of two reconstruction level sets 52 is selected in 1018 and 1020 according to the quantization state stateId, which is continuously updated based on the quantization index 58 decoded from the data stream of the previous neural network parameter. In particular, the quantization index for the current neural network parameter idx is decoded from the data stream at 10¹², where the quantization index indicates one of the selected sets of reconstruction levels for the current neural network parameter 13'. The two sets of reconstruction levels are defined by duplication at 10¹² and subsequent additions of 1 or -1 depending on the quantization state index at 10¹⁸ and 10²⁰. Here, at 10¹⁸ and 10²⁰, the current neural network parameter 13' is actually dequantized onto one of the selected sets of reconstruction levels indicated by the quantization index QuantParam[idx] for the current neural network parameter 13'. The step size stepSize is used to parameterize the sets of reconstruction levels at 3001–3003. Information regarding this given quantization step size stepSize is derived from the data stream via the syntax element qp_value. The latter may be encoded in the data stream for the entire tensor, the entire NN layer, or the entire NN, respectively.That is, the neural network 10 can include one or more NN layers 10a, 10b. For each NN layer, information on a predetermined quantization step size (QP) is derived from the data stream 14 for each NN layer, and then, for each NN layer, the parameter 13 of the neural network belonging to each NN layer is dequantized. The parameterization of a plurality of reconstruction level sets may be performed using the predetermined quantization step size derived for each NN layer as used for dequantization.

[0259] The first reconstruction level set for stateId = 0 includes zero and even multiples of a predetermined quantization step size here. As can be seen at 1018 and 1020, the second reconstruction level set for stateId = 1 includes zero and odd multiples of the predetermined quantization step size (QP). For each neural network parameter 13, at 1015 to 1021, an intermediate integer value QuantParam[idx] (IV) is derived according to the selected reconstruction level set for each neural network parameter 13 and the entropy decoding quantization index QuantParam[idx] for each neural network parameter. Next, for each neural network parameter, the intermediate value of each neural network parameter is multiplied by the predetermined quantization step size of each neural network parameter at 4001.

[0260] The selection of the reconstruction level set for the current neural network parameter 13' from two reconstruction level sets (e.g., set 0, set 1) is made according to the LSB portion of the quantization index decoded from the data stream of the previously decoded neural network parameter, as shown in 1014, and the transition table transitions from stateId to the next quantization state nextSt according to the LSB of QuantParam[idx] such that statId depends on the past sequence of quantization index 56 that has already been decoded. Thus, the state transitions depend on the result of a binary function of quantization index 56 decoded from the data stream of the previously decoded neural network parameter, i.e., its parity. In other words, for the current neural network parameter, the selection of a reconstruction level set from among several reconstruction level sets is performed by the state transition process by determining the reconstruction level set from among several reconstruction level sets in 1018 and 1020 for the current neural network parameter, depending on the state statId associated with the current neural network parameter, and by updating the stateId in 1014 for the subsequent neural network parameter whose stateId should be determined next, although not necessarily the NN parameter to be encoded / decoded next, depending on the quantization index decoded from the data stream for the previous neural network parameter, i.e., the parameter whose stateId had been determined so far. For example, here the current neural network parameter is used for the update to obtain the stateId of the NN parameter to be encoded / decoded next. The update in 1014 is performed using a binary function of the quantization index decoded from the data stream of the previous (current) neural network parameter, i.e., its parity. The state transition process is configured to transition between eight possible states. The transitions are performed via the table StateTransTab[][].In the state transition process, transitions occur between these eight possible states. For the current neural network parameters, the set of reconstruction levels from the quantization sets corresponding to the state stateId associated with the current neural network parameters is determined at 1018 and 1020. If the state belongs to the first half of the even possible states, i.e., an odd state, the first of the two reconstruction level sets is determined. If the state belongs to the second half of the even possible states, i.e., an even state, the second of the two reconstruction level sets is determined. The state statId is updated by the transition table StateTransTab[][]. The transition table maps the combination of the state statID decoded from the data stream of the previous (current) neural network parameter, the parity of the quantization index (58), and QuantParam[idx]&1 to another state associated with the subsequent neural network parameter.

[0261] The quantization index of the current neural network parameter is encoded into a data stream using arithmetic coding and decoded from the data streak, using a probabilistic model that depends on the state of the quantization stateId, i.e., the state of the current neural network parameter 13', which is a set of reconstruction levels selected for the current neural network parameter. See the third parameter when calling the function int_param in 1012. In particular, the quantization index for the current neural network parameter can be encoded into a data stream using binary arithmetic coding / decoding and decoded from the data stream, using a probabilistic model that corresponds to the state of the current neural network parameter for at least one bin of the binarized quantization index. Here, among the binarized sig_flag, sign_flag (optional), abs_level_greater_x[j], abs_level_greater_x2[j], and abs_remainder, the bin sig_flag is a significance bin that indicates whether the quantization index (56) of the current neural network parameter is equal to zero or not. The dependency of a probabilistic model involves selecting a context from a set of neural network parameter contexts using dependencies, with each context configured to associate with a given probabilistic model. Here, the context for sig_flag is selected using ctxInc as an index incrementer for indexing the context from a list of contexts, each associated with a binary probabilistic model. The model can be updated using the bins associated with the contexts; that is, a given probabilistic model associated with each context can be updated based on an arithmetic-encoded quantized index using its respective context.

[0262] (Note) It should be noted that the probabilistic model of sig_flag further depends on the quantization index of the previously decoded neural network parameter, i.e., the sig_flag of the previously decoded neural network parameter and its sign_flag - which indicates its sign. More precisely, depending on the state stateId, a subset of probabilistic models is pre-selected from among several probabilistic models, i.e., from the context incrementer states 0...23, i.e., from the eight subsets of probabilistic models for sig_flag, including three consecutive contexts from {0...23}, and the probabilistic model for the current neural network parameter is selected according to the quantization index of the previously decoded neural network parameter (121), i.e., based on the sig_flag and sign_flag of the previous NN parameter. The pre-selected subset for the first value of stateID is inconsistent with the pre-selected subset for other values ​​of stateID. The previous NN parameter in which sig_flag and sign_flag are used relates to the part of the neural network adjacent to the part to which the current neural network parameter is relevant.

[0263] Multiple embodiments have been described above. It should be noted that the aspects and features of the embodiments may be used individually or in combination. Furthermore, the aspects and features of the embodiments relating to the first and second aspects of the present invention may be used in combination.

[0264] Further embodiments configure the device such that the neural network parameters relate to one of the reconstruction layers used to represent the neural network 10, for example, an enhancement layer. The device may be configured such that the neural network is reconfigurable by combining the neural network parameters of one or more other reconstruction layers, on a neural network parameter basis, with corresponding ones, for example, those related to common neuron interconnections, or, frankly speaking, those juxtaposed with the matrix representation of the NN layer in different representation layers.

[0265] For example, as described in this embodiment, the features and aspects of the first and second aspects of the present invention may be combined. Any features of the dependent claims according to the second aspect are also transferable to bring about further embodiments.

[0266] Furthermore, the apparatus according to the embodiment of the present invention may be configured to encode the quantization index 56 for the current neural network parameter 13' into the data stream 14 using arithmetic coding with a probabilistic model that depends on the neural network parameter corresponding to the current neural network parameter.

[0267] Each further embodiment constitutes an apparatus in which the neural network parameters relate to one of the reconstruction layers in which the neural network 10 is represented, for example, an enhancement layer. The apparatus may be configured to reconstruct the neural network by combining the neural network parameters on a neural network parameter basis with the neural network parameters of one or more other reconstruction layers, for example, those related to a common neuron interconnection, or, frankly speaking, those juxtaposed in the matrix representation of the NN layer in a different representation layer.

[0268] For example, as described in this embodiment, the features and aspects of the first and second aspects of the present invention may be combined. Any features of the dependent claims according to the second aspect are also transferable to bring about further embodiments.

[0269] Furthermore, the apparatus according to the present invention may be configured to decode the quantization index 56 of the current neural network parameter 13' from the data stream 14 using arithmetic coding, using a probabilistic model that depends on the corresponding neural network parameter corresponding to the current neural network parameter.

[0270] In other words, the neural network parameters of the reconstruction layer, such as the second neural network parameters described above, can be encoded / decoded and / or quantized / dequantized according to the concepts described with respect to Figures 3 and 5 and Figures 2 and 4, respectively.

[0271] While some embodiments have been described in the context of apparatus, it is clear that these embodiments also represent descriptions of corresponding methods, where a block or apparatus corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent descriptions of features of corresponding blocks or items or corresponding apparatus.

[0272] The data stream of the present invention can be stored in a digital storage medium or transmitted via a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

[0273] Depending on specific implementation requirements, embodiments of the present invention may be implemented in hardware or in software. Implementation may be carried out using a digital storage medium, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM, or flash memory, which has electronically readable control signals stored thereon and which cooperates (or can cooperate) with a computer system that is programmable to perform each method.

[0274] Some embodiments of the present invention include a data carrier having an electronically readable control signal, which can cooperate with a programmable computer system so that one of the methods described herein is performed.

[0275] Generally, embodiments of the present invention can be implemented as a computer program product having program code, the program code being operable to perform one of the methods when the computer program product is executed on a computer. The program code can be stored, for example, on a machine-readable carrier.

[0276] Another embodiment comprises storing a computer program for performing one of the methods described herein in a machine-readable carrier.

[0277] In other words, an embodiment of the present invention is a computer program having program code for performing one of the methods described herein when the computer program is executed on a computer.

[0278] Therefore, a further embodiment of the present invention is a data carrier (or digital storage medium or computer-readable medium) having a computer program for performing one of the methods described herein recorded thereon.

[0279] A further embodiment of the present invention is a data stream or sequence of signals representing a computer program for performing one of the methods described herein. The data stream or sequence of signals may be configured to be transmitted, for example, over a data communication connection, such as the Internet.

[0280] Further embodiments include processing means, such as a computer or a programmable logic device, configured or adapted to perform one of the methods described herein.

[0281] Further embodiments include a computer on which a computer program for performing one of the methods described herein is installed.

[0282] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the method described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, the method is preferably performed by any hardware device.

[0283] The embodiments described above are merely illustrative of the principles of the present invention. Modifications and variations of the arrangements and details described herein will be obvious to those skilled in the art. Therefore, it is intended that the invention is limited only by the scope of the imminent claims and not by the specific details presented herein.

[0284] JPEG2026067854000023.jpg209170

Claims

1. A device for decoding neural network parameters (13) that define a neural network (10) from a data stream (14), For the current neural network parameters (13'), a reconstruction level set (48) is selected (54) from among multiple (50) reconstruction level sets (52) corresponding to the quantization index (58) decoded from the data stream (14) for the previous neural network parameters. A quantization index (56) for the current neural network parameters (13') is decoded from the data stream (14), where the quantization index (56) represents one of the selected reconstruction level sets (48) for the current neural network parameters. An apparatus configured to sequentially decode the neural network parameters (13) by dequantizing (62) the current neural network parameters (13') to one of the selected reconstruction levels (48) indicated by the quantization index (56) for the current neural network parameters.

2. The apparatus according to claim 1, wherein the neural network parameters (13) are related to the weights of the neuron interconnections (11) of the neural network (10).

3. The apparatus according to claim 1 or 2, wherein the number of the plurality of (50) reconstruction level sets (52) is 2.

4. The apparatus according to any one of claims 1 to 3, wherein the plurality of (50) reconstruction level sets (52) are parameterized (60) by a predetermined quantization step size (QP), and information relating to the predetermined quantization step size is derived from the data stream (14).

5. The neural network includes one or more NN layers, and the apparatus is For each NN layer (p; p-1), information regarding a predetermined quantization step size for each NN layer is derived from the data stream (14). The apparatus according to any one of claims 1 to 4, wherein each of the NN layers is configured to parameterize the plurality (50) reconstruction level sets (52) using the predetermined quantization step size derived for each of the NN layers, so as to be used to dequantize the neural network parameters belonging to each of the NN layers.

6. The number of reconstruction level sets (52) among the plurality of (50) reconstruction level sets (52) is 2, and the plurality of reconstruction level sets are, A first set of reconstruction levels (set 0) including zero and even multiples of a predetermined quantization step size, The apparatus according to any one of claims 1 to 5, comprising a second set of reconstruction levels (set 1) including zero and odd multiples of a predetermined quantization step size.

7. All reconstruction levels in all reconstruction level sets represent integer multiples of a predetermined quantization step size, and the apparatus, For each neural network parameter, an intermediate integer value is derived according to the selected reconstruction level set for each neural network parameter and the quantization index after entropy decoding for each neural network parameter, and The apparatus according to any one of claims 1 to 6, wherein the neural network parameter is dequantized by multiplying the intermediate value for each neural network parameter by a predetermined quantization step size for each neural network parameter.

8. The number of reconfiguration level sets (52) among the plurality of (50) reconfiguration level sets (52) is 2, and the apparatus is If the selected set of reconstruction levels for each of the neural network parameters is the first set, then the quantization index for each of the neural network parameters is doubled to obtain the intermediate value for each of the neural network parameters, and If the selected set of reconstruction levels for each neural network parameter is a second set, and the quantization index for each neural network parameter is equal to zero, then the intermediate value for each sample is set to equal to zero, and If the selected set of reconstruction levels for each neural network parameter is a second set, and the quantization index for each neural network parameter is greater than zero, then the quantization index for each neural network parameter is doubled, and 1 is subtracted from the result of the multiplication to obtain the intermediate value for each neural network parameter, and The apparatus according to claim 7, wherein the selected set of reconstruction levels for the current neural network parameters is a second set, and if the quantization index for each of the neural network parameters is less than zero, the apparatus is configured to derive the intermediate value for each neural network parameter by doubling the quantization index for each of the neural network parameters and adding 1 to the result of the multiplication to obtain the intermediate value for each of the neural network parameters.

9. The apparatus according to any one of claims 1 to 8, wherein the current neural network parameter (13') is configured to select (54) the reconstruction level set (48) from the plurality (50) reconstruction level sets (52) according to the LSB portion of the binarized quantization index (58) decoded from the data stream (14) for previously decoded neural network parameters or the previously decoded bin.

10. The apparatus according to any one of claims 1 to 8, wherein the current neural network parameters (13') are configured to select (54) the reconstruction level set (48) from among the plurality (50) reconstruction level sets (52) according to the result of a binary function of the quantization index (58) decoded from the data stream (14) for previously decoded neural network parameters.

11. The apparatus according to any one of claims 1 to 10, wherein the apparatus is configured to select (54) the quantization level set (48) from the plurality (50) reconstruction level sets (52) in accordance with the parity of the quantization index (58) decoded from the data stream (14) for the previously decoded neural network parameters (13') for the current neural network parameters (13').

12. The number of the multiple (50) reconstruction level sets (52) is 2, and the device is Based on the selected reconstruction level set for each of the aforementioned neural network parameters and the binary function of the quantization index for each of the aforementioned neural network parameters, a subset index for each of the aforementioned neural network parameters is derived, resulting in four possible values ​​for the subset index; and, The apparatus according to any one of claims 1 to 11, configured to select (54) the reconstruction level set (48) from the plurality (50) reconstruction level sets (52) with respect to the current neural network parameters (13') according to the subset index of previously decoded neural network parameters.

13. The aforementioned device is The apparatus according to claim 12, wherein, with respect to the current neural network parameter (13'), the apparatus is configured to select (54) the reconstruction level set (48) from the plurality (50) reconstruction level sets (52) using a selection rule corresponding to the subset index of the plurality of neural network parameters that were decoded immediately before, and to apply the selection rule to all or some of the neural network parameters.

14. The apparatus according to claim 13, wherein the number of neural network parameters that correspond to the selection rule immediately before the selection is 2.

15. The apparatus according to any one of claims 12 to 14, wherein the subset index for each neural network parameter is derived based on the selected reconstruction level set for each neural network parameter and the parity of the quantization index for each neural network parameter.

16. The aforementioned device is With respect to the current neural network parameters (13'), the quantization level set (48) is determined from among the plurality of (50) reconstruction level sets (52) according to the state associated with the current neural network parameters (13'), and By updating the state for subsequent neural network parameters according to the quantization index (58) decoded from the data stream for the immediately preceding neural network parameter, The apparatus according to any one of claims 1 to 15, configured to select (54) the quantization level set (48) from the plurality (50) reconstruction level sets (52) for the current neural network parameters (13') by a state transition process.

17. The apparatus according to claim 16, configured to update the state of subsequent neural network parameters using a binary function of the quantization index (58) decoded from the data stream for the immediately preceding neural network parameter.

18. The apparatus according to claim 16, configured to update the state for the subsequent neural network parameter using the parity of the quantization index (58) decoded from the data stream for the immediately preceding neural network parameter.

19. The apparatus according to any one of claims 16 to 18, wherein the state transition process is configured to transition between four or eight possible states.

20. The apparatus according to any one of claims 16 to 19, wherein the state transition process is configured to transition between an even number of possible states, and the number of reconstruction level sets (52) among the plurality (50) reconstruction level sets (52) is 2, and thereafter, with respect to the current neural network parameter (13'), the quantization level set (48) among the quantization set (52) is determined according to the state related to the current neural network parameter (13'), so that if the state belongs to the first half of the even number of possible states, a first reconstruction level set among the plurality (50) reconstruction level sets (52) is determined, and if the state belongs to the second half of the even number of possible states, a second reconstruction level set among the plurality (50) reconstruction level sets (52) is determined.

21. The apparatus according to any one of claims 16 to 20, configured to perform the update of the state by a transition table that maps a combination of the state and the parity of the quantization index (58) decoded from the data stream for the immediately preceding neural network parameter to another state related to the subsequent neural network parameter.

22. The apparatus according to any one of claims 1 to 21, configured to decode the quantization index (56) for the current neural network parameters (13') from the data stream (14) using arithmetic coding that uses a probabilistic model (123) corresponding to the reconstruction level set (48) selected for the current neural network parameters (13').

23. With respect to the current neural network parameters (13'), the quantization level set (48) is determined from among the plurality of (50) reconstruction level sets (52) according to the state associated with the current neural network parameters (13'), and By updating the state for subsequent neural network parameters according to the quantization index (58) decoded from the data stream for the immediately preceding neural network parameter, The state transition process selects (54) the quantization level set (48) from the plurality (50) reconstruction level sets (52) for the current neural network parameters (13'), and, The apparatus according to any one of claims 1 to 21, wherein the quantization index (56) for the current neural network parameters (13') is decoded from the data stream (14) using arithmetic coding that uses a (122) probability model corresponding to the state for the current neural network parameters (13').

24. The apparatus according to claim 23, configured to decode the quantization index (56) for the current neural network parameters from the data stream (14) using binary arithmetic coding, by using the probabilistic model (122) corresponding to the state for the current neural network parameters (13') for at least one bin (84) of the binarized (82) quantization index (56).

25. The apparatus according to claim 23, wherein at least one bin includes a significance bin indicating whether the quantization index (56) for the current neural network parameters is equal to zero.

26. The apparatus according to any one of claims 23 to 25, wherein the at least one bin includes a code bin (86) indicating whether the quantization index (56) of the current neural network parameters is greater than or less than zero.

27. The apparatus according to any one of claims 23 to 26, wherein the at least one bin includes a greater-than-X bin indicating whether the absolute value of the quantization index (56) of the current neural network parameters is greater than X, where X is an integer greater than zero.

28. The apparatus according to any one of claims 22, 23, 25 to 27, wherein the dependency of the probabilistic model includes the selection (103) of a context (87) from a context set for the neural network parameters using the dependency, and each context is configured to be associated with a predetermined probabilistic model.

29. The apparatus according to claim 28, configured to update the predetermined probability model associated with each of the contexts based on the quantization index arithmetically encoded using each of the contexts.

30. The apparatus according to any one of claims 1 to 29, wherein for at least one bin of the binarized quantization index, the quantization index (56) for the current neural network parameter (13') is decoded from the data stream (14) using binary arithmetic coding, using a probabilistic model corresponding to the reconstruction level set (48) selected for the current neural network parameter (13').

31. The apparatus according to claim 30, wherein at least one bin includes a significance bin indicating whether the quantization index (56) of the current neural network parameters is equal to zero.

32. The apparatus according to claim 30 or 31, wherein the at least one bin includes a code bin indicating whether the quantization index (56) of the current neural network parameters is greater than or less than zero.

33. The apparatus according to any one of claims 30 to 32, wherein the at least one bin includes a greater-than-X bin indicating whether the absolute value of the quantization index (56) of the current neural network parameters is greater than X, where X is an integer greater than zero.

34. The apparatus according to any one of claims 22 to 33, wherein the probabilistic model additionally corresponds to the quantization index of previously decoded neural network parameters.

35. The apparatus according to claim 34, configured to pre-select a subset of probabilistic models from a plurality of probabilistic models in accordance with the state or set (48) of the selected reconstruction level for the current neural network parameters (13'), and to select the probabilistic model for the current neural network parameters from the subset of probabilistic models in accordance with the quantization index (121) of the previously decoded neural network parameters.

36. The apparatus according to claim 35, wherein, depending on the state or set (48) of the selected reconstruction level for the current neural network parameters (13'), the apparatus is configured to pre-select a subset of probabilistic models from among the plurality of probabilistic models such that a pre-selected subset for a first state or set of reconstruction levels is coprime to any other pre-selected subset for any other state or set of reconstruction levels.

37. The apparatus according to claim 35 or 36, configured to select the probabilistic model of the current neural network parameters from a subset of probabilistic models, depending on the quantization index of a previously decoded neural network parameter relating to a portion of the neural network adjacent to the portion relating to the current neural network parameter.

38. The apparatus is configured to select the probabilistic model for the current neural network parameters from a subset of the probabilistic models, according to the characteristics of the quantization index of the previously decoded neural network parameters relating to the portion of the neural network adjacent to the portion relating to the current neural network parameters, wherein the characteristics are: The sign of the non-zero quantization index of the previously decoded neural network parameter relating to the portion of the neural network adjacent to the portion relating to the current neural network parameter, The number of quantization indices of previously decoded neural network parameters related to the portion of the neural network adjacent to the portion to which the current neural network parameters relate, and which is a non-zero number, The sum of the absolute values ​​of the quantization indices of previously decoded neural network parameters relating to the portion of the neural network adjacent to the portion relating to the current neural network parameters, The apparatus according to claim 35 or 36, comprising one or more of the following: the sum of the absolute values ​​of the quantization indices of previously decoded neural network parameters relating to the portion of the neural network adjacent to the portion of the neural network to which the current neural network parameters relate; and the difference between the number of quantization indices of previously decoded neural network parameters relating to the portion of the neural network adjacent to the portion of the neural network to which the current neural network parameters relate, and which is a non-zero number.

39. The apparatus according to claim 37 or 38, configured to position the previously decoded neural network parameters such that the previously decoded neural network parameters relate to the same neural network layer as the current neural network parameters.

40. The apparatus according to any one of claims 37 and 39, wherein one or more of the previously decoded neural network parameters are configured to position one or more of the previously decoded neural network parameters such that the neuronal interconnects referenced by the current neural network parameters relate to a neuron interconnect emerging from or toward another neuron adjacent to the neuron.

41. The apparatus according to any one of claims 1 to 40, configured to decode the quantization index (56) for the neural network parameters (13) and to perform the inverse quantization of the neural network parameters (13) in a common continuous order (14') among the neural network parameters (13).

42. The apparatus according to any one of claims 1 to 41, configured to decode the quantization index (56) for the current neural network parameter (13') from the data stream (14) using binary arithmetic coding by using the probabilistic model corresponding to previously decoded neural network parameters for one or more leading bins of the binarized quantization index, and by using equiprobability bypass-mode suffix bins of the binarized quantization index that follow the one or more leading bins.

43. The apparatus according to claim 42, wherein the suffix bin of the binarized quantization index represents a bin of a suffix binarization binarization code for binarizing a value of the quantization index whose absolute value exceeds the maximum absolute value that can be represented by one or more of the leading bins, and the apparatus is configured to select the suffix binarization in accordance with the quantization index of a previously decoded neural network parameter.

44. The neural network parameters relate to one of the reconstruction layers in which the neural network (10) is represented, and the apparatus is The apparatus according to any one of claims 1 to 43, wherein the neural network is reconstructed by combining the neural network parameters with corresponding neural network parameters of one or more other reconstruction layers on a neural network parameter basis.

45. The apparatus according to claim 44, configured to decode the quantization index (56) for the current neural network parameters (13') from the data stream (14) using arithmetic coding that uses a probabilistic model corresponding to the corresponding neural network parameters, which corresponds to the current neural network parameters.

46. A device for encoding neural network parameters that define a neural network into a data stream, For the current neural network parameters (13'), a reconstruction level set (48) is selected (54) from among multiple (50) reconstruction level sets (52), corresponding to the quantization index (58) encoded in the data stream (14) for previously encoded neural network parameters. The current neural network parameters (13') are quantized (64) onto one of the selected reconstruction level sets (48), An apparatus configured to sequentially encode the neural network parameters (13) by encoding the quantization index (56) for the current neural network parameters (13') in the data stream (14), which indicates one reconstruction level in which the quantization index (56) for the current neural network parameters is quantized.

47. The apparatus according to claim 46, wherein the neural network parameters (13) are related to the weights of the neuronal interconnections (11) of the neural network (10).

48. The apparatus according to claim 46 or 47, wherein the number of reconfiguration level sets (52) among the plurality (50) reconfiguration level sets (52) is 2.

49. The apparatus according to any one of claims 46 to 48, wherein the plurality of (50) reconstruction level sets (52) are parameterized (60) by a predetermined quantization step size (QP), and information regarding the predetermined quantization step size is inserted into the data stream (14).

50. The neural network includes one or more NN layers, and the apparatus is For each NN layer (p; p-1), information regarding a predetermined quantization step size for each NN layer is inserted into the data stream (14). The apparatus according to any one of claims 1 to 49, wherein for each NN layer, the plurality of (50) reconstruction level sets (52) are parameterized using the predetermined quantization step size derived for each NN layer, so as to be used to quantize the neural network parameters belonging to each of the NN layers.

51. The number of reconstruction level sets (52) among the plurality of (50) reconstruction level sets (52) is 2, and the plurality of reconstruction level sets are, A first set of reconstruction levels (set 0) including zero and even multiples of a predetermined quantization step size, The apparatus according to any one of claims 46 to 50, comprising a second set of reconstruction levels (set 1) including zero and odd multiples of the predetermined quantization step size.

52. All reconstruction levels in all reconstruction level sets represent integer multiples of a predetermined quantization step size, and the apparatus, For each neural network parameter, an intermediate integer value is derived according to the selected reconstruction level set for each neural network parameter and the quantization index after entropy coding for each neural network parameter, and The apparatus according to any one of claims 46 to 51, wherein the apparatus is configured to quantize each neural network parameter so that it can be dequantized by multiplying the intermediate value for each neural network parameter by a predetermined quantization step size for each neural network parameter.

53. The number of reconfiguration level sets (52) among the plurality of (50) reconfiguration level sets (52) is 2, and the apparatus is If the selected set of reconstruction levels for each of the neural network parameters is the first set, then the quantization index for each of the neural network parameters is doubled to obtain the intermediate value for each of the neural network parameters, and If the selected set of reconstruction levels for each neural network parameter is a second set, and the quantization index for each neural network parameter is equal to zero, then the intermediate value for each sample is set to equal to zero, and If the selected set of reconstruction levels for each neural network parameter is a second set, and the quantization index for each neural network parameter is greater than zero, then the quantization index for each neural network parameter is doubled, and 1 is subtracted from the result of the multiplication to obtain the intermediate value for each neural network parameter, and The apparatus according to claim 52, wherein the selected set of reconstruction levels for the current neural network parameters is a second set, and if the quantization index for each of the neural network parameters is less than zero, the apparatus is configured to derive the intermediate value for each neural network parameter by doubling the quantization index for each of the neural network parameters and adding 1 to the result of the multiplication to obtain the intermediate value for each of the neural network parameters.

54. The apparatus according to any one of claims 46 to 53, configured to select (54) the reconstruction level set (48) from the plurality (50) reconstruction level sets (52) according to the current neural network parameter (13') and the LSB portion of the binarized quantization index (58) encoded in the data stream (14) for previously encoded neural network parameters or the previously encoded bin.

55. The apparatus according to any one of claims 46 to 53, wherein, with respect to the current neural network parameters (13'), the apparatus is configured to select (54) a reconstruction level set (48) from among the plurality (50) reconstruction level sets (52) according to the result of a binary function of the quantization index (58) encoded in the data stream (14) of previously encoded neural network parameters.

56. The apparatus according to any one of claims 46 to 55, wherein the apparatus is configured to select (54) the quantization level set (48) from a plurality of (50) reconstruction level sets (52) in accordance with the parity of the quantization index (56) encoded in the data stream (14) for the previously encoded neural network parameters (13') for the current neural network parameters (13').

57. The number of the multiple (50) reconstruction level sets (52) is 2, and the device is Based on the selected set of reconstruction levels for each of the aforementioned neural network parameters and the binary function of the quantization index for each of the aforementioned neural network parameters, a subset index for each of the aforementioned neural network parameters is derived, resulting in four possible values ​​for the subset index; and, The apparatus according to any one of claims 46 to 56, configured to select (54) the reconstruction level set (48) from the plurality (50) reconstruction level sets (52) with respect to the current neural network parameters (13') according to the subset index for previously encoded neural network parameters.

58. The aforementioned device is The apparatus according to claim 57, wherein, with respect to the current neural network parameter (13'), a selection rule corresponding to the subset index for a large number of previously encoded neural network parameters is used to select (54) the reconstruction level set (48) from the plurality (50) reconstruction level sets (52), and the selection rule is configured to be used for all or some of the neural network parameters.

59. The apparatus according to claim 58, wherein the number of previously encoded neural network parameters corresponding to the selection rule is 2.

60. The apparatus according to any one of claims 57 to 59, wherein the subset index for each neural network parameter is derived based on the selected reconstruction level set for each neural network parameter and the parity of the quantization index for each neural network parameter.

61. The aforementioned device is With respect to the current neural network parameters (13'), the quantization level set (48) is determined from among the plurality of (50) reconstruction level sets (52) according to the state associated with the current neural network parameters (13'), and By updating the state of subsequent neural network parameters according to the quantization index (58) encoded in the data stream for the immediately preceding neural network parameter, The apparatus according to any one of claims 46 to 60, configured to select (54) the quantization level set (48) from among the plurality (50) reconstruction level sets (52) with respect to the current neural network parameters (13') by a state transition process.

62. The apparatus according to claim 61, configured to update the state for subsequent neural network parameters using a binary function of the quantization index (58) encoded in the data stream for the immediately preceding neural network parameter.

63. The apparatus according to claim 61, configured to update the state for the subsequent neural network parameter using the parity of the quantization index (58) encoded in the data stream for the immediately preceding neural network parameter.

64. The apparatus according to any one of claims 61 to 63, wherein the state transition process is configured to transition between four or eight possible states.

65. The apparatus according to any one of claims 61 to 64, wherein the state transition process is configured to transition between an even number of possible states, and the number of reconstruction level sets (52) among the plurality (50) reconstruction level sets (52) is 2, and thereafter, with respect to the current neural network parameter (13'), the quantization level set (48) among the quantization set (52) is determined according to the state related to the current neural network parameter (13'), so that if the state belongs to the first half of the even number of possible states, a first reconstruction level set among the plurality (50) reconstruction level sets (52) is determined, and if the state belongs to the second half of the even number of possible states, a second reconstruction level set among the plurality (50) reconstruction level sets (52) is determined.

66. The apparatus according to any one of claims 61 to 65, configured to perform the update of the state by a transition table that maps a combination of the state encoded in the data stream for the immediately preceding neural network parameter and the parity of the quantization index (58) to another state related to the subsequent neural network parameter.

67. The apparatus according to any one of claims 46 to 66, configured to encode the quantization index (56) for the current neural network parameters (13') into the data stream (14) using arithmetic coding that uses a probabilistic model (123) corresponding to the reconstruction level set (48) selected for the current neural network parameters (13').

68. With respect to the current neural network parameters (13'), the set of quantization levels (48) from the plurality of (50) reconstruction level sets (52) is determined according to the state associated with the current neural network parameters (13'), and By updating the state for subsequent neural network parameters according to the quantization index (58) encoded in the data stream for the immediately preceding neural network parameter, The state transition process selects (54) the quantization level set (48) from the plurality (50) reconstruction level sets (52) for the current neural network parameters (13'), and, The apparatus according to any one of claims 46 to 66, wherein the quantization index (56) for the current neural network parameters (13') is encoded into the data stream (14) using arithmetic coding that uses a (122) probability model corresponding to the state for the current neural network parameters (13').

69. The apparatus according to claim 68, configured to encode the quantization index (56) for the current neural network parameter (13') into the data stream (14) using binary arithmetic coding, by using the probabilistic model (122) corresponding to the state for the current neural network parameter (13') for at least one bin (84) of the binarized quantization index (56) (82).

70. The apparatus according to claim 68, wherein at least one bin includes a significance bin indicating whether the quantization index (56) for the current neural network parameters is equal to zero.

71. The apparatus according to any one of claims 68 to 70, wherein the at least one bin includes a code bin (86) indicating whether the quantization index (56) of the current neural network parameters is greater than or less than zero.

72. The apparatus according to any one of claims 68 to 71, wherein the at least one bin includes a greater-than-X bin indicating whether the absolute value of the quantization index (56) of the current neural network parameters is greater than X, where X is an integer greater than zero.

73. The apparatus according to any one of claims 67, 68, 70 to 72, wherein the dependency of the probabilistic model includes the selection (103) of a context (87) from a context set for the neural network parameters using the dependency, and each context is configured to be associated with a predetermined probabilistic model.

74. The apparatus according to claim 73, configured to update the predetermined probability model associated with each of the contexts based on the quantization index arithmetically encoded using each of the contexts.

75. The apparatus according to any one of claims 46 to 74, wherein for at least one bin of the binarized quantization index, the quantization index (56) for the current neural network parameter (13') is encoded into the data stream (14) using binary arithmetic coding, using a probabilistic model corresponding to the reconstruction level set (48) selected for the current neural network parameter (13').

76. The apparatus according to claim 75, wherein at least one bin includes a significance bin indicating whether the quantization index (56) of the current neural network parameters is equal to zero.

77. The apparatus according to claim 75 or 76, wherein the at least one bin includes a sign bin indicating whether the quantization index (56) of the current neural network parameters is greater than or less than zero.

78. The apparatus according to any one of claims 75 to 77, wherein the at least one bin includes a greater-than-X bin indicating whether the absolute value of the quantization index (56) of the current neural network parameters is greater than X, where X is an integer greater than zero.

79. The apparatus according to any one of claims 67 to 78, wherein the probabilistic model additionally corresponds to the quantization index of previously encoded neural network parameters.

80. The apparatus according to claim 79, configured to pre-select a subset of probabilistic models from a plurality of probabilistic models in accordance with the state or set (48) of the selected reconstruction level for the current neural network parameters (13'), and to select the probabilistic model for the current neural network parameters from the subset of probabilistic models in accordance with the quantization index (121) of the previously encoded neural network parameters.

81. The apparatus according to claim 80, wherein, depending on the state or set (48) of the selected reconstruction level for the current neural network parameters (13'), the apparatus is configured to pre-select a subset of probabilistic models from among the plurality of probabilistic models such that a pre-selected subset for a first state or set of reconstruction levels is coprime to any other pre-selected subset for any other state or set of reconstruction levels.

82. The apparatus according to claim 80 or 81, configured to select the probabilistic model of the current neural network parameters from a subset of probabilistic models, in accordance with the quantization index of a previously encoded neural network parameter relating to a portion of the neural network adjacent to the portion relating to the current neural network parameter.

83. The apparatus is configured to select the probabilistic model for the current neural network parameters from a subset of the probabilistic models, according to the characteristics of the quantization index of the previously encoded neural network parameters relating to the portion of the neural network adjacent to the portion relating to the current neural network parameters, wherein the characteristics are: The sign of the non-zero quantization index of the previously encoded neural network parameter relating to the portion of the neural network adjacent to the portion relating to the current neural network parameter, The number of quantization indices of previously encoded neural network parameters related to the portion of the neural network adjacent to the portion of the neural network to which the current neural network parameters relate, and which is a non-zero number, The sum of the absolute values ​​of the quantization indices of previously encoded neural network parameters relating to the portion of the neural network adjacent to the portion relating to the current neural network parameters, The apparatus according to claim 80 or 81, comprising one or more of the following: the sum of the absolute values ​​of the quantization indices of previously encoded neural network parameters relating to the portion of the neural network adjacent to the portion relating to the current neural network parameters; and the difference between the number of quantization indices of previously encoded neural network parameters relating to the portion of the neural network adjacent to the portion relating to the current neural network parameters, which is a non-zero number.

84. The apparatus according to either claim 82 or 83, configured to position the previously encoded neural network parameters such that the previously encoded neural network parameters relate to the same neural network layer as the current neural network parameters.

85. The apparatus according to any one of claims 82 and 84, wherein one or more of the previously encoded neural network parameters are configured to position one or more of the previously encoded neural network parameters such that the neuronal interconnects referenced by the current neural network parameter relate to a neuron interconnect emerging from or toward another neuron adjacent to the neuron.

86. The apparatus according to any one of claims 46 to 75, configured to encode the quantization index (56) for the neural network parameters (13) and to perform the quantization of the neural network parameters (13) in a common continuous order (14') among the neural network parameters (13).

87. The apparatus according to any one of claims 46 to 86, configured to encode the quantization index (56) for the current neural network parameter (13') into the data stream (14) using binary arithmetic coding by using the probabilistic model corresponding to previously encoded neural network parameters for one or more leading bins of the binarized quantization index, and by using equiprobability bypass-mode suffix bins of the binarized quantization index that follow the one or more leading bins.

88. The apparatus according to claim 87, wherein the suffix bin of the binarized quantization index represents a bin of a suffix binarization binarization code for binarizing a value of the quantization index whose absolute value exceeds the maximum absolute value that can be represented by one or more of the leading bins, and the apparatus is configured to select the suffix binarization in accordance with the quantization index of a previously encoded neural network parameter.

89. The neural network parameters relate to one of the reconstruction layers in which the neural network (10) is represented, and the apparatus is The apparatus according to any one of claims 46 to 88, wherein the neural network is configured to be reconfigurable by combining the neural network parameters with corresponding neural network parameters of one or more other reconstruction layers on a neural network parameter basis.

90. The apparatus according to claim 89, configured to encode the quantization index (56) for the current neural network parameter (13') into the data stream (14) using arithmetic coding that uses a probabilistic model corresponding to the corresponding neural network parameter, corresponding to the current neural network parameter.

91. The apparatus according to any one of claims 46 to 90, configured to perform the selection and / or quantization using the Viterbi algorithm and a rate-distortion cost measure.

92. A device (310) for reconstructing neural network parameters (13) that define a neural network (10), The first neural network parameters for the first reconstruction layer are derived, and for each neural network parameter (13), the neural network parameter value for the first reconstruction layer is obtained. The second neural network parameters (13) for the second reconstruction layer are decoded (312) from the data stream (14), and for each neural network parameter (13), the neural network parameter value for the second reconstruction layer is obtained. An apparatus configured to reconstruct the neural network parameters (13) (314) by combining the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer for each neural network parameter.

93. From the aforementioned data stream, or from a separate data stream, the first neural network parameters (13) for the first reconstruction layer are decoded (316), and The apparatus (310) according to claim 92, configured to decode the second neural network parameters (13) for the second reconstruction layer from the data stream by context-adaptive entropy decoding using separate probabilistic contexts for the first and second reconstruction layers.

94. The apparatus (310) according to claim 92 or 93, configured to reconstruct the neural network parameters (13) for each neural network parameter by the sum or product of the parameter units of the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer.

95. The apparatus (310) according to any one of claims 92 to 94, configured to decode the neural network parameter values ​​of the second reconstruction layer from the data stream by context-adaptive entropy decoding using a probabilistic model corresponding to the neural network parameter values ​​of the first reconstruction layer.

96. Perform context-adaptive entropy decoding, Depending on the neural network parameter values ​​of the first reconstruction layer, a probabilistic context set is selected from the set of probabilistic context sets. In accordance with the neural network parameter values ​​of the first reconstruction layer, a probabilistic context to be used is selected from the selected set of probabilistic contexts, The apparatus (310) according to any one of claims 92 to 95, configured to decode the neural network parameter values ​​of the second reconstruction layer from the data stream.

97. The set of probability context sets includes three probability context sets, and the device is If the neural network parameter value of the first reconstruction layer is negative, the first probability context set is selected from the set of probability context sets as the selected probability context set. If the neural network parameter value of the first reconstruction layer is positive, a second probability context set is selected from the set of probability context sets as the selected probability context set. The apparatus (310) according to claim 96, wherein if the neural network parameter value of the first reconstruction layer is zero, a third probability context set is selected from the set of probability context sets as the selected probability context set.

98. The set of probability context sets includes two probability context sets, and the device is If the neural network parameter value of the first reconstruction layer is greater than a predetermined value, the first probability context set is selected from the set of probability context sets as the selected probability context set; if the neural network parameter value of the first reconstruction layer is not greater than the predetermined value, the second probability context set is selected from the set of probability context sets as the selected probability context set; or The apparatus (310) according to claim 96, configured such that if the absolute value of the neural network parameter value of the first reconstruction layer is greater than the predetermined value, the first probability context set is selected from the set of probability context sets as the selected probability context set, and if the absolute value of the neural network parameter value of the first reconstruction layer is not greater than the predetermined value, the second probability context set is selected from the set of probability context sets as the selected probability context set.

99. A device (320) that encodes neural network parameters (13) that define a neural network (10) using first neural network parameters (13) for the first reconstruction layer, which include the neural network parameter values ​​of the first reconstruction layer, for each neural network parameter (13), wherein the device The system is configured to encode (322) the second neural network parameters (13) for the second reconstruction layer into a data stream (14), including the neural network parameter values ​​of the second reconstruction layer for each neural network parameter (13). The apparatus (320) allows the neural network parameters (13) to be reconstructed by combining the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer for each neural network parameter.

100. The first neural network parameters (13) for the first reconstruction layer are encoded into the data stream or a separate data stream, and The apparatus (320) according to claim 99, configured to encode the second neural network parameters (13) for the second reconstruction layer into the data stream by context-adaptive entropy coding using separate probabilistic contexts for the first and second reconstruction layers.

101. The apparatus (320) according to any one of claims 99 to 100, wherein the neural network parameters (13) can be reconstructed for each neural network parameter by the sum or product of the parameter units of the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer.

102. The apparatus (320) according to any one of claims 99 to 101, wherein the neural network parameter values ​​of the second reconstruction layer are encoded into the data stream by context-adaptive entropy coding using a probabilistic model corresponding to the neural network parameter values ​​of the first reconstruction layer.

103. Perform context-adaptive entropy coding, Depending on the neural network parameter values ​​of the first reconstruction layer, a probabilistic context set is selected from the set of probabilistic context sets. In accordance with the neural network parameter values ​​of the first reconstruction layer, a probabilistic context to be used is selected from the selected set of probabilistic contexts, The apparatus (320) according to any one of claims 99 to 102, wherein the neural network parameter values ​​of the second reconstruction layer are encoded into the data stream.

104. The set of probability context sets includes three probability context sets, and the device is If the neural network parameter value of the first reconstruction layer is negative, the first probability context set is selected from the set of probability context sets as the selected probability context set. If the neural network parameter value of the first reconstruction layer is positive, a second probability context set is selected from the set of probability context sets as the selected probability context set. The apparatus (320) according to claim 103, wherein if the neural network parameter value of the first reconstruction layer is zero, a third probability context set is selected from the set of probability context sets as the selected probability context set.

105. The set of probability context sets includes two probability context sets, and the device is If the neural network parameter value of the first reconstruction layer is greater than a predetermined value, the first probability context set is selected from the set of probability context sets as the selected probability context set; if the neural network parameter value of the first reconstruction layer is not greater than the predetermined value, the second probability context set is selected from the set of probability context sets as the selected probability context set; or The apparatus (320) according to claim 103, configured such that if the absolute value of the neural network parameter value of the first reconstruction layer is greater than the predetermined value, the first probability context set is selected from the set of probability context sets as the selected probability context set, and if the absolute value of the neural network parameter value of the first reconstruction layer is not greater than the predetermined value, the second probability context set is selected from the set of probability context sets as the selected probability context set.

106. A method (400) for decoding neural network parameters (13) that define a neural network (10) from a data stream (14), With respect to the current neural network parameters (13'), the step of selecting (54) a reconstruction level set (48) from among multiple (50) reconstruction level sets (52) corresponding to the quantization index (58) decoded from the data stream (14) for the previous neural network parameters, A step (420) of decoding (420) a quantization index (56) for the current neural network parameters (13') from the data stream (14), wherein the quantization index (56) represents one of the selected set of reconstruction levels (48) for the current neural network parameters, A method (400) comprising the step of sequentially decoding the neural network parameters (13) by dequantizing (62) the current neural network parameters (13') onto one of the selected reconstruction levels (48) indicated by the quantization index (56) for the current neural network parameters.

107. A method (500) for encoding neural network parameters that define a neural network into a data stream, With respect to the current neural network parameters (13'), the steps include selecting (54) a reconstruction level set (48) from among multiple (50) reconstruction level sets (52) corresponding to the quantization index (58) encoded in the data stream (14) for previously encoded neural network parameters, The steps include: quantizing the current neural network parameters (13') onto one of the selected reconstruction levels (48) (64); A method (500) comprising the steps of: (530) encoding the quantization index (56) for the current neural network parameter into the data stream (14) which indicates one reconstruction level in which the quantization index (56) for the current neural network parameter is quantized; and (500) sequentially encoding the neural network parameter (13).

108. A method (600) for reconstructing neural network parameters (13) that define a neural network (10), The steps include deriving (610) the first neural network parameters for the first reconstruction layer, and obtaining the neural network parameter values ​​for the first reconstruction layer for each neural network parameter (13), The steps include decoding the second neural network parameters (13) for the second reconstruction layer from the data stream, and obtaining the neural network parameter values ​​for the second reconstruction layer for each neural network parameter (13) (620), and A method (600) comprising the step (630) of reconstructing the neural network parameters (13) for each neural network parameter by combining the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer.

109. A method (700) for encoding neural network parameters (13) that define a neural network (10) using first neural network parameters (13) for the first reconstruction layer, which include the neural network parameter values ​​of the first reconstruction layer, for each neural network parameter (13), For each neural network parameter (13), the second neural network parameter (13) for the second reconstruction layer, including the neural network parameter value of the second reconstruction layer, is encoded (710) into a data stream. The neural network parameters (13) can be reconstructed for each neural network parameter by combining the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer, according to the method (700).

110. A data stream encoded by the method described in claims 107 and 109.

111. A computer program having program code for performing the method described in claim 106, 107, 108, or 109 when the program is executed on one or more computers.

Images