Concepts for encoding parameters of neural networks.

Abstract

To provide a coding and decoding device, capable of effectively compressing a neural network parameter.SOLUTION: A decoding device contains steps of: decoding a neural network parameter by selecting a reconstruction level set in accordance with a quantization index that is decoded from a data stream of a past neural network parameter or a past neural network parameter that is before the past neural network parameter, or is coded by each data stream; and combining the neural network parameter in a reconstruction layer.SELECTED DRAWING: Figure 20

Description

[Technical Field]

[0001] Embodiments according to the present invention relate to the concept of encoding neural network parameters. [Background technology]

[0002] In its most basic form, a neural network is a chain of affine transformations followed by a nonlinear function for each element. It can be represented as a directed acyclic graph, as shown in Figure 1. Figure 1 is a schematic diagram showing an example of a neural network, exemplarily illustrating a two-layer feedforward neural network. In other words, Figure 1 is a graph representation of a feedforward neural network. Specifically, this two-layer neural network is a nonlinear function that maps a four-dimensional input vector to a solid line. This neural network includes four neurons 10c in the input layer, which are the input of the neural network, corresponding to the four-dimensional input vector; five neurons 10c in the hidden layer; and one neuron 10c in the output layer, which forms the output of the neural network. The neural network further includes neuronal interconnects 11 connecting neurons in different or subsequent layers. The neuronal interconnections 11 may have associated weights, which are related to the relationship between the interconnected neurons 10c. In particular, the weights weight the activation of the neurons of a layer when they are transferred to the subsequent layer, and in turn, a sum of the inbound weighted activations is formed at each neuron of the subsequent layer—corresponding to a linear function—followed by a nonlinear scalar function being applied to the weighted sum formed at each neuron / node of the subsequent layer—corresponding to a nonlinear function. Thus, each node, e.g., neuron 10c, is associated with a specific value, which is propagated forward to the next node by multiplication with the respective weight value of the edge, e.g., neuronal interconnection 11. Afterwards, all received values ​​are simply summed.

[0003] TIFF0007783376000001.tif66151 Summary of the Invention [Problem to be solved by the invention]

[0004] Neural networks typically have millions of parameters, and their representation can require hundreds of MB (e.g., megabytes). As a result, their inference procedure involves computing many dot-product operations between large matrices, which requires high computational resources to execute. Therefore, reducing the complexity of dot-product operations is very important.

[0005] In addition to the above problems, a large number of parameters of the neural network may need to be stored and, for example, transmitted from the server to the client. Furthermore, it may be advantageous to be able to provide information about the parameterization of the neural network to an entity gradually, for example, in a federated learning environment or when providing the parameterization of the neural network at different stages of quality that can be addressed when a particular recipient pays for or uses the neural network for inference.

[0006] It is therefore desirable to provide a concept for efficient coding of neural network parameters that is more efficient, e.g., from a compression standpoint. Additionally or alternatively, it is desirable to reduce the bitstream for neural network parameters and thus the signaling cost.

[0007] This object is achieved by the subject matter of the independent claims of the present application.

[0008] Further embodiments according to the invention are defined by the subject matter of the dependent claims of the present application. [Means for solving the problem]

[0009] An embodiment according to a first aspect of the present invention is an apparatus for decoding neural network parameters defining a neural network from a data stream, the apparatus being configured to sequentially decode the neural network parameters by selecting, for a current neural network parameter, a reconstruction level set from a plurality of reconstruction level sets in response to a quantization index decoded from the data stream for a previous neural network parameter. The apparatus is further configured to sequentially decode the neural network parameters by decoding, from the data stream, a quantization index for the current neural network parameter, where the quantization index indicates a reconstruction level of the selected reconstruction level set for the current neural network parameter, and by dequantizing the current neural network parameter onto the one reconstruction level of the selected reconstruction level set indicated by the quantization index for the current neural network parameter.

[0010] A further embodiment according to the first aspect of the present invention is an apparatus for encoding neural network parameters defining a neural network into a data stream, the apparatus being configured to sequentially encode the neural network parameters by selecting, for a current neural network parameter, a reconstruction level set from among a plurality of reconstruction level sets in response to quantization indices encoded into the data stream for previously encoded neural network parameters, the apparatus being further configured to sequentially encode the neural network parameters by quantizing the current neural network parameter to one reconstruction level from the selected reconstruction level set, and encoding, into the data stream, a quantization index for the current neural network parameter indicating one reconstruction level to which the quantization index for the current neural network parameter is quantized.

[0011] A further embodiment according to the first aspect of the present invention comprises a method for decoding neural network parameters defining a neural network from a data stream, the method comprising the step of sequentially decoding the neural network parameters by selecting, for a current neural network parameter, a reconstruction level set from among a plurality of reconstruction level sets in response to a quantization index decoded from the data stream for a previous neural network parameter. The method further comprises the step of sequentially encoding the neural network parameters by decoding a quantization index for the current neural network parameter from the data stream, where the quantization index indicates a reconstruction level of the selected reconstruction level set for the current neural network parameter, and dequantizing the current neural network parameter onto the one reconstruction level of the selected reconstruction level set indicated by the quantization index for the current neural network parameter.

[0012] A further embodiment according to the first aspect of the present invention comprises a method for encoding neural network parameters defining a neural network into a data stream, the method comprising sequentially encoding the neural network parameters by selecting, for a current neural network parameter, a reconstruction level set from among a plurality of reconstruction level sets in response to quantization indices encoded into the data stream for previously encoded neural network parameters, the method further comprising sequentially encoding the neural network parameters by quantizing the current neural network parameter to one of the selected reconstruction levels and by encoding into the data stream a quantization index for the current neural network parameter indicating the one reconstruction level to which the quantization index for the current neural network parameter is quantized.

[0013] The first aspect of the present invention is based on the idea that neural network parameters can be more efficiently compressed by using a non-constant quantizer but varying the same among the encoded neural network parameters, i.e., by selecting a set of reconstruction levels depending on the quantization indexes decoded or coded from a data stream of previously coded neural network parameters, respectively. Thus, reconstruction vectors that can refer to an ordered set of neural network parameters can be packed more tightly in an N-dimensional signal space, where N denotes the number of neural network parameters in the set of samples to be processed. Such dependent quantization can be used for decoding and inverse quantization by a decoding device, or for quantization and encoding by a coding device, respectively.

[0014] An embodiment according to a second aspect of the present invention is based on the idea that more efficient neural network coding may be achieved if the parameterizations provided in these stages—called reconstruction layers to distinguish them from the layer configuration of the neural network in the neural layers—are then combined on a neural network parameter-by-parameter basis to result in an improved neural network parameterization compared to any of the stages. Thus, an apparatus for reconstructing neural network parameters defining a neural network can derive first neural network parameters, e.g., first reconstruction layer neural network parameters, for the first reconstruction layer to obtain, for each neural network parameter, neural network parameter values ​​for the first reconstruction layer. The first neural network parameters may have been previously transmitted, e.g., during an associative learning process. Furthermore, the first neural network parameters may be neural network parameter values ​​for the first reconstruction layer. The apparatus is further configured to decode second neural network parameters, e.g., neural network parameters of a second reconstruction layer, from the data stream for the second reconstruction layer to distinguish them from, e.g., final neural network parameters, and to provide, for each neural network parameter, a second reconstruction layer neural network parameter value. The second neural network parameters may not have a self-contained meaning in terms of the neural network representation, but may simply result in the neural network representation, i.e., for example, a final neural network parameter, when combined with parameters of the first representation layer. The apparatus is further configured to reconstruct the neural network parameter by combining, for each neural network parameter, the neural network parameter value of the first reconstruction layer with the neural network parameter value of the second reconstruction layer.

[0015] A further embodiment according to the second aspect of the present invention comprises an apparatus for encoding neural network parameters defining a neural network using first neural network parameters for a first reconstruction layer including, for each neural network parameter, neural network parameter values ​​of the first reconstruction layer, and further configured to encode second neural network parameters for a second reconstruction layer into a data stream, the second reconstruction layer including, for each neural network parameter, neural network parameter values ​​of the second reconstruction layer, the neural network parameters being reconfigurable by combining, for each neural network parameter, the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer.

[0016] A further embodiment according to the second aspect of the present invention comprises a method for reconstructing neural network parameters defining a neural network, the method comprising deriving, for a first reconstruction layer, first neural network parameters that may have been previously transmitted, e.g., during an associative learning process, and that may be referred to as neural network parameters of the first reconstruction layer, to yield, for each neural network parameter, neural network parameter values ​​of the first reconstruction layer.

[0017] The method further includes decoding second neural network parameters, which may be referred to as neural network parameters of a second reconstruction layer to distinguish them from final, e.g., reconstructed, neural network parameters, from the data stream for a second reconstruction layer to generate, for each neural network parameter, a neural network parameter value of the second reconstruction layer. The method then includes, for each neural network parameter, reconstructing the neural network parameter by combining the neural network parameter value of the first reconstruction layer with the neural network parameter value of the second reconstruction layer. The second neural network parameters may not have a self-contained meaning with respect to the neural representation, but may simply lead to the neural representation, i.e., final neural network parameters, when combined with the parameters of the first representation layer.

[0018] A further embodiment according to the second aspect of the present invention includes a method for encoding neural network parameters defining a neural network using first neural network parameters for a first reconstruction layer including, for each neural network parameter, neural network parameter values ​​of the first reconstruction layer, the method including encoding second neural network parameters for a second reconstruction layer into a data stream, the second reconstruction layer including, for each neural network parameter, second reconstruction layer neural network parameter values, the neural network parameters being reconfigurable by combining, for each neural network parameter, the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer.

[0019] An embodiment according to a second aspect of the present invention is based on the idea that a neural network, e.g., defined by neural network parameters, can be efficiently compressed and / or transmitted, e.g., with a small amount of data in a bitstream, using sub-layers, such as reconstruction layers, e.g., a base layer and an enhancement layer. The reconstruction layers can be defined such that the neural network parameters are reconfigurable by combining, for each neural network parameter, the neural network parameter values ​​of a first reconstruction layer with the neural network parameter values ​​of a second reconstruction layer. This distribution enables efficient coding, e.g., encoding and / or decoding, and / or transmission, of the neural network parameters. Thus, the second neural network parameters for the second reconstruction layer can be coded and / or transmitted separately in a data stream. [Brief explanation of the drawings]

[0020] The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the present invention. In the following description, various embodiments of the present invention are described with reference to the following drawings, in which embodiments of the present invention are illustrated: [Figure 1] FIG. 1 is an exemplary schematic diagram of a two-layer feedforward neural network that may be used with embodiments of the present invention. [Figure 2] FIG. 2 is a schematic diagram illustrating a concept for inverse quantization performed in an apparatus for decoding neural network parameters defining a neural network from a data stream, according to an embodiment. [Figure 3] FIG. 3 is a schematic diagram illustrating a concept for quantization performed in an apparatus for encoding neural network parameters into a data stream, according to an embodiment. [Figure 4]FIG. 4 is a schematic diagram illustrating a concept for decoding performed in an apparatus for reconstructing neural network parameters defining a neural network, according to an embodiment. [Figure 5] FIG. 5 is a schematic diagram illustrating a concept for encoding performed in an apparatus for reconstructing neural network parameters defining a neural network, according to an embodiment. [Figure 6] FIG. 6 is a schematic diagram of the concept of using a reconstruction layer for neural network parameters for use in an embodiment according to the present invention. [Figure 7] FIG. 7 is a schematic diagram showing an illustration of a uniform reconstruction quantizer according to an embodiment of the present invention. [Figure 8] FIG. 8 is a diagram illustrating examples of allowable reconstruction vector positions for the simple case of two weight parameters according to an embodiment of the present invention. [Figure 9a] FIG. 9a is a diagram illustrating an example for dependent quantization with two sets of reconstruction levels determined entirely by a single quantization step size Δ according to an embodiment of the present invention. [Figure 9b] FIG. 9b is a diagram illustrating an example for dependent quantization with two sets of reconstruction levels determined entirely by a single quantization step size Δ according to an embodiment of the present invention. [Figure 9c] FIG. 9c is a diagram illustrating an example for dependent quantization with two sets of reconstruction levels determined entirely by a single quantization step size Δ according to an embodiment of the present invention. [Figure 10] FIG. 10 is a diagram illustrating an example of pseudocode illustrating a preferred example of a neural network parameter reconfiguration process according to an embodiment of the present invention. [Figure 11] FIG. 11 is a diagram illustrating an example for dividing a reconstructed level set into two subsets according to an embodiment of the present invention. [Figure 12]FIG. 12 illustrates an example of pseudocode illustrating a preferred example of a neural network parameter reconfiguration process for a layer according to an embodiment. [Figure 13] FIG. 13 shows a preferred example of a state transition table sttab and a table setId according to an embodiment of the present invention, which specifies the quantization set associated with a state. [Figure 14] FIG. 14 shows a preferred example of a state transition table sttab and a table setId according to an embodiment of the present invention, which specifies the quantization sets associated with the states. [Figure 15] FIG. 15 illustrates pseudocode illustrating an alternative neural network parameter level reconstruction process in which quantization indices equal to 0 are excluded from state transitions and dependent scalar quantization, according to an embodiment of the present invention. [Figure 16] FIG. 16 is a diagram illustrating an example of state transitions in dependent scalar quantization as a trellis structure according to an embodiment of the present invention. [Figure 17] FIG. 17 is a diagram showing an example of a basic trellis cell according to an embodiment of the present invention. [Figure 18] FIG. 18 illustrates an example trellis for dependent scalar quantization of eight neural network parameters according to an embodiment of the present invention. [Figure 19] FIG. 19 illustrates an example trellis structure that can be utilized to determine a sequence (or block) of quantization indices that minimizes a cost measure (e.g., a Lagrangian cost measure D+λ·R) in accordance with an embodiment of the present invention. [Figure 20] FIG. 20 is a block diagram of a method for decoding neural network parameters defining a neural network from a data stream according to an embodiment of the present invention. [Figure 21] FIG. 21 is a block diagram of a method for encoding neural network parameters defining a neural network into a data stream according to an embodiment of the present invention. [Figure 22] FIG. 22 is a block diagram of a method for reconfiguring neural network parameters that define a neural network, according to an embodiment of the present invention. [Figure 23] FIG. 23 is a block diagram of a method for encoding neural network parameters that define a neural network, according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0021] Elements or components having equal or equivalent functions, even if they are depicted in different figures, are designated in the following description by equal or equivalent reference numerals.

[0022] In the following description, numerous details are set forth to provide a more thorough explanation of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the embodiments of the present invention. In addition, features of different embodiments described below in this specification may be combined with each other unless otherwise specified.

[0023] This description begins with a presentation of several embodiments of the present application. This description, while fairly general, provides the reader with an overview of the functionality on which the embodiments of the present application are based. A more detailed description of these functionalities is then presented, along with the motivation for the embodiments and how they achieve the aforementioned efficiency improvements. The details can be combined individually and in combination with the currently described embodiments.

[0024] FIG. 2 shows an embodiment of the present invention. It takes The neural network parameters that define the neural network are From the data stream Executed in the device for decoding 、 FIG. 1 is a schematic diagram illustrating the concept of inverse quantization.

[0025] The device is configured to sequentially decode the neural network parameters 13. of During the process, the quantizer (reconstruction level set) fluctuates .this Due to fluctuations ,twist Level Less (or better) As an expression, Less dense )amount Use a child generator but Possible Next door , Ultimately , Encoded Quantization index Smaller To do but Possible Next door , the quality of the neural network representation obtained from this quantization is significantly higher than that obtained using a constant quantizer compared to the required coding bit rate. too improvement vinegar This will be described in detail later. In particular, the device calculates the current neural network parameters 13' by , multiple of 50 Of the 52 reconstruction level sets (set 0, set 1) from Reconstruction level set 48 (selected set) Depending on the quantization index 58 decoded from the data stream 14 for the previous neural network parameters, By selecting 54 (reconstruction level selection), the neural network parameters 13 are sequentially decoded. transformation do.

[0026] Furthermore, the device , current Current neural network parameters 13' For Quantization index 56 And , quantization index 56 teeth Current neural network parameters Ta's Indicates one reconstruction level from a selected set of reconstruction levels 48. quantization indexes 56 are decoded from data stream 14. , and , Of the selected set of 48 reconstruction levels, Current neural network parameters For Quantization index 56 but Show vinegar One reconstruction level The current neural network parameters 13' are By inverse quantization 62, the neural network Parameters 13 sequentially To decrypt to It is composed.

[0027] The decoded neural network parameters 13 are represented, for example, by a matrix 15a, which may include deserialized neural network parameters 13 20b, which may relate to the weights of the neuron interconnections of the neural network.

[0028] Optionally, the number of reconstructed level sets 52, also sometimes referred to herein as quantizers, of the plurality (50) of reconstructed level sets 52 may be two, for example, set 0 and set 1, as shown in FIG. 2 .

[0029] Furthermore, the device may be configured to parameterize 60 the plurality (50) of reconstruction level sets 52 (e.g., set 0, set 1) by a predetermined quantization step size (QP), e.g., denoted by △ or △k, and derive the predetermined quantization step size information from the data stream 14. Thus, a decoder according to an embodiment may be adaptive to a variable step size (QP).

[0030] Further, according to an embodiment, the neural network may include one or more neural network layers, and the apparatus may be configured to derive, for each neural network layer, information of a predetermined quantization step size (QP) for the respective neural network layer from the data stream 14, and parameterize, for each neural network layer, a plurality 50 of reconstruction level sets 52 using the derived predetermined quantization step size for the respective neural network layer, to be used to dequantize neural network parameters belonging to the respective neural network layer. Adapting the step size for the neural network layer, and therefore adapting the reconstruction level sets 52, may improve coding efficiency.

[0031] According to a further embodiment, the apparatus may be configured to select 54, for a current neural network parameter 13′, a reconstructed level set 48 from the plurality of 50 reconstructed level sets 52 in response to an LSB (e.g., least significant bit) portion of a binarized quantization index 58 decoded from the data stream 14 for the previously decoded neural network parameter or a previously decoded bin (e.g., a binary decision). The LSB comparison can be performed with low computational cost. In particular, a state transition can be used. The selection 54 can be performed by a state transition process by determining, for the current neural network parameter 13′, a reconstructed level set 48 from the plurality of 50 reconstructed level sets 52 in response to a state associated with the current neural network parameter 13′, and by updating the state for the subsequent neural network parameter in response to a quantization index 58 decoded from the data stream for the immediately preceding neural network parameter. Also, alternative approaches other than state transition, for example by using a transition table, can be used as well, as defined below.

[0032] Additionally or alternatively, the device may be configured to select 54, for example, for the current neural network parameters 13′, a reconstruction level set 48 from among the plurality of reconstruction level sets 52 50 depending on the result of a binary function of quantization indices 58 decoded from the data stream 14 for the previously decoded neural network parameters. The binary function may be, for example, a parity check using a bitwise “and” operation to signal whether the quantization index 58 represents an even or odd number. This may provide information about the reconstruction level set 48 used to encode the quantization index 58, and thus, for example, for a given order of reconstruction level sets used in the corresponding encoder, for the set of reconstruction levels used to encode the current neural network parameters 13′. The parity may be used for the state transitions described above.

[0033] Further, according to an embodiment, the apparatus may be configured to, for example, select 54, for current neural network parameters 13′, a set 48 of reconstruction levels 52 from the plurality 50 of reconstruction level sets 52 depending on the parity of quantization indices 58 decoded from data stream 14 for previously decoded neural network parameters. The parity check may be performed at low computational cost, for example, using a bitwise “and” operation.

[0034] Optionally, the device can be configured to decode the quantization indices 56 for the neural network parameters 13 and perform inverse quantization of the neural network parameters 13 along a common sequential order 14′ among the neural network parameters 13. In other words, the same order can be used for both tasks.

[0035] FIG. 3 is a diagram illustrating a quantization concept implemented in an apparatus for encoding neural network parameters into a data stream according to an embodiment. FIG. 3 illustrates a neural network (NN) 10 including neural network layers 10a and 10b, each including a neuron 10c, with the neurons of the interconnected layers interconnected via neuron interconnections 11. As an example, NN layer (p-1) 10a and NN layer (p) 10b are shown, where p is the index of the NN layer and 1≦p≦NN, the number of layers. The neural network is defined or parameterized by neural network parameters 13, which may optionally be related to the weights of the neuron interconnections 11 of the neural network 10. Neurons 10c in the hidden layer of FIG. 1 may represent neurons in layer p (A, B, C, ...) of FIG. 3, and neurons in the input layer of FIG. 1 can represent neurons in layer p-1 (a, b, c, ...) shown in FIG. 3. The neural network parameters 13 may be related to the weights of the neuron interconnections 11 of FIG. 1.

[0036] The relationships between neurons 10c in different layers are represented in Fig. 1 by a matrix 15a of neural network parameters 13. For example, if the network parameters 13 relate to the weights of neuron interconnections 11, then the matrix 15a can be configured such that the matrix elements represent the weights between neurons 10c in different layers (e.g., a, b,... for layer p-1, A, B,... for layer p).

[0037] The apparatus is configured to sequentially encode the neural network parameters 13, e.g., serialize 20a. During this sequential process, the quantizer (reconstructed level set) is varied. This variation allows the use of a quantizer with fewer (or better, less dense) levels, thus allowing smaller quantization indices to be coded, improving the quality of the resulting neural network representation compared to the required coding bit rate when using a constant quantizer. This will be described in more detail below. In particular, the apparatus sequentially encodes the neural network parameters 13' by selecting 54 a reconstructed level set 48 from a plurality of reconstructed level sets 52 50 according to the quantization indices 58 coded in the data stream 14 for the previously coded neural network parameters.

[0038] Furthermore, the apparatus quantizes 64 (Q) the current neural network parameters 13' onto one reconstruction level of the selected reconstruction level set 48, and It is configured to sequentially encode the neural network parameters 13 by encoding a quantization index 56 for the current neural network parameters 13′, which indicates one reconstruction level to which the quantization index 56 for the current neural network parameters will be quantized into the data stream 14. Optionally, the number of reconstruction level sets 52, also sometimes referred to herein as quantizers, among the plurality 50 of reconstruction level sets 52 can be two, as shown, for example, with set 0 and set 1.

[0039] According to an embodiment, as shown in Figure 3, the apparatus may be configured to parameterize 60 the plurality 50 of reconstruction level sets 52 by a predetermined quantization step size (QP) and insert information about the predetermined quantization step size into the data stream 14. This may enable, for example, adaptive quantization to improve quantization efficiency, and changes in the encoding method of the neural network parameters 13 may be communicated to the decoder along with information about the predetermined quantization step size. Using a predetermined quantization step size (QP) may reduce the amount of data required for transmission of information.

[0040] Furthermore, according to an embodiment, the neural network 10 may include one or more neural network layers 10 a, 10 b. The apparatus may be configured to, for each neural network layer (p; p−1), insert information of a predetermined quantization step size (QP) for the respective neural network layer into the data stream 14 and parameterize a plurality of reconstruction level sets 52 50 using the predetermined quantization step size derived for the respective neural network layer for use in quantizing neural network parameters belonging to the respective neural network layer. As explained above, adapting the quantization depending on, for example, the neural network layer or the characteristics of the neural network layer can improve quantization efficiency.

[0041] Optionally, the apparatus may be configured to select 54, for the current neural network parameters 13′, a set 48 of reconstruction levels 52 from the plurality 50 of reconstruction level sets 52 in response to a previously coded bin or an LSB portion of a binarized quantization index 58 coded in the data stream 14 for the previously coded neural network parameters. The LSB comparison may be performed at low computational cost.

[0042] State transitions can be used similarly to the apparatus for decoding described in Fig. 2. The state transition process can perform a selection 54 for the current neural network parameters 13' among a quantization level set 48 among a plurality of 50 reconstructed level sets 52 by determining for the current neural network parameters 13' a set of reconstruction levels 48 among a plurality of 50 reconstructed level sets 52 depending on the state associated with the current neural network parameters 13', and by updating the state for the subsequent neural network parameters depending on the quantization index 58 encoded in the data stream for the previous neural network parameters. Approaches other than state transitions, for example by using a transition table, can be used as well and are defined below.

[0043] Additionally or alternatively, the device may be configured to select 54 a reconstruction level set 48 from among a plurality of 50 reconstruction level sets 52 for the current neural network parameters 13′ depending on the result of a binary function of the quantization indexes 58 encoded in the data stream 14 for the previously encoded neural network parameters. The binary function may be, for example, a parity check using a bitwise “and” operation to signal whether the quantization index 58 represents an even or odd number. This may provide information about the reconstruction level set 48 used to encode the quantization index 58, and thus, for example, for a given order of the reconstruction levels, a set of reconstruction levels 48 for the current neural network parameters 13′ can be determined such that, for the given order, a corresponding decoder can select the corresponding reconstruction level set 48. The parity may be used for the state transitions described above.

[0044] Furthermore, according to an embodiment, the apparatus may be configured to select 54, for example, for the current neural network parameters 13′, a quantization level set 48 from among the plurality 50 of reconstruction level sets 52 depending on the parity of the quantization indices 56 coded in the data stream 14 for the previously coded neural network parameters. The parity check may be performed at low computational cost, for example, using a bitwise “and” operation.

[0045] Optionally, the device may be configured to encode quantization indices (56) for the neural network parameters (13) and to perform the quantization of the neural network parameters (13) along a common sequential order (14') among the neural network parameters (13), i.e., the same order may be used for both tasks.

[0046] FIG. 4 is a schematic diagram of a concept for arithmetically decoding quantized neural network parameters according to an embodiment. This can be used within the device of FIG. 2. Therefore, FIG. 4 can be seen as a possible extension of FIG. 2. It shows a data stream 14 decoded by the device of FIG. 4, as shown by way of example, in which quantization indices 56 for the current neural network parameters 13′ are decoded by arithmetic coding, e.g., binary arithmetic coding. For example, a probabilistic model defined by a certain context is used, which depends on the reconstruction level set 48 selected for the current neural network parameters 13′, as indicated by arrow 123. Details are provided herein.

[0047] Regarding Figure 2 Ren As explained in , current Current neural network parameters 13' 、 Multiple 50 reconstruction level sets 52 out of 52 from Reconstruction level set 48 Dependent on the state associated with the current neural network parameters 13' To decide and, and ,rearThe following neural network parameters Ta's Status Depending on the quantization index 58 decoded from the data stream for the previous neural network parameters, By updating , complex Of the 50 reconstruction level sets, 52 from Set quantization level 48 By the state transition process Selection 54 is performed for the current neural network parameters 13' And Therefore, the state is used to encode / decode the current neural network parameters 13'. Ru Pointer to configuration level set 48 Like Although, however , the state is updated with finer precision by merely distinguishing a number of states corresponding to the number of reconstruction sets, so that the state essentially functions as a memory of past neural network parameters or past quantization indices. Thus, the state defines the order of the set of reconstruction levels used to encode / decode the neural network parameters 13. According to FIG. 4, for example, Ba present Current neural network parameters (13') For The quantization index (56) of the current neural network parameters (13' )of To the state dependence (122) using a probability model R Using arithmetic coding 、 The state-dependent adaptation may allow for computationally efficient adaptation with little additional data transmission.

[0048] According to a further embodiment, the device may be configured to decode the quantization indexes 56 for the current neural network parameters 13′ from the data stream 14 using binary arithmetic coding, for example using a probability model in which the binarization 82 of the quantization indexes 56 depends 122 on the state of the current neural network parameters 13′ for at least one bin 84.

[0049] Additionally or alternatively, the device may be configured such that the dependency of the probability model includes the selection 103 (derivation) of a context 87 from a set of contexts for the neural network parameters using the dependency, each context being configured to have a predetermined probability model associated with it. The better the probability estimates used, the more efficient the compression. The probability model can be updated, for example, using context-adaptive (binary) arithmetic coding.

[0050] Optionally, the device may be configured to update a predetermined probability model associated with each context based on the quantization indexes arithmetically coded with the respective context, in this way the probability model of the context is adapted to actual statistics.

[0051] Further, the apparatus may be configured to decode the quantization index 56 for the current neural network parameters 13′ from the data stream 14 using binary arithmetic coding, for example, using a probability model corresponding to the reconstruction level set 48 selected for the current neural network parameters 13′ for at least one bin of the binarized version of the quantization index.

[0052] Optionally, at least one bin may include a significance bin indicating whether the current neural network parameter quantization index 56 is equal to zero. Additionally or alternatively, at least one bin may include a sign bin indicating whether the current neural network parameter quantization index 56 is greater than or less than zero. Furthermore, at least one bin may include a greater-than-X bin indicating whether the absolute value of the current neural network parameter quantization index 56 is greater than X, where X is an integer greater than zero.

[0053] Below, in Figure 5, the opposite of the concept for decoding described with reference to Figure 4 can be explained, so all explanations and advantages can be applied to the following aspects of the concept for encoding as appropriate.

[0054] FIG. 5 is a schematic diagram illustrating a concept for arithmetically coding neural network parameters according to an embodiment. This can be used within the apparatus of FIG. 3. Therefore, FIG. 5 can be seen as a possible extension of FIG. 3. It shows a data stream 14 coded by the apparatus of FIG. 3, in which quantization indices 56 for current neural network parameters 13′ are coded by arithmetic coding, e.g., binary arithmetic coding as an example. For example, a probabilistic model defined by a certain context is used, which depends on the reconstruction level set 48 selected for the current neural network parameters 13′, as indicated by arrow 123. Details are provided herein.

[0055] As described with respect to Figure 3, selection 54 is performed for the current neural network parameters 13'. Selection 54 selects a quantization level set 48 from the plurality of 50 reconstructed level sets 52 for the current neural network parameters 13' by a state transition process, by determining a quantization level set 48 from the plurality of 50 reconstructed level sets 52 for the current neural network parameters 13' depending on the state associated with the current neural network parameters 13', and by updating the state for the subsequent neural network parameters depending on the quantization index 58 encoded in the data stream for the immediately preceding neural network parameters.

[0056] Thus, the states are quasi-pointers to the reconstruction level sets 48 to be used to encode / decode the current neural network parameters 13', but are updated with finer precision by only distinguishing a number of states corresponding to the number of reconstruction sets, so that the states essentially function as a memory of past neural network parameters or past quantization indices. Thus, the states define the order of the reconstruction level sets used to encode / decode the neural network parameters 13.

[0057] Additionally, the quantization indices 56 for the current neural network parameters 13' may be coded into the data stream 14 using arithmetic coding that uses a probability model corresponding to the states 122 for the current neural network parameters 13'.

[0058] 3, for example, quantization indexes 56 are encoded into data stream 14 using binary arithmetic coding for current neural network parameters 13′ using a probability model corresponding to state 122 for current neural network parameters 13′ for at least one bin 84 of binarized 82 of quantization indexes 56. Adapting the probability model depending on the state may improve coding efficiency, since the probability model may be suitable for estimating the probability model. Furthermore, state-dependent adaptation may allow for computationally efficient adaptation with a small amount of additional data transmitted.

[0059] Additionally or alternatively, the apparatus may be configured such that the probabilistic model dependency includes selecting 103 (deriving) a context 87 from a set of contexts for the neural network parameters using the dependency, each context having a predetermined probabilistic model associated therewith.

[0060] Optionally, the apparatus may be configured to update a predetermined probability model associated with each context based on quantization indexes arithmetically coded using the respective context.

[0061] Furthermore, the apparatus may be configured to encode the quantization indexes 56 of the current neural network parameters 13′ into the data stream 14 using binary arithmetic coding, for example, by using a probability model corresponding to the reconstruction level set 48 selected for the current neural network parameters 13′ for at least one bin of the binarized quantization indexes. To use binary arithmetic coding, the quantization indexes 56 may be binarized.

[0062] Optionally, at least one bin may include a significance bin indicating whether the current neural network parameter quantization index 56 is equal to zero. Additionally or alternatively, at least one bin may include a sign bin indicating whether the current neural network parameter quantization index 56 is greater than or less than zero. Furthermore, at least one bin may include a greater-than-X bin indicating whether the absolute value of the current neural network parameter quantization index 56 is greater than X, where X is an integer greater than zero.

[0063] The next described embodiment focuses on another aspect of the present application, where the parameterization of the neural network is coded in stages or reconstruction layers, and for each NN parameter, one value from each stage needs to be combined to obtain an improved / enhanced representation of the neural network, at least one of which may itself represent a reasonable representation of the neural network but is enhanced by one of the contributing stages of lower quality, although the latter possibility is not essential to this aspect.

[0064] 6 is a schematic diagram of the concept of using reconstruction layers for neural network parameters for use in an embodiment according to the present invention. Figure 6 shows that reconstruction layer i, e.g., the second reconstruction layer, reconstruction layer i-1, e.g., the first reconstruction layer, and neural network (NN) layer p, e.g., layer 10b from Figure 3, are layers represented in the form of an array or matrix, e.g., matrix 15a from Figure 3.

[0065] FIG. 6 conceptually illustrates an apparatus 310 for reconstructing neural network parameters 13 defining a neural network. Thus, the apparatus is configured to derive first neural network parameters 13a, which may have been previously transmitted, e.g., during an associative learning process, for a first reconstruction layer, e.g., reconstruction layer i-1, and to provide neural network parameter values ​​for the first reconstruction layer for each neural network parameter, e.g., weight or inter-neuron connection. This derivation may otherwise include decoding or receiving the first neural network parameters 13a. Furthermore, the apparatus is configured to decode 312 second neural network parameters 13b, which may be referred to as neural network parameters for the second reconstruction layer to distinguish them from, e.g., final neural network parameters, e.g., parameters 13, for the second reconstruction layer, from the data stream 14 to generate neural network parameter values ​​for the second reconstruction layer for each neural network parameter 13. Thus, two contribution values ​​of the first and second reconstruction layers may be obtained for each NN parameter, and the encoding / decoding of the first and / or second NN parameter values ​​may use dependent quantization according to Figures 2 and 3 and / or arithmetic encoding / decoding of quantization indexes as described in Figures 4 and 5. The second neural network parameters 13b may not have a self-contained meaning in terms of neural representation, but may simply lead to the neural network representation, i.e. the final neural network parameters, when combined with the parameters of the first representation layer.

[0066] Further, the apparatus is configured to reconstruct 314 the neural network parameters 13 by combining (CB) for each neural network parameter the neural network parameter values ​​of the first reconstruction layer and the second reconstruction layer neural network parameter values, for example using element-wise addition and / or multiplication.

[0067] 6 further illustrates a conceptual diagram of an apparatus 320 for encoding neural network parameters 13 defining a neural network using first neural network parameters 13a for a first reconstruction layer, e.g., reconstruction layer i-1. The first reconstruction layer includes, for each neural network parameter 13, a neural network parameter value of the first reconstruction layer. Accordingly, the apparatus is configured to encode 322 second neural network parameters 13b for a second reconstruction layer, e.g., reconstruction layer i, into a data stream. The second reconstruction layer includes, for each neural network parameter 13, a neural network parameter value of the second reconstruction layer. The neural network parameters 13 are reconfigurable by combining (CB) the neural network parameter values ​​of the first reconstruction layer with the neural network parameter values ​​of the second reconstruction layer, e.g., using element-wise addition and / or multiplication for each neural network parameter.

[0068] Optionally, the apparatus 310 may be configured to decode 316 first neural network parameters for the first reconstruction layer from the data stream 14 or from a separate data stream.

[0069] Simply put, decomposition of the neural network parameters 13 may allow for more efficient encoding and / or decoding and transmission of the parameters.

[0070] Further embodiments are disclosed below, particularly those involving neural network coding concepts. The following description provides further details that can be used individually and in combination with the above-described embodiments.

[0071] First, a method for entropy coding of parameters of a neural network with dependent scalar quantization according to an embodiment of the present invention is presented.

[0072] A parameter coding method for a set of neural network parameters 13 (also referred to as weights, weight parameters, or parameters) using dependent scalar quantization is described. The parameter coding presented herein consists of dependent scalar quantization of the parameters 13 (e.g., as described in the context of FIG. 3) and entropy coding of the resulting quantization indexes 56 (e.g., as described in the context of FIG. 5). At the decoder side, a set of reconstructed neural network parameters 13 can be obtained by entropy decoding of the quantization indexes 56 (e.g., as described in the context of FIG. 4) and dependent reconstruction of the neural network parameters 13 (e.g., as described in the context of FIG. 2). In contrast to parameter coding with independent scalar quantization and entropy coding, the set of allowable reconstruction levels for the neural network parameters 13 depends on the transmitted quantization index 56 that precedes the current neural network parameter 13′ in the reconstruction order. The following presentation additionally describes a method for entropy coding of the quantization indexes that specify the reconstruction levels used in dependent scalar quantization.

[0073] This description is primarily directed to lossy coding of neural network parameter layers in neural network compression, but is also applicable to lossy coding in other fields.

[0074] The methodology of the device is divided into different main parts:

[0075] 1. Quantization 2. Lossless coding 3. Lossless Decoding

[0076] To understand the main advantages of the embodiments described below, we first provide a brief introduction to the topic of neural networks and related methods for parameter encoding. Nevertheless, all disclosed aspects, features, and concepts can be used separately or in combination with the embodiments described herein.

[0077] 2. Related Methods for Quantization and Entropy Coding Working Draft 2 of the MPEG-7 Part 17 standard for the compression of neural networks for multimedia content description and analysis [2] applies independent scalar quantization and entropy coding to neural network parameter coding.

[0078] 2.1 Scalar Quantizer The neural network parameters are quantized using a scalar quantizer. As a result of the quantization, the set of allowable values ​​of the parameters 13 is reduced. In other words, the neural network parameters are mapped to a countable (actually finite) set of so-called reconstruction levels. The reconstruction level set represents a proper subset of the set of possible neural network parameter values. To simplify the entropy coding below, the allowable reconstruction levels are represented by quantization indices 56, which are transmitted as part of the bitstream 14. At the decoder side, the quantization indices 56 are mapped to the reconstructed neural network parameters 13. The possible values ​​of the reconstructed neural network parameters 13 correspond to the reconstruction level set 52. At the encoder side, the result of the scalar quantization is a set of (integer) quantization indices 56.

[0079] TIFF0007783376000002.tif55153

[0080] In this context, the term "independent scalar quantization" refers to the property that given a quantization index q56 for any weight parameter 13, the associated reconstructed weight parameter t'13' can be determined independently from all quantization indices for other weight parameters.

[0081] 2.1.1 Encoder Operation: Quantization Standards for neural network compression only specify the bitstream syntax and the reconstruction process. Given a set of original neural network parameters 13 and a given quantization step size (QP), the encoder has many degrees of freedom. The quantization index q of layers 10a and 10b k Given 56, entropy coding must follow a uniquely defined algorithm for writing the data into the bitstream 14 (i.e., constructing arithmetic codewords). However, given the original set of weight parameters (e.g., layers), the quantization index q k The encoding algorithm to obtain 56 is outside the scope of any standard for neural network compression. In the following description, we assume that the quantization step size (QP) of each neural network parameter 13 is known. Nevertheless, the encoder must calculate the quantization step size (QP) of each neural network (weight) parameter t k Quantizer index q for 13 k 56. The choice of quantization index determines both the distortion (or reconstruction / approximation quality) and the bit rate, so the quantization algorithm used has a substantial impact on the rate-distortion performance of the resulting bitstream 14.

[0082] TIFF0007783376000003.tif123152

[0083] The quantization process achieves good rate-distortion performance by minimizing the Lagrangian function D+λ·R, where D is the distortion of the neural network parameter set (e.g., MSE distortion or MAE distortion), R is the number of bits required to transmit the quantization index 56, and λ is the Lagrangian multiplier.

[0084] TIFF0007783376000004.tif75150

[0085] TIFF0007783376000005.tif64151

[0086] In fact, nearest neighbor quantization is the trivial case where λ=0 and is applied in Working Draft 2 of the MPEG-7 part 17 standard on Compression of Neural Networks for Multimedia Content Description and Analysis.

[0087] 2.2 Entropy coding As a result of the uniform quantization applied in the previous step, the weight parameters are mapped to a finite set of so-called reconstruction levels. These can be represented by (integer) quantizer indices 56 (also called parameter levels or weight levels) and quantization step sizes (QP), which may, for example, be fixed for all layers. To recover all quantized weight parameters of a layer, the step sizes (QP) and dimensions of the layer may be known by the decoder. These may, for example, be transmitted separately.

[0088] 2.2.1 Quantization index coding using context-adaptive binary arithmetic coding (CABAC) The quantization indexes 56 (represented as integers) are then transmitted using entropy coding techniques. Thus, the weight layer is mapped to a sequence of quantized weight levels using a scan. For example, a row-first scan order can be used, starting with the top row of the matrix and encoding the contained values ​​from left to right. In this way, all rows are encoded from top to bottom. The scan may be performed, for example, along a common sequential order 14′, as shown in FIG. 3, for a matrix 15a containing neural network parameters 13 that may be related to the weights of the neuronal interconnects 11. The matrix may represent a weight layer, for example, the weights between layer p-1 10a and layer p 10b of the neuronal interconnects 11, or the weights between the hidden layer and the input layer, as shown in FIGS. 3 and 1, respectively. Note that any other scan may be applied. For example, a matrix (e.g., matrix 15a in Figure 2 or Figure 3) can be transposed, or flipped horizontally and / or vertically, and / or rotated by 90 / 180 / 270 degrees left or right before applying row-first scanning.

[0089] 3 and 5, an apparatus according to an embodiment may be configured to encode the quantization index 56 for the current neural network parameters 13′ into the data stream 14 using binary arithmetic coding, using a probability model corresponding to the state 122 for the current neural network parameters 13′ for at least one bin 84 of the binarized quantization index 56. The binary arithmetic coding using the probability model may be context-adaptive binary arithmetic coding (CABAC).

[0090] In other words, according to the embodiment, CABAC is used for level encoding. For details, refer to [3]. Thus, the quantized weight level q56 may be decomposed into a series of binary symbols or syntax elements, such as bins (binary decisions), and then passed to a binary arithmetic coder (CABAC). In the first step, a binary syntax element sig_flag is derived for the quantized weight level, which specifies whether the corresponding level is equal to zero. In other words, at least one bin of the binarized 82 quantization index 56 shown in FIG. 4 may include a significance bin indicating whether the quantization index 56 of the current neural network parameters is equal to zero.

[0091] If sig_flag is equal to 1, a further binary syntax element sign_flag is derived. This bin indicates whether the current weight level is positive (e.g., bin = 0) or negative (e.g., bin = 1). In other words, at least one bin of the binarized 82 quantization index 56 shown in FIG. 4 may include a sign bin 86 indicating whether the quantization index 56 of the current neural network parameters is greater than or less than zero.

[0092] Next, the unary sequence of bins is encoded, followed by encoding of the fixed-length sequence as follows.

[0093] The variable k is initialized with a non-negative integer, and X is initialized with 1 << k.

[0094] One or more syntax elements, called abs_level_greater_X, indicating that the absolute value of the quantized weight level is greater than X, are encoded. When abs_level_greater_X is equal to 1, the variable k is updated (e.g., incremented by 1), then 1<<k is added to X, and then abs_level_greater_X is encoded again. This procedure continues until abs_level_greater_X is equal to 0. Thereafter, the quantization index can be encoded with a fixed-length code of length k. For example, the variable rem = X - |q| can be encoded using k bits. Alternatively, the variable rem' can be defined as rem'=(1<<k)-rem-1, which is encoded using k bits. Other mappings of the variable rem to a fixed-length code of k bits can also be used.

[0095] In other words, at least one bin of the binarized 82 quantization index 56 shown in FIG. 4 can include a greater-than-X bin indicating whether the absolute value of the quantization index 56 of the current neural network parameters is greater than X, where X is an integer greater than zero.

[0096] When k is incremented by 1 after each abs_level_greater_X, this approach is identical to applying exponential Golomb coding (when sign_flag is not considered).

[0097] Also, when the maximum absolute value abs_max is known on both the encoder side and the decoder side, for the next abs_Level_greater_X to be transmitted, when X>=abs_max holds, the encoding of the abs_Level_greater_X syntax element can be terminated.

[0098] <00e0475>TIFF0007783376000006.tif91152

[0099] In other words, as described with respect to Figures 2 and 4, an apparatus according to an embodiment can be configured to decode the quantization index 56 for the current neural network parameter 13' from the data stream 14 using binary arithmetic coding by using a probability model corresponding to the state 122 for the current neural network parameter 13' for at least one bin 84 of the binarized quantization index 56.

[0100] At least one bin of the binarized quantization index 56 82 shown in Figure 5 may include a significance bin indicating whether the current neural network parameter quantization index 56 is equal to zero. Additionally or alternatively, at least one bin may include a sign bin 86 indicating whether the current neural network parameter quantization index 56 is greater than or less than zero. Furthermore, at least one bin may include a greater-than-X bin indicating whether the absolute value of the current neural network parameter quantization index 56 is greater than X, where X is an integer greater than zero.

[0101] In a preferred embodiment, k is initialized to 0 and updated as follows: After each abs_level_greater_X equals 1, any necessary updates to k are made according to the following rule: if X > X', then k is incremented by 1, where X' is a constant corresponding to the application. For example, X' is a number (e.g., between 0 and 100) derived by the encoder and communicated to the decoder.

[0102] 2.2.3 Context Modeling In CABAC entropy coding, most syntax elements with quantized weight levels 56 are coded using binary probability modeling. Each binary decision (bin) is associated with a context. The context represents a probability model for the class of coded bins. The probability of one of two possible bin values ​​is estimated for each context based on the bin values ​​already coded in the corresponding context. Depending on the application, different context modeling approaches can be applied. Typically, for some bins associated with quantized weight coding, the context used for coding is selected based on the syntax elements already transmitted. Depending on the actual application, different probability estimators can be selected, such as those for SBMP 0, HEVC 0, or VTM-4.0 0. This choice affects, for example, compression efficiency and complexity.

[0103] In other words, a probability model such as that described with respect to FIG. 5, eg, context 87, additionally relies on the quantization indices of previously coded neural network parameters.

[0104] Respectively, the probability models as described with respect to FIG. 4, for example context 87, further depend on the quantization indices of previously decoded neural network parameters.

[0105] A context modeling scheme that is compatible with a wide range of neural networks is described as follows: To decode the quantized weight level q56 at a particular position (x,y) of a weight matrix (layer), a local template is applied to the current position. This template contains a number of other (ordered) positions, e.g., (x-1,y), (x,y-1), (x-1,y-1), etc. For each position, a status identifier is derived.

[0106] TIFF0007783376000007.tif34152

[0107] For a particular template, we derive a sequence of status identifiers and map each possible constellation of status identifier values ​​to a context index that identifies the context in which it is used. The template and mapping may be different for different syntax elements. For example, from a template containing (ordinal) positions (x-1,y), (x,y-1), (x-1,y-1), we derive the sequence of status identifiers s x-1,y , s x,y-1 , s x-1,y-1 For example, this sequence is derived for a context index C=s x-1,y +3*s x,y-1 +9*s x-1,y-1 For example, a context index C can be used to identify multiple contexts for a sig_flag.

[0108] In a preferred embodiment (called Approach 1), the quantized weight level q at position (x,y) x,y The local template for a sig_flag or for a sign_flag consists of only one position (x-1,y) (i.e., the left neighbor). The associated status identifier s x-1,y is derived according to the preferred embodiment Si1.

[0109] For sig_flag, x-1,y One of three contexts is selected depending on the value of x-1,y Depending on the value of , one of the other three contexts is selected.

[0110] In another preferred embodiment (called Approach 2), the local template for the sig flag contains three ordered positions (x-1, y), (x-2, y), (x-3, y). x-1,y ,s x-2,y ,s x-3,y The associated sequence of is derived according to the preferred embodiment Si2.

[0111] For sig_flag, the context index C is derived as follows:

[0112] s x-1,y If ≠0, then C=0, otherwise s x-2,y If ≠0 then C=1, otherwise s x-3,y If ≠0, C=2, otherwise, s x-3,y If ≠ 0, then C = 2. Otherwise, C = 3.

[0113] TIFF0007783376000008.tif24132

[0114] Similarly, the number of left neighbors may be increased or decreased so that the context index C is equal to the distance to the next non-zero weight on the left (not exceeding the template size).

[0115] Each abs_level_greater_X flag can apply, for example, a unique set of two contexts, one of which is selected depending on the value of sign_flag.

[0116] In a preferred embodiment, for abs_level_greater_X flags where X is less than a predefined number X', different contexts are distinguished depending on the value of X and / or sign_flag.

[0117] In a preferred embodiment, for abs_level_greater_X flags where X is greater than or equal to a predefined number X', different contexts are distinguished only in correspondence to X.

[0118] In another preferred embodiment, abs_level_greater_X flags with X greater than or equal to a predefined number X' are coded using a fixed code length of 1 (eg, using the bypass mode of the arithmetic coder).

[0119] Furthermore, some or all of the syntax elements may be encoded without using the context. Instead, they are encoded with a fixed length of 1 bit, for example, using the so-called bypass bins of CABAC.

[0120] In another preferred embodiment, the fixed-length remainder rem is encoded using the bypass mode.

[0121] In another preferred embodiment, the encoder determines a predefined number X', distinguishes two contexts according to the sign for each syntax element abs_level_greater_X where X < X', and uses one context for each abs_level_greater_X where X >= X'.

[0122] In other words, a probability model as described with respect to FIG. 5, for example context 87, can be selected 103 for the current neural network parameters from among a subset of the probability models according to the quantization index of the previously encoded neural network parameters related to the part of the neural network adjacent to the part to which the current neural network parameters are related.

[0123] This part can be defined, for example, by a template as described above, a template that includes the (ordered) positions (x - 1, y), (x, y - 1), (x - 1, y - 1).

[0124] Respectively, a probability model as described with respect to FIG. 5 can be selected for the current neural network parameters from among a subset of the probability models according to the quantization index of the previously decoded neural network parameters related to the part of the neural network adjacent to the part to which the current neural network parameters are related.

[0125] 3 Additional Methods Below we describe an additional, and therefore optional, method for compressing / transmitting neural network 10 in which a reconstructed layer, e.g., neural network layer p from FIG. 6, is a composition of different sub-layers, such as reconstructed layer i-1 and reconstructed layer i from FIG. 6, which may be transmitted separately.

[0126] 3.1 Base and Enhancement Layer Concepts This concept introduces two types of sub-layers, called base layer and enhancement layer. Then, the reconstruction process (e.g., adding all sub-layers) defines how the reconstructed layer can be obtained from the sub-layers. The base layer contains base values, which can be chosen, for example, to be efficiently represented or compressed / transmitted in the first step. The enhancement layer contains enhancement information, e.g., difference values ​​that can be added to the (base) layer values ​​to reduce distortion metrics (e.g., with respect to the original layer). In another example, the base layer contains coarse values ​​(from training with a small training set) and the enhancement layer contains refinement values ​​(based on the complete training set or, more generally, on a separate training set). The sub-layers may be stored / transmitted separately.

[0127] In a preferred embodiment, the layer L to be compressed R , for example, neural network parameters, e.g., neural network weights, such as the weights that may be represented by matrix 15a in Figs. 2 and 3, are used in the base layer L B and one or more enhancement layers L E,1 ,L E,2 ,···,L E,N Then, in the first step, the base layer is compressed / transmitted, and in the subsequent step, the enhancement layer L E,1 ,L E,2 ,...,L E,N is compressed / transmitted (separately).

[0128] TIFF0007783376000009.tif38152

[0129] TIFF0007783376000010.tif36152

[0130] In other words, the present embodiment provides a reconstructed layer L R or in the form of a reconstructed layer L R and a device configured to reconstruct the neural network parameters 13 by a parameter-wise sum or a parameter-wise product of the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer, for each neural network parameter, using a parameter-wise sum or a parameter-wise product of the neural network parameter values ​​of the first reconstruction layer and the second reconstruction layer.

[0131] Respectively, in the case of the device for encoding neural network parameters 13 according to the embodiment, the neural network parameters 13 are reconfigurable for each neural network parameter by a parameter-wise sum or a parameter-wise product of the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer.

[0132] In a further preferred embodiment, the methods of 2.1 and / or 2.2 are applied to a subset or all sublayers.

[0133] In a particularly preferred embodiment, an entropy coding scheme using context modeling (e.g., identical or similar to 2.2.3) is applied, but with the addition of one or more sets of context models according to one or more of the following rules:

[0134] a) each sub-layer applies its own set of contexts. In other words, embodiments of the present invention include an apparatus configured to encode / decode first neural network parameters 13a for a first reconstruction layer to / from a data stream or another data stream, and to encode / decode second neural network parameters 13b for a second reconstruction layer to / from a data stream by context-adaptive entropy coding using different probability contexts for the first and second reconstruction layers.

[0135] b) The context set selected for a parameter of an enhancement layer to be coded depends on the value of the parameter at the same position in a preceding layer (e.g., the base layer) in coding order. The first set of context models is selected whenever the parameter at the same position is equal to zero, and the second set is selected otherwise. In other words, an embodiment according to the present invention comprises an apparatus configured to code neural network parameter values ​​of a second reconstruction layer, e.g., parameters of an enhancement layer, into a data stream by context-adaptive entropy coding using a probabilistic model corresponding to the neural network parameter values ​​of a first reconstruction layer, e.g., the parameter at the same position in a preceding layer (e.g., the base layer) in coding order. A further embodiment includes an apparatus configured to code second reconstruction layer neural network parameter values ​​into a data stream by context-adaptive entropy coding by selecting a probabilistic context set from a set of probabilistic context sets depending on the first reconstruction layer neural network parameter values ​​and by selecting a probabilistic context to use from the selected probabilistic context set depending on the first reconstruction layer neural network parameter values. Respectively, for an apparatus for decoding neural network parameters 13 according to an embodiment, the apparatus may be configured to decode second reconstruction layer neural network parameter values ​​from a data stream by context-adaptive entropy decoding using a probability model corresponding to a first reconstruction layer neural network parameter value. Respectively, further embodiments include an apparatus configured to decode second reconstruction layer neural network parameter values ​​from a data stream by context-adaptive entropy decoding by selecting a probability context set from a set of probability context sets corresponding to the first reconstruction layer neural network parameter values ​​and by selecting a probability context to use from the selected probability context set corresponding to the first reconstruction layer neural network parameter value.

[0136] c) The context set selected for a parameter of an enhancement layer to be coded corresponds to the value of the parameter at the same position in a preceding layer (e.g., the base layer) in coding order. The first set of context models is selected whenever the parameter at the same position is less than zero (negative), the second set is selected whenever the parameter at the same position is greater than zero (positive), and the third set is selected otherwise. In other words, an embodiment according to the present invention includes, for example, an apparatus for coding. The set of probabilistic context sets includes three probabilistic context sets. The apparatus is configured to select a first probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set when the neural network parameter value of the first reconstruction layer is negative, to select a second probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set when the neural network parameter value of the first reconstruction layer is positive, and to select a third probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set when the neural network parameter value of the first reconstruction layer is zero. Respectively, for an apparatus for decoding neural network parameters 13 according to an embodiment, the set of probabilistic context sets may include three probabilistic context sets, and the apparatus may be configured to select a first probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set when the neural network parameter value of the first reconstruction layer is negative, to select a second probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set when the neural network parameter value of the first reconstruction layer is positive, and to select a third probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set when the neural network parameter value of the first reconstruction layer is zero.

[0137] d) The context set selected for a parameter of an enhancement layer to be coded corresponds to the value of the parameter at the same position in the preceding layer (e.g., base layer) in coding order. The first set of context models is selected whenever the (absolute) value of the parameter at the same position is greater than X (X is the parameter), otherwise the second set is selected. In other words, an embodiment according to the present invention includes an apparatus, wherein the set of probabilistic context sets includes two probabilistic context sets, and the apparatus is configured to select a first probabilistic context set from the set of probabilistic context sets as a selected probabilistic context set if a parameter value of a neural network of a first reconstruction layer, e.g., a value of a parameter at the same position in a preceding layer (e.g., a base layer) in the coding order, is greater than a predetermined value, e.g., X; and to select a second probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set if the neural network parameter value of the first reconstruction layer is not greater than the predetermined value; or to select the first probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set if the absolute value of the neural network parameter value of the first reconstruction layer is greater than the predetermined value; and to select the second probabilistic context set from the set of probabilistic context sets as the selected probabilistic context set if the absolute value of the neural network parameter value of the first reconstruction layer is not greater than the predetermined value.Respectively, for an apparatus for decoding neural network parameters 13 according to an embodiment, the set of probabilistic contexts may include two probabilistic context sets, and the apparatus may be configured to select a first probabilistic context set from the set of probabilistic context sets as a selected probabilistic context set if the neural network parameter value of the first reconstruction layer is greater than a predetermined value, for example X, and to select a second probabilistic context set from the set of probabilistic context sets as a selected probabilistic context set if the neural network parameter value of the first reconstruction layer is not greater than the predetermined value; or to select a first probabilistic context set from the set of probabilistic context sets as a selected probabilistic context set if the absolute value of the neural network parameter value of the first reconstruction layer is greater than the predetermined value, and to select a second probabilistic context set from the set of probabilistic context sets as a selected probabilistic context set if the absolute value of the neural network parameter value of the first reconstruction layer is not greater than the predetermined value.

[0138] 4 Neural Network Parameter Coding with Dependent Scalar Quantization This section discloses further optional aspects and features to the concepts and embodiments according to the present invention as described in the context of FIGS.

[0139] A modified concept of neural network parameter coding is described. The main change to the previously described neural network parameter coding is that the neural network parameters 13 are not independently quantized and reconstructed. Instead, the allowed reconstruction levels of a neural network parameter 13 correspond to the selected quantization indexes 56 of the preceding neural network parameter in the reconstruction order. The concept of dependent scalar quantization is combined with modified entropy coding, in which the probability model selection (or alternatively, codeword table selection) for the neural network parameters depends on the set of allowed reconstruction levels. However, it should be noted that the previously described embodiment may use and / or incorporate and / or extend any of the features described below, separately or in combination.

[0140] 4.1 Advantages over related neural network parameter coding An advantage of dependent quantization of neural network parameters is that the allowable reconstruction vectors are tightly packed in an N-dimensional signal space (where N represents the set of samples being processed, e.g., the number of samples or neural network parameters 13 in layers 10a and 10b). The reconstruction vectors of a neural network parameter set refer to the ordered reconstructed neural network parameters (or, alternatively, the ordered reconstructed samples) of the neural network parameter set. The effect of dependent scalar quantization is illustrated in FIG. 8 for the simplest case of two neural network parameters. FIG. 8 shows an example of the positions of allowable reconstruction vectors for the simple case of two weight parameters. FIG. 8(a) is an example of independent scalar quantization, and FIG. 8(b) is an example of dependent scalar quantization. FIG. 8a shows allowable reconstruction vectors 201 (representing points on a two-dimensional plane) for independent scalar quantization. As can be seen, the set of allowable values ​​for the second neural network parameter t1′13 is independent of the value selected for the first reconstructed neural network parameter t0′13. FIG. 8(b) shows an example of dependent scalar quantization. Note that, in contrast to independent scalar quantization, the selectable reconstruction values ​​for the second neural network parameter t1'13 depend on the reconstruction level selected for the first neural network parameter t0'13. In the example of FIG. 8b, there are two different sets 52 of available reconstruction levels for the second neural network parameter t1'13 (illustrated by different colors). If the quantization index 56 for the first neural network parameter t0'13 is even (..., -2, 0, 2,...), any reconstruction level 201a in the first set (blue dots) can be selected for the second neural network parameter t1'13.Then, if the quantization index 56 for the first neural network parameter t0' is odd (..., -3, -1, 1, 3,...), any reconstruction level 201b of the second set (red dot) can be selected for the second neural network parameter t1' 13. In this example, the reconstruction levels of the first and second sets are shifted by half the quantization step size (any reconstruction level of the second set is located between two reconstruction levels of the first set).

[0141] Dependent scalar quantization of neural network parameters 13 has the effect that, for a given average number of reconstructed vectors 201 per N-dimensional unit volume, the expected value of the distance between a given input vector of neural network parameters 13 and the nearest available reconstructed vector is reduced. As a result, the average distortion between the input vector of neural network parameters and the vector-reconstructed neural network parameters can be reduced for a given average number of bits. In vector quantization, this effect is called space-filling gain. Using dependent scalar quantization for neural network parameter sets 13 allows us to exploit most of the potential space-filling gain of high-dimensional vector quantization. And, in contrast to vector quantization, the implementation complexity of the reconstruction process (or decoding process) is comparable to that of related neural network parameter encoding using independent scalar quantizers.

[0142] 4.2 Overview The main change is the dependency quantization, as mentioned above. The reconstructed neural network parameters t k '13 is the associated quantization index q k 56 as well as the quantization indices q0,q1,···,q of the preceding neural network parameters in the reconstruction order. k-1It should be noted that in dependent quantization, the reconstruction order of the neural network parameters 13 must be uniquely determined. k The overall performance of a neural network codec can typically be improved if knowledge of the reconstruction level set associated with 56 is also utilized in the entropy coding. That is, it is typically preferable to switch contexts (probability models) or codeword tables based on the reconstruction level set applied to the neural network parameters.

[0143] Entropy coding is usually uniquely specified given the entropy decoding process, but as with related neural network parameter coding, there is a lot of freedom in the choice of quantization indexes given the original neural network parameters.

[0144] The embodiments described herein are not limited to layer-wise neural network coding, but are also applicable to neural network parameter coding of any finite set of neural network parameters 13.

[0145] In particular, the method can also be applied to sub-layers such as those described in sec. 3.1.

[0146] 4.3 Dependent quantization of neural network parameters Dependent quantization of neural network parameters 13 refers to the concept that the available reconstruction level sets for a neural network parameter 13 depend on the chosen quantization index for the preceding neural network parameter in the reconstruction order (e.g., within the same set of neural network parameters, such as a layer or sublayer).

[0147] In a preferred embodiment, multiple sets of reconstruction levels are predefined, and one of the predefined sets is selected for reconstructing the current neural network parameters based on the quantization index of the preceding neural network parameter in the coding order. In other words, an apparatus according to an embodiment can be configured to select 54 a reconstruction level set 48 from a plurality 50 of reconstruction level sets 52 for the current neural network parameters 13 depending on the quantization index (58) for the previous, e.g., preceding, neural network parameter.

[0148] A preferred embodiment for defining a set of reconstruction levels is described in Section 4.3.1. Identification and signaling of the selected reconstruction level is described in Section 4.3.2. Section 4.3.3 describes a preferred embodiment for selecting one of a predefined set of reconstruction levels for the current neural network parameter (based on the selected quantization index of the preceding neural network parameter in the reconstruction order).

[0149] 4.3.1 Reconstructed level set In a preferred embodiment, the set of allowable reconstruction levels for the current neural network parameters is selected (based on the quantization index of the preceding neural network parameter in the coding order) from among a set of predefined sets 52 of reconstruction levels (two or more sets, e.g., Set 0 and Set 1 from Figures 2 and 3).

[0150] In a preferred embodiment, the parameter determines the quantization step size Δ(QP), and all reconstruction levels (in all sets of reconstruction levels) represent integer multiples of the quantization step size Δ. Note, however, that each set of reconstruction levels contains only a subset of integer multiples of the quantization step size Δ(QP). Such a configuration for dependent quantization, in which all possible reconstruction levels for all sets of reconstruction levels represent integer multiples of the quantization step size (QP), can be considered an extension of uniform reconstruction quantizers (URQs). Its fundamental advantage is that the reconstructed neural network parameters 13 can be calculated by an algorithm with very low computational complexity (described in more detail below).

[0151] The reconstruction level sets can be completely discontinuous, but it is also possible for one or more reconstruction levels to be included in multiple sets (although the sets may still differ at other reconstruction levels).

[0152] In a preferred embodiment, the dependent scalar quantization for the neural network parameters uses exactly two different sets of reconstruction levels, e.g., set 0 and set 1. And in a particularly preferred embodiment, the dependent scalar quantization for the neural network parameters t k All the reconstruction levels of the two sets for 13 are calculated by the quantization step size Δ k (QP). Note that the quantization step size Δ k Note that (QP) simply represents a scaling factor for both sets of acceptable reconstruction values. The same two sets of reconstruction levels are used for all neural network parameters 13.

[0153] Figure 9 illustrates three preferred configurations ((a)-(c)) for two sets of reconstruction levels (Set 0 and Set 1). Figure 9 shows an example of dependent quantization with two sets of reconstruction levels completely determined by a single quantization step size, Δ(QP). The two available sets of reconstruction levels are highlighted in different colors (Set 0 is blue, Set 1 is red). Example quantization indices indicating the reconstruction levels within the set are indicated by numbers below the circles. Hollow and filled circles indicate two different subsets within the reconstruction level set, which can be used to determine the reconstruction level set for the next neural network parameter in the reconstruction order. The figure shows three preferred configurations with two reconstruction level sets: (a) the two sets are disjoint and symmetric about zero; (b) both sets contain a reconstruction level equal to zero but are otherwise disjoint, and the sets are asymmetric around zero; and (c) both sets contain a reconstruction level equal to zero but are otherwise disjoint, and both sets are symmetric around zero. Note that all reconstruction levels lie on a grid given by integer multiples (IV) of the quantization step size Δ. Furthermore, it should be noted that a particular reconstruction level may be included in both sets.

[0154] The two sets depicted in Figure 9(a) are discontinuous. Each integer multiple of the quantization step size Δ(QP) is included in only one of the sets. The first set (Set 0) includes all even integer multiples (IV) of the quantization step size, while the second set (Set 1) includes all odd integer multiples of the quantization step size. In both sets, the distance between two adjacent reconstruction levels is twice the quantization step size. These two sets are typically suitable for high-rate quantization, i.e., settings where the variance of the neural network parameters is significantly larger than the quantization step size (QP). However, in neural network parameter coding, the quantizer usually operates in the low-rate domain. Typically, the absolute values ​​of many original neural network parameters 13 are closer to zero than any non-zero multiple of the quantization step size (QP). In this case, it is typically preferable for zero to be included in both quantization sets (reconstruction level sets).

[0155] The two quantization sets shown in Figure 9(b) both contain zero. In set 0, the distance between the reconstructed level equal to zero and the first reconstructed level greater than zero is equal to the quantization step size (QP), while all other distances between two adjacent reconstructed levels are equal to twice the quantization step size. Similarly, in set 1, the distance between the reconstructed level equal to zero and the first reconstructed level less than zero is equal to the quantization step size, while all other distances between two adjacent reconstructed levels are equal to twice the quantization step size. Note that both reconstructed sets are asymmetric around zero. This can lead to inefficiencies because it makes it difficult to accurately estimate code probabilities.

[0156] A preferred arrangement of two sets of reconstruction levels is shown in Figure 9(c). The reconstruction levels included in the first quantization set (labeled Set 0 in the figure) represent even integer multiples of the quantization step size (note that this set is actually the same as Set 0 in Figure 9(a)). The second quantization set (labeled Set 1 in the figure) includes all odd integer multiples of the quantization step size, and also has reconstruction levels equal to 0. Note that both reconstruction sets are symmetric about zero. Reconstruction levels equal to zero are included in both reconstruction sets; otherwise, the reconstruction sets are discontinuous. The combination of both reconstruction sets includes all integer multiples of the quantization step size.

[0157] In other words, according to an embodiment, for example, an apparatus for encoding / decoding neural network parameters 13 comprises a plurality 50 of reconstruction level sets 52, the number of which is two (e.g., set 0, set 1), and the plurality of reconstruction level sets includes a first reconstruction level set (set 0) that includes zero and an even multiple of a predetermined quantization step size, and a second reconstruction level set (set 1) that includes zero and an odd multiple of the predetermined quantization step size.

[0158] Furthermore, all reconstruction levels of all reconstruction level sets may represent integer multiples (IV) of a predetermined quantization step size (QP), and an apparatus for decoding, for example, neural network parameters 13 according to an embodiment may be configured to dequantize the neural network parameters 13 by deriving, for each neural network parameter, an intermediate integer value, for example, an integer multiple (IV) according to the reconstruction level set selected for the respective neural network parameter and the entropy decoding quantization index 58 for the respective neural network parameter 13′, and by multiplying, for each neural network parameter 13, the intermediate value of the respective neural network parameter by a predetermined quantization step size for the respective neural network parameter 13.

[0159] Respectively, all reconstruction levels of all reconstruction level sets may represent integer multiples (IV) of a predetermined quantization step size (QP), and the apparatus, e.g., according to an embodiment, may be configured to quantize the neural network parameters 13 in order to encode the neural network parameters by deriving, for each neural network parameter, an intermediate integer value depending on the selected reconstruction level set of the respective neural network parameter and the entropy coding quantization index of the respective neural network parameter, and by multiplying, for each neural network parameter, the intermediate value of the respective neural network parameter by the predetermined quantization step size of the respective neural network parameter, such that the same can be inverse quantized.

[0160] The embodiments defined herein are not limited to the configuration shown in FIG. 9. Any other two different sets of reconstruction levels can be used. Multiple reconstruction levels may be included in both sets. Alternatively, the combination of both quantization sets may not include all possible integer multiples of the quantization step size. Furthermore, it is possible to use more than two sets of reconstruction levels for dependent scalar quantization of neural network parameters.

[0161] 4.3.2 Signaling the selected reconfiguration level The reconstruction level that the encoder selects from among the allowed reconstruction levels must be indicated within the bitstream 14. Similar to conventional independent scalar quantization, this can be achieved using so-called quantization indices 56, also called weight levels. The quantization indices 56 (or weight levels) are integer values ​​that uniquely identify an available reconstruction level within the quantization set 52 (i.e., within the set of reconstruction levels). The quantization indices 56 are sent to the decoder as part of the bitstream 14 (using any entropy coding technique). At the decoder side, the reconstructed neural network parameters 13 can be uniquely calculated based on the current set of reconstruction levels 48 (which is determined by the preceding quantization indices in the encoding / reconstruction order) and the transmitted quantization indices 56 for the current neural network parameters 13'.

[0162] In a preferred embodiment, the assignment of quantization indices 56 to reconstruction levels within a set of reconstruction levels (or quantization set) follows the following rules: For illustrative purposes, the reconstruction levels in FIG. 9 are labeled with associated quantization indices 56 (the quantization indices are given by the numbers below the circles representing the reconstruction levels). If the reconstruction level set includes a reconstruction level equal to 0, a quantization index equal to 0 is assigned to the reconstruction level equal to 0. A quantization index equal to 1 is assigned to the smallest reconstruction level greater than 0, a quantization index equal to 2 is assigned to the next reconstruction level greater than 0 (i.e., the second smallest reconstruction level greater than 0), and so on. Or, in other words, the reconstruction levels greater than 0 are labeled with integer values ​​greater than 0 (i.e., 1, 2, 3, etc.) in ascending order of their value. Similarly, a quantization index −1 is assigned to the largest reconstruction level less than 0, a quantization index −2 is assigned to the next (i.e., second largest) reconstruction level less than 0, and so on. Or, in other words, reconstruction levels less than 0 are labeled with integer values ​​less than 0 in descending order of value (i.e., −1, −2, −3, etc.). The example in Figure 9 illustrates the described quantization index assignments for all quantization sets except for set 1 in Figure 9(a) (which does not contain a reconstruction level equal to 0).

[0163] For quantization sets that do not include a reconstruction level equal to 0, one way to assign quantization indices 56 to reconstruction levels is as follows: All reconstruction levels greater than 0 are assigned quantization indices greater than 0 (in ascending order of value), and all reconstruction levels less than 0 are assigned quantization indices less than 0 (in descending order of value). Thus, the assignment of quantization indices 56 essentially follows the same concept as for quantization sets that include a reconstruction level equal to 0, except that there are no quantization indices equal to 0 (see the label for Quantizer Set 1 in FIG. 9(a)). The entropy coding of the quantization indices 56 must take this into account. For example, the quantization indices 56 are often transmitted by encoding their absolute values ​​(ranging from 0 to the maximum supported value), and for absolute values ​​that are not equal to 0, the sign of the quantization indices 56 is additionally coded. If no quantization indices 56 are equal to 0, the entropy coding can be modified so that the absolute level minus 1 is transmitted (the value of the corresponding syntax element ranges from 0 to the maximum supported value), and a sign is always transmitted. Alternatively, the assignment rule for assigning quantization indexes 56 to reconstruction levels can be modified. For example, one of the reconstruction levels close to 0 can be labeled with a quantization index equal to 0. The remaining reconstruction levels are then labeled according to the following rule: Reconstruction levels greater than the reconstruction level with a quantization index equal to 0 are assigned quantization indexes greater than 0 (the quantization indexes increase with the value of the reconstruction level); and quantization indexes less than 0 are assigned to reconstruction levels less than the reconstruction level with a quantization index equal to 0 (the quantization indexes decrease with the value of the reconstruction level). One possibility for such assignment is shown by the numbers in parentheses in FIG. 9( a) (if no number in parentheses is given, other numbers apply).

[0164] As mentioned above, in a preferred embodiment, two different sets of reconstruction levels (also referred to as quantization sets) are used, and the reconstruction levels within both sets represent integer multiples of the quantization step size (QP), including cases where the quantization step size is changed on a layer-by-layer basis (e.g., by sending layer quantization parameters within the bitstream 14) or for another finite set (e.g., block) of neural network parameters 13 (e.g., by sending block quantization parameters within the bitstream 14).

[0165] The use of reconstruction levels representing integer multiples of the quantization step size (QP) allows for a computationally inexpensive and complex algorithm for the reconstruction of the neural network parameters 13 at the decoder side. This is described below based on the preferred example of FIG. 9(c) (similar simple algorithms exist for other configurations, particularly the settings shown in FIG. 9(a) and FIG. 9(b)). In the configuration shown in FIG. 9(c), the first quantization set includes all even integer multiples of the quantization step size (QP), and the second quantization set includes all odd integer multiples of the quantization step size plus a reconstruction level equal to 0 (which is included in both quantization sets). The neural network parameter reconstruction process can be implemented similarly to the algorithm specified in the pseudocode of FIG. 10. FIG. 10 shows an example of pseudocode illustrating a preferred example of the neural network parameter 13 reconstruction process. k represents an index specifying the order of reconstruction of the current neural network parameters 13′, the quantization index 56 of the current neural network parameters is represented by level[k] 210, and the quantization step size Δ k (QP) is expressed as quant_step_size[k], and trec[k]220 is the parameter t of the reconstructed neural network. kThe variable setId[k] 240 specifies the set of reconstruction levels to be applied to the current neural network parameters 13'. It is determined based on the neural network parameters preceding it in the reconstruction order; possible values ​​of setId[k] are 0 and 1. The variable n specifies the integer factor, e.g., intermediate value IV, of the quantization step size (QP); it is given by the set of selected reconstruction levels (i.e., the value of setId[k]) and the transmitted quantization index level[k].

[0166] TIFF0007783376000011.tif115154

[0167] When the second quantization set is used, the variable n is equal to twice the quantization index level[k] minus the sign function sign(level[k]) of the quantization index. This can be expressed as the reconstruction level of the second quantization set Set1 in FIG. 9(c), where Set1 includes all odd integer multiples of the quantization step size (QP).

[0168] Once the variable n (which specifies an integer multiple of the quantization step size) is determined, the quantization step size Δ k By multiplying by , the reconstructed neural network parameters t k ' is required.

[0169] In other words, the number of reconstruction level sets 52 among the plurality 50 of reconstruction level sets 52 may be two, and an apparatus for decoding and / or encoding, for example, neural network parameters 13 according to an embodiment of the present invention may be configured to derive intermediate values ​​of each neural network parameter as follows: If the selected reconstruction level set for each neural network parameter is the first set, double the quantization index for each neural network parameter to obtain an intermediate value for each neural network parameter; and If the selected reconstruction level set for each neural network parameter is a second set and the quantization index of each neural network parameter is equal to zero, set the median value of each sample to zero; and If the selected reconstruction level set for each neural network parameter is a second set and the quantization index for each neural network parameter is greater than zero, multiply the quantization index for each neural network parameter by two and subtract one from the multiplication result to obtain an intermediate value for each neural network parameter; and If the selected reconstruction level set for the current neural network parameters is the second set and the quantization index for each neural network parameter is less than zero, the quantization index for each neural network parameter is doubled and the result of the multiplication is added by 1 to obtain an intermediate value for each neural network parameter.

[0170] 4.3.3 Dependent Reconfiguration of Neural Network Parameters In addition to the selection of the set of reconstruction levels discussed in Sections 4.3.1 and 4.3.2, another important design aspect of dependent scalar quantization in neural network parameter coding is the algorithm used to switch between the defined quantization sets (sets of reconstruction levels). The algorithm used determines the "packing density" that can be achieved in the N-dimensional space of neural network parameters (and therefore the N-dimensional space of reconstructed samples). The higher the packing density, the better the final coding efficiency.

[0171] A preferred method for determining the set of reconstruction levels for the next neural network parameters is based on dividing the quantizer sets, as shown in FIG. 11. FIG. 11 illustrates an example for dividing the set of reconstruction levels into two subsets according to an embodiment of the present invention. The two quantizer sets shown are the quantizer sets of the preferred example of FIG. 9(c). The two subsets of quantizer set 0 are labeled with "A" and "B," and the two subsets of quantizer set 1 are labeled with "C" and "D." Note that the quantizer sets shown in FIG. 11 are the same quantizer sets as those of FIG. 9(c). Each of the two (or more) quantizer sets is divided into two subsets. In the preferred example of FIG. 11, the first quantizer set (labeled set 0) is divided into two subsets (labeled A and B), and the second quantizer set (labeled set 1) is also divided into two subsets (labeled C and D). Although not the only possibility, the division for each quantization set is preferably made so that immediately adjacent reconstruction levels (and therefore adjacent quantization indices) are associated with different subsets. In a preferred embodiment, each quantization set is divided into two subsets. In Figure 9, the division of the quantization set into subsets is indicated by hollow and filled circles.

[0172] In a particularly preferred embodiment shown in Figure 11 and Figure 9(c), the following splitting rules apply: Subset A consists of all even quantization indices of quantization set 0. Subset B consists of all odd quantization indices of quantization set 0. Subset C consists of all even quantization indices of quantization set 1. Subset D consists of all odd quantization indices of quantization set 1.

[0173] Note that the subset used is typically not explicitly indicated inside the bitstream 14. Instead, it can be derived based on the quantization set used (e.g., set 0 or set 1) and the quantization indices 56 that were actually transmitted. For the preferred partitioning shown in Figure 11, the subsets can be derived by a bitwise "and" operation of the transmitted quantization index levels and 1. Subset A consists of all quantization indices in set 0 for which (level&1) equals 0, subset B consists of all quantization indices in set 0 for which (level&1) equals 1, subset C consists of all quantization indices in set 1 for which (level&1) equals 0, and subset D consists of all quantization indices in set 1 for which (level&1) equals 1.

[0174] In a preferred embodiment, the quantization set (set of allowable reconstruction levels) used to reconstruct the current neural network parameters 13′ is determined based on the subset associated with the last two or more quantization indexes 56. An example in which the last two subsets (given by the last two quantization indexes) are used is shown in Table 1. The determination of the quantization set specified by this table represents a preferred embodiment. In other embodiments, the quantization set for the current neural network parameters 13′ is determined by the subset associated with the last three or more quantization indexes 56. For the first neural network parameter (or subset of neural network parameters) of a layer, there is no data regarding the subset of the preceding neural network parameters (because there are no preceding neural network parameters). In a preferred embodiment, a predefined value is used in such cases. In a particularly preferred embodiment, a subset A is inferred for all neural network parameters that are not available. That is, when reconstructing the first neural network parameter, the two preceding subsets are inferred as "AA" (or "AAA" if the three preceding neural network parameters are considered), and therefore, according to Table 1, quantization set 0 is used. For the second neural network parameter, the subset of the preceding quantization index is determined by its value (set 0 is used for the first neural network parameter, so the subset is either A or B), while the subset for the second and last quantization index (which does not exist) is inferred to be equal to A. Of course, any other rule can be used to infer default values ​​for non-existent quantization indexes. Also, other syntax elements can be used to derive the default subset for non-existent quantization indexes. As a further alternative, the last quantization index 56 of the preceding set of neural network parameters 13 can be used for initialization.

[0175] Table 1: Example for determining the quantization set (set of available reconstruction levels) to be used for the next neural network parameters based on the subsets associated with the two last quantization indexes according to an embodiment of the present invention. The subsets are shown in the left table column and are uniquely determined by the quantization set used (for the two last quantization indexes) and the so-called path (which may be determined by the parity of the quantization indexes). The quantization set and, in parentheses, the path for the subset are listed in the second column from the left. The third column specifies the associated quantization set. The last column shows the values ​​of so-called state variables, which can be used to simplify the process for determining the quantization set.

[0176] [Table 1]

[0177] It should be noted that the subset (A, B, C, or D) of quantization indexes 56 is determined by the quantization set to be used (set 0 or set 1) and the subset to be used within the quantization set (e.g., A or B for set 0, C or D for set 1). The subset selected within the quantization set is also called a path (because it specifies a path when representing the dependent quantization process with a trellis structure, as will be described later). In our convention, a path is equal to either 0 or 1. Then, subset A corresponds to path 0 in set 0, subset B corresponds to path 1 in set 0, subset C corresponds to path 0 in set 1, and subset D corresponds to path 1 in set 1. Therefore, the quantization set for the next neural network parameter is also uniquely determined by the quantization set (set 0 or set 1) and path (path 0 or path 1) associated with the last two (or more) quantization indexes. In Table 1, the associated quantization sets and paths are specified in the second column.

[0178] TIFF0007783376000013.tif50158

[0179] In other words, the number of reconstruction level sets 52 among the plurality 50 reconstruction level sets 52 may be two, for example, set 0 and set 1, and an apparatus for decoding, for example, neural network parameters 13 according to an embodiment of the present invention may be configured to derive a subset index for each neural network parameter based on the selected set of reconstruction levels for each neural network parameter and a binary function of the quantization index for each neural network parameter, resulting in four possible values, for example, A, B, C, or D for the subset index, and to select, for the current neural network parameters 13′, a set 48 of reconstruction levels among the plurality 50 reconstruction level sets 52 depending on the subset index for the previously decoded neural network parameter.

[0180] A further embodiment according to the present invention comprises an apparatus configured to select 54 a reconstruction level set 48 from a plurality of 50 reconstruction level sets 5) for current neural network parameters 13′ using a selection rule corresponding to a subset index of a number of previously decoded neural network parameters, e.g., as shown in the first column of Table 1, and to use the selection rule for all or a portion of the neural network parameters.

[0181] According to a further embodiment, the number of previously decoded neural network parameters to which the selection rule corresponds is two, eg, a subset of the two last quantization indexes, as shown in Table 1.

[0182] According to an additional embodiment, the subset index for each neural network parameter is derived based on the parity of the selected set of reconstruction levels for each neural network parameter and the quantization index for each neural network parameter, e.g., using path=(level[k]&1).

[0183] Respectively, for an apparatus for encoding neural network parameters 13 according to an embodiment, the number of reconstruction level sets 52 of the plurality 50 reconstruction level sets 52 may be two, for example, set 0 and set 1, and the apparatus may be configured to derive a subset index for each neural network parameter based on the selected set of reconstruction levels for each neural network parameter and a binary function of the quantization index for each neural network parameter, resulting in four possible values ​​for the subset index, for example, A, B, C, D, and to select 54 a reconstruction level set 48 from the plurality 50 reconstruction level sets 52 for the current neural network parameters 13′ depending on the subset index for the previously encoded neural network parameter.

[0184] A further embodiment according to the present invention comprises an apparatus configured to select 54 a set of reconstruction levels 48 from a plurality of 50 reconstruction level sets 52 using a selection rule corresponding to a subset index of a number of previously encoded neural network parameters for a current neural network parameter 13′, e.g., as shown in the first column of Table 1, and to use the selection rule for all or a portion of the neural network parameters.

[0185] According to a further embodiment, the number of previously coded neural network parameters to which the selection rule corresponds is two, a subset of the two last quantization indexes, for example as shown in Table 1.

[0186] According to an additional embodiment, the subset index for each neural network parameter is derived based on the selected set of reconstruction levels for the respective neural network parameter and the parity of the quantization index of the respective neural network parameter, e.g., using path=(level[k]&1).

[0187] The transitions between quantization sets 52 (set 0 and set 1) can also be elegantly represented by state variables. An example of such a state variable is shown in the last column of Table 1. In this example, the state variable has four possible values ​​(0, 1, 2, and 3). The state variable, in turn, specifies the quantization set to be used for the current neural network parameters 13'. In the preferred example of Table 1, quantization set 0 is used only if the state variable is equal to 0 or 2, and quantization set 1 is used only if the state variable is equal to 1 or 3. The state variable, in turn, specifies the possible transitions between quantization sets. Using state variables, the rules of Table 1 can be written in smaller state transition tables. As an example, Table 2 defines a state transition table for the rules given in Table 1. This represents a preferred embodiment. Given the current state, a quantization set is defined for the current neural network parameters (second column). It further specifies the state transitions based on the path associated with the selected quantization index 56 (the path specifies the subset A, B, C, or D to be used, given a quantization set). Note that by using the concept of state variables, it is not necessary to keep track of the actual selected subset: when reconstructing the parameters of a layer of the neural network, it is sufficient to update the state variables and determine the path of quantization indices to be used.

[0188] Table 2: A preferred example of a state transition table for a configuration having four states, according to an embodiment of the present invention. [Table 2]

[0189] That is, for example, an apparatus for decoding neural network parameters according to an embodiment may be configured to select 54 a quantization level set 48 from the plurality of 50 reconstructed level sets 52 for a current neural network parameter 13′ by a state transition process by determining a set of quantization levels 48 from the plurality of 50 reconstructed level sets 52 depending on a state associated with the current neural network parameter 13′, and by updating the state of the subsequent neural network parameter depending on a quantization index 58 decoded from a data stream of the immediately preceding neural network parameter.

[0190] Respectively, for an apparatus for encoding neural network parameters 13 according to an embodiment, the apparatus may be configured to select 54 a set 48 of reconstruction levels 48 of the plurality 50 of reconstruction level sets 52 for a current neural network parameter 13′ by a state transition process by determining a set 48 of reconstruction levels 48 of the plurality 50 of reconstruction level sets 52 depending on a state associated with the current neural network parameter 13′, and by updating a state for a subsequent neural network parameter depending on a quantization index 58 encoded in the data stream of the immediately preceding neural network parameter.

[0191] TIFF0007783376000015.tif46160

[0192] In other words, an apparatus according to an embodiment, for example for decoding neural network parameters, can be configured to update the state of a subsequent neural network parameter using a binary function of quantization indexes 58 decoded from the data stream for the immediately preceding neural network parameter, for example according to Table 2.

[0193] Furthermore, the apparatus according to the embodiment may be configured to update the state for the subsequent neural network parameter using the parity of the quantization index 58 decoded from the data stream 14 for the immediately preceding neural network parameter, e.g., path=(level[k]&1).

[0194] Respectively, for an apparatus for encoding neural network parameters 13 according to an embodiment, the apparatus may be configured to update the state for a subsequent neural network parameter using a binary function of the quantization indexes 58 encoded in the data stream for the immediately preceding neural network parameter.

[0195] Furthermore, an apparatus for encoding, for example, neural network parameters 13 according to an embodiment may be configured to use the parity of the quantization indexes 58 encoded in the data stream for the immediately preceding neural network parameter to update the state for the subsequent neural network parameter, for example according to Table 2.

[0196] In a preferred embodiment, a state variable having four possible values ​​is used. In other embodiments, state variables having a different number of possible values ​​are used. Of particular interest are state variables whose number of possible values ​​represents an integer power of two, i.e., 4, 8, 16, 32, 64, etc. Note that in the preferred configuration (as given in Tables 1 and 2), a state variable having four possible values ​​is equivalent to an approach in which the current quantization set is determined by a subset of the two last quantization indexes. A state variable having eight possible values ​​corresponds to a similar approach in which the current quantization set is determined by a subset of the three last quantization indexes. A state variable having 16 possible values ​​corresponds to an approach in which the current quantization set is determined by a subset of the last four quantization indexes. Although it is generally preferred to use state variables whose number of possible values ​​is equal to an integer power of two, embodiments are not limited to this configuration.

[0197] In a particularly preferred embodiment, a state variable is used that has eight possible values: 0, 1, 2, 3, 4, 5, 6, and 7. In the preferred example, Table 3, quantization set 0 is used only if the state variable is equal to 0, 2, 4, or 6, and quantization set 1 is used only if the state variable is equal to 1, 3, 5, or 7.

[0198] 1 is a preferred example of a state transition table for a configuration having eight states, according to an embodiment. [Table 3]

[0199] That is, in embodiments of the present invention, the state transition process is configured to transition between four or eight possible states.

[0200] Further, according to an embodiment, the apparatus for decoding / encoding neural network parameters 13 can be configured to transition between an even number of possible states in the state transition process, and the number of the plurality of 50 reconstruction level sets 52 is two, wherein for the current neural network parameters 13', a set of quantization levels 48 from the quantization set 52 is determined according to the state associated with the current neural network parameters 13', such that if the state belongs to the first half of the even number of possible states, a first reconstruction level set from the plurality of 50 reconstruction level sets 52 is determined, and if the state belongs to the second half of the even number of possible states, a second reconstruction level set from the plurality of 50 reconstruction level sets 52 is determined.

[0201] According to a further embodiment, for example, an apparatus for decoding neural network parameters 13 may be configured to perform state updates by means of a transition table that maps a combination of state and parity of a quantization index 58 decoded from a data stream of a previous neural network parameter onto another state associated with a subsequent neural network parameter.

[0202] Accordingly, an apparatus for encoding neural network parameters 13 according to an embodiment can be configured to perform state updates via a transition table that maps a combination of state and parity of quantization index 58 encoded in the data stream for a previous neural network parameter to another state associated with a subsequent neural network parameter.

[0203] Using the concept of state transitions, the current state, and therefore the current quantization set, is uniquely determined by the previous state and previous quantization index 56 (in the reconstruction order). However, for the first neural network parameter 13 of a finite set (e.g., a layer), there is no previous state and no previous quantization index. Therefore, it is required that the state for the first neural network parameter of a layer is uniquely defined. There are various possibilities. Preferred options are as follows: The first state for a layer is always set equal to a fixed predefined value. In the preferred embodiment, the first state is set equal to 0. The value of the first state is explicitly transmitted as part of the bitstream 14. This includes an approach where only a subset of the possible state values ​​can be indicated by the corresponding syntax element. The value of the first state is derived based on other syntax elements of that layer, meaning that even if the corresponding syntax element (or syntax elements) are used to signal other aspects to the decoder, they are additionally used to derive the first state for the dependent scalar quantization.

[0204] The concept of state transitions for dependent scalar quantization allows for a low-complexity implementation for the reconstruction of neural network parameters 13 in a decoder. A preferred example of a single-layer neural network parameter reconstruction process is shown in C-style pseudocode in FIG. 12. FIG. 12 shows an example of pseudocode illustrating a preferred example of a layer neural network parameter reconstruction process 13 according to an embodiment of the present invention. Note that the derivation of the quantization indexes and the derivation of the reconstructed values, e.g., using the quantization step size or alternatively using a codebook, may be performed in separate loops. In other words, the derivation of “n” and state update may be performed in a first loop, and the derivation of “trec” may be performed in a second separate loop. Array level 210 represents the transmitted neural network parameter levels (quantization indexes 56) for that layer, and array trec 220 represents the corresponding reconstructed neural network parameters 13. The quantization step size Δk (QP) applied to the current neural network parameters 13′ is denoted by quant_step_size[k]. 2d table sttab 230 specifies a state transition table according to, for example, one of Table 1, Table 2, and / or Table 3, and table setId 240 specifies a quantization set associated with state 250.

[0205] In the pseudocode of FIG. 12, the index k specifies the reconstruction order of the neural network parameters. The final index, layerSize, specifies the reconstruction index of the last reconstructed neural network parameter. The variable layerSize may be set equal to the number of neural network parameters in the layer. The reconstruction process for each single neural network parameter is the same as in the example of FIG. 10. For the example of FIG. 10, the quantization index is represented by level[k] 210, and the associated reconstructed neural network parameter is represented by trec[k] 220. Also, the state variable is represented by state 210. Note that in the example of FIG. 12, state is set equal to 0 at the beginning of the layer. However, as mentioned above, other initializations (e.g., based on the values ​​of some syntax elements) are possible. The 1d table setId[] 240 specifies the quantization sets associated with different values ​​of the state variables, and the 2d table sttab[][] 230 specifies the state transitions given the current state (first argument) and the path (second argument). In this example, the path is given by the parity of the quantization index (bitwise using the & operator), but other concepts are possible. In C-style syntax, example tables are shown in Figures 13 and 14 (these tables are identical to Tables 2 and 3, and in other words, can provide the representations of Tables 2 and 3).

[0206] 13 is a diagram showing a preferred example of a state transition table sttab 230 and a table setId 240 specifying the quantization set associated with a state 250 according to an embodiment of the present invention. The table, given in C-style syntax, represents the table defined in Table 2.

[0207] 14 is a diagram showing a preferred example of a state transition table sttab 230 and a table setId 240 specifying the quantization set associated with a state 250 according to an embodiment of the present invention. The table, given in C-style syntax, represents the table defined in Table 3.

[0208] In another embodiment, all quantization indices 56 equal to 0 are excluded from the state transitions and dependent reconstruction process. The information of whether a quantization index 56 is equal to or not equal to 0 is simply used to partition the neural network parameters 13 into zero and non-zero neural network parameters. The reconstruction process for dependent scalar quantization is applied only to the ordered set of non-zero quantization indices 56. All neural network parameters associated with quantization indices equal to 0 are simply set equal to 0. Corresponding pseudocode is shown in FIG. 15. FIG. 15 shows pseudocode illustrating an alternative neural network parameter level reconstruction process in which quantization indices equal to 0 are excluded from state transitions and dependent scalar quantization, according to an embodiment of the present invention.

[0209] State transitions in dependent quantization can also be represented using a trellis structure, as shown in FIG. 16. FIG. 16 illustrates an example of state transitions in dependent scalar quantization as a trellis structure according to an embodiment of the present invention. The horizontal axis represents different neural network parameters 13 in the reconstruction order. The vertical axis represents different possible states 250 in the dependent quantization and reconstruction process. The connections shown specify the available paths between states for different neural network parameters. The trellis shown in this figure corresponds to the state transitions specified in Table 2. For each state 250, there are two paths connecting the state for the current neural network parameter 13′ with the two possible states for the next neural network parameter 13 in the reconstruction order. The paths are labeled Path 0 and Path 1, and the numbers correspond to the path variables introduced above (in a preferred embodiment, the path variables are equal to the parity of the quantization indexes). Note that each path uniquely specifies a subset (A, B, C, or D) for the quantization indexes. The subsets are specified in brackets in Figure 16. Given an initial state (say state 0), the path through the trellis is uniquely specified by the transmitted quantization index 56.

[0210] In the example of Figure 16, state (0, 1, 2, 3) has the following properties: State 0: The previous quantization index level[k-1] specifies the reconstruction level of set 0, and the current quantization index level[k] specifies the reconstruction level of set 0. State 1: The previous quantization index level[k-1] specifies the reconstruction level of set 0, and the current quantization index level[k] specifies the reconstruction level of set 1. State 2: The previous quantization index level[k-1] specifies the reconstruction level of set 1, and the current quantization index level[k] specifies the reconstruction level of set 0. State 3: The previous quantization index level[k-1] specifies the reconstruction level of set 1, and the current quantization index level[k] specifies the reconstruction level of set 1.

[0211] A trellis is composed of a concatenation of so-called basic trellis cells. An example of such a basic trellis cell is shown in FIG. 17. FIG. 17 illustrates an example of a basic trellis cell according to an embodiment of the present invention. Note that the present invention is not limited to a trellis having four states 250. In other embodiments, the trellis can have more states 250. In particular, any number of states representing an integer power of two is preferred. In a particularly preferred embodiment, the number of states 250 is equal to eight, as in Table 3. Even when the trellis has more than two states 250, each node for the current neural network parameters 13' is typically connected to two states for the previous neural network parameters 13 and two states for the next neural network parameters 13. However, it is also possible for a node to be connected to more than two states of the previous neural network parameters or more than two states of the next neural network parameters. Note that a fully connected trellis (each state 250 is connected to every state 250 of the previous neural network parameter 13 and every state 250 of the next neural network parameter 13) would correspond to independent scalar quantization.

[0212] In a preferred embodiment, the initial state cannot be freely chosen (since some side information rate is required to transmit this decision to the decoder). Instead, the initial state is either set to a predefined value or its value is derived based on other syntax elements. In this case, not all paths and states 250 are available for the first neural network parameter. As an example of a four-state trellis, FIG. 18 shows the trellis structure when the initial state is equal to 0. FIG. 18 illustrates an example trellis for dependent scalar quantization of eight neural network parameters according to an embodiment of the present invention. The first state (on the left) represents the initial state and is set equal to 0 in this example.

[0213] 4.4 Entropy Coding The quantization indexes obtained by the dependent quantization are coded using an entropy coding method. Any entropy coding method can be used for this. In a preferred embodiment of the present invention, the entropy coding method according to section 2.2 (see sections 2.2.1 for the coding method and 2.2.2 for the decoding method) using Context-Adaptive Binary Arithmetic Coding (CABAC) is used. For this purpose, the non-binary values ​​are first mapped onto a series of binary decisions (so-called bins) (binarization) in order to transmit the quantization indexes as absolute values, as shown for example in Fig. 5.

[0214] It should be noted that any of the concepts described here can be combined with the methods and related concepts in Section 3, especially those related to context modeling.

[0215] 4.4.1 Context Modeling for Dependent Scalar Quantization A key aspect of dependent scalar quantization is that there are different sets of allowable reconstruction levels (also called quantization sets) for neural network parameters 13. The quantization set for the current neural network parameters 13' is determined based on the value of the quantization index 56 for the preceding neural network parameters. Considering the preferred example of FIG. 11 and comparing the two quantization sets, it is clear that the distance between a reconstruction level equal to zero and an adjacent reconstruction level is greater in set 0 than in set 1. Thus, the probability that a quantization index 56 is equal to zero is greater when set 0 is used and less when set 1 is used. In preferred embodiments, this effect is exploited in entropy coding by switching codeword tables or probability models based on the quantization set (or state) used for the current quantization index.

[0216] It should be noted that for proper switching of codeword tables or probability models, the paths (associations with the subsets of the quantization set used) of all preceding quantization indexes must be known when entropy decoding the current quantization index (or the corresponding binary decision of the current quantization index). Therefore, it is necessary that the neural network parameters 13 be coded in a reconstruction order. Therefore, in a preferred embodiment, the coding order of the neural network parameters 13 is equal to their reconstruction order. Besides that aspect, any coding / reconstruction order of the quantization indexes 56 is possible, e.g., the order specified in Section 2.2.1 is any other uniquely defined order.

[0217] In other words, embodiments according to the present invention include an apparatus for encoding neural network parameters using a probability model that additionally depends on, for example, the quantization indices of previously encoded neural network parameters.

[0218] Respectively, embodiments according to the present invention include, for example, an apparatus for decoding neural network parameters, which uses a probability model that additionally depends on the quantization indices of previously decoded neural network parameters.

[0219] At least some of the bins for absolute levels are typically coded using adaptive probability models (also called contexts). In a preferred embodiment of the present invention, the probability model for one or more bins is selected based on the quantization set of the corresponding neural network parameters (or, more generally, the corresponding state variables, e.g., a relationship according to any of Tables 1-3). The selected probability model can depend on multiple parameters or characteristics of the already transmitted quantization indexes 56, one of which is the quantization set or state applied to the quantization indexes being coded.

[0220] In other words, an apparatus according to an embodiment, for example for encoding neural network parameters 13, can be configured to pre-select a subset of probability models from a plurality of probability models depending on the state or set 48 of reconstruction levels selected for the current neural network parameters 13′, and to select a probability model for the current neural network parameters from the subset 121 of probability models depending on the quantization index of the previously encoded neural network parameters.

[0221] In the embodiment Kaka For example, the device for decoding the neural network parameters 13 is each , Status or The reconstruction level selected for the current neural network parameters 13' No Se Depending on the set 48, several probability models are used. from A subset of the probabilistic model pre-selection Selection , certain Among the subset of rate models from The probabilistic model of the current neural network parameters is Depending on the quantization index of the previously decoded neural network parameters,It may be configured to select.

[0222] In combination with the inventive concept as described, for example, in the context of FIG. 9, an embodiment according to the present invention, for example for encoding and / or decoding neural network parameters 13, includes an apparatus configured to pre-select a subset of probabilistic models from among a plurality of probabilistic models depending on a state or set of reconstruction levels 48 selected for a current neural network parameters 13′, such that the pre-selected subset for a first state or reconstruction level set is disjoint from the pre-selected subsets for any other state or reconstruction level set.

[0223] In a particularly preferred embodiment, the syntax for transmitting the quantization index of a layer includes a bin, e.g., the aforementioned sig_flag, that specifies whether the quantization index is equal to or not equal to 0. The probability model used to encode this bin is selected from a set of two or more probability models. The choice of the probability model used depends on the quantization set (i.e., the set of reconstruction levels) applied to the corresponding quantization index 56. In another embodiment of the present invention, the probability model used depends on the current state variable (the state variable means the quantization set used).

[0224] In a further embodiment, the syntax for transmitting the quantization index of a layer includes a bin, such as the aforementioned sign_flag, that specifies whether the quantization index is greater than or less than zero. That is, the bin indicates the sign of the quantization index. The selection of the probability model to be used depends on the quantization set (i.e., the set of reconstruction levels) applied to the corresponding quantization index. In another embodiment, the probability model to be used depends on the current state variable (the state variable means the quantization set to be used).

[0225] In a further embodiment, the syntax for transmitting the quantization index includes a bin that specifies whether the absolute value of the quantization index (parameter level of the neural network) is greater than X, e.g., the aforementioned abs_level_greater_X (see Section 0 for details). The probability model used to encode this bin is selected from a set of two or more probability models. The choice of the probability model used depends on the quantization set (i.e., the reconstruction level set) applied to the corresponding quantization index 56. In another embodiment, the probability model used depends on the current state variable (the state variable means the quantization set used).

[0226] One advantageous aspect of the embodiments discussed herein is that dependent quantization of neural network parameters 13 is combined with entropy coding, and the selection of a probabilistic model for one or more bins (also called quantization levels) of the binary representation of the quantization indexes depends on the quantization set (set of allowable reconstruction levels) or corresponding state variable for the current quantization index. The quantization set 52 (or state variable) is given by the quantization index 56 (or subset of bins representing the quantization index) of the preceding neural network parameter in the encoding and reconstruction order.

[0227] In a preferred embodiment, the described choice of probability model is combined with one or more of the following entropy coding aspects: The absolute values ​​of the quantization indices are transmitted using a binary scheme consisting of a number of bins that are coded using an adaptive probability model, and a suffix part that is coded in bypass mode of the arithmetic coding engine (a non-adaptive probability model with pmf (e.g. probability mass function) (0.5, 0.5) for all bins) if the adaptively coded bins do not already fully specify the absolute values. In a preferred embodiment, the binarization used for the suffix part depends on the values ​​of the quantization indices already transmitted. The binarization of the absolute value of the quantization index includes an adaptively coded bin that specifies whether the quantization index is not equal to 0. The probability model (called a context) used to code this bin is selected from a set of candidate probability models. The selected candidate probability model is determined not only by the quantization set (set of allowable reconstruction levels) or state variables for the current quantization index 56, but also by the quantization indexes already transmitted for that layer. In a preferred embodiment, the quantization set (or state variables) determines a subset of the available probability models (also called a context set), and the values ​​of the already coded quantization indexes determine the probability model used within this subset (context set).

[0228] In an embodiment, the probability model to be used within a context set is determined based on the values ​​of already coded quantization indices in a local neighborhood of the current neural network parameters, e.g., a template as described in 2.2.3. Below we list some exemplary measures that can be derived based on the values ​​of quantization indices in a local neighborhood and then used to select a probability model for a pre-determined context set. The sign of the quantization index that is not equal to 0 within the local neighborhood. The number of quantization indices in the local neighborhood that are not equal to 0. This number may be clipped (shortened) to a maximum value. The sum of the absolute values ​​of the quantization indices in a local neighborhood. This number may be clipped to a maximum value. The difference between the sum of the absolute values ​​of the quantization indices in a local neighborhood and the number of quantization indices in the local neighborhood that are not equal to 0. This number may be clipped to a maximum value.

[0229] In other words, embodiments according to the present invention include an apparatus for encoding neural network parameters configured to select a probabilistic model for a current neural network parameter from a subset of probability models depending on characteristics of quantization indices of previously encoded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameter relates, for example, the characteristics including one or more of the following: The signs of the non-zero quantization indices of previously coded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameters are associated. the number of quantization indices of previously coded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameters are associated, the number being non-zero; the sum of the absolute values ​​of the quantization indices of previously coded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameters are associated the sum of the absolute values ​​of the quantization indices of previously coded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameter is associated; and a non-zero number of quantization indices of previously coded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameters are associated; and The difference between.

[0230] Respectively, embodiments according to the present invention include, for example, an apparatus for decoding neural network parameters, configured to select a probability model for a current neural network parameter from a subset of probability models depending on properties of quantization indices of previously decoded neural network parameters associated with portions of the neural network neighboring the portion to which the current neural network parameter is associated, the properties including one or more of the following: The signs of the non-zero quantization indices of the previously decoded neural network parameters associated with the portions of the neural network adjacent to the portion to which the current neural network parameters are associated. The number of quantization indices of previously decoded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameters are associated, the number being non-zero. The sum of the absolute values ​​of the quantization indices of previously decoded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameters are associated. the sum of the absolute values ​​of the quantization indices of previously decoded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameter is associated; and the number of quantization indices of previously decoded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameters are associated, the number being non-zero; The difference.

[0231] The binarization of the absolute value of the quantization index includes adaptively coded bins, e.g., abs_level_greater_X, that specify whether the absolute value of the quantization index is greater than X. The probability model (called abs_level_greater_X) used to code these bins is selected from a set of candidate probability models. The selected probability model is determined not only by the quantization set (set of allowable reconstruction levels) or state variables for the current quantization index, but also by the quantization indexes already transmitted for the layer, e.g., using a template as described above. In a preferred embodiment, the quantization set (or state variables) determines a subset of available probability models (also called a context set), and data on previously coded quantization indexes can be used to determine, in other words, the probability model to be used within this subset (context set). The selection of the probability model can use any of the methods described above (for bins specifying whether the quantization index is not equal to 0).

[0232] Additionally, the apparatus according to the present invention can be configured to position the previously coded neural network parameters 13 so that they are associated with the same neural network layer as the current neural network parameters 13'.

[0233] Furthermore, an apparatus for encoding, for example, neural network parameters according to the present invention can be configured to position one or more of the previously encoded neural network parameters such that the one or more previously encoded neural network parameters are associated with a neuron interconnection emerging from or directed towards the neuron 10c to which the neuron interconnection 11 to which the current neural network parameter refers is associated or another neuron adjacent to that neuron.

[0234] In a further embodiment, the apparatus can be configured to encode the quantization indexes 56 for the current neural network parameters 13′ into the data stream 14 using binary arithmetic coding by using a probability model corresponding to previously coded neural network parameters for one or more leading bins of the binarized quantization indexes, and by using equi-probable bypass mode suffix bins of the binarized quantization indexes following the one or more leading bins.

[0235] The suffix bins of the binarized quantization indexes represent the bins of the binarization codes of the suffix binarizations for binarizing the values ​​of the quantization indexes, and their absolute values ​​may exceed the maximum absolute value that can be represented by one or more leading bins. Thus, an apparatus according to an embodiment of the present invention may be configured to select a suffix binarization depending on the quantization indexes 56 of the previously coded neural network parameters 13.

[0236] Respectively, an apparatus according to the present invention, for example for decoding neural network parameters, may be configured to position previously decoded neural network parameters 13 such that the previously decoded neural network parameters are associated with the same neural network layer as the current neural network parameters 13′.

[0237] According to a further embodiment, for example, an apparatus for decoding neural network parameters according to the present invention can be configured to position one or more of the previously decoded neural network parameters 13 in such a way that the one or more previously decoded neural network parameters are associated with a neuron interconnection 11 emerging from or directed towards the neuron 10c to which the neuron interconnection referred to by the current neural network parameter is associated or another neuron adjacent to said neuron.

[0238] In a further embodiment, the apparatus can be configured to decode the quantization indexes 56 for the current neural network parameters 13′ from the data stream 14 using binary arithmetic coding by using a probability model corresponding to previously decoded neural network parameters for one or more leading bins of the binarized quantization indexes, and by using equal probability bypass mode suffix bins of the binarized quantization indexes following one or more leading bins.

[0239] The suffix bin of the binarization of the quantization index represents a bin of the binarization code of the suffix binarization for binarizing the value of the quantization index, and its absolute value exceeds the maximum absolute value representable by one or more leading bins. Thus, an apparatus according to an embodiment can be configured to select the suffix binarization depending on the quantization index of the previously decoded neural network parameter.

[0240] TIFF0007783376000017.tif94156

[0241] TIFF0007783376000018.tif143153

[0242] In other words, embodiments according to the present invention include apparatus configured to use the Viterbi algorithm and a rate-distortion cost metric to perform selection and / or quantization.

[0243] An exemplary encoding algorithm for selecting appropriate quantization indexes for a layer can consist of the following major steps: 1. Set the rate-distortion cost to 0 in the initial state. 2. For all neural network parameters 13 in the encoding order, do the following: a. For each subset A, B, C, D, determine the quantization index 56 that minimizes the distortion given the original neural network parameters 13. b. For all trellis nodes (0, 1, 2, 3) for the current neural network parameters 13', do the following: i Calculate the rate-distortion cost of the two paths connecting the previous neural network parameter 13 state and the current state. The cost is calculated by dividing the cost for the previous state by the cost for D k +λ R k where D k and R k represents the distortion and rate for selecting the quantization indexes of the subset (A, B, C, D) associated with the connection considered. ii. Assign the minimum of the calculated cost to the current node and remove connections to previous neural network parameter 13 states that do not represent the minimum cost path. NOTE: After this step, all nodes for the current neural network parameters 13' have a single connection to any node for the preceding neural network parameters 13. 3. Compare the costs of the four final nodes (for the last parameter in the encoding order) and select the node with the smallest cost. Note that this node is associated with a unique path through the trellis (all other connections were removed in the previous step). 4. Follow the selected path (specified by the final node) in reverse order and collect the quantization indices 56 associated with the connections between the trellis nodes.

[0244] Note that determining the quantization indexes 56 based on the Viterbi algorithm is substantially less complex than rate-distortion optimized quantization (RDOQ) for independent scalar quantization. Nevertheless, simpler coding algorithms for dependent quantization also exist. For example, starting from a predefined initial state (or quantization set), the quantization indexes 56 can be determined in the encoding / reconstruction order by minimizing an arbitrary cost quantity that considers only the effect of the current quantization index. Given the determined quantization index for the current parameter (and all preceding quantization indexes), the quantization set for the next neural network parameter 13 is known. Thus, this algorithm can be applied to all neural network parameters in the encoding order.

[0245] The method according to the following embodiment is illustrated in FIGS.

[0246] 20 is a block diagram of a method 400 for decoding neural network parameters defining a neural network from a data stream. The method 400 includes sequentially decoding neural network parameters by selecting 54, for a current neural network parameter, a reconstruction level set from a plurality of reconstruction level sets in response to a quantization index decoded from the data stream for a previous neural network parameter, by decoding a quantization index 420 of the current neural network parameter from the data stream, where the quantization index indicates one reconstruction level of the selected set of reconstruction levels for the current neural network parameter, and by dequantizing 62 the current neural network parameter onto one reconstruction level of the selected set of reconstruction levels indicated by the quantization index for the current neural network parameter.

[0247] 21 is a block diagram of a method 500 for encoding neural network parameters defining a neural network from a data stream. The method 500 includes successively encoding neural network parameters by selecting 54 for a current neural network parameter a reconstruction level set from among a plurality of reconstruction level sets in response to quantization indices encoded into the data stream for previously encoded neural network parameters, by quantizing 64 the current neural network parameter to one reconstruction level from the selected set of reconstruction levels, and by encoding 530 into the data stream a quantization index for the current neural network parameter indicating one reconstruction level to which the quantization index of the current neural network parameter is quantized.

[0248] 22 is a block diagram of a method for reconstructing neural network parameters defining a neural network according to an embodiment of the present invention. The method 600 includes deriving first neural network parameters 610 for a first reconstruction layer to provide, for each neural network parameter, a neural network parameter value of the first reconstruction layer. The method 600 further includes decoding 620 second neural network parameters of a second reconstruction layer from the data stream (e.g., as indicated by arrow 312 in FIG. 6 ) to provide, for each neural network parameter, a neural network parameter value of the second reconstruction layer, and reconstructing 630 the neural network parameters by combining, for each neural network parameter, the neural network parameter values ​​of the first reconstruction layer and the neural network parameter values ​​of the second reconstruction layer (e.g., as indicated by arrow 314 in FIG. 6 ).

[0249] 23 is a block diagram of a method for encoding neural network parameters defining a neural network according to an embodiment of the present invention. The method 700 includes encoding 710, for each neural network parameter, a first neural network parameter of a first reconstruction layer including a neural network parameter value of the first reconstruction layer and a second neural network parameter of a second reconstruction layer into a data stream (e.g., as shown by arrow 322 in the figure), the second reconstruction layer including, for each neural network parameter, a second reconstruction layer neural network parameter value, and the neural network parameters are reconfigurable by combining, for each neural network parameter, the first reconstruction layer neural network parameter value and the second reconstruction layer neural network parameter value.

[0250] TIFF0007783376000019.tif236169

[0251] For example, the two-dimensional integer array StateTransTab[][] shown on line 1014 specifies the state transition table for dependent scalar quantization, and is as follows:

[0252] TIFF0007783376000020.tif225170

[0253] The inputs to this process are: The variable tensorDims specifies the dimensions of the tensor to be decoded. A variable entryPointOffset that indicates whether an entry point exists for decoding, and the entry point offset if one exists. · A variable codebookId that indicates whether a codebook is applied and, if so, which codebook to use. The output of this process is a variable recParam of type TENSOR_FLOAT with dimensions equal to tensorDims.

[0254] TIFF0007783376000021.tif88146

[0255] Regarding the derivation process of ctxInc, which indicates the context or probability estimation to be used for the syntax element sig_flag,

[0256] The input to this process is the sig_flag decoded before the current sig_flag, the state value stateId, and the associated sign_flag (if any). If there is no sig_flag decoded before the current sig_flag, it is assumed to be 0. Also, if the sign_flag associated with a previously decoded sig_flag has not been decoded, it is assumed to be 0.

[0257] The output of this process is the variable ctxInc. The variable ctxInc is derived as follows: If sig_flag is 0, ctxIn c is stateId*3 to It is set. Otherwise, if sign_flag is equal to 0, ctxInc is set to stateId*3+1. Otherwise, ctxInc is set to stateId*3+2.

[0258] The above example illustrates the concept of encoding / decoding neural network parameters 13 to / from a data stream 14, where the neural network parameters 13 may relate to the weights of the neuron interconnections 11 of the neural network 10, e.g., the weights of a weight tensor. The decoding / encoding of the neural network parameters 13 is performed sequentially. See the for-next loop 1000, which cycles through the weights of the tensor with as many weights as the product of the number of weights per dimension of the tensor. The weights are scanned in a predetermined order, TensorIndex(dimensions, i, scan_order). For the current neural network parameters idx13', one of the two reconstruction level sets 52 is selected according to the quantization state stateId, which is continuously updated based on the quantization index 58 decoded from the data stream of the previous neural network parameters, at 1018 and 1020. In particular, a quantization index for the current neural network parameter idx is decoded from the data stream in 1012, and the quantization index indicates one reconstruction level of the selected reconstruction level set for the current neural network parameter 13′. The two reconstruction level sets are defined by duplication in 1016, followed by the addition of 1 or minus 1 depending on the quantization state index in 1018 and 1020. Here, in 1018 and 1020, the current neural network parameter 13′ is actually dequantized onto one reconstruction level of the selected reconstruction level set indicated by the quantization index QuantParam[idx] for the current neural network parameter 13′. A step size, stepSize, is used to parameterize the reconstruction level set in 3001-3003. Information about this predetermined quantization step size, stepSize, is derived from the data stream via the syntax element qp_value. The latter may be coded into the data stream for the entire tensor, the entire NN layer, or the entire NN, respectively.That is, the neural network 10 may include one or more neural network layers 10a, 10b, and for each neural network layer, information of a predetermined quantization step size (QP) for the respective neural network layer may be derived from the data stream 14, and then, for each neural network layer, multiple reconstruction level sets may be parameterized using the predetermined quantization step size derived for the respective neural network layer, to be used to dequantize the neural network parameters 13 belonging to the respective neural network layer.

[0259] The first reconstruction level set for stateId=0 here includes zero and even multiples of the predetermined quantization step size, and the second reconstruction level set for stateId=1 includes zero and odd multiples of the predetermined quantization step size (QP), as can be seen at 1018 and 1020. For each neural network parameter 13, at 1015-1021 an intermediate integer value QuantParam[idx] (IV) is derived according to the reconstruction level set selected for the respective neural network parameter 13 and the entropy-decoded quantization index QuantParam[idx] for the respective neural network parameter, and then, for each neural network parameter, the intermediate value for the respective neural network parameter is multiplied by the predetermined quantization step size for the respective neural network parameter at 4001.

[0260] The selection of one of the two reconstruction level sets (e.g., set 0, set 1) for the current neural network parameters 13' is made according to the LSB portion of the quantization indexes decoded from the data stream of previously decoded neural network parameters, as shown at 1014, and the transition table transitions from stateId to the next quantization state nextSt according to the LSB of QuantParam[idx] such that statId depends on the past sequence of already decoded quantization indexes 56. Thus, the state transition depends on the result of a binary function of the quantization indexes 56 decoded from the data stream of previously decoded neural network parameters, i.e., their parity. In other words, for a current neural network parameter, the selection of a reconstruction level set among the plurality of reconstruction level sets is performed by the state transition process by determining, for the current neural network parameter, a reconstruction level set among the plurality of reconstruction level sets according to the state statId associated with the current neural network parameter in 1018 and 1020, and updating, in 1014, the stateId for the subsequent neural network parameter for which the stateId is to be determined next, not necessarily the NN parameter to be next coded / decoded, according to the quantization index decoded from the data stream for the previous neural network parameter, i.e., the parameter for which the stateId was previously determined. For example, here, the current neural network parameter is used for updating to provide the stateId of the NN parameter to be next coded / decoded. The update in 1014 is performed using a binary function of the quantization index decoded from the data stream for the previous (current) neural network parameter, i.e., its parity. The state transition process is configured to transition among eight possible states. The transitions are performed via a table StateTransTab[][].In the state transition process, transitions are made between these eight possible states, and for the current neural network parameters, a set of reconstruction levels from the quantization set in 1018 and 1020 is determined according to the state stateId associated with the current neural network parameters. If the state belongs to the first half of the even possible states, i.e., an odd state, a first reconstruction level set from the two reconstruction level sets is determined. If the state belongs to the second half of the even possible states, i.e., an even state, a second reconstruction level set from the two reconstruction level sets is determined. The state statId is updated by a transition table, StateTransTab[][]. The transition table maps the combination of the state statId decoded from the data stream of the previous (current) neural network parameters and the parity of the quantization index (58), QuantParam[idx]&1, to another state associated with the subsequent neural network parameters.

[0261] The quantization indices of the current neural network parameters are encoded into and decoded from the data stream using arithmetic coding, using a probability model that depends on the reconstruction level set selected for the current neural network parameters, or more precisely, the quantization state stateId, i.e., the state for the current neural network parameters 13'. See the third parameter in the call to function int_param 1012. In particular, the quantization indices for the current neural network parameters can be encoded into and decoded from the data stream using binary arithmetic coding / decoding, using a probability model that corresponds to the state for the current neural network parameters for at least one bin of the binarization of the quantization indices. Here, the bin sig_flag among the binarization sig_flag, sign_flag (optional), abs_level_greater_x[j], abs_level_greater_x2[j], and abs_remainder is a significance bin that indicates whether the quantization indices (56) of the current neural network parameters are equal to zero. The probability model dependency includes selecting a context from a set of neural network parameter contexts using the dependency, where each context is configured to be associated with a predetermined probability model. Here, the context of sig_flag is selected using ctxInc as an index incrementer to index the context from the list of contexts, each of which is associated with a binary probability model. The model can be updated using the bins associated with the context. That is, the predetermined probability model associated with each context can be updated based on the quantization index arithmetically coded using the respective context.

[0262] Note that the probability model for sig_flag further depends on the quantization index of the previously decoded neural network parameter, i.e., the sig_flag of the previously decoded neural network parameter, and its sign_flag, which indicates its sign. More precisely, depending on the state stateId, a subset of probability models is preselected from among the context incrementer states 0...23, i.e., eight of {0...23}, including three consecutive contexts. Among the subset of probability models for sig_flag, the probability model for the current neural network parameter is selected according to the quantization index of the previously decoded neural network parameter (121), i.e., based on the sig_flag and sign_flag of the previous NN parameter. The subset preselected for the first value of stateId is inconsistent with the subset preselected for the other values ​​of stateId. The previous NN parameters for which sig_flag and sign_flag are used relate to parts of the neural network adjacent to the part to which the current neural network parameter relates.

[0263] Several embodiments have been described above. It should be noted that aspects and features of the embodiments may be used individually or in combination. Furthermore, aspects and features of the embodiments according to the first and second aspects of the present invention may be used in combination.

[0264] In a further embodiment, the device is configured such that the neural network parameters are associated with one of the reconstruction layers, e.g., an enhancement layer, in which the neural network 10 is represented. The device may be configured such that the neural network is reconfigurable by combining the neural network parameters of one or more other reconstruction layers, corresponding to, for example, those associated with common neuron interconnections, or more simply, those collocated in the matrix representation of the NN layer in a different representation layer.

[0265] For example, as described in this embodiment, features and aspects of the first and second aspects of the present invention may be combined, and any features of the dependent claims from the second aspect are intended to be transferable hereto to yield further embodiments.

[0266] Additionally, an apparatus according to an aspect of the present invention may be configured to encode the quantization indexes 56 for the current neural network parameters 13′ into the data stream 14 using arithmetic coding with a probability model that depends on the neural network parameters corresponding to the current neural network parameters.

[0267] Respectively, further embodiments may comprise an apparatus in which neural network parameters are associated with one of the reconstruction layers in which the neural network 10 is represented, e.g., an enhancement layer. The apparatus may be configured to reconstruct the neural network by combining, neural network parameter by neural network parameter, neural network parameters of one or more other reconstruction layers that correspond to, e.g., common neuronal interconnections, or more frankly, are collocated in a matrix representation of the NN layer in a different representation layer.

[0268] For example, as described in this embodiment, features and aspects of the first and second aspects of the present invention may be combined, and any features of the dependent claims from the second aspect are intended to be transferable hereto to yield further embodiments.

[0269] Additionally, an apparatus according to an aspect of the present invention may be configured to decode quantization indices 56 of current neural network parameters 13′ from data stream 14 using arithmetic coding, using a probability model that depends on corresponding neural network parameters corresponding to the current neural network parameters.

[0270] In other words, the neural network parameters of the reconstruction layer, e.g., the second neural network parameters as described, can be encoded / decoded and / or quantized / dequantized according to the concepts described with respect to Figures 3 and 5 and Figures 2 and 4, respectively.

[0271] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0272] The data stream of the present invention can be stored on a digital storage medium or can be transmitted over a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

[0273] Depending on particular implementation requirements, embodiments of the present invention can be implemented in hardware or in software. Implementation can be performed using a digital storage medium, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM or flash memory, having electronically readable control signals stored thereon, which cooperates (or can cooperate) with a programmable computer system to perform the respective methods.

[0274] Some embodiments according to the present invention comprise a data carrier having an electronically readable control signal, which data carrier is capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0275] Generally, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product is run on a computer, which program code can be stored on, for example, a machine-readable carrier.

[0276] Another embodiment comprises the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0277] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0278] A further embodiment of the inventive method is, therefore, a data carrier (or digital storage medium, or computer readable medium) having recorded thereon the computer program for performing one of the methods described herein.

[0279] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, The data stream or the sequence of signals may for example be adapted to be transmitted via a data communication connection, for example the Internet.

[0280] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0281] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0282] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0283] The above-described embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. It is therefore intended to be limited only by the scope of the appended claims, and not by the specific details presented by the description and explanation of the embodiments herein.

[0284] TIFF0007783376000022.tif187152

Claims

1. 1. An apparatus for decoding neural network parameters (13) defining a neural network (10) from a data stream (14), comprising: decoding quantization indices (56) for the current neural network parameters (13') from said data stream (14); and Dequantizing (62) the current neural network parameters (13'). configured to sequentially decode the neural network parameters (13) by The device comprises: performing the inverse quantization using dependent quantization based on a state transition process; decoding the quantization indexes (56) for the current neural network parameters (13') from the data stream (14) using arithmetic coding using a probability model that depends (122) on the state of the state transition process of the current neural network parameters (13') and on the quantization indexes of previously decoded neural network parameters; preselecting a subset of probability models from among a plurality of probability models depending on the state of the state transition process of the current neural network parameters (13'); and selecting the probability model for the current neural network parameters from among the subset of probability models depending on the quantization indexes (121) for previously decoded neural network parameters; selecting a probability model for the current neural network parameter from among a subset comprising first, second and third probability models depending on values ​​of the quantization indexes of previously decoded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameter relates; It is configured as follows: wherein if the previously decoded neural network parameters are less than zero, a first probability model is selected; a second probability model is selected if the previously decoded neural network parameters are greater than zero; If the previously decoded neural network parameters are equal to zero, the third probability model is selected. Device.

2. The device described in claim 1, configured to decode the quantization indexes (56) for the current neural network parameters (13') from the data stream (14) using arithmetic coding with a probability model that depends (123) on a set of reconstruction levels (48) for inverse quantizing the current neural network parameters (13').

3. An apparatus for encoding neural network parameters defining a neural network into a data stream, comprising: quantizing (64) the current neural network parameters (13'); and encoding quantization indices (56) for the current neural network parameters into the data stream (14); configured to sequentially encode the neural network parameters (13') by The device comprises: performing the quantization using dependent quantization based on a state transition process; encoding the quantization indexes (56) for the current neural network parameters (13') into the data stream (14) using arithmetic coding using a probability model that depends (122) on the state of the state transition process of the current neural network parameters (13') and on the quantization indexes of previously coded neural network parameters; preselecting a subset of probability models from among a plurality of probability models depending on the state of the state transition process of the current neural network parameters (13'); and selecting the probability model for the current neural network parameters from among the subset of probability models depending on the quantization indexes (121) for previously coded neural network parameters; selecting a probability model for the current neural network parameter from among a subset comprising first, second and third probability models depending on values ​​of the quantization indexes of previously coded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameter relates; It is configured as follows: wherein the first probability model is selected if the previously encoded neural network parameters are less than zero; if the previously encoded neural network parameters are greater than zero, the second probability model is selected; If the previously encoded neural network parameters are equal to zero, the third probability model is selected. Device.

4. A method (400) for decoding neural network parameters (13) defining a neural network (10) from a data stream (14), comprising: decoding (420) quantization indices (56) for the current neural network parameters (13') from said data stream (14); and Dequantizing (62) the current neural network parameters (13'). and sequentially decoding the neural network parameters (13) by The method further comprises: performing the inverse quantization using dependent quantization based on a state transition process; decoding the quantization indices (56) for the current neural network parameters (13') from the data stream (14) using arithmetic coding that uses a probability model that depends (122) on the state of the state transition process of the current neural network parameters (13') and on the quantization indices of previously decoded neural network parameters; preselecting a subset of probability models from among a plurality of probability models depending on the state of the state transition process of the current neural network parameters (13'), and selecting the probability model for the current neural network parameters from among the subset of probability models depending on the quantization indexes (121) for previously decoded neural network parameters; selecting a probability model for the current neural network parameter from a subset comprising first, second and third probability models depending on values ​​of the quantization indices of previously decoded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameter relates; further comprising wherein the first probability model is selected if the previously decoded neural network parameters are less than zero; if the previously decoded neural network parameters are greater than zero, the second probability model is selected; If the previously decoded neural network parameters are equal to zero, the third probability model is selected. method.

5. A method (500) for encoding neural network parameters defining a neural network into a data stream, comprising: quantizing (64) the current neural network parameters (13'); and encoding (530) quantization indices (56) for the current neural network parameters into the data stream (14); sequentially encoding the neural network parameters (13) by The method further comprises: performing the quantization using dependent quantization based on a state transition process; encoding the quantization indices (56) for the current neural network parameters (13') into the data stream (14) using arithmetic coding using a probability model that depends (122) on the state of the state transition process of the current neural network parameters (13') and on the quantization indices of previously coded neural network parameters; preselecting a subset of probability models from among a plurality of probability models depending on the state of the state transition process of the current neural network parameters (13'), and selecting the probability model for the current neural network parameters from among the subset of probability models depending on the quantization indexes (121) for previously coded neural network parameters; selecting a probability model for the current neural network parameters from a subset comprising first, second and third probability models in dependence on values ​​of the quantization indices of previously coded neural network parameters associated with portions of the neural network adjacent to the portion to which the current neural network parameters relate; Including, wherein the first probability model is selected if the previously encoded neural network parameters are less than zero; if the previously encoded neural network parameters are greater than zero, the second probability model is selected; If the previously encoded neural network parameters are equal to zero, the third probability model is selected. method.

6. A non-transitory digital storage medium storing a computer program for performing the method of claim 4 or 5 when the program is run on one or more computers.

Images