Apparatus, method and computer program for decoding neural network parameters using updated model, and apparatus, method and computer program for encoding neural network parameters

Abstract

To provide an apparatus, method, program and coded representation for decoding neural network parameters that define a neural network.SOLUTION: A method includes decoding (310) an updated model that defines modifications to one or more layers of the neural network, modifying (320) parameters of a base model of the neural network using the updated model to obtain an updated model, and evaluating (330) skip information that indicates whether a sequence of parameters of the updated model is zero.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] Embodiments according to the present invention relate to an apparatus, method and computer program for decoding neural network parameters and an apparatus, method and computer program for encoding neural network parameters using an update model.

[0002] A further embodiment according to the invention relates to a method for entropy coding of parameters of incremental updates of a neural network. [Background technology]

[0003] Neural networks (NNs) are used in a wide variety of applications. With ever-increasing computing power, NNs of increasing complexity and therefore increasing numbers of neural network parameters, e.g., weights, can be used.

[0004] Particularly computationally expensive training processes may be performed on dedicated training equipment, and updated neural network parameters may therefore need to be transmitted from such training devices to end-user devices.

[0005] Furthermore, a NN may be trained on multiple devices, e.g., multiple end user devices, and it may be advantageous to provide an aggregated version of the multiple training results. Therefore, each training result may need to be transmitted for subsequent aggregation, and the aggregated updated parameter set may be retransmitted to each of the devices. Summary of the Invention [Problem to be solved by the invention]

[0006] Therefore, there is a need for a concept for coding, e.g., encoding and / or decoding, neural network parameters that achieves a good compromise between efficiency, complexity, and computational cost of the concept. [Means for solving the problem]

[0007] This is achieved by the subject matter of the independent claims of the present application.

[0008] Further embodiments according to the invention are defined by the subject matter of the dependent claims of the present application.

[0009] An embodiment according to the invention includes an apparatus for decoding neural network parameters defining a neural network, which may optionally be configured to obtain, e.g., decode, parameters of a base model of the neural network, e.g., NB, that define one or more layers, e.g., a base layer, of the neural network.

[0010] Further, the apparatus is configured to decode an updated model, e.g., NU1 to NUK, that defines modifications of one or more layers, e.g., a base layer, of the neural network, and the apparatus is configured to modify parameters of a base model of the neural network using the updated model to obtain an updated model, e.g., designated as a "new model," that includes a new model layer LNkj.

[0011] Furthermore, the apparatus is configured to evaluate skip information, such as skip_row_flag and / or skip_column_flag, which indicates whether a sequence, such as a row, or a column, or a block, of parameters of the updated model is zero.

[0012] The inventors have recognized that neural network parameters can be efficiently transmitted using a base model and an updated model. In training a neural network, only a portion of the neural network parameters may be significantly changed compared to the base parameters, e.g., default or initial parameters. Therefore, the inventors have recognized that it may be advantageous to transmit only the change information, e.g., correction information in the form of an updated model. As an example, the base model may be stored in the decoder, thereby eliminating the need for transmission. Alternatively, such a base model may be transmitted, for example, only once.

[0013] Furthermore, the inventors have recognized that such an update model approach can be further improved by using skip information. The skip information can contain information about the structure of the update model with respect to the information distribution within the model. Thus, the skip information can indicate that a particular sequence of parameters of the update model does not contain update information, in other words, is zero. Therefore, instead of such a sequence of parameters, only the skip information need be transmitted.

[0014] Furthermore, based on the skip information, the evaluation and application of such parameters (eg, application to the base model) can be skipped at the decoder.

[0015] Additionally, it should be noted that the base model and update model may address neural network parameters of the entire neural network or of a layer thereof, or other subsets or portions of the neural network parameters of the neural network.

[0016] According to a further embodiment of the invention, the updated model describes difference values, and the apparatus is configured to combine the difference values ​​additively or subtractively with values ​​of parameters of the base model to obtain, e.g., corresponding values ​​of parameters of the updated model.

[0017] The inventors have recognized that additive or subtractive correction information can enable efficient parameter updates and even computationally inexpensive parameter adaptation.

[0018] According to a further embodiment of the present invention, the apparatus further comprises: a difference value or difference tensor L associated with the j-th layer of the neural network; Uk,j Let L be the base value parameter or base value tensor L, which represents the value of the parameter in the j-th layer of the neural network base model. Bj And, L Nkj =L Bj +L Uk,j (for all j, or for all j in which the updated model includes a layer), thereby obtaining an updated model value parameter or an updated model value tensor L, which represents the value of the parameter of the j-th layer of the updated model having model index k of the neural network. Nkj where, for example, "+" may define an element-wise addition operation between two tensors.

[0019] The inventors have recognized that neural network parameters can be efficiently represented using, for example, tensors, and further, that the combination of update information and base information in the form of tensors can be performed in a computationally inexpensive manner.

[0020] According to a further embodiment of the invention, the updated model describes scaling factor values, and the apparatus is configured to scale values ​​of parameters of the base model using the scaling factor values ​​to obtain, for example, corresponding values ​​of parameters of the updated model.

[0021] The inventors have recognized that using scaling factors allows parameter updates to be represented using fewer bits, thereby enabling such information to be transmitted using fewer transmission resources. Furthermore, application of the scaling factors can be performed at low computational cost.

[0022] According to a further embodiment of the invention, the apparatus further comprises: a scaling value or scaling tensor L associated with the j-th layer of the neural network; Uk,j Let L be the base value parameter or base value tensor L, which represents the value of the parameter in the j-th layer of the neural network base model. Bj And, L Nkj =L Bj L Uk,j (for all j, or for all j in which the updated model includes a layer), thereby obtaining an updated model value parameter or an updated model value tensor L, which represents the value of the parameter of the j-th layer of the updated model having model index k of the neural network. Nkj where, for example, "·" may define an element-wise multiplication operation between two tensors.

[0023] The inventors have recognized that a combination of tensor and multiplicative scaling can enable efficient neural network parameter updates.

[0024] According to a further embodiment of the invention, the updated model describes replacement values, and the apparatus is configured to use the replacement values ​​to replace values ​​of parameters of the base model, e.g. to obtain corresponding values ​​of parameters of the updated model.

[0025] The inventors have recognized that in some cases it may be more efficient to represent parameter updates by replacing values ​​of the base model with values ​​from the updated model, for example, instead of additive or multiplicative modifications.

[0026] According to a further embodiment of the present invention, the neural network parameters include weight values ​​that define weights of neuronal interconnections emanating from or leading towards the neurons.

[0027] Therefore, the weight values ​​of the NN can be decoded efficiently.

[0028] According to a further embodiment of the invention, the sequence of neural network parameters comprises weight values ​​associated with rows or columns of a matrix, for example a two-dimensional matrix or even a higher dimensional matrix.

[0029] The inventors have recognized that a row-by-row or column-by-column arrangement of a sequence of neural network parameters can allow for efficient processing of the sequence, including, for example, scanning a matrix.

[0030] According to a further embodiment of the invention, the skip information comprises a flag indicating, for example using a single bit, whether all parameters of a sequence, e.g., a row, of parameters of the updated model are zero or not.

[0031] The inventors have recognized that a flag dedicated to a sequence of neural network parameters can enable a decoder to make a separate assessment of how to efficiently handle the corresponding sequence. As an example, if a flag indicates that the corresponding parameter of the updated model is zero, processing of such a sequence may be skipped.

[0032] Therefore, according to a further embodiment of the invention, the device is configured to selectively skip decoding of sequences, e.g. rows, of parameters of the updated model depending on the skip information.

[0033] According to a further embodiment of the invention, the apparatus is adapted to selectively set values ​​of the sequence of parameters of the updated model to predetermined values, such as zero, depending on the skip information.

[0034] As an example, instead of a sequence of parameters, only skip information may be transmitted to the decoder, based on which the decoder may conclude that the neural network parameters of the sequence have predetermined values ​​and may therefore reconstruct these values.

[0035] According to a further embodiment of the present invention, the skip information may comprise an array of skip flags, e.g., using a single bit, indicating whether all parameters of a respective sequence, e.g., row, of parameters of the update model are zero, e.g., each flag may be associated with one sequence of parameters of the update model.

[0036] The inventors have recognized that using an array of skip flags can allow for providing compact information that addresses multiple sequences of neural network parameters in an update model.

[0037] Therefore, according to a further embodiment of the present invention, the apparatus is configured to selectively skip decoding of a plurality of respective sequences, e.g. rows, of parameters of the updated model (or for example of one respective sequence), depending on a respective skip flag associated with the respective sequence of parameters.

[0038] According to a further embodiment of the invention, the apparatus is configured to evaluate, e.g., decode, and use array size information, e.g., N, describing the number of entries in the array of skip flags. This can provide good flexibility and good efficiency.

[0039] According to a further embodiment of the present invention, the apparatus is configured to decode one or more skip flags using a context model, and the apparatus is configured to select a context model for decoding the one or more skip flags in dependence on one or more previously decoded symbols, such as in dependence on one or more previously decoded skip flags.

[0040] The inventors have recognized that using a context model can enable skip flags to be efficiently encoded and correspondingly decoded.

[0041] According to a further embodiment of the invention, the apparatus is configured to apply a single context model for decoding all skip flags associated with a layer of the neural network.

[0042] This can allow for simple decoding of the skip flag with low computational effort.

[0043] According to a further embodiment of the invention, the device is configured to select a context model for decoding the skip flag, e.g. from a set of two context models, depending on a previously decoded skip flag.

[0044] The inventors have recognized that the correlation between corresponding skip flags can be exploited through the selection of a context model to improve coding efficiency.

[0045] According to a further embodiment of the present invention, the device is configured to select a context model for decoding the skip flag, e.g. from among a set of two context models, depending on the value of a corresponding skip flag (e.g. associated with the same neural network parameters (e.g. defining the same neuronal interconnections) as the currently considered skip flag is associated with, e.g. in the same position, e.g. associated with the corresponding parameter sequence of the update model) in a previously decoded neural network model, e.g. a previously decoded update model or a previously decoded base model.

[0046] The inventors have realised that for decoding skip flags, one can take advantage of the correlation of previously decoded neural networks with corresponding skip flags by selecting the context model accordingly.

[0047] According to a further embodiment of the present invention, the device is configured to select a set of selectable context models for decoding the skip flag, e.g. from among a set of two context models, depending on the value of a corresponding skip flag (e.g. associated with the same neural network parameters (e.g. defining the same neuronal interconnections) as the currently considered skip flag is associated with, e.g. in the same position, e.g. associated with the corresponding parameter sequence of the update model) in a previously decoded neural network model, e.g. a previously decoded update model or a previously decoded base model.

[0048] The present inventors have recognized that, to improve coding efficiency, a set of context models can be used to decode and correspondingly encode skip flags. Furthermore, the present inventors have recognized that correlations between previously decoded neural network models and currently decoded neural network models can be utilized for selecting such a set of context models.

[0049] According to a further embodiment of the present invention, the device is configured to select a set of selectable context models for decoding skip flags, for example from among a set of two context models, depending on the presence of a corresponding layer in a previously decoded neural network model, for example in a previously decoded update model or a previously decoded base model, where optionally the previously decoded neural network model does not include a certain layer, but this certain layer may be present in the currently considered layer. This may be true, for example, when the topology of the neural network is changed, for example, by adding a layer. This may also be true, for example, when a certain layer of the neural network was not changed in a previous update and therefore information about this certain layer was not included in the previous update.

[0050] As an example, the decoder may be configured to evaluate whether there is a correlation between corresponding skip flags. The absence of a corresponding layer may indicate that there is no skip flag in a previously decoded neural network model that may correspond to the current skip flag to be decoded. This information can therefore be used for selecting a set of context models.

[0051] According to a further embodiment of the invention, the apparatus is configured to select a context model from the selected set of context models in dependence on one or more previously decoded symbols of the currently decoded update model, such as in dependence on one or more previously decoded skip flags.

[0052] Therefore, it should be noted that, according to an embodiment, several decisions, and therefore degrees of freedom, can be incorporated. Information correlations between a previously decoded neural network model and a currently decoded model, and also between a previously decoded symbol of a currently decoded updated model and a currently decoded symbol, may be utilized. These correlations can then be used to first select a set of context models, and then select a context model from the set of context models, and therefore decode the current symbol. In short, the inventors have recognized that several layers of information correlations can be utilized to improve coding efficiency.

[0053] A further embodiment according to the invention comprises an apparatus for decoding neural network parameters defining a neural network, optionally configured to obtain, for example, decoding, parameters of a base model, e.g., NB, of the neural network, which define one or more layers, e.g., a base layer, of the neural network.

[0054] Furthermore, the device is configured to decode a current update model, e.g., NU1 or NUK, which defines modifications of one or more layers, e.g., the base layer, of the neural network, e.g., LB,j, or modifications of one or more hidden layers or of the neural network, e.g., LUK-1,j.

[0055] Further, the apparatus is configured to modify parameters of the base model of the neural network, e.g., parameters of LB,j, using a current update model, e.g., NU1 or NUK, or intermediate parameters, e.g., parameters of LUK-1,j, derived from the base model of the neural network, using one or more intermediate update models, e.g., using NU1 to NUK-1, to obtain an updated model, e.g., designated as a "new model" including a new model layer LN1,j or LNK,j.

[0056] Additionally, the apparatus is configured to entropy decode one or more parameters of the current update model, for example using context-adaptive binary arithmetic coding, and the apparatus is configured to adapt a context used for entropy decoding of the one or more parameters of the current update model depending on one or more previously decoded parameters of the base model and / or depending on one or more previously decoded parameters of the intermediate update model, for example to exploit correlations between the current update model and the base model and / or correlations between the current update model and the intermediate update models.

[0057] The inventors have recognized that the correlation between a previously decoded neural network model, e.g., a base model or an intermediate model, and a current update model, which is the neural network model currently being decoded, can be utilized for adaptation of a context model, which is used for entropy decoding of one or more parameters of the current update model.

[0058] As an example, in an iterative training procedure for a neural network, an updated, e.g., improved, model can be obtained, e.g., after each training, based on a base model (e.g., including or associated with default or initial neural network parameters). The inventors have recognized that changes or modifications of neural network parameters, e.g., during a training cycle, can be correlated. There may be several sets of neural network parameters that can be correlated through previous and subsequent training. Therefore, by utilizing such correlations, coding efficiency can be improved. An intermediate model may represent an updated neural network between a base model, e.g., an initial model, and a current model, e.g., associated with the most recent training cycle.

[0059] Therefore, the inventors have recognized that context adaptation for decoding and corresponding encoding can be advantageous in order to incorporate information about such correlations.

[0060] According to a further embodiment of the invention, the device is configured to decode a quantized and binarized representation of one or more parameters of the current updated model, such as the difference values ​​L k,j or the scaling factor values ​​L k,j or the replacement values ​​L k,j, using context-based entropy decoding.

[0061] The inventors have recognized that using a quantized and binarized representation for the parameters of the current update model can further improve the coding efficiency of the inventive approach. As an example, using a binary representation can keep complexity low and allow simple probability modeling for the more frequently used bits of any symbol.

[0062] According to a further embodiment of the invention, the device is configured for entropy decoding at least one significance bin associated with a currently considered parameter value of a current updated model, the significance bin describing whether a quantization index of the currently considered parameter value is equal to zero or not.

[0063] This can allow saving bits used for encoding and / or decoding neural network parameters: if a significant bin indicates that the parameter is zero, then further bins may be unnecessary and therefore can be used for other information.

[0064] According to a further embodiment of the invention, the device is configured to entropy decode at least one code bin associated with a currently considered parameter value of a current updated model, the code bin describing whether a quantization index of the currently considered parameter value is greater than or less than zero.

[0065] The inventors have recognised that the use of sign bins allows for providing compact, low complexity information about the signs of parameter values.

[0066] According to a further embodiment of the invention, the device is configured for entropy decoding a unary sequence associated with a currently considered parameter value of the current updated model, the bins of the unary sequence describing whether the absolute value of a quantization index of the currently considered parameter value is greater than a respective bin weight, e.g. X, or not.

[0067] The inventors have recognised that the use of such unary arrays allows for an efficient representation of the parameter values ​​currently under consideration.

[0068] According to a further embodiment of the invention, the device is configured to entropy decode one or more bins greater than X indicating whether the absolute value of the quantization index of the currently considered parameter value is greater than X, where X is an integer greater than zero.

[0069] The inventors have recognized that spaced subsequent indices for the quantization indexes allow for efficient representation of their absolute values.

[0070] According to a further embodiment of the present invention, the device is configured to select a context model, e.g. from a set of two context models, for decoding one or more bins of quantization indices of the currently considered parameter value depending on the value of a corresponding previously decoded parameter value (e.g., associated with the same neural network parameters (e.g., defining the same neuronal interconnections) as the currently considered parameter value is associated with, e.g., at the same position, e.g., associated with the corresponding parameter sequence of the update model) in a previously decoded neural network model, e.g., a previously decoded update model or a previously decoded base model, e.g., in a corresponding layer of the previously decoded base model or the previously decoded update model.

[0071] The inventors have recognized that correlations between a currently decoded model and previously decoded models can be utilized. The inventors have recognized that such correlations can be advantageously utilized, for example, to provide improved coding efficiency, by selecting a context model for decoding quantization indices for a currently considered parameter value depending on the values ​​of corresponding previously decoded parameter values ​​in previously decoded neural network models. The inventors have recognized that, for example, the correlation of corresponding quantization indices for corresponding parameter values ​​of subsequent neural network trainings can be incorporated into the selection of a context model.

[0072] According to a further embodiment of the present invention, the apparatus is configured to select a set of selectable context models, e.g., from among a set of two context models, for decoding one or more bins of quantization indices of the currently considered parameter value depending on the value of a previously decoded corresponding (e.g., co-located, e.g., associated with a sequence of corresponding parameters of the update model) parameter value in a previously decoded neural network model, e.g., a previously decoded update model or a previously decoded base model, e.g., in a corresponding layer of the previously decoded base model or the previously decoded update model, which may be associated with the same neural network parameter (e.g., defining the same neuronal interconnection) as the currently considered parameter value is associated with, e.g., which may be associated with the same neural network parameter (e.g., defining the same neuronal interconnection) between two given neurons, e.g., the currently considered parameter value.

[0073] The inventors have recognized that the correlation between the currently decoded model and previously decoded models can be further utilized, e.g., for the selection of a set of context models, and thus, as an example, multiple context models, for a bin of quantization indexes, e.g., for improved coding efficiency. The use of the full set of context models can make it possible to implement another degree of freedom, allowing for better context selection and therefore improved coding efficiency.

[0074] According to a further embodiment of the invention, the device is configured to select a context model for decoding one or more bins of quantization indices of the currently considered parameter value depending on the absolute values ​​of corresponding previously decoded parameter values ​​in a previously decoded neural network model.

[0075] Alternatively, the apparatus is configured to select a set of context models for decoding one or more bins of quantization indices of the currently considered parameter value depending on the absolute values ​​of previously decoded corresponding parameter values ​​in previously decoded neural network models.

[0076] The inventors have recognized that information correlation can optionally be utilized based on the absolute values ​​of corresponding previously decoded parameter values ​​in previously decoded neural network models. Therefore, a context model or set of context models may be selected, where the selected set or context model may include contexts that well, or for example, best, represent the correlation of quantization index bins based on the corresponding previously decoded absolute values.

[0077] According to a further embodiment of the present invention, the apparatus is configured to compare corresponding previously decoded parameter values ​​in the previously decoded neural network model with one or more threshold values, e.g. T1, T2, etc.

[0078] Furthermore, the device is configured to select a context model for decoding one or more bins of quantization indices of the currently considered parameter value depending on the result of the comparison.

[0079] Alternatively, the device is configured to select a set of context models for decoding one or more bins of quantization indices of the currently considered parameter value depending on the result of the comparison, e.g., the first set is selected if the corresponding or co-located parameters are lower than a first threshold T1, the second set is selected if the corresponding or co-located parameters are equal to or greater than the first threshold T1, and the third set is selected if the corresponding or co-located parameters are equal to or greater than a threshold T2.

[0080] The inventors have recognized that thresholds can enable a computationally inexpensive way of selecting a context model or set of context models. Using multiple thresholds can, for example, enable providing differentiated information about which context model or set of context models to choose or select.

[0081] According to a further embodiment of the present invention, the apparatus is configured to compare corresponding previously decoded parameter values ​​in the previously decoded neural network model with a single threshold value, e.g., T1 (or a single threshold value).

[0082] Furthermore, the device is configured to select a context model for decoding one or more bins of quantization indices of the parameter value currently under consideration depending on the result of the comparison with the single threshold.

[0083] Alternatively, the device is configured to select a set of context models for decoding one or more bins of quantization indices of the currently considered parameter value depending on the result of the comparison with a single threshold.

[0084] The inventors have realised that the use of a single threshold may make it possible to provide a good compromise between the amount of information extracted from or used based on corresponding previously decoded parameter values ​​and computational cost.

[0085] According to a further embodiment of the present invention, the apparatus is configured to compare the absolute values ​​of corresponding previously decoded parameter values ​​in the previously decoded neural network model with one or more threshold values, e.g. T1, T2, etc.

[0086] Furthermore, the device is configured to select a context model for decoding one or more bins of quantization indices of the currently considered parameter value depending on the result of the comparison.

[0087] Alternatively, the device is configured to select a set of context models for decoding one or more bins of quantization indices of the currently considered parameter value depending on the result of the comparison, e.g., the first set is selected if the corresponding or co-located parameters are lower than a first threshold T1, the second set is selected if the corresponding or co-located parameters are equal to or greater than the first threshold T1, and the third set is selected if the corresponding or co-located parameters are equal to or greater than a threshold T2.

[0088] The inventors have realised that an intelligent selection of a context model or set of context models can be performed based on a computationally inexpensive comparison of the absolute values ​​of corresponding previously decoded parameter values ​​with one or more thresholds.

[0089] According to a further embodiment of the present invention, the device is configured to entropy decode at least one significance bin associated with a currently considered parameter value of a current updated model, the significance bin describing whether a quantization index of the currently considered parameter value is equal to zero or not, and to select a context for entropy decoding of the at least one significance bin or a set of contexts for entropy decoding of the at least one significance bin depending on a value, such as an absolute value or a signed value, of a previously decoded corresponding (e.g., a "corresponding" parameter value in a previously decoded neural network model, which may be associated with the same neural network parameter (e.g., defining the same neuronal interconnection) as the currently considered parameter value is associated with, e.g., a parameter value that is at the same position, e.g., associated with a sequence of corresponding parameters of the updated model), e.g., a "corresponding" parameter value associated with or defining the same neuronal interconnection between two given neurons, such as the currently considered parameter value, wherein, for example, the corresponding parameter value is compared with a single threshold value to select the context or the set of contexts, or, for example, the corresponding parameter value is compared with two threshold values, e.g., T1=1 and T2=2, to select the context or the set of contexts.

[0090] The inventors have recognized that the use of significance bins can improve coding efficiency. When a parameter value is zero, only the significance bin needs to be transmitted to indicate this. Thus, for example, for an updated model that includes only a few changed values ​​for a fraction of the neural network parameters of the base model or intermediate models, significance bins can reduce the amount of bits that may need to be transmitted to represent the updated model. Furthermore, significance bins can be efficiently coded and therefore decoded using a context model, and the inventors have recognized that the selection of the context can be performed based on corresponding previously decoded parameter values ​​in a previously decoded neural network model to take advantage of the correlation between the current updated model and the previously decoded model.

[0091] According to a further embodiment of the present invention, the apparatus is configured to entropy decode at least one code bin associated with a currently considered parameter value of a current updated model, the code bin describing whether a quantization index of the currently considered parameter value is greater than or less than zero, and to select a context for entropy decoding of the at least one code bin or a set of contexts for entropy decoding of the at least one code bin depending on the value of a previously decoded corresponding (e.g., associated with the same neural network parameter (e.g., defining the same neuronal interconnection) as the currently considered parameter value is associated with, e.g., co-located, e.g., associated with the corresponding parameter sequence of the updated model) parameter value in a previously decoded neural network model, e.g., a "corresponding" parameter value associated with or defining the same neuronal interconnection between two given neurons such as the currently considered parameter value, wherein, for example, the corresponding parameter value is compared to a single threshold value for selecting the context or the set of contexts, or, for example, the corresponding parameter value is compared to two threshold values, e.g., T1=0 and T2=1, for selecting the context or the set of contexts.

[0092] As explained above, the inventors have recognized that the use of code bins can, for example, enable providing compact, low-complexity information about the signs of parameter values. Furthermore, code bins can be effectively encoded and therefore decoded using, for example, a context model, and the inventors have recognized that the selection of the context can be performed based on parameter values ​​in a previously decoded neural network model, for example, to exploit correlations between a current updated model and a previously decoded model.

[0093] According to a further embodiment of the present invention, the apparatus is configured to entropy decode one or more bins greater than X indicating whether the absolute value of a quantization index of a currently considered parameter value is greater than X, where X is an integer greater than zero, and to select a context for entropy decoding of the at least one bin greater than X or a set of contexts for entropy decoding of the at least one bin greater than X depending on a value, e.g., absolute value or signed value, of a previously decoded corresponding (e.g., associated with the same neural network parameter (e.g., defining the same neuronal interconnection) as the currently considered parameter value is associated with, e.g., co-located, e.g., associated with the corresponding parameter sequence of the updated model) parameter value in a previously decoded neural network model, e.g., a "corresponding" parameter value associated with or defining the same neuronal interconnection between two given neurons such as the currently considered parameter value, wherein for example the corresponding parameter value is compared with a single threshold, e.g., T1=X, to select the context or the set of contexts.

[0094] As explained above, the inventors have recognized that spaced subsequent indices for the quantization indexes allow for efficient representation of their absolute values. Furthermore, bins larger than X can be efficiently coded and therefore decoded using context models, and the inventors have recognized that the selection of the context can be performed based on corresponding previously decoded parameter values ​​in a previously decoded neural network model, for example, to exploit correlations between the current updated model and the previously decoded model.

[0095] According to a further embodiment of the invention, the apparatus is configured to select a context model from among the selected set of context models depending on one or more previously decoded bins or parameters of the current updated model (or, for example, depending on the current updated model).

[0096] We also consider the correlation of parameters or bins in the current updated model as, for example, to select or choose a context model for efficient encoding and decoding respectively. I realized I could use it.

[0097] A further embodiment according to the invention comprises an apparatus for encoding neural network parameters defining a neural network, the apparatus optionally being configured to obtain and / or provide, e.g., encode, parameters of a base model, e.g., NB, of the neural network that define one or more layers, e.g., a base layer, of the neural network.

[0098] Further, the device is configured to encode an update model, e.g., NU1 to NUK, that defines modifications of one or more layers, e.g., a base layer, of the neural network. Further, for example, the device is configured to provide an update model, e.g., such that the update model enables a decoder, e.g., a decoding device as defined above, to modify parameters of a base model of the neural network using the update model to obtain an updated model, e.g., designated as a "new model", including a new model layer LNkj.

[0099] Further, the apparatus is configured to provide and / or determine and / or encode skip information, such as skip_row_flag and / or skip_column_flag, indicating whether a sequence, such as a row, or a column, or a block, of parameters of the updated model is zero or not.

[0100] The encoders described above may be based on the same considerations as the decoders described above, and may include all (e.g., all corresponding or all similar) features and functionality described with respect to the decoders.

[0101] According to a further embodiment of the invention, the update model describes difference values ​​that allow the decoder to combine them additively or subtractively with values ​​of parameters of the base model to obtain, e.g., corresponding values ​​of parameters of the updated model.

[0102] According to a further embodiment of the invention, the apparatus is configured to determine difference values ​​as or using the differences between the values ​​of the parameters of the updated model and, for example, the corresponding values ​​of the parameters of the base model.

[0103] According to a further embodiment of the present invention, the apparatus further comprises: a difference value or difference tensor L associated with the j-th layer of the neural network; Uk,j Let L be the base value parameter or base value tensor L, which represents the value of the parameter in the j-th layer of the neural network base model. Bj With, L Nkj =L Bj +L Uk,j (for all j, or for all j whose update model contains a layer) Uk,j represents the value of the parameter of the j-th layer of the updated model with model index k of the neural network. Nkj where, for example, "+" may define an element-wise addition operation between two tensors.

[0104] According to a further embodiment of the invention, the updated model describes scaling factor values, and the apparatus is configured to provide the scaling factor values ​​such that scaling of values ​​of parameters of the base model using the scaling factor values ​​results in, for example, corresponding values ​​of parameters of the updated model.

[0105] According to a further embodiment of the invention, the apparatus is configured to determine a scaling factor value as a scaling factor between a value of a parameter of the updated model and, e.g., a corresponding value of a parameter of the base model.

[0106] According to a further embodiment of the invention, the apparatus further comprises: a scaling value or scaling tensor L associated with the j-th layer of the neural network; Uk,j Let L be the base value parameter or base value tensor L, which represents the value of the parameter in the j-th layer of the neural network base model. Bj With, L Nkj =L Bj L Uk,j The combination of scaling values ​​or scaling tensors according to (for all j, or for all j in which the updated model includes a layer) represents the parameters of the j-th layer of the updated model with model index k of the neural network, and is the updated model value parameter or updated model value tensor L Nkj where, for example, "·" may define an element-wise addition operation between two tensors.

[0107] According to a further embodiment of the invention, the updated model describes replacement values, and the device is configured to provide the replacement values ​​such that replacement of values ​​of parameters of the base model using the replacement values ​​allows obtaining e.g. corresponding values ​​of parameters of the updated model.

[0108] According to a further embodiment of the invention, the apparatus is configured to determine a replacement value.

[0109] According to a further embodiment of the present invention, the neural network parameters include weight values ​​that define weights of neuronal interconnections emanating from or leading towards the neurons.

[0110] According to a further embodiment of the invention, the sequence of neural network parameters comprises weight values ​​associated with rows or columns of a matrix, for example a two-dimensional matrix or even a higher dimensional matrix.

[0111] According to a further embodiment of the invention, the skip information comprises a flag indicating, for example using a single bit, whether all parameters of a sequence, e.g., a row, of parameters of the updated model are zero or not.

[0112] According to a further embodiment of the invention, the apparatus is configured to provide skip information signaling the skipping of the decoding of a sequence, e.g., a row, of parameters of the updated model.

[0113] According to a further embodiment of the invention, the apparatus is configured to provide skip information comprising information whether the sequence of parameters of the updated model has a predetermined value, such as zero.

[0114] According to a further embodiment of the present invention, the skip information may comprise an array of skip flags, e.g., using a single bit, indicating whether all parameters of a respective sequence, e.g., row, of parameters of the update model are zero, e.g., each flag may be associated with one sequence of parameters of the update model.

[0115] According to a further embodiment of the invention, the apparatus is configured to provide a skip flag associated with each sequence of parameters, e.g., rows, for signaling a skip of decoding of the respective sequence of parameters of the updated model.

[0116] According to a further embodiment of the present invention, the apparatus is configured to provide, e.g., encode and / or determine, array size information, e.g., N, describing the number of entries in the array of skip flags.

[0117] According to a further embodiment of the present invention, the apparatus is configured to encode one or more skip flags using a context model, and the apparatus is configured to select a context model for encoding the one or more skip flags in dependence on one or more previously encoded symbols, such as in dependence on one or more previously encoded skip flags.

[0118] According to a further embodiment of the invention, the apparatus is configured to apply a single context model for the encoding of all skip flags associated with a layer of the neural network.

[0119] According to a further embodiment of the invention, the device is configured to select a context model for encoding the skip flag, e.g. from a set of two context models, depending on a previously encoded skip flag.

[0120] According to a further embodiment of the present invention, the device is configured to select a context model for encoding the skip flag, e.g. from a set of two context models, depending on the value of a corresponding skip flag (e.g. associated with the same neural network parameters (defining the same neuronal interconnections) as the currently considered skip flag is associated with, e.g. in the same position, e.g. associated with the corresponding parameter sequence of the update model) in a previously coded neural network model, e.g. a previously coded update model or a previously coded base model.

[0121] According to a further embodiment of the present invention, the device is configured to select a set of selectable context models for encoding the skip flag, e.g. from among a set of two context models, depending on the value of a corresponding skip flag (e.g. associated with the same neural network parameters (defining the same neuronal interconnections) as the currently considered skip flag is associated with, e.g. in the same position, e.g. associated with the corresponding parameter sequence of the update model) in a previously coded neural network model, e.g. a previously coded update model or a previously coded base model.

[0122] According to a further embodiment of the present invention, the device is configured to select a set of selectable context models for encoding skip flags, for example, from among a set of two context models, depending on the presence of a corresponding layer in a previously coded neural network model, for example, in a previously coded update model or a previously coded base model, where, for example, the previously coded neural network model does not include a certain layer, but this certain layer may be present in the currently considered layer. This may be true, for example, when the topology of the neural network is changed, for example, by adding a layer. This may also be true when a certain layer of the neural network was not changed in a previous update, and therefore information about this certain layer was not included in the previous update.

[0123] According to a further embodiment of the invention, the apparatus is configured to select a context model from the selected set of context models in dependence on one or more previously coded symbols of the currently coded update model, such as in dependence on one or more previously coded skip flags.

[0124] A further embodiment according to the invention comprises an apparatus for encoding neural network parameters defining a neural network, optionally configured to obtain and / or provide, e.g., encode, parameters of a base model, e.g., NB, of the neural network that define one or more layers, e.g., a base layer, of the neural network.

[0125] Furthermore, the device is configured to encode a current update model, e.g., NU1 or NUK, that defines modifications of one or more layers, e.g., the base layer, of the neural network, e.g., LB,j, or modifications of one or more hidden layers or layers, e.g., LUK-1,j, of the neural network.

[0126] Further, for example, the device is configured to provide an update model that enables a decoder, for example a decoding device as defined above, to modify parameters of the base model of the neural network, for example the parameters of LB,j, using a current update model, for example NU1 or NUK, or intermediate parameters, for example the parameters of LUK-1,j, derived from the base model of the neural network, using one or more intermediate update models, for example using NU1 to NUK-1, to obtain an updated model, for example designated as a "new model", where the updated model includes a new model layer LN1,j or LNK,j.

[0127] Additionally, the apparatus is configured to entropy encode one or more parameters of the current update model, e.g., using context-adaptive binary arithmetic coding, and the apparatus is configured to adapt a context used for entropy encoding of the one or more parameters of the current update model depending on one or more previously encoded parameters of the base model and / or depending on one (or, e.g., more) previously encoded parameters of the intermediate update model, e.g., to exploit a correlation between the current update model and the base model and / or a correlation between the current update model and the intermediate update model.

[0128] An encoder as described above may be based on the same considerations as the decoder described above, whereby the encoder may comprise all (e.g. all corresponding or all similar) features and functions that are also described with respect to the decoder.

[0129] According to a further embodiment of the invention, the device is configured to encode a quantized and binarized representation of one or more parameters of the current updated model, such as the difference values ​​L k,j or the scaling factor values ​​L k,j or the replacement values ​​L k,j, using context-based entropy coding.

[0130] According to a further embodiment of the invention, the device is configured for entropy coding at least one significance bin associated with a currently considered parameter value of the current updated model, the significance bin describing whether a quantization index of the currently considered parameter value is equal to zero or not.

[0131] According to a further embodiment of the invention, the device is configured for entropy coding at least one code bin associated with a currently considered parameter value of the current updated model, the code bin describing whether a quantization index of the currently considered parameter value is greater than or less than zero.

[0132] According to a further embodiment of the invention, the device is configured for entropy coding a unary sequence associated with a currently considered parameter value of the current updated model, wherein a bin of the unary sequence describes whether the absolute value of a quantization index of the currently considered parameter value is greater than a respective bin weight, e.g. X, or not.

[0133] According to a further embodiment of the invention, the device is configured for entropy coding one or more bins greater than X indicating whether the absolute value of the quantization index of the currently considered parameter value is greater than X, where X is an integer greater than zero.

[0134] According to a further embodiment of the present invention, the device is configured to select a context model, e.g. from a set of two context models, for encoding one or more bins of quantization indices of the currently considered parameter value depending on the value of a corresponding previously coded parameter value (associated with the same neural network parameters (defining the same neuronal interconnections) as the currently considered parameter value is associated with, e.g. at the same position, e.g. associated with the corresponding parameter sequence of the update model) in a previously coded neural network model, e.g. a previously coded update model or a previously decoded base model, e.g. in a corresponding layer of the previously coded base model or the previously coded update model.

[0135] According to a further embodiment of the present invention, the apparatus is configured to select a set of selectable context models, for example from among a set of two context models, for encoding one or more bins of quantization indices of the currently considered parameter value, depending on a corresponding (e.g., a parameter value associated with the same neural network parameter (defining the same neuronal interconnection) as the currently considered skip flag is associated with, e.g., at the same position, e.g., associated with the corresponding parameter sequence of the update model) previously coded parameter value in a previously coded neural network model, e.g., a previously coded update model or a previously coded base model, e.g., in a corresponding layer of the previously coded base model or the previously coded update model, e.g., a “corresponding” parameter value associated with or defining the same neuronal interconnection between two given neurons, e.g., the currently considered parameter value.

[0136] According to a further embodiment of the invention, the device is configured to select a context model for encoding one or more bins of quantization indices of the currently considered parameter value depending on the absolute values ​​of previously encoded corresponding parameter values ​​in a previously encoded neural network model.

[0137] Alternatively, the apparatus is configured to select a set of context models for encoding one or more bins of quantization indices of the currently considered parameter value depending on the absolute values ​​of previously encoded corresponding parameter values ​​in previously encoded neural network models.

[0138] According to a further embodiment of the invention, the device is configured to compare previously coded corresponding parameter values ​​in a previously coded neural network model with one or more threshold values, such as T1, T2, etc., and the device is configured to select a context model for coding one or more bins of quantization indices of the currently considered parameter value depending on the result of the comparison.

[0139] Alternatively, the device is configured to select a set of context models for encoding one or more bins of quantization indices of the currently considered parameter value depending on the result of the comparison, e.g., the first set being selected if the corresponding or co-located parameters are below a first threshold T1, the second set being selected if the corresponding or co-located parameters are equal to or greater than the first threshold T1, and the third set being selected if the corresponding or co-located parameters are equal to or greater than a threshold T2.

[0140] According to a further embodiment of the present invention, the apparatus is configured to compare corresponding previously coded parameter values ​​in the previously coded neural network model with a single threshold value, e.g., T1.

[0141] Furthermore, the device is configured to select a context model for encoding one or more bins of quantization indices of the currently considered parameter value depending on the result of the comparison with the single threshold.

[0142] Alternatively, the device is configured to select a set of context models for encoding one or more bins of quantization indices of the currently considered parameter value depending on the result of the comparison with a single threshold.

[0143] According to a further embodiment of the present invention, the apparatus is configured to compare the absolute values ​​of corresponding previously coded parameter values ​​in the previously coded neural network model with one or more threshold values, e.g. T1, T2, etc.

[0144] Furthermore, the device is configured to select a context model for encoding one or more bins of quantization indices of the currently considered parameter value depending on the result of the comparison.

[0145] Alternatively, the device is configured to select a set of context models for encoding one or more bins of quantization indices of the currently considered parameter value depending on the result of the comparison, e.g., the first set being selected if the corresponding or co-located parameters are below a first threshold T1, the second set being selected if the corresponding or co-located parameters are equal to or greater than the first threshold T1, and the third set being selected if the corresponding or co-located parameters are equal to or greater than a threshold T2.

[0146] According to a further embodiment of the present invention, the device is configured to entropy code at least one significance bin associated with a currently considered parameter value of a current updated model, the significance bin describing whether a quantization index of the currently considered parameter value is equal to zero or not, and to select a context for entropy coding of the at least one significance bin or a set of contexts for entropy coding of the at least one significance bin depending on a value, such as an absolute value or a signed value, of a previously coded corresponding (e.g., a "corresponding" parameter value in a previously coded neural network model, which may be associated with the same neural network parameter (e.g., defining the same neuronal interconnection) as the currently considered parameter value is associated with, e.g., a parameter value that is at the same position, e.g., associated with a sequence of corresponding parameters in the updated model), e.g., a "corresponding" parameter value associated with or defining the same neuronal interconnection between two given neurons, such as the currently considered parameter value, wherein, for example, the corresponding parameter value is compared with a single threshold value to select the context or the set of contexts, or, for example, the corresponding parameter value is compared with two threshold values, e.g., T1=1 and T2=2, to select the context or the set of contexts.

[0147] According to a further embodiment of the present invention, the device is configured to entropy encode at least one code bin associated with a currently considered parameter value of a current updated model, the code bin describing whether a quantization index of the currently considered parameter value is greater than or less than zero, and to select a context for entropy encoding of the at least one code bin or a set of contexts for entropy encoding of the at least one code bin depending on the value of a previously encoded corresponding (e.g., a "corresponding" parameter value in a previously encoded neural network model, which may be associated with the same neural network parameter (e.g., defining the same neuronal interconnection) as the currently considered parameter value is associated with, e.g., a "corresponding" parameter value associated with or defining the same neuronal interconnection between two given neurons, e.g., the currently considered parameter value, wherein, for example, the corresponding parameter value is compared with a single threshold value for selecting the context or the set of contexts, or, for example, the corresponding parameter value is compared with two threshold values, e.g., T1=0 and T2=1, for selecting the context or the set of contexts.

[0148] According to a further embodiment of the present invention, the apparatus is configured to entropy encode one or more bins greater than X indicating whether the absolute value of a quantization index of a currently considered parameter value is greater than X, where X is an integer greater than zero, and to select a context for entropy encoding of the at least one bin greater than X or a set of contexts for entropy encoding of the at least one bin greater than X depending on a value, such as an absolute value or a signed value, of a previously encoded corresponding (e.g., associated with the same neural network parameter (e.g., defining the same neuronal interconnection) as the currently considered parameter value is associated with, e.g., co-located, e.g., associated with the corresponding parameter sequence of the updated model) parameter value in a previously encoded neural network model, e.g., a "corresponding" parameter value associated with or defining the same neuronal interconnection between two given neurons such as the currently considered parameter value, wherein for example the corresponding parameter value is compared with a single threshold, e.g., T=X, to select the context or the set of contexts.

[0149] According to a further embodiment of the invention, the apparatus is configured to select a context model from the selected set of context models depending on one or more previously coded bins or parameters (e.g. of the current updated model) or the current updated model.

[0150] A further embodiment according to the invention includes a method for decoding neural network parameters defining a neural network, the method optionally including obtaining, e.g., decoding, parameters of a base model, e.g., NB, of the neural network, which define one or more layers, e.g., a base layer, of the neural network. The method further includes decoding an updated model, e.g., NU1-NUK, which defines modifications to one or more layers, e.g., a base layer, of the neural network, modifying parameters of the base model of the neural network using the updated model to obtain an updated model, e.g., designated as a "new model," including a new model layer LNkj, and evaluating skip information, e.g., skip_row_flag and / or skip_column_flag, which indicate whether a sequence, e.g., a row, or a column, or a block, of the parameters of the updated model is zero.

[0151] A further embodiment according to the invention includes a method for decoding neural network parameters defining a neural network, the method optionally including obtaining, e.g., decoding, parameters of a base model, e.g., NB, of the neural network, defining one or more layers, e.g., a base layer, of the neural network. The method further includes decoding a current update model, e.g., NU1 or NUK, that defines modifications to one or more layers, e.g., the base layer, of the neural network, e.g., LB,j, or modifications to one or more hidden layers or layers, e.g., LUK-1,j, of the neural network, and modifying parameters of the base model of the neural network, e.g., the parameters of LB,j, or intermediate parameters, e.g., the parameters of LUK-1,j, derived from the base model of the neural network, using one or more intermediate update models, e.g., NU1 through NUK-1, using the current update model, e.g., NU1 or NUK, to obtain an updated model, e.g., designated "new model," including new model layers LN1,j or LNK,j.

[0152] Further, the method includes entropy decoding one or more parameters of the current update model, e.g., using context-adaptive binary arithmetic coding, and adapting a context used for entropy decoding of the one or more parameters of the current update model depending on one or more previously decoded parameters of the base model and / or depending on one or previously decoded parameters of the intermediate update models, e.g., to exploit correlations between the current update model and the base model and / or correlations between the current update model and the intermediate update models.

[0153] Further embodiments according to the invention include a method for encoding neural network parameters defining a neural network, the method optionally including obtaining and / or providing, e.g., encoding, parameters of a base model, e.g., NB, of the neural network that define one or more layers, e.g., a base layer, of the neural network.

[0154] The method further includes encoding an update model, e.g., NU1 through NUK, that defines modifications to one or more layers, e.g., a base layer, of the neural network, and providing the update model to modify parameters of the base model of the neural network using the update model to obtain an updated model, e.g., designated "new model," that includes a new model layer LNkj.

[0155] Further, the method includes providing and / or determining and / or encoding skip information, such as skip_row_flag and / or skip_column_flag, indicating whether a sequence, such as a row, or a column, or a block, of parameters of the updated model is zero.

[0156] Further embodiments according to the invention include a method for encoding neural network parameters defining a neural network, the method optionally including obtaining and / or providing, e.g., encoding, parameters of a base model, e.g., NB, of the neural network that define one or more layers, e.g., a base layer, of the neural network.

[0157] The method further includes modifying parameters of the base model of the neural network, such as the parameters of LB,j, or intermediate parameters, such as the parameters of LUK-1,j, derived from the base model of the neural network, using the current update model, such as NU1 or NUK, to obtain an updated model, such as designated a "new model," including a new model layer LN1,j or LNK,j, encoding the current update model, such as NU1 or NUK, that defines modification of one or more layers, such as the base layer, of the neural network, such as that of LB,j, or modification of one or more hidden layers or layers, such as LUK-1,j, of the neural network, using the current update model, such as NU1 or NUK, to obtain an updated model, such as designated a "new model," including a new model layer LN1,j or LNK,j.

[0158] Further, the method includes entropy coding one or more parameters of the current update model, e.g., using context-adaptive binary arithmetic coding, and adapting a context used for entropy coding of the one or more parameters of the current update model depending on one or more previously coded parameters of the base model and / or depending on one or more previously coded parameters of the intermediate update models, e.g., to exploit correlations between the current update model and the base model and / or correlations between the current update model and the intermediate update models.

[0159] It should be noted that the above-described method may be based on the same considerations as the above-described decoder and encoder, and the method may comprise all (e.g., all corresponding or all similar) features and functions that are also described with respect to the decoder and encoder.

[0160] A further embodiment according to the invention comprises a computer program for performing any of the above methods disclosed herein when the computer program runs on a computer.

[0161] Further embodiments according to the invention include a coded representation of neural network parameters, e.g., a bitstream, including an update model, e.g., NU1-NUK, that defines modifications to one or more layers, e.g., a base layer, of the neural network, and skip information, e.g., skip_row_flag and / or skip_column_flag, that indicates whether a sequence, e.g., a row, or a column, or a block, of parameters of the update model is zero.

[0162] The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings: [Brief explanation of the drawings]

[0163] [Figure 1] 1 is a schematic diagram illustrating an apparatus for encoding neural network parameters and an apparatus for decoding neural network parameters according to an embodiment of the present invention; [Figure 2] 4A and 4B are schematic diagrams illustrating a second apparatus for encoding neural network parameters and a second apparatus for decoding neural network parameters according to an embodiment of the present invention; [Figure 3] FIG. 2 illustrates a method for decoding neural network parameters defining a neural network according to an embodiment of the present invention. [Figure 4] FIG. 2 illustrates a method for decoding neural network parameters defining a neural network according to an embodiment of the present invention. [Figure 5] FIG. 2 illustrates a method for encoding neural network parameters that define a neural network, according to an embodiment of the present invention. [Figure 6] FIG. 2 illustrates a method for encoding neural network parameters that define a neural network, according to an embodiment of the present invention. [Figure 7]FIG. 2 illustrates an example of a graphical representation of a feedforward neural network, e.g., a feedforward neural network, according to an embodiment of the present invention. [Figure 8] FIG. 1 shows an example of a uniform reconstruction quantizer illustration, according to an embodiment of the present invention. [Figure 9] FIG. 10 illustrates an example of allowable reconstruction vector locations according to an embodiment of the present invention. [Figure 10] FIG. 2 illustrates an example of a division of a set of reconstruction levels into two subsets according to an embodiment of the present invention. [Figure 11] 1 shows a preferred example of a state transition table for a configuration having eight states. DETAILED DESCRIPTION OF THE INVENTION

[0164] Identical or equivalent elements, or elements with identical or equivalent functionality, are designated by the same or equivalent reference numerals in the following description, even if they appear in different figures.

[0165] In the following description, numerous details are set forth to provide a more thorough explanation of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form, rather than in detail, to avoid obscuring embodiments of the present invention. In addition, features of different embodiments described hereinafter may be combined with each other unless specifically stated otherwise.

[0166] FIG. 1 shows a schematic diagram of an apparatus for encoding neural network parameters and an apparatus for decoding neural network parameters according to an embodiment of the present invention.

[0167] 1 shows an apparatus 100 for encoding neural network (NN) parameters that define a neural network. The apparatus 100 comprises an updated model providing unit 110 and an encoding unit 120.

[0168] For simplicity, the encoding device 100 is referred to as an encoder 100. As an optional feature, the encoder 100, for example the updated model providing unit 110, may be provided with NN parameters .

[0169] Based on this, the updated model providing unit 110 may be configured to provide updated model information 112 that is or includes the updated model, so that the updated model enables the decoder 150 to use the updated model to modify parameters of the base model of the neural network to obtain an updated model. As an example, the updated model may be associated with or represented by the NN parameters 102.

[0170] Alternatively, as an example, the encoder 100 may be provided with an updated model, e.g., in the form of updated model information 112, instead of e.g., the NN parameters 102, and may therefore be configured to encode the received updated model, thereby enabling the decoder 150 to use the updated model to modify the parameters of the base model, e.g., to obtain an updated model such as 108.

[0171] The updated model information 112 is provided to an encoding unit 120 configured to encode the updated model. The updated model may define modifications to one or more layers of the neural network.

[0172] As an optional feature, the encoder 100, e.g., the updated model providing unit 110, may be provided with reference model information 104. As another optional feature, e.g., alternatively, the encoder 100 may comprise a reference unit 130 configured to optionally provide the reference model information 104 to the updated model providing unit 110 and / or the encoding unit 120.

[0173] The reference model information 104 may include information about a base model of the neural network, such as neural network parameters of the base model, which define one or more layers of the neural network. Thus, as an optional feature, the encoder 100 may be configured to obtain the reference model information 104, for example, using the updated model providing unit 110 and / or for example, using the reference unit 130.

[0174] As an example, based on the reference model information 104, the updated model providing unit 110 may determine a difference between the base model and a model associated with the neural network parameters 102, such as between the base model and an updated model provided to the encoder 100, e.g., between the neural network parameters of the base model, e.g., represented by the reference model information, and the corresponding NN parameters 102 of an updated version of the base model, e.g., those of the updated model. This difference or difference information may be provided in the form of updated model information 112, e.g., as an updated model.

[0175] As another example, instead of, for example, the NN parameters 102, the updated model providing unit 110 may be configured to receive updated model information, such as, for example, 108 or equivalent to 108, and the updated model providing unit 110 may be configured to provide updated model information 112 as difference information between, for example, the updated model or including the updated model, and reference information, for example, the base model or including the base model.

[0176] Therefore, if the base model is available, the corresponding decoder 150 can use the updated model (e.g., parameters or their parameter values) to modify the parameters of the base model, e.g., to obtain an updated model that includes or is associated with the NN parameters 102, without requiring transmission of all the NN parameters 102.

[0177] Further, by way of example, using the encoding unit 120, the encoder 100 may be configured to optionally provide reference model information to a corresponding decoder 150, e.g., as part of the encoded bitstream 106. The decoder 150 may therefore be provided with reference information, e.g., reference parameters of the base model in the reference model information 104, and correction information, e.g., updated model information 112.

[0178] Additionally, the updated model providing unit 110 may be configured to determine skip information 114 indicating whether a sequence of parameters of the updated model is zero (alternatively, the skip information 114 may optionally be provided to the encoder 100 from an external source). Therefore, the encoding unit 120 may be configured to provide and / or encode the skip information 114 in an encoded bitstream for the corresponding decoder 150. The skip information may be, for example, a flag or an array of flags. Therefore, the updated model information 112 may be compressed by using flags to represent its NN parameters, which may be zero or have no significant impact, so that these parameters do not need to be transmitted explicitly.

[0179] 1 further shows an apparatus 150 for decoding neural network parameters that define a neural network. For simplicity, the apparatus 150 is referred to as a decoder 150. The decoder 150 comprises a decoding unit 160 and a modification unit 170.

[0180] As shown, decoder 150, or for example, decoding unit 160, may be configured to receive encoded bitstream 106, which includes updated model information and skip information (e.g., equal to or equivalent to skip information 114). Decoding unit 160 may be configured to decode bitstream 106 to provide updated model information 162 (e.g., equal to or equivalent to updated model information 112), which includes or is an updated model that defines modifications of one or more layers of the neural network. The decoded updated model information 162 may be provided to modification unit 170.

[0181] The modification unit 170 is configured to modify the parameters of the base model of the neural network using the updated model information 162 to obtain updated model information 108, e.g., including or being an updated model.

[0182] Therefore, as an optional feature, the modification unit 170 may be provided with reference model information, eg information about the base model of the neural network, eg neural network parameters of the base model.

[0183] As an example, the decoder 150, e.g., the decoding unit 160, may be configured to obtain, e.g., from the encoded bitstream 106, reference model information 184 (e.g., equal to or equivalent to the reference model information 104), which includes or is parameters of a base model of the neural network that defines one or more layers of the neural network.

[0184] By way of example, the reference model information 184 may be stored, for example, in the optional reference unit 180. Optionally, the reference unit 180 may include the reference model information 184, for example, independently of its transmission.

[0185] Therefore, optionally, the decoding unit 160 and / or the reference unit 180 may provide reference model information 184 to the modification unit 170. Thus, the parameters of the base model included in the reference model information 104 may be adapted or modified or updated using the updated model information 112 to provide updated model information 108 that includes or is an updated model.

[0186] Further, decoder 150, e.g., decoding unit 160, may optionally be configured to decode bitstream 106. Decoder 150 to provide skip information 164 (e.g., equal to or equivalent to skip information 114). As an example, decoding unit 160 or modification unit 170 may be configured to evaluate skip information 164 to indicate whether a sequence of parameters of an updated model is zero.

[0187] As an example, after evaluating the skip information 164, the decoding unit 160 may adapt the update model information 162 accordingly, such that parameters of the update model indicated to be zero by the skip information 114 are set to zero.

[0188] As another example, the modification unit 170 may modify the base model according to the updated model information 162 taking into account the skip information 164 to obtain or provide the updated model information 108 .

[0189] As an optional feature, the updated model information 112, 162 includes or is an updated model, which describes the difference values.

[0190] Therefore, the difference value may, for example, enable the decoder 150, e.g., the modification unit 170, to combine the difference value additively or subtractively with the value of the parameter of the base model to obtain, e.g., a corresponding value of the parameter of the updated model, e.g., one of the updated model information 108.

[0191] Thus, the decoder 100, e.g., the modification unit 170, may be configured to additively or subtractively combine the difference values ​​with values ​​of the parameters of the base model (e.g., from the reference model information 184) to obtain, e.g., corresponding values ​​of the parameters of the updated model.

[0192] Therefore, as another optional feature, the encoder 100, e.g., the updated model providing unit 110, may be configured to determine a difference value as the difference between the value of a parameter of the updated model, e.g., determined or represented by the NN parameters 102, and the value of a parameter of the base model, e.g., included in the reference model information 104.

[0193] As another optional feature, the encoder 100, e.g., the updated model providing unit 110, may provide a difference value or difference tensor L associated with the j-th layer of the neural network. Uk,j may be configured to determine L Nkj =L Bj +L Uk,j A base value parameter or base value tensor L, which represents the value of the parameter of the jth layer of the neural network base model according to (for all j, or for all j whose update model contains a layer) Bj (for example, included in the reference model information 104) Uk,j represents the value of the parameter of the j-th layer of the updated model with model index k of the neural network. Nkj (thus, for example, allowing for the determination of updated model information 108).

[0194] Thus, the decoder 150, e.g., the correction unit 170, calculates the difference value or difference tensor L associated with the j-th layer of the neural network. Uk,jLet L be the base value parameter or base value tensor L, which represents the value of the parameter in the j-th layer of the neural network base model. Bj And, L Nkj =L Bj +L Uk,j (for all j, or for all j in which the updated model includes a layer), thereby obtaining an updated model value parameter or updated model value tensor L representing the value of the parameter of the j-th layer of the updated model having model index k of the neural network. Nkj (Thus, for example, obtaining updated model information 108).

[0195] Therefore, the updated model providing unit 110 and / or the modifying unit 170 may be configured to perform element-wise addition between tensors. However, it should be noted that subtraction may also be performed accordingly.

[0196] As another optional feature, the update model, such as 112, 162, may describe or include scaling factor values.

[0197] Therefore, the encoder 100, e.g., the updated model providing unit 110, may be configured to provide scaling factor values, whereby scaling of the values ​​of the parameters of the base model (e.g., included in the reference model information 104) using the scaling factor values ​​results in the values ​​of the parameters of the updated model, e.g., 108.

[0198] Therefore, the decoder 150, e.g., the modification unit 170, may be configured to scale the values ​​of the parameters of the base model using the scaling factor value to obtain the values ​​of the parameters of the updated model, e.g., 108 or 102.

[0199] Thus, the encoder 100, e.g., the updated model providing unit 110, may be configured to determine a scaling factor value as a scaling factor between the value of the parameter of the updated model, e.g., 108 or 102, and the value of the parameter of the base model, e.g., based on the reference model information 104. As explained above, by way of example, the updated model may be represented by the NN parameters 102. As another optional feature, the updated model may be provided to the encoder 100.

[0200] As another optional feature, the encoder 100, e.g., the updated model providing unit 110, may provide a scaling value or scaling tensor L associated with the j-th layer of the neural network. Uk,j may be configured to determine L Nk,j =L Bj L Uk,j (for all j, or for all j in which the updated model contains a layer) Bj The combination of the scaling values ​​or scaling tensors with the updated model value parameters or updated model value tensor L represents the parameters of the j-th layer of the updated model, e.g., 108, having model index k of the neural network. Nkj This allows for the determination of

[0201] Thus, the decoder 150, e.g., the correction unit 170, calculates the scaling value or scaling tensor L associated with the j-th layer of the neural network. Uk,j Let L be the base value parameter or base value tensor L, which represents the value of the parameter in the j-th layer of the neural network base model. Bj And, L Nk,j =L Bj L Uk,j(for all j, or for all j in which the updated model includes a layer), thereby obtaining an updated model value parameter or updated model value tensor L, which represents the value of the parameter of the j-th layer of the updated model, e.g., 108, having a model index k of the neural network. Nkj get.

[0202] Therefore, the updated model providing unit 110 and / or the modifying unit 170 may be configured to perform element-wise multiplication between tensors. However, it should be noted that division may also be performed accordingly.

[0203] As another optional feature, the update model, eg, 112, 162, describes replacement values.

[0204] Therefore, the encoder 100, e.g. the updated model providing unit 110, may be configured to provide substitution values, which allow substitution of values ​​of parameters of the base model, e.g. 184, using substitution values, e.g. 162, to obtain values ​​of parameters of the updated model, e.g. included in the updated model information 108.

[0205] Thus, the decoder 150, e.g., the modification unit 170, may be configured to replace values ​​of parameters of the base model, e.g., included in the reference information 184, using replacement values, e.g., included in 162, to obtain values ​​of parameters of the updated model, e.g., included in 108.

[0206] Therefore, the encoder 100, for example the updated model providing unit 110, may be configured to determine the replacement values.

[0207] As another optional example, neural network parameters such as 102 include weight values ​​that define the weights of neuronal interconnections that emanate from or lead to neurons.

[0208] As another optional feature, the sequence of neural network parameters includes weight values ​​associated with rows or columns of a matrix. The inventors have recognized that row-wise or column-wise processing can be performed efficiently. As an example, the encoder 1010, e.g., the update model providing unit 110, and / or the decoder 150, e.g., the modifying unit 170, may be configured to process matrices efficiently or may be optimized for processing matrices.

[0209] As another optional feature, the skip information 114 and / or 164 includes a flag indicating whether all parameters of the sequence of parameters of the updated model, e.g., 112, 162, are zero. Therefore, instead of the zero sequence, only the skip information 114 may be coded into the bitstream 106 using the coding unit 120, requiring fewer transmission resources. At the decoder side, based on evaluation of the skip information 164, modifications of base model parameters associated with update values, e.g., weights, that are zero may be skipped. Thus, the decoder 150, e.g., the modification unit 170, may be configured to selectively skip decoding of the sequence of parameters of the updated model depending on the skip information 164.

[0210] Therefore, as another optional feature, the encoder 100, e.g., the updated model providing unit 110, may be configured to provide skip information 114 to signal a skip in the decoding of a sequence of parameters of the updated model, e.g., 112.

[0211] As another optional feature, the encoder 100, e.g., the updated model providing unit 110, may be configured to provide skip information 114 including information on whether a sequence of parameters of the updated model, e.g., 112, has a predetermined value or not.

[0212] Thus, the decoder 150, eg the modification unit 170, may be configured to selectively set the values ​​of the sequence of parameters of the updated model, eg 162, to predetermined values ​​depending on the skip information.

[0213] Therefore, the skip information may provide differentiated information: neural network parameters may be flagged as zero or non-zero, and if non-zero, even a predetermined value may be indicated, or it may be indicated that a set or sequence of parameters can be represented by a predetermined value, e.g., as an approximation of the neural network parameters.

[0214] As another optional feature, skip information 114 and / or 164 comprises an array of skip flags that indicate whether all parameters in each sequence of parameters of the updated model, e.g., 108, are zero. The inventors have recognized that indicators for multiple sequences, such as for rows and columns of a neural network parameter matrix, can be summarized in an array of skip flags. Such an array can be efficiently coded, transmitted, and decoded.

[0215] As another optional feature, the encoder 100, e.g., the updated model providing unit 110, may be configured to provide a skip flag, e.g., included in the skip flag information 114, associated with each sequence of parameters to signal a skip in the decoding of a respective sequence of parameters of the updated model, e.g., 112. The flag may be represented by a few bits for a simple indication of the skip of the parameter.

[0216] Thus, the decoder 150, e.g., the decoding unit 160, may be configured to selectively skip decoding of each sequence of parameters of the updated model, e.g., from the encoded bitstream 106, depending on a respective skip flag associated with the respective sequence of parameters, e.g., included in the skip information 164.

[0217] As another example, the encoder 100, e.g., the updated model providing unit 110, may be configured to provide array size information describing the number of entries in the array of skip flags. Optionally, the updated model information 114 may include the array size information.

[0218] Therefore, decoder 150, eg, correction unit 170, may be configured to evaluate array size information, eg, included in update model information 162, that describes the number of entries in the array of skip flags.

[0219] As another optional example, encoder 100, e.g., encoding unit 120, may be configured to use a context model to encode one or more skip flags, e.g., included in skip information 114, and to select a context model for encoding the one or more skip flags depending on one or more previously encoded symbols, e.g., symbols of update model information 112 and / or skip information 114. Thus, encoding unit 120 may include one or more context models, or may be provided with one context model for selecting whether to use a context model, or with more context models for selecting from. As an optional feature, encoder 100 and / or decoder 150 may include a context unit, which may include a context model for selecting from, which may be provided to each coding unit (encoding / decoding), e.g., as further described in the context of FIG. 2.

[0220] Accordingly, decoder 150, e.g., decoding unit 160, may be configured to decode one or more skip flags using a context model and to select a context model for decoding one or more skip flags depending on one or more previously decoded symbols. Accordingly, decoding unit 160 may include one or more context models or may be provided with one or more context models, e.g., via encoded bitstream 106. Therefore, optionally, encoder 100, e.g., encoding unit 120, may be configured to encode and / or transmit one or more context models.

[0221] As another optional feature, encoder 100, e.g., encoding unit 120, may be configured to apply a single context model for encoding all skip flags associated with layers of the neural network. Accordingly, decoder 150, e.g., decoding unit 160, may be configured to apply a single context model for decoding all skip flags associated with layers of the neural network. Therefore, encoding and / or decoding can be performed at low computational cost.

[0222] As another optional feature, the encoder 100, e.g., the encoding unit 120, may be configured to select a context model for encoding a skip flag depending on previously encoded skip flags. Accordingly, the decoder 150, e.g., the decoding unit 160, may be configured to select a context model for decoding a skip flag depending on previously decoded skip flags. Therefore, the inventive encoder and decoder can be configured to utilize or exploit correlations between subsequent skip flags. This can enable improved coding efficiency. The correlation information can be used in the form of a context model.

[0223] Additionally, generally, and as another optional feature, encoding unit 120 may be configured to store information about previously encoded information, and decoding unit 160 may be configured to store information about previously decoded information.

[0224] As another optional feature, the encoder 100, e.g., the encoding unit 120, may be configured to select a context model for encoding a skip flag, e.g., included in the skip information 114, depending on the value of the corresponding skip flag in a previously encoded neural network model. Accordingly, the decoder 150, e.g., the decoding unit 160, may be configured to select a set of selectable context models for decoding a skip flag depending on the value of the corresponding skip flag in a previously decoded neural network model. The inventors recognized that to improve coding efficiency, correlations can be used or exploited not only between subsequent skip flags of a single model, but also between corresponding skip flags of different models (e.g., between a current model and a previously encoded / decoded updated model or a previously encoded / decoded base model). This correlation may be mapped to the respective context models and used by selecting the appropriate context model.

[0225] As another optional feature, the encoder 100, e.g., the encoding unit 120, may be configured to select a set of selectable context models for encoding a skip flag depending on the value of a corresponding skip flag in a previously encoded neural network model. Accordingly, the decoder 150, e.g., the decoding unit 160, may be configured to select a set of selectable context models for decoding a skip flag depending on the value of a corresponding skip flag in a previously decoded neural network model. As an additional degree of freedom, the set of context models may be selected, e.g., before selecting each context model from the set of context models. Therefore, a method for selecting a good or best-matching context may be provided to provide good coding efficiency. As described above, the encoding unit 120 and / or the decoding unit 160 may be configured to store information about previously encoded / decoded information for subsequent context selection.

[0226] As another optional feature, the encoder 100, e.g., the encoding unit 120, may be configured to select a set of selectable context models for encoding a skip flag depending on the existence of a corresponding layer in a previously encoded neural network model. Accordingly, the decoder 150, e.g., the decoding unit 160, may be configured to select a set of selectable context models for decoding a skip flag depending on the existence of a corresponding layer in a previously decoded neural network model. Therefore, the inventive approach can cope with topology changes of the neural network, e.g., during the training step. Therefore, coding can be performed efficiently and even flexible network topologies can be achieved.

[0227] As another optional feature, encoder 100, e.g., encoding unit 120, may be configured to select a context model from the selected set of context models depending on one or more previously encoded symbols of the currently encoded update model. Accordingly, decoder 150, e.g., decoding unit 160, may be configured to select a context model from the selected set of context models depending on one or more previously decoded symbols of the currently decoded update model.

[0228] FIG. 2 shows a schematic diagram of a second apparatus for encoding neural network parameters and a second apparatus for decoding neural network parameters according to an embodiment of the present invention.

[0229] 2 shows an apparatus 200 for encoding neural network parameters that define a neural network. For simplicity, the apparatus 200 will be referred to as an encoder 200.

[0230] The encoder 200 includes an updated model providing unit 210 and an encoding unit 220. As an optional example, the encoder 200, e.g., the updated model providing unit 210, may be configured to receive updated model information 202, e.g., being or including an updated model.

[0231] Alternatively, as described, for example, in the context of FIG. 1, the updated model providing unit may be configured to receive, for example, NN parameters associated with or of the updated model.

[0232] Based thereon, the updated model providing unit 210 may be configured to provide updated model information, e.g., being or including a current, e.g., recent, updated model, whereby the updated model enables the decoder to modify the parameters of the base model of the neural network using the current updated model, or intermediate parameters derived from the base model of the neural network using one or more intermediate updated models, to obtain an updated model.

[0233] Therefore, as another optional feature, the encoder 200 may optionally comprise a reference unit 230 configured to provide reference model information including, e.g., a reference model whose parameters are modified using updated model information 212, e.g., including information about, e.g., a base model or intermediate parameters derived from the base model, or about, e.g., an intermediate updated model (e.g., a partially updated model based on the base model).

[0234] As another optional feature, the encoder 100 may be configured to receive updated model information, e.g., that is or includes the current updated model, e.g., instead of the updated model information 202. In this case, the encoder 200 may not include the updated model providing unit 210. The encoding unit 220 may encode the updated model information, which may define modifications of one or more layers of a neural network, or modifications of one or more intermediate layers or neural networks. Thus, the encoder 200 may provide the updated model information, such that the updated model enables a decoder, e.g., 250, to modify parameters of a base model of the neural network using the current updated model, or intermediate parameters derived from the base model of the neural network using one or more intermediate updated models, to obtain an updated model, e.g., 208.

[0235] As an example, the optional reference unit 230 may include such reference model information 204 or may, for example, be provided with such reference model information (not shown). As another example, the update model providing unit 210 may optionally be configured to receive the reference model information 204.

[0236] For example, based on the reference model information, the updated model providing unit 210 may be configured to provide or even determine an updated model, for example as a model showing the difference between the base model and the updated model.

[0237] For example, the updated model information 212, which is or includes such an updated model, may then be provided to the encoding unit 220. The encoding unit 220 is configured to entropy encode one or more parameters of the current updated model. Thus, the updated model information 212 or parts thereof, such as its parameters, parameter values, flags, symbols, may be encoded into the bitstream 206.

[0238] Furthermore, the encoding unit 220 is configured to adapt the context used for entropy encoding of one or more parameters of the current update model depending on one or more previously encoded parameters of the base model and / or depending on one or more previously encoded parameters of the intermediate update model.

[0239] 2, the encoder 200 may include a context unit 240 that includes information about one or more context models for encoding the update model information 212. Based on optional encoding information 222, which may include or be, for example, one or more previously encoded parameters and / or one or more previously encoded parameters of an intermediate update model, the context unit 240 may provide context information 224, which may include or be, for example, a context or a context model, to the encoding unit 220.

[0240] Therefore, encoding unit 220 may optionally store information about such previously encoded parameters.

[0241] As explained above, the encoder 200 may optionally be configured to obtain reference model information, e.g., parameters of or including parameters of base modes of a neural network defining one or more layers of the neural network. This information 204 may then be optionally provided to the encoding unit 220, e.g., to be provided to a corresponding decoder. The reference model information 204 may then be encoded into the bitstream 206.

[0242] As another optional example, encoding unit 220 may optionally be configured to encode context information 240 into encoded bitstream 206 .

[0243] 2 further shows an apparatus 250 for decoding neural network parameters that define a neural network. For simplicity, the apparatus 250 is referred to as a decoder 250. The decoder 250 comprises a decoding unit 260 and a modification unit 270.

[0244] As optionally shown, the decoder 200, e.g., a decoding unit 260, may receive an encoded bitstream 206. The bitstream may include or be an encoded version of the updated model information 212.

[0245] The decoding unit 260 is configured to decode a current updated model (e.g., by decoding the updated model information encoded in the bitstream 206), which defines modifications to one or more layers of a neural network or modifications to one or more hidden layers or to the neural network. The decoding unit 260 may therefore provide updated model information 262, which is or includes the current updated model. The updated model information 262 may, for example, be the same as or equivalent to the updated model information 212.

[0246] The decoding unit 260 is configured to entropy decode one or more parameters of the current updated model. Accordingly, the updated model information 262 may include these decoded parameters. The updated model information 262 may be, for example, the same as or equivalent to the updated model information 212.

[0247] Furthermore, the decoding unit 260 is configured to adapt the context used for entropy decoding of one or more parameters of the current update model depending on one or more previously decoded parameters of the base model and / or depending on one or more previously decoded parameters of the intermediate update model.

[0248] Therefore, the decoder 250 comprises a context unit 290. Based on optional decoding information, which for example is or includes one or more previously decoded parameters of a base model and / or one or more previously decoded parameters of an intermediate update model, the context unit 290 may provide context information 264, which for example is or includes a context or a corresponding context model. Optionally, the context information 264 may be equal to or comparable to the context information 224.

[0249] Therefore, the decoding unit 260 may optionally be configured to store information about such previously decoded parameters.

[0250] Further, the updated model information 262 is provided to a modification unit 270. The modification unit 270 is configured to modify the parameters of the base model of the neural network using the current update model, or intermediate parameters derived from the base model of the neural network using one or more intermediate update models, to obtain an updated model 208. As shown, the modification unit 270 may be configured to provide updated model information 208 that includes or is the updated model. Furthermore, the updated model information 208 may be, for example, the same as or equivalent to the updated model information 202.

[0251] As explained above, the updated model information 262 may be or include the current updated model. As optionally shown, the modification unit 270 may be provided with, for example, reference model information 284. The reference model information 284 may be or include a base model or an intermediate model, or parameters of a base model or intermediate parameters (or their respective values) of, for example, a neural network.

[0252] Additionally, the reference model information 284 may be the same as or equivalent to the reference model information 204, for example.

[0253] As an optional feature, the decoder 250 may comprise a reference unit configured to provide reference model information 284 to the modification unit 270, for example.

[0254] As another optional feature, the decoding unit 260 may receive reference model information 284, for example via the bitstream 206, and provide the information 284 to the modification unit 270. In this case, the reference unit 280 may, for example, not be present.

[0255] As another example, the decoding unit 260 may receive the reference model information 284, for example via the bitstream 206, and provide the reference model information 284, for example once, to the reference unit 280 for storage therein.

[0256] Therefore, decoder 250 may optionally be configured to obtain, eg, decode, parameters of a base model of the neural network that defines one or more layers of the neural network.

[0257] As an optional feature, the encoder 200, e.g., the encoding unit 220, may be configured to encode the quantized and binarized representation of one or more parameters of the current updated model, e.g., 212, i.e., included in the updated model information 212, e.g., using context-based entropy coding, e.g., using the context information 224.

[0258] The inventors have recognized that context-based entropy coding can make it possible to provide a good trade-off between computational effort and coding efficiency.

[0259] Thus, the decoder 250, e.g., the decoding unit 260, may be configured to decode the quantized and binarized representation of one or more parameters of the current updated model, e.g., encoded in the bitstream 206, using, e.g., context-based entropy decoding, such as using the context information 264.

[0260] As an optional feature, the encoder 200, e.g., the encoding unit 220, may be configured to entropy encode at least one significance bin associated with a currently considered parameter value of a current updated model, e.g., 212, where the significance bin describes whether a quantization index of the currently considered parameter value is equal to zero or not. The updated model information 212 may, for example, include the at least one significance bin. The significance bin may, for example, be encoded into the bitstream 206.

[0261] Thus, the decoder 250, e.g., the decoding unit 260, may be configured to, e.g., entropy decode at least one significance bin associated with the currently considered parameter value of the current updated model, the significance bin describing whether the quantization index of the currently considered parameter value is equal to zero or not. Optionally, the updated model information 262 may, e.g., include the at least one decoded significance bin.

[0262] As an optional feature, the encoder 200, e.g., the encoding unit 220, may be configured to entropy encode at least one code bin associated with a currently considered parameter value of a current updated model, e.g., 212, where the code bin describes whether a quantization index of the currently considered parameter value is greater than or less than zero. The updated model information 212 may include, e.g., the at least one code bin. The code bin may be encoded into, e.g., the bitstream 206.

[0263] Thus, decoder 250, e.g., decoding unit 260, may be configured to, e.g., entropy decode at least one code bin associated with a currently considered parameter value of a current updated model, where the code bin describes whether a quantization index of the currently considered parameter value is greater than or less than zero. Optionally, updated model information 262 may, e.g., include at least one decoded code bin.

[0264] As an optional feature, the encoder 200, e.g., the encoding unit 220, may be configured to entropy code a unary sequence associated with a currently considered parameter value of a current updated model, e.g., 212, where the bins of the unary sequence describe whether the absolute value of the quantization index of the currently considered parameter value is greater than the respective bin weight. The updated model information 212 may include, e.g., the unary sequence. The unary sequence may be encoded into the bitstream 206.

[0265] Thus, the decoder 250, e.g., the decoding unit 260, may be configured to entropy decode a unary sequence associated with the currently considered parameter value of the current updated model, where the bins of the unary sequence describe whether the absolute value of the quantization index of the currently considered parameter value is greater than the respective bin weight. Optionally, the updated model information 262 may include, e.g., the decoded unary sequence.

[0266] As an optional feature, the encoder 200, e.g., the encoding unit 220, may be configured to entropy encode one or more bins greater than X, e.g., indicating whether the absolute value of the quantization index of the currently considered parameter value is greater than X, where X is an integer greater than zero. The updated model information 212 may include, e.g., the one or more bins greater than X. The one or more bins greater than X may be encoded into the bitstream 206, e.g.

[0267] Therefore, the decoder 250, e.g., the decoding unit 260, may be configured to entropy decode one or more bins greater than X, e.g., indicating whether the absolute value of the quantization index of the currently considered parameter value is greater than X, where X is an integer greater than zero. Optionally, the updated model information 262 may include, e.g., one or more decoded bins greater than X.

[0268] As an optional feature, the encoder 200, e.g., the encoding unit 220 and / or the context unit 240, may be configured to select a context model, e.g., 224, for encoding one or more bins of quantization indices of a currently considered parameter value depending on the values ​​of previously encoded corresponding parameter values ​​in, e.g., previously encoded neural network models.

[0269] Therefore, the encoding unit 220 may optionally be configured to store or include information about previously coded values ​​of corresponding parameter values ​​in previously coded neural network models. The optional coding information 222 may, for example, include the previously coded values ​​of corresponding parameter values. The updated model information 212 may, for example, include one or more bins of quantization indices for the currently considered parameter value.

[0270] Thus, the decoder 250, e.g., the decoding unit 260 and / or the context unit 290, may be configured to select a context model, e.g., 264, for decoding one or more bins of quantization indices of the currently considered parameter value depending on the values ​​of previously decoded corresponding parameter values ​​in, e.g., previously decoded neural network models.

[0271] Therefore, the decoding unit 260 may optionally be configured to store or include information about the values ​​of corresponding previously decoded parameter values ​​in previously decoded neural network models. The optional decoding information 292 may, for example, include the values ​​of corresponding previously decoded parameter values ​​in previously decoded neural network models, for example for the selection of a context in the context unit 290.

[0272] As an optional feature, the encoder 200, e.g., the encoding unit 220 and / or the context unit 240, may be configured to select a set of context models, e.g., 224, selectable for encoding one or more bins of quantization indices of the currently considered parameter value depending on, e.g., the values ​​of previously encoded corresponding parameter values ​​in previously encoded neural network models.

[0273] Therefore, the encoding unit 220 may optionally be configured to store or include information about the values ​​of corresponding previously encoded parameter values ​​in previously encoded neural network models.

[0274] The optional encoding information 222 may include, for example, previously encoded values ​​of corresponding parameter values ​​in previously encoded neural network models. The context information 224 may optionally include a set of selected context models. The updated model information 212 may include, for example, one or more bins of quantization indices for the parameter values ​​currently being considered.

[0275] Thus, the decoder 250, e.g., the decoding unit 260 and / or the context unit 290, may be configured to select a set of context models, e.g., 224, selectable for decoding one or more bins of quantization indices for the currently considered parameter value depending on the values ​​of previously decoded corresponding parameter values ​​in, e.g., previously decoded neural network models.

[0276] Therefore, the decoding unit 260 may optionally be configured to store or include information about the values ​​of corresponding previously decoded parameter values ​​in previously decoded neural network models.

[0277] The context information 264 may include a set of selected context models. The updated model information 212 may include, for example, one or more decoded bins of quantization indices for the currently considered parameter value. The optional decoding information 292 may include, for example, values ​​of previously decoded corresponding parameter values ​​in previously decoded neural network models.

[0278] As an optional feature, the encoder 200, e.g., the encoding unit 220 and / or the context unit 240, may be configured to select a context model, e.g., 224, for encoding one or more bins of quantization indices of a currently considered parameter value depending on, e.g., the absolute values ​​of previously encoded corresponding parameter values ​​in a previously encoded neural network model.

[0279] Alternatively, the encoder 200, e.g., the encoding unit 220 and / or the context unit 240, may be configured to select a set of context models, e.g., H.264, for encoding one or more bins of quantization indices of a currently considered parameter value depending on the absolute values ​​of previously encoded corresponding parameter values ​​in previously encoded neural network models.

[0280] The context information 224 may include, for example, a selected context model or a selected set of context models. The coding information 222 may include, for example, absolute values ​​of corresponding previously coded parameter values ​​in previously coded neural network models. Therefore, the coding unit 220 may optionally be configured to store or include information about absolute values ​​of corresponding previously coded parameter values ​​in previously coded neural network models. The updated model information 212 may include, for example, one or more bins of quantization indices of the currently considered parameter value.

[0281] Thus, the decoder 250, e.g., the decoding unit 260 and / or the context unit 290, may be configured to select a context model, e.g., 264, for decoding one or more bins of quantization indices of the currently considered parameter value depending on the absolute values ​​of corresponding previously decoded parameter values ​​in a previously decoded neural network model.

[0282] Alternatively, the decoder 250, e.g., the decoding unit 260 and / or the context unit 290, may be configured to select a set of context models for decoding one or more bins of quantization indices of the currently considered parameter value depending on, e.g., the absolute values ​​of previously decoded corresponding parameter values ​​in previously decoded neural network models.

[0283] The context information 264 may include, for example, a selected context model or a set of context models. The decoding information 222 may include, for example, absolute values ​​of previously decoded corresponding parameter values ​​in previously decoded neural network models. Therefore, the decoding unit 260 may optionally be configured to store or include information about absolute values ​​of previously decoded corresponding parameter values ​​in previously decoded neural network models. The updated model information 262 may include, for example, one or more decoded bins of quantization indices of the currently considered parameter value.

[0284] As an optional feature, the encoder 200, e.g., the updated model providing unit 210, may be configured to compare previously coded corresponding parameter values, e.g., in a previously coded neural network model, with one or more thresholds.

[0285] Optionally, the encoder 200 may be configured to select a context model, such as 224, for encoding one or more bins of quantization indices of the currently considered parameter value depending on the result of the comparison.

[0286] Alternatively, the encoder 200 may be configured to select a set of context models, such as 224, for encoding one or more bins of quantization indices for the currently considered parameter value, for example depending on the result of the comparison.

[0287] As an example, the first set may be selected if the corresponding or concentric parameter is less than a first threshold T1, the second set may be selected if the corresponding or concentric parameter is greater than or equal to the first threshold T1, and the third set may be selected if the corresponding or concentric parameter is greater than or equal to a threshold T2.

[0288] The context information 224 may include, for example, a selected context model or a set of context models. The updated model information 212 may include, for example, one or more bins of quantization indices for the currently considered parameter value. Thus, the encoding unit 220 may be configured to store or include, for example, information about previously encoded corresponding parameter values ​​and / or about one or more thresholds. Furthermore, the encoding information 222 may include, for example, the results of the comparison.

[0289] Thus, the decoder 250, e.g., the decoding unit 260 and / or the context unit 290, may be configured to compare corresponding previously decoded parameter values ​​in, e.g., previously decoded neural network models, with one or more thresholds.

[0290] Optionally, the decoder 250 may be configured to select a context model, such as 224, for decoding one or more bins of quantization indices for the currently considered parameter value, for example depending on the result of the comparison.

[0291] Alternatively, the decoder 250 may be configured to select a set of context models for decoding one or more bins of quantization indices for the currently considered parameter value depending on, for example, the result of the comparison.

[0292] The context information 264 may include, for example, a selected context model or a set of context models. The updated model information 212 may include, for example, one or more decoded bins of quantization indices for the currently considered parameter value. Furthermore, the decoding information 292 may include, for example, the results of the comparison. Thus, the decoding unit 260 may be configured to store or include, for example, information about previously decoded corresponding parameter values ​​and / or about one or more thresholds.

[0293] As an optional feature, the encoder 200, e.g., the encoding unit 220, may be configured to compare previously encoded corresponding parameter values ​​in, e.g., previously encoded neural network models, with a single threshold value.

[0294] Optionally, the encoder 200, e.g., the encoding unit 220 and / or the context unit 240, may be configured to select a context model, e.g., 224, for encoding one or more bins of quantization indices of the currently considered parameter value, e.g., depending on the result of the comparison with a single threshold.

[0295] Alternatively, the encoder 200, e.g., the encoding unit 220 and / or the context unit 240, may be configured to select a set of context models, e.g., 224, for encoding one or more bins of quantization indices of the currently considered parameter value, e.g., depending on the result of a comparison with a single threshold.

[0296] Therefore, the encoding unit 220 may optionally be configured to store or include information about previously encoded corresponding parameter values ​​and / or may include threshold values.

[0297] The context information 264 may include, for example, a selected context model or a set of context models. The updated model information 212 may optionally include one or more bins of quantization indices for the parameter value currently being considered. The coding information 222 may optionally include the result of the comparison with a single threshold.

[0298] Thus, the decoder 250, e.g., the decoding unit 260, may be configured to compare corresponding previously decoded parameter values ​​in, e.g., previously decoded neural network models, with a single threshold value.

[0299] Optionally, the decoder 250, e.g., the decoding unit 260 and / or the context unit 290, may be configured to select a context model, e.g., 264, for decoding one or more bins of quantization indices of the currently considered parameter value, e.g., depending on the result of the comparison with a single threshold.

[0300] Alternatively, the decoder 250, e.g., the decoding unit 260 and / or the context unit 290, may be configured to select a set of context models, e.g., H.264, for decoding one or more bins of quantization indices of the currently considered parameter value, e.g., depending on the result of a comparison with a single threshold.

[0301] Therefore, the decoding unit 260 may optionally be configured to store or include information about previously decoded corresponding parameter values ​​and / or about threshold values.

[0302] The context information 264 may include, for example, a selected context model or a set of context models. The updated model information 262 may optionally include one or more decoded bins of quantization indices for the parameter value currently under consideration. The decoded information 292 may optionally include the result of a comparison with a single threshold.

[0303] As an optional feature, the encoder 200, e.g., the encoding unit 220, may be configured to compare the absolute values ​​of corresponding previously encoded parameter values ​​in, e.g., previously encoded neural network models, with one or more thresholds.

[0304] Optionally, the encoder 200, e.g., the encoding unit 220 and / or the context unit 240, may be configured to select a context model, e.g., 224, for encoding one or more bins of quantization indices for the currently considered parameter value, e.g., depending on the result of the comparison.

[0305] Alternatively, the encoder 200, e.g., the encoding unit 220 and / or the context unit 240, may be configured to select a set of context models, e.g., 224, for encoding one or more bins of quantization indices of the currently considered parameter value, e.g., depending on the result of the comparison.

[0306] Therefore, the encoding unit 220 may optionally be configured to store the absolute values ​​of corresponding previously encoded parameter values, which may include one or more threshold values.

[0307] The context information 224 may include, for example, a selected context model or a set of context models. The updated model information 212 may optionally include one or more bins of quantization indices for the parameter value currently being considered. The coding information 222 may optionally include the results of a comparison with one or more thresholds.

[0308] Thus, the decoder 250, e.g., the decoding unit 260, may be configured to compare the absolute values ​​of corresponding previously decoded parameter values ​​in, e.g., previously decoded neural network models, with one or more thresholds.

[0309] Optionally, the decoder 250, e.g., the decoding unit 260 and / or the context unit 290, may be configured to select a context model, e.g., 224, for decoding one or more bins of quantization indices of the currently considered parameter value, e.g., depending on the result of the comparison.

[0310] Alternatively, the decoder 250, e.g., the decoding unit 260 and / or the context unit 290, may be configured to select a set of context models, e.g., 224, for decoding one or more bins of quantization indices for the currently considered parameter value, e.g., depending on the result of the comparison.

[0311] Therefore, the decoding unit 260 may optionally be configured to store or include information about the absolute values ​​of corresponding previously decoded parameter values ​​and / or about one or more thresholds.

[0312] The context information 264 may include, for example, a selected context model or a set of context models. The updated model information 262 may optionally include one or more decoded bins of quantization indices for the currently considered parameter value. The decoded information 292 may optionally include, for example, the results of a comparison with one or more thresholds for the selection of the context information.

[0313] As an optional feature, the encoder 200, e.g., the encoding unit 220, may be configured to, e.g., entropy encode at least one significance bin associated with a currently considered parameter value of a current updated model, e.g., 212, where the significance bin describes whether a quantization index of the currently considered parameter value is equal to zero or not, and to select a context, e.g., 224, or a set of contexts, e.g., 224, for entropy encoding of the at least one significance bin depending on a value of a previously encoded corresponding parameter value in a previously encoded neural network model.

[0314] The context information 224 may include, for example, a selected context model or a set of context models. Therefore, the encoding unit 220 may optionally be configured to store information about previously encoded values ​​of corresponding parameter values. The updated model information 212 may optionally include at least one significance bin. The encoding information 222 may optionally include previously encoded values ​​of corresponding parameter values ​​for context selection, for example using the context unit 240.

[0315] Thus, the decoder 250, e.g., the decoding unit 260, may be configured, for example, to entropy decode at least one significance bin associated with a currently considered parameter value of a current updated model, the significance bin describing whether a quantization index of the currently considered parameter value is equal to zero or not, and to select a context, e.g., 264, for entropy decoding of the at least one significance bin or a set of contexts, e.g., 264, for entropy decoding of the at least one significance bin depending on the value of a previously decoded corresponding parameter value in a previously decoded neural network model.

[0316] The context information 264 may include, for example, a selected context model or a set of context models. Therefore, the decoding unit 260 may optionally be configured to store information about values ​​of corresponding parameter values ​​that have been previously decoded.

[0317] The updated model information 262 may optionally include at least one decoded significant bin. The decoding information 292 may optionally include values ​​of corresponding previously decoded parameter values ​​for context selection, for example using the context unit 290.

[0318] As an optional feature, the encoder 200, e.g., the encoding unit 220, may be configured, e.g., to entropy encode at least one code bin associated with a currently considered parameter value of the current updated model, the code bin describing whether a quantization index of the currently considered parameter value is greater than or less than zero, and to select a context, e.g., 224, or a set of contexts, e.g., 224, for entropy encoding of the at least one code bin depending on a value of a previously encoded corresponding parameter value in a previously encoded neural network model.

[0319] The context information 224 may, for example, include a selected context model or a set of context models. Therefore, the encoding unit 220 may optionally be configured to store information about values ​​of corresponding parameter values ​​that have been previously encoded.

[0320] The updated model information 212 may optionally include at least one code bin. The coding information 222 may optionally include values ​​of corresponding parameter values ​​previously coded for context selection, for example using the context unit 240.

[0321] Thus, decoder 250, e.g., decoding unit 260, may be configured, for example, to entropy decode at least one code bin associated with a currently considered parameter value of a current updated model, where the code bin describes whether a quantization index of the currently considered parameter value is greater than or less than zero, and to select a context, e.g., 224, or a set of contexts, e.g., 224, for entropy decoding of the at least one code bin depending on the value of a previously decoded corresponding parameter value in a previously decoded neural network model.

[0322] The context information 264 may include, for example, a selected context model or a set of context models. Therefore, the decoding unit 260 may optionally be configured to store information about values ​​of corresponding parameter values ​​that have been previously decoded.

[0323] The updated model information 262 may optionally include at least one decoded code bin. The decoding information 292 may optionally include values ​​of corresponding previously decoded parameter values ​​for context selection, for example, using the context unit 290.

[0324] As an optional feature, the encoder 200, e.g., the encoding unit 220, may be configured to entropy encode one or more bins greater than X, e.g., indicating whether the absolute value of a quantization index of the currently considered parameter value is greater than X, where X is an integer greater than zero, and to select a context, e.g., 224, for entropy encoding of the at least one bin greater than X, or a set of contexts, e.g., 224, for entropy encoding of the at least one bin greater than X, depending on values ​​of previously encoded corresponding parameter values ​​in previously encoded neural network models.

[0325] The context information 224 may, for example, include a selected context model or a set of context models. Therefore, the encoding unit 220 may optionally be configured to store information about values ​​of corresponding parameter values ​​that have been previously encoded.

[0326] The updated model information 212 may optionally include one or more bins greater than X. The encoding information 222 may optionally include values ​​of corresponding parameter values ​​previously encoded for context selection, for example using the context unit 240.

[0327] Thus, the decoder 250, e.g., the decoding unit 260, may be configured, for example, to entropy decode one or more bins greater than X indicating whether the absolute value of the quantization index of the currently considered parameter value is greater than X, where X is an integer greater than zero, and to select a context, e.g., 264, for entropy decoding of the at least one bin greater than X, or a set of contexts, e.g., 264, for entropy decoding of the at least one bin greater than X, depending on the value of a corresponding previously decoded parameter value in a previously decoded neural network model.

[0328] The context information 264 may include, for example, a selected context model or a set of context models. Therefore, the decoding unit 260 may optionally be configured to store information about values ​​of previously decoded corresponding parameter values ​​in previously decoded neural network models.

[0329] The updated model information 262 may optionally include one or more bins greater than X. The decoding information 292 may optionally include values ​​of corresponding previously decoded parameter values ​​for context selection, for example using the context unit 290.

[0330] As another optional feature, the encoder 200, e.g., the encoding unit 220 and / or the context unit 240, may be configured to choose a context model from among a selected set of context models, e.g., depending on one or more previously encoded bins or parameters of the current updated model.

[0331] Therefore, the context information 224 may include a selected set of context models, and the encoding unit 220 may choose one context model from the set of context models. Alternatively, the context information 224 may be or include the selected context model. As an example, one or more previously encoded bins or parameters of the current updated model may be provided to the context unit 240 for context selection using the encoding information 222.

[0332] Thus, decoder 250, e.g., decoding unit 260 and / or context unit 290, may be configured to choose a context model from among a selected set of context models, e.g., depending on one or more previously decoded bins or parameters of the current updated model.

[0333] It should therefore be noted that in general, the encoding unit 220 may be configured to store information about previously encoded information, such as, for example, symbols, models, values, magnitudes and / or bins.

[0334] It should therefore be noted that in general, the decoding unit 260 may be configured to store information about previously decoded information, such as, for example, symbols, models, values, magnitudes and / or bins.

[0335] 3 illustrates a method for decoding neural network parameters defining a neural network according to an embodiment of the present invention. The method 300 includes decoding (310) an update model defining modifications to one or more layers of the neural network, modifying (320) parameters of a base model of the neural network using the update model to obtain an updated model, and evaluating (330) skip information indicating whether a sequence of parameters of the updated model is zero.

[0336] 4 illustrates a method for decoding neural network parameters defining a neural network according to an embodiment of the present invention. Method 400 includes decoding (410) a current update model that defines modifications to one or more layers of the neural network or to one or more intermediate layers or neural network, modifying (420) parameters of a base model of the neural network using the current update model or intermediate parameters derived from the base model of the neural network using one or more intermediate update models to obtain an updated model, entropy decoding (430) one or more parameters of the current update model, and adapting (440) a context used for entropy decoding of one or more parameters of the current update model depending on one or more previously decoded parameters of the base model and / or depending on one or more previously decoded parameters of the intermediate update models.

[0337] 5 illustrates a method for encoding neural network parameters defining a neural network according to an embodiment of the present invention. The method 500 includes encoding (510) an update model defining modifications to one or more layers of the neural network, providing (520) the update model to modify parameters of a base model of the neural network using the update model to obtain an updated model, and providing and / or determining (530) skip information indicating whether a sequence of parameters of the update model is zero.

[0338] 6 illustrates a method for encoding neural network parameters defining a neural network according to an embodiment of the present invention. Method 600 includes encoding (610) a current update model that defines modifications to one or more layers of the neural network or modifications to one or more intermediate layers or neural network parameters using the current update model to modify parameters of a base model of the neural network or intermediate parameters derived from the base model of the neural network using one or more intermediate update models to obtain an updated model; entropy encoding (620) one or more parameters of the current update model; and adapting (630) a context used for entropy encoding one or more parameters of the current update model depending on one or more previously encoded parameters of the base model and / or depending on one or more previously encoded parameters of the intermediate update models.

[0339] Further embodiments according to the present invention include temporal context adaptation. Embodiments may include, for example, adapting a context model or context information over time.

[0340] Furthermore, it should be noted that embodiments may be applied to compressing entire neural networks, and some of them may also be applied to compressing differential updates of a neural network relative to a base network. Such differential updates are useful, for example, when a model is redistributed after fine-tuning or transfer learning, or when providing versions of a neural network with different compression ratios.

[0341] Embodiments may further address the use, eg, manipulation or modification, of a base neural network, eg, a neural network on which differential updates are based.

[0342] Embodiments may further address, include, or provide an updated neural network, e.g., a neural network resulting from modifying a base neural network. Note: An updated neural network may be reconstructed, for example, by applying differential updates to a base neural network.

[0343] Further embodiments in accordance with the present invention may include syntax elements in the form of NNR units, which may be, for example, data structures carrying neural network data and / or associated metadata that may be compressed or represented in accordance with embodiments of the present invention.

[0344] The NNR unit may carry at least one of compressed information about neural network metadata, uncompressed information about neural network metadata, topology information, full or partial layer data, filters, kernels, biases, quantized weights, tensors, etc.

[0345] An NNR unit may, for example, include or consist of the following data elements:

[0346] NNR unit size (optional): This data element may signal the total byte size of the NNR unit, including the NNR unit size.

[0347] NNR Unit Header: This data element may contain or include information about the NNR unit type and / or associated metadata.

[0348] NNR Unit Payload: This data element may contain or encompass compressed or uncompressed data related to a neural network.

[0349] As an example, an embodiment may include (or use) the following bitstream syntax (eg, numBytesInNNRUnit may indicate the size of an nnr_unit bitstream element):

[0350] [Table 1]

[0351] [Table 2]

[0352] [Table 3]

[0353] The parent node identifier may include, for example, one or more of the syntax elements listed above, such as device_id, parameter_id, and / or put_node_depth.

[0354] [Table 4]

[0355] [Table 5]

[0356] Using decode_compressed_data_unit_payload(), we can modify the parameters of the neural network's base model to obtain an updated model.

[0357] A node_id_present_flag equal to 1 may indicate that the syntax elements device_id, parameter_id, and / or put_node_depth are present.

[0358] The device_id may, for example, uniquely identify the device that generated the current NDU.

[0359] The parameter_id may, for example, uniquely identify a parameter of a model to which a tensor stored in the NDU pertains. If the parent_node_id_type is equal to ICNN_NDU_ID, the parameter_id may, for example, be or shall be equal to the parameter_id of the associated parent NDU.

[0360] The put_node_depth may be, for example, the tree depth where the current NDU is located. A depth of 0 may correspond to the root node. If parent_node_id_type is equal to ICNN_NDU_ID, then put_node_depth-1 may be, for example, or must even be equal to the put_node_depth of the associated parent NDU.

[0361] A parent_node_id_present_flag equal to 1 may indicate, for example, that the syntax element parent_node_id_type is present.

[0362] parent_node_id_type can, for example, specify the parent node id type. This may indicate the presence of further syntax elements that uniquely identify the parent node. Example allowed values ​​for parent_node_id_type are defined in Table 2.

[0363] [Table 6]

[0364] The temporal_context_modeling_flag may, for example, specify whether temporal context modeling is enabled. A temporal_context_modeling_flag equal to 1 may indicate that temporal context modeling is enabled. If temporal_context_modeling_flag is not present, it is inferred to be 0.

[0365] The parent_device_id may be equal to the syntax element device_id of the parent NDU, for example.

[0366] The parent_node_payload_sha256 may be, for example, the SHA256 hash of the nnr_compressed_data_unit_payload of the parent NDU.

[0367] The parent_node_payload_sha512 may be, for example, the SHA512 hash of the nnr_compressed_data_unit_payload of the parent NDU.

[0368] Additionally, embodiments in accordance with the present invention may include a row skipping feature. As an example, when enabled by the flag row_skip_flag_enabled_flag, the row skipping technique signals one flag row_skip_list[i] for each value i along the first axis of the parameter tensor. When the flag row_skip_list[i] is 1, all elements of the parameter tensor whose index on the first axis is equal to i are set to zero. When the flag row_skip_list[i] is 0, all elements of the parameter tensor whose index on the first axis is equal to i are individually encoded.

[0369] Furthermore, embodiments in accordance with the present invention may include context modeling. As an example, context modeling may correspond to associating three types of flags, sig_flag, sign_flag, and abs_level_greater_x / x2, with a context model. In this way, flags with similar statistical behavior may or should be associated with the same context model so that a probability estimator (inside the context model) can, for example, adapt to the underlying statistics.

[0370] The context modeling of the presented approach may be, for example, as follows:

[0371] For example, 24 context models can be distinguished for sig_flag depending on the state value and whether the left adjacent quantized parameter level is zero, less than zero, or greater than zero.

[0372] If dq_flag is 0, for example, only the first three context models can be used.

[0373] Three other context models can be distinguished for sign_flag, for example, depending on whether the left adjacent quantized parameter level is zero, less than zero, or greater than zero.

[0374] For the abs_level_greater_x / x2 flag, each x can use, for example, either one or two distinct context models. If x≦maxNumNoRemMinus1, two context models are distinguished depending on sign_flag. If x>maxNumNoRemMinus1, for example, only one context model can be used.

[0375] Furthermore, embodiments in accordance with the present invention may include temporal context modeling. As an example, if enabled by the flag temporal_context_modeling_flag, an additional context model set of flags sig_flag, sign_flag, and abs_level_greater_x may be available. The derivation of ctxIdx may then also be based on the value of the quantized collocated parameter level in a previously encoded parameter update tensor, which may be uniquely identified, for example, by a parameter update tree. If the collocated parameter level is not available or is equal to zero, for example, context modeling as previously described may be applied. Otherwise, if the collocated parameter level is not equal to zero, the temporal context modeling of the presented approach may be as follows:

[0376] The 16 context models can be distinguished for sig_flag, for example, depending on the state value and whether the absolute value of the quantized collocated parameter level is greater than 1 or not.

[0377] If dq_flag is 0, only the first two additional context models can be used.

[0378] Furthermore, two context models can be distinguished for sign_flag, for example, depending on whether the quantized collocated parameter level is less than or greater than zero.

[0379] For the abs_level_greater_x flag, each x can use two separate context models, which can be distinguished, for example, depending on whether the absolute value of the quantized collocated parameter level is greater than or equal to x-1.

[0380] Embodiments in accordance with the present invention may optionally include the following tensor syntax, for example, quantized tensor syntax:

[0381] [Table 7]

[0382] The skip information may include, for example, any or all of the row skip information described above, eg, row_skip_enabled_flag and / or row_skip_list.

[0383] As an example, row_skip_enabled_flag may specify whether row skipping is enabled: row_skip_enabled_flag equal to 1 may indicate that row skipping is enabled.

[0384] row_skip_list can specify a list of flags, and the i-th flag row_skip_lsit[i] can indicate whether all tensor elements of QuantParam whose first dimension index is equal to i are zero. If row_skip_list[i] is equal to 1, all tensor elements of QuantParam whose first dimension index is equal to i may be zero.

[0385] Embodiments in accordance with the present invention may further include, for example, a quantized parameter syntax, such as the syntax defined below (all elements may be considered optional):

[0386] [Table 8]

[0387] sig_flag may, for example, specify whether the quantized weight QuantParam[i] is non-zero. A sig_flag equal to 0 may, for example, indicate that QuantParam[i] is 0. A sign_flag may, for example, specify whether the quantized weight QuantParam[i] is positive or negative. A sign_flag equal to 1 may, for example, indicate that QuantParam[i] is negative. Abs_level_greater_x[j] may, for example, indicate whether the absolute level of QuantParam[i] is greater than j+1.

[0388] abs_level_greater_x2[j] may, for example, contain the unary part of the exponential-Golomb residue.

[0389] abs_remainder may indicate, for example, a fixed length remainder.

[0390] Further embodiments in accordance with the present invention may include, for example, the following shift parameter index syntax (all elements may be considered optional):

[0391] [Table 9]

[0392] Further embodiments in accordance with the present invention include an entropy decoding process, as described below.

[0393] In general, the input to this process can be, for example, a request for the value of a syntax element and the value of a previously parsed syntax element.

[0394] The output of this process can be, for example, the value of a syntax element.

[0395] Parsing of syntax elements can proceed, for example, as follows:

[0396] For each requested value of a syntax element, for example, a binarization can be derived.

[0397] The binarization of the syntax elements and the sequence of parsed bins can determine, for example, the decoding process flow.

[0398] Example of initialization process according to embodiment:

[0399] In general, the output of this process can be, for example, initialized DeepCABAC internal variables.

[0400] The context variables of the arithmetic decoding engine can be initialized, for example, as follows:

[0401] The decoding engine may, for example, register IvlCurrRange and IvlOffset, both with 16-bit register precision, and may be initialized, for example, by invoking an arithmetic decoding engine initialization process.

[0402] Embodiments in accordance with the present invention may include an initialization process for probability estimation parameters, for example as described below.

[0403] The output of this process may be, for example, the initialized probability estimation parameters shift0, shift1, pStateIdx0, and pStateIdx1 for each context model of the syntax elements sig_flag, sign_flag, abs_level_greater_x, and abs_level_greater_x2.

[0404] The 2D array CtxParameterList[][] can be initialized, for example, as follows:

[0405] CtxParameterList[][]={{1,4,0,0},{1,4,-41,-654},{1,4,95,1519},{0,5,0, 0},{2,6,30,482},{2,6,95,1519},{2,6,-21,-337},{3,5,0,0},{3,5,30,482}}

[0406] If dq_flag is equal to 1 and temporal_context_modeling_flag is equal to 1, for example, for each of the 40 context models of the syntax element sig_flag, the associated context parameter shift0 may be set, for example, to CtxParameterList[setId][0], shift1 may be set, for example, to CtxParameterList[setId][1], pStateIdx0 may be set, for example, to CtxParameterList[setId][2], and pStateIdx1 may be set, for example, to CtxParameterList[setId][3], where i may be, for example, an index of the context model and setId may be equal, for example, to ShiftParameterIdsSigFlag[i].

[0407] If dq_flag== is equal to 1 and temporal_context_modeling_flag is equal to 0, then for each of the first, e.g., 24, context models of the syntax element sig_flag, the associated context parameter shift0 may be set, e.g., to CtxParameterList[setId][0], shift1 may be set, e.g., to CtxParameterList[setId][1], pStateIdx0 may be set, e.g., to CtxParameterList[setId][2], and pStateIdx1 may be set, e.g., to CtxParameterList[setId][3], where i may be, e.g., the index of the context model and setId may be equal, e.g., to ShiftParameterIdsSigFlag[i].

[0408] If dq_flag is equal to 0 and temporal_context_modeling_flag is equal to 1, then for each of, for example, the first three context models of the syntax element sig_flag and, for example, context models 24 and 25, the associated context parameter shift0 may be set, for example, to CtxParameterList[setId][0], shift1 may be set, for example, to CtxParameterList[setId][1], pStateIdx0 may be set, for example, to CtxParameterList[setId][2], and pStateIdx1 may be set, for example, to CtxParameterList[setId][3], where i may be, for example, an index of the context model and setId may, for example, be equal to ShiftParameterIdsSigFlag[i].

[0409] If temporal_context_modeling_flag is equal to 1, then for each of the, e.g., five context models of the syntax element sign_flag, the associated context parameter shift0 may be set, e.g., to CtxParameterList[setId][0], shift1 may be set, e.g., to CtxParameterList[setId][1], pStateIdx0 may be set, e.g., to CtxParameterList[setId][2], and pStateIdx1 may be set, e.g., to CtxParameterList[setId][3], where i may be, e.g., an index of the context model, and setId may be equal, e.g., to ShiftParameterIdsSignFlag[i].

[0410] Otherwise (temporal_context_modeling_flag==0), for example, for each of the first, e.g., three, context models of the syntax element sign_flag, the associated context parameter shift0 may be set, e.g., to CtxParameterListsetId[0], shift1 may be set, e.g., to CtxParameterList[setId][1], pStateIdx0 may be set, e.g., to CtxParameterList[setId][2], and pStateIdx1 may be set, e.g., to CtxParameterList[setId][3], where i may be, e.g., the index of the context model, and setId may be equal, e.g., to ShiftParameterIdsSignFlag[i].

[0411] If temporal_context_modeling_flag is equal to 1, then, for example, for each of the 4*(cabac_unary_length_minus1+1) context models of the syntax element abs_level_greater_x, the associated context parameter shift0 may be set, for example, to CtxParameterList[setId][0], shift1 may be set, for example, to CtxParameterList[setId][1], pStateIdx0 may be set, for example, to CtxParameterList[setId][2], and pStateIdx1 may be set, for example, to CtxParameterList[setId][3], where i may be, for example, an index of the context model and setId may, for example, be equal to ShiftParameterIdsAbsGrX[i].

[0412] Otherwise (temporal_context_modeling_flag==0), for example, for each of the first, e.g., 2*(cabac_unary_length_minus1+1) context models of the syntax element abs_level_greater_x, the associated context parameter shift0 may be set, e.g., to CtxParameterList[setId][0], shift1 may be set, e.g., to CtxParameterList[setId][1], pStateIdx0 may be set, e.g., to CtxParameterList[setId][2], and pStateIdx1 may be set, e.g., to CtxParameterList[setId][3], where i may be, e.g., the index of the context model and setId may, e.g., be equal to ShiftParameterIdsAbsGrX[i].

[0413] Further embodiments in accordance with the present invention may include, for example, a decoding process flow as described below.

[0414] In general, the input to this process may be, for example, all the bin strings of the binarization of the requested syntax element.

[0415] The output of this process may be, for example, the value of a syntax element.

[0416] This process may, for example, specify for each syntax element, how each bin of the bin string is to be parsed, e.g., after parsing each bin, the resulting bin string may, for example, be compared with all bin strings of a binarization of the syntax element, and the following may apply:

[0417] If the bin string is equal to one of the bin strings, the corresponding value of the syntax element may be, for example, output. If not (the bin string is not equal to one of the bin strings), the next bit may be parsed, for example.

[0418] While analyzing each bin, the variable binIdx may be incremented by 1, for example, starting with binIdx being set equal to 0 for the first bin.

[0419] The analysis of each bin can be specified, for example, by the following two ordered steps:

[0420] 1. For example, the derivation process for ctxIdx and bypassFlag can be invoked, for example, with binIdx as input and ctxIdx and bypassFlag as outputs. 2. The arithmetic decoding process can be called, for example, with ctxIdx and bypassFlag as input and the bin value as output.

[0421] Further embodiments according to the present invention may include a derivation process for ctxInc of the syntax element sig_flag.

[0422] The input to this process can be, for example, the sig_flag decoded before the current sig_flag, the state value stateId, the associated sign_flag, if present, and the co-located parameter level (coLocParam) from the incremental update decoded before the current incremental update, if present. If a sig_flag was not decoded before the current sig_flag, it can be inferred to be, for example, 0. If a sign_flag associated with a previously decoded sig_flag was not decoded, it can be inferred to be, for example, 0. If a co-located parameter level from an incremental update decoded before the current incremental update is not available, it is inferred to be 0. The co-located parameter level refers to the parameter level in the same tensor at the same position in the previously decoded incremental update.

[0423] The output of this process is the variable ctxInc.

[0424] The variable ctxInc is derived as follows:

[0425] If coLocParam is equal to 0, the following applies: If sig_flag is equal to 0, ctxInc is set to stateId*3. Otherwise, if sign_flag is equal to 0, ctxInc is set to stateId*3+1. Otherwise, ctxInc is set to stateId*3+2. If coLocParam is not equal to 0, the following applies: If coLocParam is greater than 1 or less than -1, ctxInc is set to stateId*2+24. Otherwise, ctxInc is set to stateId*2+25.

[0426] Further embodiments according to the present invention may include a derivation process for ctxInc of the syntax element sign_flag.

[0427] The input to this process can be, for example, the sig_flag decoded before the current sig_flag, the associated sign_flag, if present, and the co-located parameter level (coLocParam) from the incremental update decoded before the current incremental update, if present. If a sig_flag was not decoded before the current sig_flag, it can be inferred to be, for example, 0. If a sign_flag associated with a previously decoded sig_flag was not decoded, it can be inferred to be, for example, 0. If a co-located parameter level from an incremental update decoded before the current incremental update is not available, it can be inferred to be, for example, 0. The co-located parameter level refers to the parameter level in the same tensor at the same position in the previously decoded incremental update.

[0428] The output of this process may be, for example, the variable ctxInc.

[0429] The variable ctxInc may be derived, for example, as follows:

[0430] If coLocParam is equal to 0, the following can be applied: If sig_flag is equal to 0, then ctxInc may be set to 0, for example. Otherwise, if sign_flag is equal to 0, ctxInc may be set to 1, for example. If not, ctxInc may be set to 2, for example. If coLocParam is not equal to 0, the following can be applied: If coLocParam is less than 0, ctxInc may be set to 3, for example. If not, ctxInc may be set to 4, for example.

[0431] Further embodiments may include a derivation process for ctxInc of the syntax element abs_level_greater_x[j].

[0432] The input to this process can be, for example, the sign_flag decoded before the current syntax element abs_level_greater_x[j] and, if present, the collocated parameter level (coLocParam) from the incremental update decoded before the current incremental update. If the collocated parameter level from the incremental update decoded before the current incremental update is not available, it can be inferred to be, for example, 0. The collocated parameter level means the parameter level in the same tensor at the same position in the previously decoded incremental update.

[0433] The output of this process may be, for example, the variable ctxInc.

[0434] The variable ctxInc may be derived, for example, as follows:

[0435] If coLocParam is equal to zero, the following can be applied:

[0436] If sign_flag is equal to 0, ctxInc may be set to, for example, 2*j. Otherwise, ctxInc may be set to, for example, 2*j+1. If coLocParam is not equal to zero, the following may apply: If coLocParam is greater than or equal to j or less than or equal to -j, ctxInc may be set to, for example, 2*j+2*maxNumNoRemMinus1. Otherwise, ctxInc may be set to, for example, 2*j+2*macNumNoRemMinus1+1.

[0437] Further notes: In the following, various embodiments and aspects of the present invention are described in the sections "Areas of Application", "Aspects of Embodiments According to the Invention" and "Aspects of the Invention".

[0438] Further embodiments are defined by the appended claims.

[0439] It should be noted that any embodiment defined by the claims may be supplemented by any of the details (features and functions) described in the above sections and / or subsections, respectively, and / or by any of the details (features and functions) described in the above disclosure.

[0440] Furthermore, the embodiments described in each of the above chapters and / or subchapters can be used individually and can also be supplemented by any of the features of each of the other chapters and / or subchapters or by any features included in the claims.

[0441] It should also be noted that the individual aspects described herein can be used individually or in combination, and thus details can be added to each individual aspect without adding details to another one of the aspects.

[0442] It should also be noted that this disclosure explicitly or implicitly describes features that can be used in neural network parameter encoders or neural network parameter update encoders (devices that provide coded representations of neural network parameters or updates thereto) and neural network parameter decoders or neural network parameter update decoders (devices that provide decoded representations of neural network parameters or neural network parameter updates based on coded representations). Thus, any of the features described herein can be used in the context of a neural network encoder as well as in the context of a neural network decoder.

[0443] Furthermore, features and functions disclosed herein with respect to a method may also be used in an apparatus (configured to perform such functions). Furthermore, any feature and function disclosed herein with respect to an apparatus may also be used in the corresponding method. In other words, the method disclosed herein may be complemented by any of the features and functions described with respect to the apparatus.

[0444] Additionally, any of the features and functions described herein may be implemented in hardware or software, or using a combination of hardware and software, as described in the "Alternative Implementations" section.

[0445] The following section may be entitled "Methods for Entropy Coding of Parameters for Incremental Updates of Neural Networks" and may include, for example, subsections or chapters 1-3.

[0446] In the following, aspects of embodiments of the present invention are disclosed. The following may provide a general overview of aspects of embodiments of the present invention. It should be noted that any embodiment defined by the claims may optionally be supplemented by any of the details (features and functions) described below. Also, the embodiments and aspects thereof described below may be used individually, and may optionally be supplemented by any of the features in each separate chapter and / or subchapter, or by any feature included in the claims, and / or by any of the details (features and functions) described in the above disclosure. An embodiment may include such aspects and / or features alone or in combination.

[0447] Embodiments and / or aspects of the present invention may describe methods for parameter coding of incremental updates to a set of neural network parameters (e.g., also referred to as weights, or weight parameter(s)), e.g., using entropy coding methods. For example, similar to coding of (e.g., complete) neural network parameters, the methods may include quantization, lossless encoding, and / or lossless decoding methods. For example, the incremental updates may not usually be sufficient to reconstruct a neural network model, but may provide, e.g., differential updates to an existing model. For example, many or even all existing methods for neural network compression (e.g., as provided in the MPEG-7 Part 17 Standard for Compression of Neural Networks for Multimedia Content Description and Analysis [2]) may be applicable, e.g., because their architectures, e.g., update architectures, may be similar or even identical to the associated complete neural network model.

[0448] The basic structure of having a base model and one or more incremental updates can enable new methods, for example, in context modeling for entropy coding as described in this disclosure. In other words, embodiments in accordance with the present invention may include a base model and one or more incremental updates using context modeling methods for entropy coding.

[0449] Embodiments and / or aspects of the present invention may be primarily directed to, for example, lossy coding of layers of neural network parameters in neural network compression, but may also be applied to other areas of lossy coding, i.e., embodiments in accordance with the present invention may, for example, additionally, include methods for lossy coding.

[0450] For example, a methodology or apparatus according to an embodiment of the present invention may be divided into different main parts, which may include or consist of at least one of the following: 1. Quantization 2. Lossless encoding 3. Lossless Decoding

[0451] In order to understand the main advantages of the embodiments of the present invention, a brief introduction to the topic of neural networks and related methods for parameter encoding is disclosed below. It should be noted that any aspect and / or feature disclosed below may be incorporated into the embodiments of the present invention and / or the embodiments of the present invention may be supplemented by such features and aspects.

[0452] 1. Application fields In its most basic form, neural networks may comprise, for example, a concatenation of affine transformations followed by, for example, element-wise nonlinear functions. These can be represented as directed acyclic graphs, as depicted in FIG. 7. FIG. 7 shows an example of a graph representation of a feedforward neural network, e.g., a feedforward neural network. Specifically, this two-layer neural network is a nonlinear function that maps a four-dimensional input vector onto the real number line. Each node 710 may contain a specific value that can be forward propagated to the next node, e.g., by multiplication with the respective weight value of the edge 720. All input values ​​may then be simply aggregated, for example.

[0453] Mathematically, the above neural network may or could calculate the output as follows, for example: Output = σ(W2 σ(W1 Input)) Here, W2 and W1 may be weight parameters (edge ​​weights) of the neural network, and σ may be some nonlinear function. For example, so-called convolutional layers may also be used, e.g., by casting as a matrix-matrix product, as described in [1]. The incremental updates may, for example, typically aim to provide updates to the weights W1 and / or W2, and may, for example, be the result of an additional training process. The updated versions of W2 and W1 may, for example, typically lead to revised outputs. From here on, the procedure of computing outputs from given inputs will be referred to as inference. Intermediate results will also be referred to as hidden layers or hidden activations, which may, for example, constitute a linear transformation plus element-wise nonlinearity, such as the first dot product plus nonlinearity computation above.

[0454] For example, neural networks typically may have millions of parameters and thus require hundreds of MB to represent. As a result, their inference procedures may require high computational resources to execute, for example, because they may involve many dot-product operations between large matrices. Therefore, reducing the complexity of performing these dot products may be of great importance.

[0455] 2. Aspects of the embodiment according to the present invention 2.1 Related Methods of Quantization and Entropy Coding The MPEG-7 Part 17 standard for the compression of neural networks for the description and analysis of multimedia content [2] provides various methods for the quantization of neural network parameters, such as independent scalar quantization and dependent scalar quantization (DQ or trellis-coded quantization (TCQ)). In addition, it also specifies an entropy quantization scheme known as deepCABAC [7]. For a deeper understanding, these methods are briefly summarized. More details can be found in [2]. It should be noted that embodiments according to the present invention (e.g., those described in Section 3 and defined by the claims) may include any features and / or aspects of the method or standard, and in particular may include the features and / or aspects described below, either alone or in combination.

[0456] 2.1.1 Scalar Quantizer (Optional, all details are optional) Neural network parameters may be quantized, for example, using a scalar quantizer. As a result of the quantization, the set of allowable values ​​for the parameters may be reduced, for example. In other words, the neural network parameters may be mapped to a countable set (e.g., in practice, a finite set) of so-called reconstruction levels. The set of reconstruction levels may represent a proper subset of the set of possible neural network parameter values. To simplify the entropy coding below, the allowable reconstruction levels may be represented by quantization indexes, which may be transmitted, for example, as part of the bitstream. On the decoder side, the quantization indexes may be mapped, for example, to reconstructed neural network parameters. The possible values ​​for the reconstructed neural network parameters may correspond to the set of reconstruction levels. On the encoder side, the result of the scalar quantization may be a set of (integer) quantization indexes.

[0457] According to an embodiment, for example, uniform reconstruction quantizers (URQs) can be used in this application. Their basic design is shown in Fig. 8. Fig. 8 is an illustrative example of a uniform reconstruction quantizer according to an embodiment of the present invention. URQs can have the property that the reconstruction levels are equally spaced. The distance Δ between two adjacent reconstruction levels is called the quantization step size. One of the reconstruction levels may, for example, be equal to 0. Therefore, the complete set of available reconstruction levels can, for example, be uniquely specified by the quantization step size Δ. The decoder mapping of the quantization index q to the reconstructed weight parameter t' can, in principle, be, for example, t'=q Δ It can be given by the simple formula:

[0458] In this context, the term “independent scalar quantization” may refer to the property that, for example, for a given quantization index q for any weight parameter, the associated reconstructed weight parameter t′ can be determined, for example, independently of all quantization indexes for other weight parameters.

[0459] 2.1.2 Dependent Scalar Quantization (Optional, all details are optional) In dependent scalar quantization (DQ), the allowable reconstruction level for a neural network parameter may depend, for example, on the quantization index selected for the preceding neural network parameter, for example, in the reconstruction order. The concept of dependent scalar quantization may be combined with modified entropy coding, in which the probability model selection (or, for example, alternatively, codeword table selection) for a neural network parameter may depend, for example, on the set of allowable reconstruction levels. An advantage of dependent quantization of neural network parameters may be, for example, that allowable reconstruction vectors can be packed more tightly into an N-dimensional signal space (where N indicates the number of samples or neural network parameters in a set of samples to be processed, e.g., a layer). A reconstruction vector for a set of neural network parameters may refer to ordered reconstructed neural network parameters (or, for example, alternatively, ordered reconstructed samples) of the set of neural network parameters. An example of the effect of dependent scalar quantization is shown in FIG. 9, for example, for the simplest case of two neural network parameters. FIG. 9 shows examples of allowable reconstruction vector locations for the simplest case of, for example, two weighting parameters: (a) independent scalar quantization (example) and (b) dependent scalar quantization, according to an embodiment of the present invention. FIG. 9(a) shows examples of allowable reconstruction vectors 910 (representing points in a 2D plane) for independent scalar quantization. As can be seen, the set of allowable values ​​for the second neural network parameter t1′ can be independent of the value chosen for the first reconstructed neural network parameter t0′. FIG. 9(b) shows an example of dependent scalar quantization. Note that, in contrast to independent scalar quantization, selectable reconstruction values ​​for the second neural network parameter t1′ can depend on the reconstruction level chosen for the first neural network parameter t0′.In the example of Figure 9(b), there are two different sets 920, 930 of available reconstruction levels for the second neural network parameter t1' (shown with different colors, hatching, or symbols of different types). If the quantization index for the first neural network parameter t0' is even (..., -2, 0, 2,...), any reconstruction level in the first set 920 (e.g., blue points, points with the first hatching, or symbols of the first type) may be selected for the second neural network parameter t1'. Also, if the quantization index for the first neural network parameter t0' is odd (..., -3, -1, 1, 3,...), any reconstruction level in the second set 930 (e.g., red points, points with the second hatching, or symbols of the second type) may be selected for the second neural network parameter t1'. In this example, the reconstruction levels for the first and second sets are shifted by half the quantization step size (any reconstruction level in the second set is located between two reconstruction levels in the first set).

[0460] Dependent scalar quantization of neural network parameters may have the effect of reducing the expected distance between a given input vector of neural network parameters and the closest available reconstructed vector, for example, for a given average number of reconstructed vectors per N-dimensional unit volume. For example, as a result, the average distortion between the input vector of neural network parameters and the vector-reconstructed neural network parameters may be reduced, for example, for a given average number of bits. In vector quantization, this effect may be referred to as a space-filling gain. Using dependent scalar quantization for a set of neural network parameters, for example, most of the potential space-filling gain of high-dimensional vector quantization may be utilized. Also, in contrast to vector quantization, the implementation complexity of the reconstruction process (or, for example, the decoding process) may be comparable to that of the associated neural network parameter coding, for example, using independent scalar quantizers.

[0461] As a result of the above-mentioned aspects, DQ, for example, can typically achieve the same distortion level at a lower bit rate.

[0462] 2.1.3 DQ in MPEG-7 Part 17 (Optional, all details optional) The MPEG-7 Part 17 standard for neural network compression for multimedia content description and analysis employs two quantizers, Q1 and Q2, with different sets of reconstruction levels. Each set may include, for example, integer multiples of the quantization step size Δ. Q1 may include, for example, all even multiples of the quantization step size and 0, while Q2 may include, for example, all odd multiples of the quantization step size and 0. This division of the reconstruction sets is illustrated in FIG. 10 , which shows an example of the division of a set of reconstruction levels into two subsets according to an embodiment of the present invention. The two subsets of quantization set 0 are labeled using “A” and “B,” and the two subsets of quantization set 1 are labeled using “C” and “D.”

[0463] The process for switching between sets may determine the quantizer to be applied based, for example, on the quantization index chosen for the preceding neural network parameter in the reconstruction order, or, for example, more precisely, on the parity of the previously coded quantization index. This switching process may be realized, for example, by a finite state machine having eight states (as depicted in FIG. 11), where each state may be associated, for example, with one of the quantizers Q1 or Q2. FIG. 11 shows a preferred example of a state transition table for a configuration having eight states.

[0464] Using the concept of state transitions, for example, the current state and thus the current quantization set may be uniquely determined by the previous state (e.g., in reconstruction order) and, for example, the previous quantization index.

[0465] 2.1.4 Entropy Coding (Optional, all details optional) For example, as a result of quantization applied in a previous step, the weight parameters may be mapped to a finite set of so-called reconstruction levels. These may be represented, for example, by (e.g., integer) quantizer indices (also called parameter levels or weight levels) and quantization step sizes, which may be fixed for the entire layer. For example, to reconstruct all quantized weight parameters of a layer, the step size and dimension of that layer may be known to the decoder. These may be transmitted separately, for example.

[0466] 2.1.4.1 Coding Quantization Indices Using Context-Adaptive Binary Arithmetic Coding (CABAC) The quantization indexes (integer representations) may then be transmitted, for example, using an entropy coding technique. For example, a layer of weights may then be mapped onto a sequence of quantized weight levels, for example, using a scan. For example, a row-first scan order may be used, starting from the top row of the matrix and encoding the contained values ​​from left to right. In this way, for example, all rows may be encoded from top to bottom. It should be noted that any other scan may be applied, for example. For example, before applying the row-first scan, the matrix may be transposed, or may be flipped horizontally and / or vertically, and / or rotated 90 degrees / 180 degrees / 270 degrees left or right.

[0467] For level coding, for example, CABAC (Context-Adaptive Binary Arithmetic Coding) may be used. For details, see for example [2]. Thus, the quantized weight level q may for example be decomposed into a series of binary symbols or syntax elements and then passed to for example a binary arithmetic coder (CABAC).

[0468] In the first step, for example, a binary syntax element sig_flag may be derived for the quantized weight level, which may, for example, specify whether the corresponding level is equal to zero. If sig_flag is equal to 1, for example, a further binary syntax element sign_flag may be derived. The bin may, for example, indicate whether the current weight level is positive (e.g., bin = 0) or negative (e.g., bin = 1).

[0469] For example, next, a sequence of bins may be followed by a fixed-length sequence, for example, and encoded as follows.

[0470] The variable k may, for example, be initialized with a non-negative integer, and X may, for example, be initialized with 1 << k.

[0471] One or more syntax elements abs_level_greater_X may, for example, be encoded, which can indicate that the absolute value of the quantized weight level is greater than X. If abs_level_greater_X is equal to 1, the variable k may, for example, be updated (e.g., incremented by 1), and then, for example, 1 << k may be added to X, and for example, a further abs_level_greater_X may be encoded. This procedure may continue until abs_level_greater_X becomes equal to 0. Thereafter, to complete the encoding of the quantizer index, a fixed-length code of length k may be sufficient. For example, the variable rem = X - |q| may, for example, be encoded using k bits or may be encoded. Or alternatively, the variable rem' may be defined as rem' = (1 << k) - rem - 1 or may be defined, which can be encoded using, for example, k bits. Alternatively, any other mapping of the variable rem to a fixed-length code of k bits may be used according to embodiments of the present invention.

[0472] When incrementing k by 1 after each abs_level_greater_X, this approach can be the same as applying exponential Golomb coding (for example, when the sign_flag is not considered).

[0473] In addition to this, when the maximum absolute value abs_max is known on both the encoder and decoder sides, for example, for the next abs_Level_greater_X to be transmitted, when X≥abs_max holds, the encoding of the abs_level_greater_X syntax element may be terminated.

[0474] 2.1.4.2 Decoding of Quantized Indexes Using Context-Adaptive Binary Arithmetic Coding (CABAC) The decoding of quantized weight levels (for example, integer representation) may operate in a similar manner to encoding, for example. The decoder may first decode the sig_flag. If this is equal to 1, a sequence of the sign_flag and abs_level_greater_X may follow, and the update of k (and, for example, thus the increment of X) may follow the same rules as in the encoder, or, for example, must follow. For example, finally, a fixed-length code of k bits may be decoded and interpreted as an integer (for example, as rem or rem’, depending on which of the two was encoded). The absolute value |q| of the decoded quantized weight level may then be reconstructed from X and form the fixed-length part. For example, if rem is used as the fixed-length part, |q| = X - rem. Or alternatively, if rem’ was encoded, |q| = X + 1 + rem’ - (1 << k). For example, as a final step, a sign may be applied, or, for example, depending on the decoded sign_flag, may need to be applied to |q|, for example resulting in the quantized weight level q. For example, finally, the quantized weight w may be reconstructed, for example, by multiplying the quantized weight level q by the step size Δ.

[0475] In one implementation variant, k may be initialized, for example, at 0 and updated as follows: after each abs_level_greater_X is equal to 1, for example, the required update of k may be done according to the following rule: if X>X', then k may be incremented by 1, and X' may be a constant that depends on the application. For example, X' may be a number (e.g., between 0 and 100) that may be derived, for example, by the encoder and signaled to the decoder.

[0476] 2.1.4.3 Context Modeling In CABAC entropy coding, for example, most syntax elements of quantized weight levels may be coded using binary probability modeling. Each binary decision (bin) may be associated with a context. The context may, for example, represent a probability model for the class of the coded bin. The probability of one of two possible bin values ​​may be estimated for each context, for example, based on the values ​​of the bins already coded using the corresponding context. For example, depending on the application, different context modeling techniques may be applied. For example, for some bins normally associated with quantized weight coding, the contexts that can be used for coding may be selected based on previously transmitted syntax elements. For example, depending on the actual application, different probability estimators can be chosen, such as SBMP [4], HEVC [5], or VTM-4.0 [6]. This choice may affect, for example, compression efficiency and / or complexity.

[0477] A context modeling scheme that can be adapted to a wide range of neural networks is described below. To decode a quantized weight level q, for example, at a particular position (x, y) in a weight matrix (layer), a local template may be applied to the current position. This template may include several other (ordered) positions, such as (x-1, y), (x, y-1), (x-1, y-1), etc. For example, for each position, a state identifier may be derived.

[0478] In one variant of implementation (denoted for example as Si1), the state identifier s for the position (x, y) x,y may be derived as follows: if the position (x,y) points outside the matrix, or if the quantized weight level q at the position (x,y) x,y If s is not yet decoded or is equal to zero, the state identifier s x,y = 0. Otherwise, the state identifier is s x,y =q x,y It can or should be <0?1:2.

[0479] For a particular template, a sequence of state identifiers may be derived, and each possible constellation of state identifier values ​​may be mapped to a context index that identifies the context in which it should be used. The templates and mappings may be different, for example, for different syntax elements. For example, a template with (e.g., ordered) positions (x-1, y), (x, y-1), (x-1, y-1) may be used to derive an ordered sequence of state identifiers s x-1,y , s x,y-1 , s x-1,y-1 For example, this sequence can be derived as follows: x-1,y +3*s x,y-1 +9*s x-1,y-1 For example, a context index C may be used to identify several contexts for sig_flag.

[0480] In one implementation variant (denoted as Scheme 1), the quantized weight level q at position (x,y) is x,y The local template for sig_flag or sign_flag of may, for example, consist of only one position (x-1, y) (i.e., for example, the left neighbor). x-1,y may be derived according to implementation variant Si1.

[0481] For sig_flag, for example, s x-1,y Depending on the value of , one of the three contexts may be selected, or for sign_flag, e.g., s x-1,y Depending on the value of , one of three other contexts may be selected.

[0482] In another implementation variation (denoted as Scheme 2), the local template for the sig flag may include three ordered positions (x-1, y), (x-2, y), (x-3, y). The sequence of associated state identifiers s x-1,y , s x-2,y , s x-3,y may be derived according to implementation variation Si2.

[0483] For sig_flag, the context index C may be derived, for example, as follows: s x-1,y If ≠0, then C=0. Otherwise, s x-2,y If ≠0, then C=1. Otherwise, s x-3,y If ≠ 0 then C=2. Otherwise C=3.

[0484] This can also be expressed by the following formula: C=(s x-1,y ≠0)?0:((s x-2,y ≠0)?1:((s x-3,y ≠0)?2:3)).

[0485] For example, in the same manner, the number of left neighbors may be increased or decreased so that the context index C is equal to the distance to the next non-zero weight left neighbor (eg, not exceeding the template size).

[0486] For example, each abs_level_greater_X flag may apply its own set of, for example, two contexts. Then, one of the two contexts may be chosen depending on, for example, the value of sign_flag.

[0487] In one implementation variant, different contexts may be distinguished, for example depending on X and / or on the value of sign_flag, for abs_level_greater_X flags where X is smaller than a predefined number X'.

[0488] In one implementation variant, different contexts may be distinguished, eg only depending on X, for abs_level_greater_X flags, where X is greater than or equal to a predefined number X'.

[0489] In another implementation variant, the abs_level_greater_X flags, where X is greater than or equal to a predefined number X', may be coded using a fixed code length of 1 (e.g., using the bypass mode of the arithmetic coder).

[0490] Furthermore, some or all of the syntax elements may be coded without the use of a context, for example, or may instead be coded with a fixed length of 1 bit, for example using the so-called bypass bins of CABAC.

[0491] In another implementation variant, a fixed length remainder part rem may be encoded, for example using a bypass mode.

[0492] In another implementation variant, the encoder may determine a predefined number X', and for example, depending on the code, may distinguish two contexts for each syntax element abs_level_greater_X where X < X', and for example, for each abs_level_greater_X where X >= X', one context may be used.

[0493] 2.1.4.4 Context Modeling for Dependent Scalar Quantization One of the main aspects of dependent scalar quantization, or for example the main aspect, may be that there can be different sets of acceptable reconstruction levels for neural network parameters (also called quantization sets for example). The current quantization set of neural network parameters may be determined based on, for example, the values of the quantization indices for the previous neural network parameters. Considering the preferred example in Figure 10 and comparing two quantization sets, it is clear that the distance between the reconstruction level equal to zero and the adjacent reconstruction level is greater in set 0 than in set 1. Therefore, the probability that the quantization index is equal to 0 increases when set 0 is used and decreases when set 1 is used. In one variant of the implementation, this effect may be utilized in entropy coding by switching the codeword table or probability model, for example, based on the quantization set (or state) used for the current quantization index.

[0494] It should be noted that, for example, for proper switching of codeword tables or probability models, the paths (e.g., associations with subsets of the quantization set used) of all preceding quantization indexes may be known, or, for example, must be known, when entropy decoding the current quantization index (or, for example, the corresponding binary decision of the current quantization index). For example, it may therefore be beneficial or even necessary for the neural network parameters to be coded in reconstruction order. Therefore, in one implementation variant, the coding order of the neural network parameters may be equal to their reconstruction order. Besides that aspect, any coding / reconstruction order of the quantization indexes may be possible, such as those specified in section 2.1.4.1 and / or any other, e.g., uniquely defined, order.

[0495] For example, at least some of the bins for absolute levels may typically be coded using adaptive probability models (also called contexts). In one implementation variant, the probability model for one or more bins may be selected based on, for example, a quantization set (or, for example, more generally, a corresponding state variable) for the corresponding neural network parameters. The chosen probability model may, for example, depend on multiple parameters or characteristics of the already transmitted quantization indexes, one of the parameters being a quantization set or state that can be applied to the quantization index being coded.

[0496] In another implementation variant, the syntax for transmitting the quantization index of a layer may include a bin that specifies whether the quantization index is equal to 0 or not. The probability model that can be used to code this bin may, for example, be selected from a set of two or more probability models. The selection of the probability model to be used may, for example, depend on the quantization set (i.e., for example, the set of reconstruction levels) that can be applied to the corresponding quantization index. In another implementation variant, the probability model to be used may, for example, depend on the current state variable (which may, for example, mean the quantization set to be used).

[0497] In a further implementation variant, the syntax for transmitting the quantization index of a layer may, for example, include a bin specifying whether the quantization index is greater than or less than zero. In other words, the bin may indicate the sign of the quantization index. The selection of the probability model to be used may, for example, depend on the quantization set (i.e., for example, the set of reconstruction levels) that can be applied to the corresponding quantization index. In another implementation variant, the probability model to be used may depend on the current state variable (which may represent the quantization set to be used).

[0498] In a further implementation variant, the syntax for transmitting the quantization index may include a bin that may specify whether the absolute value of the quantization index (e.g., neural network parameter level) is greater than X (see Section 2.1.4.1 for any details). The probability model that can be used to code this bin may, for example, be selected from a set of two or more probability models. The selection of the probability model used may depend on the quantization set (i.e., for example, the set of reconstruction levels) that can be applied to the corresponding quantization index. In another implementation variant, the probability model used may depend on the current state variable (e.g., the state variable refers to the quantization set used).

[0499] In one aspect according to an embodiment, dependent quantization of neural network parameters can be combined with entropy coding, in which case the selection of a probability model for one or more bins (also called quantization levels) of the binary representation of the quantization indexes may depend, for example, on a quantization set (e.g., a set of allowable reconstruction levels) and / or a corresponding state variable, for example, for the current quantization index. The quantization set (and / or state variable) may, for example, be given by the quantization index (and / or a subset of bins representing the quantization index) for the preceding neural network parameter in the coding order and reconstruction order.

[0500] In one variant of implementation, the described choice of probability model may be combined with, for example, one or more of the following entropy coding aspects: The absolute values ​​of the quantization indices may be transmitted using a binarization scheme, which may consist of, for example, several bins that can be coded using an adaptive probability model and, if the adaptively coded bins do not already completely specify the absolute values, a suffix part that can be coded in a bypass mode of the arithmetic coding engine (for example, a non-adaptive probability model with pmf(0.5,0.5) for all bins). In one implementation variant, the binarization used for the suffix part may, for example, depend on the values ​​of the quantization indices already transmitted. The binarization of the absolute value of a quantization index may, for example, include an adaptively coded bin that specifies whether the quantization index is not equal to 0 or not. The probability model (called a context) used to code this bin may be selected from a set of candidate probability models. The selected candidate probability model may be determined not only by the quantization set (e.g., a set of allowable reconstruction levels) and / or state variables for the current quantization index, but also, for example, by the quantization indexes already transmitted for the layer. In one implementation variant, the quantization set (and / or state variables) may determine a subset of available probability models (e.g., also called a context set), and the values ​​of already coded quantization indexes may determine the probability model used within this subset (context set). In other words, for example, according to an embodiment of the present invention, the subset of available probability models (e.g., also called a context set) may be determined based on the quantization set (and / or state variables), and / or the probability model used within this subset (context set) may be determined based on, for example, the values ​​of already coded quantization indexes.

[0501] In one implementation variant, the probability model to be used within a context set may be determined based on the values ​​of the quantization indices already coded, for example, in a local neighborhood of the current neural network parameters. In the following, some exemplary measures are listed that can be derived, for example, based on the values ​​of the quantization indices in a local neighborhood and then used, for example, to select a probability model for a given context set: - The sign of a quantization index that is not equal to 0, for example within a local neighborhood. The number of quantization indices, for example within a local neighborhood, that are not equal to 0. This number may possibly be clipped, for example, to a maximum value. the sum of the absolute values ​​of the quantization indices, for example in a local neighborhood. This value may be clipped to a maximum value, for example. the difference between the sum of the absolute values ​​of the quantization indices, for example in a local neighborhood, and the number of quantization indices, for example within the local neighborhood, that are not equal to 0. This number may for example be clipped to a maximum value.

[0502] The binarization of the absolute value of the quantization index may include one or more adaptively coded bins, which may, for example, specify whether the absolute value of the quantization index is greater than X. For example, the probability model (called a context) used to code these bins may be selected from a set of candidate probability models. The selected probability model may be determined not only by the quantization set (e.g., a set of allowable reconstruction levels) and / or state variables for the current quantization index, but also by the already transmitted quantization indexes for the layer. In one implementation variant, the quantization set (or state variables) may determine a subset of available probability models (also called a context set), and data on already coded quantization indexes may determine the probability model used within this subset (e.g., the context set). To select a probability model, embodiments of the present invention may use any of the methods described above (e.g., for bins specifying whether the quantization index is not equal to 0).

[0503] 3. Aspects of the Invention Embodiments in accordance with the present invention describe and / or include methods for encoding incremental updates of a neural network, e.g., a reconstructed network layer may be a combination of an existing base layer (e.g., of a base model) and one or more incremental update layers, which may, for example, be separately coded and / or transmitted.

[0504] 3.1 Concept of base model and updated model The concept according to the embodiment of the present invention introduces, for example, a neural network model according to clause 1, which can be considered as a complete model in the sense that for a given input, an output can be calculated. In other words, the embodiment according to the present invention may include a neural network model according to clause 1. This model is referred to as a base model N B Each base model is denoted as a base layer L B1 , L B2 , ..., L BJ The base layer may comprise a base value, which may be chosen, for example, to be efficiently represented and / or compressed / transmitted, for example, in a first step, such as in the first step of a method according to an embodiment of the present invention. For example, in addition to this, this concept may also include an updated model (N) that may have a similar or, for example, identical architecture to the base model. U1 , N U2 , ..., N UK ) is introduced. In other words, the embodiment in accordance with the present invention introduces an update model (N U1 , N U2 , ..., N UK ) The updated model may, for example, not be a complete model in the sense described above. Instead, it may be combined with the base model, for example using a synthesis method, so that, for example, the base model and the updated model form a new complete model N B1 This model itself can, for example, serve as a base model for further updated models. Uk is the update layer L Uk,1 , L Uk,2 , ..., L Uk,J The update layer may comprise layers denoted as: The update layer may, for example, include base values ​​that may be chosen so that they can be efficiently represented and / or compressed / transmitted separately.

[0505] The updated model may, for example, be the result of a (e.g., additional) training process applied to the base model, for example, at the encoder side. For example, several synthesis methods may be applied depending on the type of updates provided by the updated model. Note that the methods described within this invention are not limited to any particular type of update / synthesis method, and may, for example, be applicable to any architecture that uses a base model / updated model approach.

[0506] In a preferred embodiment, the kth updated model N Uk is the layer L with differential values Uk,j (also referred to as incremental update), and this difference value can be, for example, Bj is added to the corresponding layer of Nk,j Can be formed: L Nkj =L Bj +L Uk,j (for all j)

[0507] The new model layer can form a new (e.g., updated) model, which can then serve as the base model for the next incremental update, which can then be transmitted separately.

[0508] In a further preferred embodiment, the kth updated model is a layer L with scaling factor values. Uk,j , where the scaling factor values ​​can be, for example, the scaling factor values ​​for the corresponding base layer L Bj multiplied by the value of the new model L according to Nk,j Can be formed: L Nk,j =L Bj L Uk,j (for all j)

[0509] The new model layer can form a new (updated) model, which can then serve as the base model for the next incremental update, for example, which can then be transmitted separately.

[0510] Note that in some cases, the updated model may also include new layers that can replace one or more existing layers, e.g., instead of updating the layers as described above (i.e., for layer k, L Nk,j =L Uk,j (for all j)). However, according to an embodiment, any combination of the above mentioned updates may be performed.

[0511] 3.2 Neural network parameter coding for incremental updates The concept of a base model and one or more incremental updates according to embodiments of the present invention may be utilized, for example, in the entropy coding stage, for example, to improve coding efficiency. Layer parameters may, for example, typically be represented by multidimensional tensors. For the coding process, for example, several or all tensors may be mapped to a 2D matrix, for example, typically with row and column entities. This 2D matrix may, for example, be scanned in a predefined order, and the parameters may be coded / transmitted. Note that the method described below is not limited to 2D matrices. The method according to the embodiments can be applied to any representation of neural network parameters that provides parameter entities of known size, such as rows, columns, blocks, etc., and / or combinations thereof. In the following, a 2D matrix representation is used for a deeper understanding of the method. In general, according to embodiments, tensors containing information about neural network parameters may, for example, be mapped to rows and columns of a multidimensional matrix.

[0512] In a preferred embodiment, the layer parameters may be expressed as a 2D matrix, which may provide value entities such as rows and columns.

[0513] 3.2.1 Line or Channel Skip Mode For example, typically, the magnitude of the values ​​in the updated model may be small, e.g., compared to the full (base) model. For example, in many cases, a significant number of values ​​may be zero, which can be further amplified by the quantization process. For example, as a result, a layer to be transmitted may contain long sequences of zeros, which may mean that some of the rows of a 2D matrix may be entirely zero.

[0514] This may be exploited, for example, by introducing, for each row, a flag (skip_row_flag) that can specify whether all parameters in the row are equal to zero. If the flag is equal to 1 (skip_row_flag==1), no further parameters may be coded for that row. On the decoder side, if the flag is equal to 1, no parameters for this row may be decoded. Instead, these, for example, such parameters may be assumed to be 0.

[0515] A variation according to the embodiment herein is to arrange all skip_row_flags in an array of flags skip_row_flag[N], where N is the number of rows, and in one variation N can be or be signaled before the array.

[0516] For example, if the flag is equal to 0, parameters may be encoded and decoded periodically for this row.

[0517] For example, each skip_row_flag may be associated with a probability model or a context model, which may be selected from among a set of context models based on, for example, previously coded symbols (e.g., preceding coded parameters and / or skip_row_flag).

[0518] In a preferred embodiment, a single context model may be applied to all skip_row_flags of a layer.

[0519] In another preferred embodiment, a context model may be selected from a set of two context models, for example, based on the value of a previously coded skip_row_flag, which may be a first context model if the value of the preceding skip_row_flag is equal to zero and a second context model if the value is equal to 1. In other words, according to the embodiment, the first context model may be selected, for example, if the value of the preceding skip_row_flag is equal to zero, and the second context model may be selected, for example, if the value is equal to 1.

[0520] In a further preferred embodiment, a context model may be selected from a set of two context models based on, for example, the value of skip_row_flag at the same position in a corresponding layer of a previously coded update model and / or base model, where the context model may be the first context model if the preceding value of skip_row_flag is equal to zero and the second context model if the value is equal to 1. In other words, according to the embodiment, the first context model may be selected, for example, if the preceding value of skip_row_flag is equal to zero, and the second context model may be selected, for example, if the value is equal to 1.

[0521] In another preferred embodiment, for example, a given number of context models as in the above embodiments may be doubled to form two sets of context models. For example, then, the set of context models may be selected based on, for example, the value of the co-located skip_row_flag in the corresponding layer of a particular previously encoded update model and / or base model. This means that, for example, the first set may be selected if the value of the preceding skip_row_flag is equal to 0, and the second set may be selected if the value is equal to 1.

[0522] A further preferred embodiment may be the same as the above embodiments, but the first set of context models may be selected when there is no corresponding layer in a particular previously encoded update model and / or base model. For example, as a result, the second set may be selected when there is a corresponding layer in a particular previously encoded update model and / or base model.

[0523] The particular described mechanism for skipping rows may be similarly applicable to columns in the case of a 2D matrix and may also be applicable in the case of a generalized tensor having N parameter dimensions. For example, it should be noted that using the described mechanism of skip_flag and / or skip_flag_array, sub-blocks or sub-rows of a smaller dimension K (K < N) may be skipped, for example.

[0524] 3.2.2 Improved Context Modeling for the Base Model Update Model Structure The concept of a base model and one or more update models may be utilized, for example, in the entropy encoding stage. The method according to the embodiments described herein can be applicable to any entropy encoding scheme that uses a context model such as that described in Section 2.1.4.

[0525] For example, typically, separate update models (and, e.g., base models) may be correlated and may be available, e.g., at the encoder and decoder sides, which may be used in the context modeling stage, e.g., to improve coding efficiency, e.g., by providing new context models and / or methods for context model selection.

[0526] In a preferred embodiment, binarization (eg, including these flags, sig_flag, sign_flag, etc.), context modeling and / or encoding schemes according to section 2.1.4.1 may be applied.

[0527] In another preferred embodiment, a given number of context models (e.g., context sets) for a symbol to be coded may be replicated to form two or more sets of context models. For example, a set of context models may then be selected based on, for example, the values ​​of parameters at the same position in, for example, a particular previously coded update model and / or corresponding layer of the base model. This means that the first set may be selected if the parameters at the same position are less than a first threshold T1, the second set may be selected if the values ​​are greater than or equal to threshold T1, the third set may be selected if the values ​​are greater than or equal to threshold T2, and so on. This procedure may be applied with more or fewer thresholds, for example, any number of thresholds, e.g., a number that can be chosen depending on the particular application.

[0528] In a preferred embodiment, which may be equivalent to the above embodiment, a single threshold T1=0 may be used.

[0529] In another preferred embodiment, a given number of context models (e.g., context sets) for a symbol to be coded may be replicated to form two or more sets of context models. For example, the set of context models may then be selected based on a set of values, e.g., consisting of co-located parameters and / or neighboring values ​​(e.g., one or several spatial neighbors of the co-located parameters), in a corresponding layer of a particular previously coded update model and / or base model.

[0530] In a preferred embodiment, e.g., equivalent or partially equivalent to the above embodiment, if the sum of the values ​​(or, e.g., absolute values) in a template, e.g., a template of sets of values ​​consisting of co-located parameters and / or neighboring values, is less than a first threshold T1, then a first set, e.g., a first set of context models, is selected; if the sum is greater than or equal to threshold T1, then a second set, e.g., a second set of context models, is selected; if the sum is greater than or equal to threshold T2, then a third set is selected, etc. This procedure may be applied with more or fewer thresholds, e.g., any number of thresholds, e.g., a number that can be chosen depending on the particular application.

[0531] In a particularly preferred embodiment, eg, equivalent or partially equivalent to the above embodiment, the template may include the co-located parameter and the left neighbor of the co-located parameter, and a single threshold T1=0 may be used.

[0532] In another preferred embodiment, a context model from the set of context models for a syntax element may be selected based on a set of values, e.g., consisting of co-located parameters and / or neighboring values ​​(e.g., one or several spatial neighbors of the co-located parameters), e.g., in the corresponding layer of a particular previously coded update model and / or base model.

[0533] In a preferred embodiment, e.g., equivalent or equivalent in part to the above embodiments, if the sum of the values ​​(or, e.g., absolute values) in a template, e.g., a template of a set of values ​​consisting of co-located parameters and / or neighboring values, is less than a first threshold T1, a first context model may be selected, if the sum is greater than or equal to threshold T1, a second context model may be selected, if the sum is greater than or equal to threshold T2, a third context model may be selected, etc. This procedure may be applied with more or fewer thresholds, e.g., any number of thresholds, e.g., a number that can be chosen depending on the particular application.

[0534] In a particularly preferred embodiment, eg, equivalent or partially equivalent to the above embodiment, the template may include the co-located parameter and the left neighbor of the co-located parameter, and a single threshold T1=0 may be used.

[0535] In a further preferred embodiment, a given number of context models (e.g., context sets) for a symbol to be coded may be replicated to form two or more sets of context models. For example, a set of context models may then be selected based on the absolute values ​​of parameters at the same position, e.g., in the corresponding layer of a particular previously coded update model and / or base model. This means that the first set may be selected if the absolute values ​​of the parameters at the same position are smaller than a first threshold T1, the second set may be selected if the absolute values ​​are equal to or greater than the threshold T1, the third set may be selected if the absolute values ​​are equal to or greater than the threshold T2, and so on. This procedure may be applied with more or fewer thresholds, e.g., any number of thresholds, e.g., a number that can be selected depending on the particular application.

[0536] In a preferred embodiment that may be equivalent to the above embodiment, a sig_flag that can indicate whether the current value to be coded is equal to zero or not may be coded, which may utilize a set of context models. This embodiment may use a single threshold T1=1. According to the embodiment, the set of context models may be chosen depending on, for example, the sig_flag that indicates whether the current value to be coded is equal to zero or not.

[0537] Another preferred embodiment may be equal to the above embodiment, but instead of sig_flag, a sign_flag may be coded which can indicate the sign of the current value to be coded.

[0538] A further preferred embodiment may be equal to the above embodiment, but instead of sig_flag, abs_level_greater_X may be coded, which can indicate whether the current value to be coded is greater than X or not.

[0539] In a further preferred embodiment, a given number of context models (e.g., context sets) for symbols to be coded may be doubled to form two sets of context models. For example, the set of context models may then be selected depending on whether there is a corresponding previously coded update (and / or base) model. If there is no corresponding previously coded update (and / or base) model, the first set of context models may be selected, otherwise the second set may be selected.

[0540] In another preferred embodiment, a context model from the set of context models for a syntax element may be selected based on the value of a parameter at the same position, for example, in a particular corresponding previously coded update (and / or base) model. This means that a first model may be selected if the parameter at the same position is less than a first threshold T1, a second model may be selected if the value is greater than or equal to threshold T1, a third model may be selected if the value is greater than or equal to threshold T2, etc. This procedure may be applied with more or fewer thresholds, for example, any number of thresholds, e.g., a number that can be chosen depending on the particular application.

[0541] For example, in a preferred embodiment, which is equivalent to the above embodiment, a sign_flag may be encoded which may indicate the sign of the current value to be encoded. The first threshold for the context model selection process may be T1=0 and the second threshold may be T2=1.

[0542] In another preferred embodiment, a context model from the set of context models for a syntax element may be selected based on the absolute value of the parameter at the same position in a particular corresponding previously coded update (and / or base) model. This means that if the absolute value of the parameter at the same position is less than a threshold T1, a first model may be selected, if the value is greater than or equal to threshold T1, a second model may be selected, if the value is greater than or equal to threshold T2, a third model may be selected, etc. This procedure may be applied with more or fewer thresholds, e.g., any number of thresholds, e.g., a number that can be chosen depending on the particular application.

[0543] In a preferred embodiment, such as equivalent to the above embodiment, a sig_flag may be encoded that can indicate whether the current value to be encoded is equal to zero. For example, a device according to this embodiment may employ a first threshold value set to T1=1 and a second threshold value set to T2=2.

[0544] For example, in another preferred embodiment, which is equivalent to the above embodiment, instead of sig_flag, abs_level_greater_X may be coded, which may indicate whether the current value to be coded is greater than X. In addition to this, only one threshold may be employed, which may be set to T1=X.

[0545] It should be noted that any of the above-described embodiments and aspects and features thereof may be combined with one or more of the other embodiments and aspects and features thereof.

[0546] While some aspects are described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0547] The present coded representation of neural network parameters may be stored on a digital storage medium or transmitted over a transmission medium, such as a wireless or wired transmission medium, such as the Internet.

[0548] Depending on certain implementation requirements, embodiments of the present invention can be implemented in hardware or in software. Implementations can be carried out using a digital storage medium, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM or flash memory, on which electronically readable control signals are stored that cooperate (or can cooperate) with a programmable computer system so that the respective methods are performed.

[0549] Some embodiments according to the invention comprise a data carrier having an electronically readable control signal, which is capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0550] Generally, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine-readable carrier.

[0551] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0552] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0553] A further embodiment of the inventive method is, therefore, a data carrier (or digital storage medium, or computer readable medium) having recorded thereon a computer program for performing one of the methods described herein.

[0554] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, The data stream or the sequence of signals may for example be adapted to be transmitted via a data communication connection, for example via the Internet.

[0555] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0556] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0557] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0558] The described embodiments merely exemplify the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to others skilled in the art. It is therefore intended to be limited only by the scope of the appended claims, and not by the specific details presented by the description and illustration of the embodiments herein.

[0559] References [1] S. Chetlur et al., “cuDNN: Efficient Primitives for Deep Learning,”arXiv: 1410.0759, 2014 [2] MPEG, “Text of ISO / IEC DIS 15938-17 Compression of Neural Networks for Multimedia Content Description and Analysis”, Document of ISO / IEC JTC1 / SC29 / WG11, w19764, OnLine, Oct. 2020 [3] D. Marpe, H. Schwarz und T. Wiegand, “Context-Based Adaptive Binary Arithmetic Coding in the H.264 / AVC Video Compression Standard,” IEEE transactions on circuits and systems for video technology, Vol. 13, No. 7, pp. 620-636, July 2003. [4] H. Kirchhoffer, J. Stegemann, D. Marpe, H. Schwarz und T. Wiegand, “JVET-K0430-v3 - CE5-related: State-based probalility estimator,”in JVET, Ljubljana, 2018. [5] ITU - International Telecommunication Union, “ITU-T H.265 High efficiency video coding,”Series H: Audiovisual and multimedia systems - Infrastructure of audiovisual services - Coding of moving video, April 2015. [6] B. Bross, J. Chen und S. Liu, “JVET-M1001-v6 - Versatile Video Coding (Draft 4),”in JVET, Marrakech, 2019. [7] S. Wiedemann et al., “DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks,”in IEEE Journal of Selected Topics in Signal Processing, vol. 14, no. 4, pp. 700-714, May 2020, doi: 10.1109 / JSTSP.2020.2969554.

Claims

1. 1. An apparatus (150, 250) for decoding neural network parameters defining a neural network, comprising: the device is configured to decode an updated model (112, 162, 212, 262) defining modifications to one or more layers of the neural network; the apparatus is configured to modify parameters of a base model (184, 284) of the neural network using the updated model to obtain an updated model (108, 208); the apparatus is configured to evaluate skip information (164) indicating whether a sequence of parameters of the updated model is zero. Device.

2. 2. The device (150, 250) of claim 1, The updated model (112, 162, 212, 262) describes the difference values, the device is configured to additively or subtractively combine the difference values ​​with values ​​of the parameters of the base model (184, 284) to obtain values ​​of the parameters of the updated model (108, 208). Device.

3. 3. The device (150, 250) according to claim 1 or 2, The device includes a difference value or difference tensor L associated with the jth layer of the neural network. Ｕｋ，ｊ is a base value parameter or base value tensor L representing the value of the parameter of the j-th layer of the base model (184, 284) of the neural network. Ｂｊ and, L Ｎｋｊ =L Ｂｊ +L Ｕｋ，ｊ (for all j, or for all j for which the updated model contains a layer) and thereby generating an updated model value parameter or an updated model value tensor L representing the value of the parameter of the j-th layer of the updated model (108, 208) having a model index k of the neural network. Ｎｋｊ is configured to obtain Device.

4. An apparatus (150, 250) according to any one of claims 1 to 3, comprising: The updated model (112, 162, 212, 262) describes the scaling factor values; the apparatus is configured to scale values ​​of the parameters of the base model (184, 284) using the scaling factor values ​​to obtain values ​​of the parameters of the updated model (108, 208). Device.

5. An apparatus (150, 250) according to any one of claims 1 to 4, comprising: The apparatus includes a scaling value or scaling tensor L associated with the j-th layer of the neural network. Ｕｋ，ｊ is a base value parameter or base value tensor L representing the value of the parameter of the j-th layer of the base model (184, 284) of the neural network. Ｂｊ and, L Ｎｋｊ =L Ｂｊ ・L Ｕｋ，ｊ (for all j, or for all j for which the updated model contains a layer) and thereby generating an updated model value parameter or an updated model value tensor L representing the value of the parameter of the j-th layer of the updated model (108, 208) having a model index k of the neural network. Ｎｋｊ is configured to obtain Device.

6. An apparatus (150, 250) according to any one of claims 1 to 5, The updated model (112, 162, 212, 262) describes a replacement value; the apparatus is configured to use the substitution values ​​to substitute values ​​of the parameters of the base model to obtain values ​​of the parameters of the updated model (108, 208). Device.

7. An apparatus (150, 250) according to any one of claims 1 to 6, comprising: The neural network parameters include weight values ​​that define the weights of neuronal interconnections that arise from or lead to neurons. Device.

8. An apparatus (150, 250) according to any one of claims 1 to 7, comprising: The sequence of neural network parameters includes weight values ​​associated with rows or columns of a matrix. Device.

9. An apparatus (150, 250) according to any one of claims 1 to 8, comprising: the skip information (164) includes a flag indicating whether all parameters in the sequence of parameters of the updated model (112, 162, 212, 262) are zero; Device.

10. An apparatus (150, 250) according to any one of claims 1 to 9, comprising: the device is configured to selectively skip decoding a sequence of parameters of the updated model (112, 162, 212, 262) depending on the skip information (164). Device.

11. An apparatus (150, 250) according to any one of claims 1 to 10, comprising: the device is configured to selectively set values ​​of a sequence of parameters of the updated model (112, 162, 212, 262) to predetermined values ​​depending on the skip information (164). Device.

12. An apparatus (150, 250) according to any one of claims 1 to 11, comprising: the skip information (164) includes an array of skip flags indicating whether all parameters in each sequence of parameters of the updated model (112, 162, 212, 262) are zero; Device.

13. An apparatus (150, 250) according to any one of claims 1 to 12, comprising: the apparatus is configured to selectively skip decoding of respective sequences of parameters of the updated model (112, 162, 212, 262) depending on respective skip flags associated with each sequence of parameters. Device.

14. An apparatus (150, 250) according to any one of claims 1 to 13, comprising: the apparatus is configured to evaluate array size information describing the number of entries in the array of skip flags. Device.

15. An apparatus (150, 250) according to any one of claims 1 to 14, comprising: the apparatus is configured to decode one or more skip flags using a context model (264); the apparatus is configured to select a context model for decoding one or more skip flags in dependence on one or more previously decoded symbols. Device.

16. An apparatus (150, 250) according to any one of claims 1 to 15, comprising: the apparatus is configured to apply a single context model (264) for decoding all skip flags associated with layers of the neural network. Device.

17. An apparatus (150, 250) according to any one of claims 1 to 16, comprising: the apparatus is configured to select a context model (264) for decoding a skip flag depending on a previously decoded skip flag. Device.

18. An apparatus (150, 250) according to any one of claims 1 to 17, comprising: the apparatus is configured to select a context model (264) for decoding a skip flag depending on the value of a corresponding skip flag in a previously decoded neural network model. Device.

19. An apparatus (150, 250) according to any one of claims 1 to 18, comprising: the apparatus is configured to select a set of selectable context models (264) for decoding a skip flag depending on the value of a corresponding skip flag in a previously decoded neural network model. Device.

20. An apparatus (150, 250) according to any one of claims 1 to 19, comprising: the apparatus is configured to select a set of selectable context models (264) for decoding a skip flag depending on the existence of a corresponding layer in a previously decoded neural network model. Device.

21. An apparatus (150, 250) according to any one of claims 1 to 20, comprising: the apparatus is configured to select a context model (264) from the selected set of context models depending on one or more previously decoded symbols of the currently decoded update model (112, 162, 212, 262). Device.

22. 1. An apparatus (150, 250) for decoding neural network parameters defining a neural network, comprising: the device is configured to decode a current update model (112, 162, 212, 262) that defines modifications to one or more layers of the neural network, or modifications to one or more hidden layers of the neural network; the apparatus is configured to modify parameters of a base model (184, 284) of the neural network using the current update model, or intermediate parameters derived from the base model of the neural network using one or more intermediate update models, to obtain an updated model (108, 208); the apparatus is configured to entropy decode one or more parameters of the current updated model; the device is configured to adapt a context used for entropy decoding of one or more parameters of the current update model depending on one or more previously decoded parameters of the base model (184, 284) and / or depending on one or more previously decoded parameters of an intermediate update model (184, 284). Device.

23. 23. The device (150, 250) of claim 22, the apparatus is configured to decode the quantized and binarized representation of one or more parameters of the current updated model (112, 162, 212, 262) using context-based entropy decoding. Device.

24. 24. An apparatus (150, 250) according to claim 22 or 23, comprising: the device is configured to entropy decode at least one significance bin associated with a currently considered parameter value of the current updated model (112, 162, 212, 262), the significance bin describing whether a quantization index of the currently considered parameter value is equal to zero or not; Device.

25. An apparatus (150, 250) according to any one of claims 22 to 24, comprising: the apparatus is configured to entropy decode at least one code bin associated with a currently considered parameter value of the current updated model (112, 162, 212, 262), the code bin describing whether a quantization index of the currently considered parameter value is greater than or less than zero; Device.

26. An apparatus (150, 250) according to any one of claims 22 to 25, comprising: the device is configured to entropy decode a sequence of unary terms associated with a currently considered parameter value of the current updated model (112, 162, 212, 262), the bins of the sequence of unary terms describing whether the absolute value of a quantization index of the currently considered parameter value is greater than a respective bin weight or not; Device.

27. An apparatus (150, 250) according to any one of claims 22 to 26, comprising: the apparatus is configured to entropy decode one or more bins greater than X, which indicate whether the absolute value of a quantization index of a parameter value currently under consideration is greater than X, where X is an integer greater than zero; Device.

28. An apparatus (150, 250) according to any one of claims 22 to 27, comprising: the device is configured to select a context model (264) for decoding one or more bins of quantization indices of a currently considered parameter value depending on values ​​of previously decoded corresponding parameter values ​​in a previously decoded neural network model; Device.

29. An apparatus (150, 250) according to any one of claims 22 to 28, comprising: the device is configured to select a set of selectable context models (264) for decoding one or more bins of quantization indices of a parameter value currently under consideration depending on values ​​of corresponding previously decoded parameter values ​​in a previously decoded neural network model; Device.

30. An apparatus (150, 250) according to any one of claims 22 to 29, comprising: the device is configured to select a context model (264) for decoding one or more bins of quantization indices of a parameter value currently under consideration depending on the absolute values ​​of corresponding previously decoded parameter values ​​in a previously decoded neural network model, or the device is configured to select a set of context models for decoding one or more bins of quantization indices of the currently considered parameter value depending on absolute values ​​of previously decoded corresponding parameter values ​​in previously decoded neural network models. Device.

31. An apparatus (150, 250) according to any one of claims 22 to 30, comprising: the apparatus is configured to compare corresponding previously decoded parameter values ​​in the previously decoded neural network model with one or more thresholds; the device is configured to select a context model (264) for decoding one or more bins of quantization indexes of the currently considered parameter value depending on the result of the comparison, or the device is configured to select a set of context models for decoding one or more bins of quantization indexes of the currently considered parameter value depending on a result of the comparison. Device.

32. An apparatus (150, 250) according to any one of claims 22 to 31, comprising: the apparatus is configured to compare corresponding previously decoded parameter values ​​in the previously decoded neural network model with a single threshold; the device is configured to select a context model (264) for decoding one or more bins of quantization indexes of the parameter value currently under consideration depending on the result of the comparison with the single threshold value, or the device is configured to select a set of context models for decoding one or more bins of quantization indexes of the currently considered parameter value depending on the result of the comparison with the single threshold. Device.

33. An apparatus (150, 250) according to any one of claims 22 to 32, comprising: the apparatus is configured to compare absolute values ​​of corresponding previously decoded parameter values ​​in the previously decoded neural network model with one or more thresholds; the device is configured to select a context model (264) for decoding one or more bins of quantization indexes of the currently considered parameter value depending on the result of the comparison, or the device is configured to select a set of context models for decoding one or more bins of quantization indexes of the currently considered parameter value depending on a result of the comparison. Device.

34. An apparatus (150, 250) according to any one of claims 22 to 33, comprising: The apparatus entropy decodes at least one significance bin associated with a currently considered parameter value of the current updated model (112, 162, 212, 262), the significance bin describing whether a quantization index of the currently considered parameter value is equal to zero or not; selecting a context for the entropy decoding of the at least one significant bin or a set of contexts for the entropy decoding of the at least one significant bin depending on values ​​of corresponding previously decoded parameter values ​​in a previously decoded neural network model; configured to: Device.

35. An apparatus (150, 250) according to any one of claims 22 to 34, comprising: The apparatus entropy decodes at least one code bin associated with a currently considered parameter value of the current updated model (112, 162, 212, 262), the code bin describing whether a quantization index of the currently considered parameter value is greater than or less than zero; selecting a context for the entropy decoding of the at least one code bin or a set of contexts for the entropy decoding of the at least one code bin depending on values ​​of corresponding previously decoded parameter values ​​in a previously decoded neural network model; configured to: Device.

36. An apparatus (150, 250) according to any one of claims 22 to 35, comprising: The apparatus entropy decodes one or more bins greater than X, which indicate whether the absolute value of a quantization index of a parameter value currently under consideration is greater than X, where X is an integer greater than zero; selecting a context for the entropy decoding of at least one bin greater than X or a set of contexts for the entropy decoding of the at least one bin greater than X depending on values ​​of corresponding previously decoded parameter values ​​in a previously decoded neural network model; configured to: Device.

37. An apparatus (150, 250) according to any one of claims 22 to 36, comprising: the device is configured to select a context model (264) from among a selected set of context models depending on one or more previously decoded bins or parameters of the current updated model (112, 162, 212, 262). Device.

38. 1. An apparatus (100, 210) for encoding neural network parameters defining a neural network, comprising: the device is configured to encode an update model (112, 162, 212, 262) defining modifications to one or more layers of the neural network; the device is configured to provide an updated model such that the updated model enables a decoder to use the updated model to modify parameters of a base model (104, 204) of the neural network to obtain an updated model (108, 208); the device is configured to provide and / or determine skip information (114) indicating whether a sequence of parameters of the updated model is zero or not; Device.

39. 39. The device (100, 210) of claim 38, the updated model (112, 162, 212, 262) describes difference values ​​that allow the decoder to combine the difference values ​​additively or subtractively with the parameter values ​​of the base model (104, 204) to obtain the parameter values ​​of the updated model (108, 208); Device.

40. 40. The device (100, 210) of claim 39, the device is configured to determine the difference value as a difference between a value of a parameter of the updated model (108, 208) and a value of a parameter of the base model (104, 204). Device.

41. 41. An apparatus (100, 210) according to claim 39 or 40, comprising: The device includes a difference value or difference tensor L associated with the jth layer of the neural network. Ｕｋ，ｊ is a base value parameter or base value tensor L representing the value of the parameter of the j-th layer of the base model (104, 204) of the neural network. Ｂｊ With, L Ｎｋｊ =L Ｂｊ +L Ｕｋ，ｊ (for all j, or for all j for which the updated model contains a layer) The difference value or difference tensor L Ｕｋ，ｊ represents the value of the parameter of the j-th layer of the updated model (108, 208) having model index k of the neural network. Ｎｋｊ configured to determine, so as to enable determination of Device.

42. An apparatus (100, 210) according to any one of claims 38 to 41, comprising: The updated model (112, 162, 212, 262) describes the scaling factor values; the apparatus is configured to provide the scaling factor values ​​such that scaling of values ​​of parameters of the base model (104, 204) using the scaling factor values ​​results in values ​​of parameters of the updated model (108, 208). Device.

43. 43. The device (100, 210) of claim 42, the device is configured to determine the scaling factor value as a scaling factor between the value of the parameter of the updated model (108, 208) and the value of the parameter of the base model (104, 204). Device.

44. 44. The device (100, 210) of claim 43, The apparatus includes a scaling value or scaling tensor L associated with the j-th layer of the neural network. Ｕｋ，ｊ is a base value parameter or base value tensor L representing the value of the parameter of the j-th layer of the base model (104, 204) of the neural network. Ｂｊ With, L Ｎｋｊ =L Ｂｊ ・L Ｕｋ，ｊ (for all j, or for all j for which the updated model contains a layer) the combination of scaling values ​​or scaling tensors according to: Ｎｋｊ configured to determine, so as to enable determination of Device.

45. An apparatus (100, 210) according to any one of claims 38 to 44, comprising: The updated model (112, 162, 212, 262) describes a replacement value; the device is configured to provide the replacement values ​​such that replacement of values ​​of parameters of the base model (104, 204) using the replacement values ​​allows obtaining values ​​of parameters of the updated model (108, 208). Device.

46. 46. ​​The device (100, 210) of claim 45, the device is configured to determine the replacement value; Device.

47. An apparatus (100, 210) according to any one of claims 38 to 46, comprising: The neural network parameters include weight values ​​that define the weights of neuronal interconnections that arise from or lead to neurons. Device.

48. An apparatus (100, 210) according to any one of claims 38 to 47, comprising: The sequence of neural network parameters includes weight values ​​associated with rows or columns of a matrix. Device.

49. An apparatus (100, 210) according to any one of claims 38 to 48, comprising: the skip information (114) includes a flag indicating whether all parameters in the sequence of parameters of the updated model (112, 162, 212, 262) are zero; Device.

50. An apparatus (100, 210) according to any one of claims 38 to 49, comprising: the device is configured to provide skip information (114) signaling a skip of decoding a sequence of parameters of the updated model (112, 162, 212, 262). Device.

51. An apparatus (100, 210) according to any one of claims 38 to 50, comprising: the device is configured to provide skip information including information on whether a sequence of parameters of the updated model (112, 162, 212, 262) has a predetermined value; Device.

52. An apparatus (100, 210) according to any one of claims 38 to 51, comprising: the skip information (114) includes the array of skip flags indicating whether all parameters in each sequence of parameters of the updated model are zero; Device.

53. An apparatus (100, 210) according to any one of claims 38 to 52, comprising: the device is configured to provide a skip flag associated with each sequence of parameters of the updated model (112, 162, 212, 262) for signaling a skip of decoding of the respective sequence of parameters. Device.

54. An apparatus (100, 210) according to any one of claims 38 to 53, comprising: the apparatus is configured to provide array size information describing the number of entries in an array of skip flags; Device.

55. An apparatus (100, 210) according to any one of claims 38 to 54, comprising: the apparatus is configured to encode one or more skip flags using a context model (264); the apparatus is configured to select a context model for encoding one or more skip flags in dependence on one or more previously encoded symbols. Device.

56. An apparatus (100, 210) according to any one of claims 38 to 55, comprising: the apparatus is configured to apply a single context model (264) for encoding all skip flags associated with layers of the neural network. Device.

57. An apparatus (100, 210) according to any one of claims 38 to 56, comprising: the apparatus is configured to select a context model (264) for encoding a skip flag in dependence on a previously encoded skip flag. Device.

58. An apparatus (100, 210) according to any one of claims 38 to 57, comprising: the apparatus is configured to select a context model (264) for encoding a skip flag depending on the value of a corresponding skip flag in a previously encoded neural network model. Device.

59. An apparatus (100, 210) according to any one of claims 38 to 58, comprising: the apparatus is configured to select a set of selectable context models (264) for encoding a skip flag depending on the value of a corresponding skip flag in a previously encoded neural network model. Device.

60. An apparatus (100, 210) according to any one of claims 38 to 59, comprising: the apparatus is configured to select a set of selectable context models (264) for encoding a skip flag depending on the existence of a corresponding layer in a previously encoded neural network model. Device.

61. An apparatus (100, 210) according to any one of claims 38 to 60, comprising: the device is configured to select a context model (264) from among a selected set of context models depending on one or more previously coded symbols of a currently coded update model (112, 162, 212, 262). Device.

62. 1. An apparatus (100, 210) for encoding neural network parameters defining a neural network, comprising: the device is configured to encode a current update model (112, 162, 212, 262) that defines modifications to one or more layers of the neural network, or modifications to one or more hidden layers of the neural network; the device is configured to provide the update model (112, 162, 212, 262) such that the update model enables a decoder to modify parameters of the base model (104, 204) of the neural network using the current update model or intermediate parameters derived from the base model (104, 204) of the neural network using one or more intermediate update models to obtain an updated model (108, 208); the apparatus is configured to entropy encode one or more parameters of the current updated model; the device is configured to adapt a context used for entropy coding of one or more parameters of the current update model depending on one or more previously coded parameters of the base model (104, 204) and / or depending on one or more previously coded parameters of intermediate update models. Device.

63. 63. The device (100, 210) of claim 62, the apparatus is configured to encode a quantized and binarized representation of one or more parameters of the current updated model (112, 162, 212, 262) using context-based entropy coding. Device.

64. 64. Apparatus (100, 210) according to claim 62 or 63, comprising: the device is configured to entropy code at least one significance bin associated with a currently considered parameter value of the current updated model (112, 162, 212, 262), the significance bin describing whether a quantization index of the currently considered parameter value is equal to zero or not; Device.

65. Apparatus (100, 210) according to any one of claims 62 to 64, comprising: the apparatus is configured to entropy code at least one code bin associated with a currently considered parameter value of the current updated model (112, 162, 212, 262), the code bin describing whether a quantization index of the currently considered parameter value is greater than or less than zero; Device.

66. 66. Apparatus (100, 210) according to any one of claims 62 to 65, comprising: the device is configured to entropy code a unary sequence associated with a currently considered parameter value of the current updated model (112, 162, 212, 262), the bins of the unary sequence describing whether the absolute value of a quantization index of the currently considered parameter value is greater than a respective bin weight or not; Device.

67. 67. Apparatus (100, 210) according to any one of claims 62 to 66, comprising: the apparatus is configured to entropy encode one or more bins greater than X, the bins indicating whether the absolute value of a quantization index of a parameter value currently under consideration is greater than X, where X is an integer greater than zero; Device.

68. 68. Apparatus (100, 210) according to any one of claims 62 to 67, comprising: the device is configured to select a context model (224) for encoding one or more bins of quantization indices of a parameter value currently under consideration depending on values ​​of previously encoded corresponding parameter values ​​in a previously encoded neural network model. Device.

69. Apparatus (100, 210) according to any one of claims 62 to 68, comprising: the device is configured to select a set of selectable context models (224) for encoding one or more bins of quantization indices of a parameter value currently under consideration depending on values ​​of corresponding previously encoded parameter values ​​in a previously encoded neural network model; Device.

70. 70. Apparatus (100, 210) according to any one of claims 62 to 69, comprising: the device is configured to select a context model (224) for encoding one or more bins of quantization indices of a parameter value currently under consideration depending on the absolute value of a corresponding previously encoded parameter value in a previously encoded neural network model, or the device is configured to select a set of context models for encoding one or more bins of quantization indices of the currently considered parameter value depending on absolute values ​​of previously encoded corresponding parameter values ​​in previously encoded neural network models. Device.

71. An apparatus (100, 210) according to any one of claims 62 to 70, comprising: the apparatus is configured to compare corresponding previously coded parameter values ​​in a previously coded neural network model with one or more thresholds; the device is configured to select a context model (264) for the coding of one or more bins of quantization indexes of the parameter value currently under consideration depending on the result of the comparison, or the device is configured to select a set of context models (224) for encoding one or more bins of quantization indexes of the currently considered parameter value depending on the result of the comparison. Device.

72. An apparatus (100, 210) according to any one of claims 62 to 71, comprising: the apparatus is configured to compare corresponding previously coded parameter values ​​in a previously coded neural network model with a single threshold; the device is configured to select a context model (224) for the encoding of one or more bins of quantization indexes of the currently considered parameter value depending on the result of the comparison with the single threshold value, or the device is configured to select a set of context models for encoding one or more bins of quantization indexes of the currently considered parameter value depending on the result of the comparison with the single threshold. Device.

73. An apparatus (100, 210) according to any one of claims 62 to 72, comprising: the apparatus is configured to compare absolute values ​​of corresponding previously coded parameter values ​​in the previously coded neural network model with one or more thresholds; the device is configured to select a context model (224) for the encoding of one or more bins of quantization indexes of the parameter value currently under consideration depending on the result of the comparison, or the device is configured to select, depending on a result of the comparison, a set of context models for encoding one or more bins of quantization indexes of the currently considered parameter value. Device.

74. Apparatus (100, 210) according to any one of claims 62 to 73, comprising: The apparatus entropy encodes at least one significance bin associated with a currently considered parameter value of the current updated model (112, 162, 212, 262), the significance bin describing whether a quantization index of the currently considered parameter value is equal to zero or not; selecting a context (224) for the entropy coding of the at least one significant bin or a set of contexts for the entropy coding of the at least one significant bin depending on values ​​of corresponding previously coded parameter values ​​in a previously coded neural network model; configured to: Device.

75. Apparatus (100, 210) according to any one of claims 62 to 74, comprising: The apparatus entropy encodes at least one code bin associated with a currently considered parameter value of the current updated model (112, 162, 212, 262), the code bin describing whether a quantization index of the currently considered parameter value is greater than or less than zero; selecting a context (224) for the entropy coding of the at least one code bin or a set of contexts for the entropy coding of the at least one code bin depending on values ​​of corresponding previously coded parameter values ​​in a previously coded neural network model; configured to: Device.

76. An apparatus (100, 210) according to any one of claims 62 to 75, comprising: The apparatus entropy encodes one or more bins greater than X, which indicate whether the absolute value of a quantization index of a parameter value currently under consideration is greater than X, where X is an integer greater than zero; selecting a context (224) for the entropy coding of at least one bin greater than X, or a set of contexts for the entropy coding of the at least one bin greater than X, depending on values ​​of corresponding previously coded parameter values ​​in a previously coded neural network model; configured to: Device.

77. An apparatus (100, 210) according to any one of claims 62 to 76, comprising: the device is configured to select a context model (224) from among a selected set of context models depending on one or more previously coded bins or parameters of the current updated model (112, 162, 212, 262). Device.

78. 1. A method (300) for decoding neural network parameters defining a neural network, the method comprising: Decoding (310) an updated model (112, 162, 212, 262) defining modifications to one or more layers of the neural network; modifying (320) parameters of the base model of the neural network using the updated model to obtain an updated model (108, 208); evaluating (330) skip information (164) indicating whether the sequence of parameters of the updated model is zero; A method comprising:

79. 1. A method (400) for decoding neural network parameters defining a neural network, the method comprising: Decoding (410) a current update model (112, 162, 212, 262) that defines modifications to one or more layers of the neural network, or modifications to one or more hidden layers or to the neural network; modifying (420) parameters of a base model of the neural network using the current update model, or intermediate parameters derived from the base model of the neural network using one or more intermediate update models, to obtain an updated model (108, 208); entropy decoding (430) one or more parameters of the current updated model; adapting (440) a context (264) used for entropy decoding of one or more parameters of the current update model depending on one or more previously decoded parameters of the base model and / or depending on one or more previously decoded parameters of intermediate update models; A method comprising:

80. 1. A method (500) for encoding neural network parameters defining a neural network, the method comprising: encoding (510) an update model (112, 162, 212, 262) defining modifications to one or more layers of the neural network; providing (520) the updated model to modify parameters of the base model (104, 204) of the neural network using the updated model to obtain an updated model (108, 208); providing and / or determining (530) skip information (114) indicating whether the sequence of parameters of the updated model is zero; A method comprising:

81. 1. A method (600) for encoding neural network parameters defining a neural network, the method comprising: Encoding (610) a current updated model (112, 162, 212, 262) that defines modifications to one or more layers of the neural network, or modifications to one or more hidden layers or to the neural network; using the current update model to modify parameters of the base model (104, 204) of the neural network, or intermediate parameters derived from the base model of the neural network using one or more intermediate update models, to obtain an updated model (108, 208); entropy encoding (620) one or more parameters of the current updated model; adapting (630) a context (224) used for entropy coding of one or more parameters of the current update model (112, 162, 212, 262) depending on one or more previously coded parameters of the base model and / or depending on one or more previously coded parameters of intermediate update models; A method comprising:

82. A computer program for performing the method according to any one of claims 78 to 81 when said computer program runs on a computer.

83. an encoded representation of neural network parameters, an update model (112, 162, 212, 262) that defines modifications to one or more layers of the neural network; skip information indicating whether the sequence of parameters of the updated model is zero; and The encoded representation includes:

Images