Neural network representation format

Abstract

To provide a concept for the efficient utilization of a neural network and / or, efficient transmission and / or update of a neural network.SOLUTION: A data stream (45) is an encoded representation of a neural network (NN). The data stream (45) includes serialization parameters (102) indicating an encoding order (104) in which neural network parameters (32) defining neural network (NN) neuron interconnections are encoded in the data stream (45).SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a concept regarding the representation format of a neural network. [Background technology]

[0002] Neural networks (NNs) are currently making breakthroughs in many applications. -Detection and classification of objects in image / video data - Voice recognition and keyword recognition -Speech synthesis -Optical character recognition - Language Translation - etc.

[0003] However, the amount of data required to represent a NN can be huge, making it difficult to apply in some use cases. This data often consists of two types of parameters: "weights" and "biases" that represent the connections between neurons. Weights are usually parameters that perform some linear transformation (e.g., dot product or convolution) on the input values, in other words, parameters that weight the input of the neuron, and biases are offsets that are added after a linear calculation, in other words, to offset the aggregation of weighted messages received by the neuron. More specifically, these weights, biases, and further parameters that characterize each connection between two of the potentially very large number of neurons (up to tens of millions) in each layer of the NN (up to hundreds) occupy the majority of the data associated with a particular NN. Moreover, these parameters are typically composed of floating-point date types with large sizes. These parameters are usually represented as large tensors that contain all the parameters of each layer. If an application requires frequent transmission or updating of the associated NN, the required data rate becomes a serious bottleneck. Therefore, lossy compression of these matrices is a promising approach to reduce the encoding size of the NN representation.

[0004] Typically, parameter tensors are stored in a container format (e.g., ONNX (Open Neural Network Exchange), Pytorch, TensorFlow) that carries all the data needed to fully reconstruct and run the NN (e.g., the parameter matrix above) as well as further properties (e.g., parameter tensor dimensionality, layer types, operations, etc.).

[0005] It would be advantageous to have at hand a concept that makes the transmission / update of machine learning predictors, or alternatively, machine learning models such as neural networks, more efficient in terms of preserving inference quality while simultaneously reducing the coded size of the NN representation, the computational inference complexity, the complexity of writing or storing the NN representation, or that allows for more frequent transmission / update of the NN than currently, or that improves inference quality for the particular task at hand and / or for the particular local input data statistics. Furthermore, it would be advantageous to provide neural network representations, derivations of such neural network representations, and uses of such neural network representations in making predictions based on neural networks, such that the use of neural networks is more effective than currently possible. Summary of the Invention [Problem to be solved by the invention]

[0006] It is therefore an object of the present invention to provide a concept for an efficient utilization of neural networks and / or for an efficient transmission and / or updating of neural networks. This object is achieved by the subject matter of the independent claims of the present application. [Means for solving the problem]

[0007] Further embodiments according to the invention are defined by the subject matter of the dependent claims of the present application.

[0008] The idea underlying the first aspect of the present invention is that a neural network (NN) can be used more efficiently if serialization parameters are coded / decoded into a data stream in which a representation of the NN is coded. The serialization parameters indicate a coding order in which the NN parameters defining the neuronal interconnections of the NN are coded into the data stream. The neuronal interconnections may represent connections between neurons of different NN layers of the NN. In other words, the NN parameters may define connections between a first neuron associated with a first layer of the NN and a second neuron associated with a second layer of the NN. The decoding can use the coding order to assign the NN parameters successively decoded from the data stream to the neuronal interconnections.

[0009] In particular, it is found that the serialization parameters can be used to efficiently split the bit string into a meaningful contiguous subset of NN parameters. The serialization parameters may indicate a grouping of NN parameters that allows efficient execution of the NN. This may be done depending on the application scenario of the NN. For different application scenarios, the encoding may scan the NN parameters using different encoding orders. Thus, the NN parameters may be encoded using individual encoding orders that depend on the application scenario of the NN, and the decoding may reconstruct the NN parameters accordingly during decoding by the information provided by the serialization parameters. The NN parameters may represent entries of one or more parameter matrices or tensors, and the parameter matrices or tensors may be used for the inference procedure. It is found that one or more parameter matrices or tensors of the NN are efficiently reconstructed by the decoder based on the decoded NN parameters and the serialization parameters.

[0010] In this way, serialized parameters can use different application-specific encoding orders, allowing for flexible encoding and decoding with improved efficiency. For example, encoding parameters along different dimensions may result in compression performance benefits, as the entropy coder can better capture dependencies between them. In another example, it may be desirable to group parameters according to some application-specific criteria, i.e., which part of the input data they relate to, or whether they are jointly executable, so that they can be decoded / input in parallel. A further example is encoding parameters according to the General Matrix Matrix (GEMM) product scan order, which supports efficient memory allocation of decoded parameters when performing dot product operations (Andrew Kerr, 2017).

[0011] Further embodiments are directed to encoder-side selective reordering of the data, for example to achieve energy compaction of the NN parameters to be encoded, and then process / serialize / encode the resulting reordered data according to the resulting order. Thus, the reordering can reorder the parameters to have the same increase or decrease steadily along the encoding order.

[0012] According to a second aspect of the present application, the inventors of the present application provide a method for encoding / decoding numerical computation representation parameters to / from a data stream in which the representation of the NN is encoded, the method comprising: The inventors have noticed that the use of the NN (Non- ...

[0013] This embodiment is based on the idea that it is advantageous to express the NN parameters and the activation values ​​obtained as a result of using the NN parameters in inference using a NN with the same numerical representation and bit size. Based on the numerical calculation representation parameter, it is possible to efficiently compare the indicated numerical representation and bit size of the NN parameters with the numerical representation and bit size of the activation values. This is considered to be particularly advantageous when the numerical calculation representation parameter indicates a fixed-point representation as the numerical representation. This is because if both the NN parameters and the activation values ​​can be expressed in fixed-point representation, inference can be performed efficiently by fixed-point arithmetic.

[0014] According to a third aspect of the present application, the inventors of the present application have realized that a neural network can be used more efficiently when an NN layer type parameter is coded / decoded to / from a data stream in which a representation of the NN is coded. The NN layer type parameter indicates an NN layer type (e.g., a convolutional layer type or a fully connected layer type) of a given NN layer of the NN. The data stream is structured into one or more individually accessible parts, each individually accessible part representing a corresponding NN layer of the NN. The given NN layer represents one of the NN layers of the neural network. Optionally, an NN layer type parameter is coded / decoded to / from the data stream for each of two or more given NN layers of the NN, and the NN layer type parameter can be different between at least some of the given NN layers.

[0015] This embodiment is based on the idea that it may be useful for the data stream to include NN layer type parameters of the NN layers, for example to understand the meaning of the dimensions of the parameter tensors / matrices. Furthermore, different layers may be treated differently in encoding important information that the decoder should know before decoding, for example by using different sets or modes of context models, to better capture the data dependencies and lead to higher coding efficiency.

[0016] Similarly, it may be advantageous to encode / decode a type parameter into / from the data stream indicating a parameter type of the NN parameter. The type parameter may indicate whether the NN parameter represents a weight or a bias. The data stream is structured into one or more individually accessible portions, each individually accessible portion representing a corresponding NN layer of the NN. The individually accessible portions representing a corresponding given NN layer may be further structured into individually accessible subportions. Each individually accessible subportion is completely scanned by the coding order before a subsequent individually accessible subportion is scanned by the coding order. Each individually accessible subportion may have, for example, a NN parameter and a type parameter encoded and decoded. The NN parameters of a first individually accessible subportion may be of a different parameter type than the NN parameters of a second individually accessible subportion, or may be of the same parameter type. Different types of NN parameters associated with the same NN layer may be encoded / decoded into / from different individually accessible subportions associated with the same individually accessible portion. The distinction of parameter types may be beneficial for encoding / decoding, for example, when different types of dependencies can be used for each type of parameter, or when parallel decoding is desired. For example, the same N It is possible to encode / decode different types of NN parameters associated with N layers in parallel, which may result in better compression performance since it improves the efficiency of encoding / decoding the NN parameters and also allows the entropy coder to better capture dependencies between NN parameters.

[0017] According to a fourth aspect of the present application, the inventors have discovered that transmitting / updating a neural network is very efficient if the representation of the NN is encoded / decoded into / from the encoded data stream by a pointer. This is because the data stream is organized into individually accessible parts and for each of the one or more predefined individually accessible parts, a pointer points to the beginning of the respective predefined individually accessible part. Not all individually accessible parts need to be predefined individually accessible parts, but all individually accessible parts may represent predefined individually accessible parts. The one or more predefined individually accessible parts may be set by default or may be set depending on the application of the NN encoded into the data stream. The pointer indicates the beginning of each predefined individually accessible part, for example as a data stream position in bytes or as an offset, for example relative to the beginning of the data stream or relative to the beginning of the part corresponding to the NN layer to which each predefined individually accessible part belongs. The pointer may be encoded / decoded into / from a header part of the data stream. According to an embodiment, for each of one or more predetermined individually accessible portions, a pointer is encoded / decoded in a header portion of the data stream if each predetermined individually accessible portion represents a corresponding NN layer of a neural network, and a pointer is encoded / decoded in a parameter set portion of the corresponding portion of the NN layer if each predetermined individually accessible portion represents an NN layer of a NN. The NN portion of the NN layer of a NN may represent a baseline portion of the respective NN layer or may represent an advanced section of the respective layer. The use of pointers allows efficient access to the predetermined individually accessible portions of the data stream, for example, to enable parallelization of layer processing or packaging of the data stream into respective container formats.The use of pointers allows easier, faster and better access to given individually accessible parts, facilitating applications that require parallel or partial decoding and execution of NNs.

[0018] According to a fifth aspect of the present application, the inventors of the present application have realized that transmission / updating of a neural network becomes highly efficient if a start code, pointer, and / or data stream length parameter is encoded into / decoded from an individually accessible sub-portion of the data stream in which a representation of the NN is encoded. The data stream is structured into one or more individually accessible portions, each individually accessible portion representing a corresponding NN layer of the neural network. Furthermore, the data stream is further structured into individually accessible sub-portions within one or more predefined individually accessible portions, each individually accessible sub-portion representing a corresponding NN portion of a respective NN layer of the neural network. The device is configured to encode / decode into / from the data stream, for each of the one or more predefined individually accessible sub-portions, a start code at which each predefined individually accessible sub-portion begins, and / or a pointer to the beginning of each predefined individually accessible sub-portion, and / or a data stream length parameter indicating the data stream length of each predefined individually accessible sub-portion to skip each predefined individually accessible sub-portion when parsing the DS. The start code, pointer, and / or data stream length parameters allow efficient access to predetermined, individually accessible sub-portions. This allows the NN parameters to be partially or partially This is particularly useful for applications that may rely on grouping of NN parameters in a NN layer in a certain configurable way, since it is beneficial to have them decoded / processed / input in parallel. Thus, access to an individually accessible portion by individually accessible sub-portions helps to access needed data in parallel or to filter out unwanted data portions. It has also been found that it is sufficient to indicate it using a start code. This is based on the knowledge that the amount of data per NN layer, i.e. the individually accessible portions, is usually less than if the NN layer were detected by a start code in the entire data stream. Nevertheless, it is also advantageous to use pointers and / or data stream length parameters to improve access to the individually accessible portions. According to an embodiment, one or more individually accessible sub-portions in an individually accessible portion of a data stream are indicated by a pointer indicating the data stream position in bytes within the parameter set portion of the individually accessible portion. The data stream length parameter may indicate the run length of the individually accessible sub-portion. The data stream length parameter may be encoded / decoded to / from the header portion of the data stream or encoded / decoded to / from the parameter set portion of the individually accessible portion. The data stream length parameter may be used to facilitate segmentation of each individually accessible sub-portion for the purpose of packaging the one or more individually accessible sub-portions in a suitable container. According to an embodiment, an apparatus for decoding a data stream is configured to use a start code and / or a pointer and / or the data stream length parameter to access the data stream for one or more predetermined individually accessible sub-portions.

[0019] According to a sixth aspect of the present application, the inventors of the present application have realized that the neural network can be used more efficiently if a processing option parameter is coded / decoded from a data stream in which a representation of the NN is coded. The data stream is structured into individually accessible parts, and for each of one or more predetermined individually accessible parts, the processing option parameter indicates one or more processing options that must be used or can optionally be used when using the neural network for inference. The processing option parameter may indicate one of various processing options that determine whether and how a client accesses an individually accessible part (P) and / or an individually accessible sub-part (SP). For example, for each of P and / or SP, the parallel processing capacity of each P or SP and / or the parallel processing capacity per sample of each P or SP and / or the parallel processing capacity per channel of each P or SP and / or the parallel processing capacity per classification category of each P or SP and / or other processing options. The processing option parameter enables the client to make appropriate decisions, thereby enabling a highly efficient use of the NN.

[0020] According to a seventh aspect of the present application, the inventors of the present application have discovered that transmission / update of a neural network can be made more efficient if the reconstruction rule for dequantizing the NN parameters depends on the NN part to which the NN parameters belong. The NN parameters representing the neural network are quantized into quantization indexes and coded into a data stream. The apparatus for decoding is configured, for example, to dequantize the quantization indexes to reconstruct the NN parameters using the reconstruction rule. The NN parameters are coded into the data stream such that the NN parameters of different NN parts of the NN are different, the data stream indicating, for each of the NN parts, a reconstruction rule for dequantizing the NN parameters associated with the respective NN part. The apparatus for decoding is configured, for each of the NN parts, to dequantize the NN parameters in the respective NN part using the reconstruction rule indicated by the data stream for the respective NN part. The NN parts may, for example, be one or more NN layers of the NN and / or a predetermined NN layer of the NN. The NN layer includes a portion of the NN layer that has been subdivided.

[0021] According to an embodiment, a first reconstruction rule for dequantizing NN parameters associated with a first NN part is delta coded with respect to a second reconstruction rule for dequantizing NN parameters associated with a second NN part and coded into the data stream. The first NN part may consist of a first NN layer and the second NN part may consist of a second layer, the first NN layer being different from the second NN layer. Alternatively, the first NN part may consist of a first NN layer and the second NN part may consist of one part of the first NN layer. In this alternative, a reconstruction rule associated with NN parameters in a part of a given NN layer, e.g., the second reconstruction rule, is delta coded with respect to a reconstruction rule associated with a given NN layer, e.g., the first reconstruction rule. This special delta coding of the reconstruction rules may allow for the use of only a small number of bits to signal the reconstruction rules, resulting in efficient transmission / update of the neural network.

[0022] According to an eighth aspect of the present application, the inventors of the present application have realized that transmission / update of a neural network can be made highly efficient when a reconstruction rule for dequantizing a NN parameter depends on the magnitude of a quantization index associated with the NN parameter. The NN parameter representing the neural network is quantized into a quantization index and encoded into a data stream. The device for decoding is configured to, for example, dequantize the quantization index to reconstruct the NN parameter using the reconstruction rule. The data stream comprises a quantization step size parameter indicating a quantization step size and a parameter set defining a quantization index-reconstruction level mapping to indicate the reconstruction rule for dequantizing the NN parameter. The reconstruction rule of the NN parameter in a given NN part is defined by a quantization step size for quantization indexes within a given index interval, and is defined by a quantization index-reconstruction level mapping for quantization indexes outside the given index interval. For each NN parameter, for example, each NN parameter corresponding to a quantization index within a predetermined index interval is reconstructed by multiplying the respective quantization index by a quantization step size, and for example, each NN parameter corresponding to a quantization index outside the predetermined index interval is reconstructed by mapping the respective quantization index onto a reconstruction level using a quantization index-reconstruction level mapping. The decoder may be configured to determine the quantization index-reconstruction level mapping based on a parameter set in the data stream. According to an embodiment, the parameter set defines the quantization index-reconstruction level mapping by pointing to one quantization index-reconstruction level mapping of the set of quantization index-reconstruction level mappings, and the set of quantization index-reconstruction level mappings may not be part of the data stream, and may be stored, for example, at the encoder side and the decoder side.Defining the reconstruction rule based on the magnitude of the quantization index allows signaling of the reconstruction rule with a small number of bits.

[0023] According to a ninth aspect of the present application, the inventors of the present application have realised that transmitting / updating a neural network can be very efficient if an identification parameter is encoded / decoded into / from an individually accessible portion of a data stream in which a representation of the NN is encoded. The data stream is structured into individually accessible portions, and for each of one or more predefined individually accessible portions, an identification parameter for identifying the respective predefined individually accessible portion is encoded / decoded into / from the data stream. The identification parameter may indicate a version of the predefined individually accessible portion. This is particularly advantageous in scenarios such as distributed learning, where many clients independently further train the NN and send relative NN updates back to a central entity. The identification parameters may be used to identify the NNs of individual clients by a versioning scheme, thereby allowing a central entity to identify the NN on which a NN update is being built. Additionally or alternatively, the identification parameters may indicate whether a given individually accessible portion is associated with a baseline portion of the NN or with an advanced / enhanced / complete portion of the NN. This may be advantageous in use cases such as scalable NNs, where a baseline portion of the NN may be run, for example, to generate preliminary results, before a complete or enhanced NN is run to receive complete results. Furthermore, transmission errors or involuntary changes in the parameter tensors reconfigurable based on the NN parameters representing the NN are easily recognizable using the identification parameters. The identification parameters allow operations to be more error-robust when each pre-determined individually accessible portion can be checked for consistency and verified based on the NN characteristics.

[0024] According to a tenth aspect of the present application, the inventors of the present application have realized that encoding / decoding different versions of the NN into / from the data stream using delta coding or using a compensation scheme can greatly improve the efficiency of transmitting / updating the neural network. The data stream is coded with a representation of the NN in a layered manner such that different versions of the NN are coded into the data stream. The data stream is structured into one or more individually accessible parts, each individually accessible part being associated with a corresponding version of the NN. The data stream is, for example, delta coded with a first version of the NN coded into a first part with respect to a second version of the NN coded into a second part. Additionally or alternatively, the data stream may have, for example, a first version of the NN coded into a first part in the form of one or more compensation NN parts, each of which is executed in addition to executing a corresponding NN part of the second version of the NN coded into the second part to perform inference based on the first version of the NN, and the outputs of each compensation NN part and the corresponding NN part are summed. These encoded versions of the NN in the data stream may allow a client, e.g., a decoder, to tailor its processing capabilities or perform inference first on a first version, e.g., a baseline, before processing a second version, e.g., a more complex advanced NN. Furthermore, by applying / using delta coding and / or compensation schemes, different versions of the NN can be encoded into the DS using fewer bits.

[0025] According to an eleventh aspect of the present application, the inventors of the present application have realized that the neural network can be used more efficiently if the supplementary data is encoded in / from separately accessible portions of the data stream in which the representation of the NN is encoded. The data stream is structured in separately accessible portions, and the data stream includes, for each of one or more predetermined separately accessible portions, supplementary data for supplementing the representation of the NN. This supplementary data is usually not necessary for the decoding / reconstruction / inference of the NN, but may be essential from the application point of view. It is therefore advantageous to mark this supplementary data as irrelevant for the decoding of the NN, which is solely intended for inference, so that clients, e.g. decoders, that do not need the supplementary data can skip this portion of the data.

[0026] According to a twelfth aspect of the present application, the inventors of the present application have realized that the use of neural networks can be made very efficient if hierarchical control data is encoded / decoded into a data stream in which a representation of the NN is encoded. The data stream comprises hierarchical control data structured into a sequence of control data portions, the control data portions providing information about the NN with increasing detail along the sequence of the control data portions. Hierarchically structuring the control data Coding the control data into a single level is advantageous because a decoder may only require control data up to a certain level of aspects and can therefore skip control data that provides further aspects. Thus, depending on the use case and knowledge of the environment, different levels of control data may be required and the above-described scheme of presenting such control data allows efficient access to the required control data for different use cases.

[0027] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent method descriptions in which blocks or apparatus correspond to method steps or features of method steps.Embodiments relate to computer programs having program code for performing such methods, when the computer programs are executed on a computer.

[0028] Embodiments of the invention are the subject of dependent claims. Preferred embodiments of the present invention are described below with reference to the figures. The drawings are not necessarily to scale, emphasis instead generally being placed on illustrating the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings, in which: [Brief description of the drawings]

[0029] [Figure 1] FIG. 1 is a diagram illustrating an example of an encoding / decoding pipeline for encoding / decoding a neural network. [Diagram 2] FIG. 2 illustrates a neural network that may be encoded / decoded according to one embodiment. [Diagram 3] FIG. 3 is a diagram illustrating serialization of parameter tensors of a layer of a neural network, according to an embodiment. [Figure 4] FIG. 4 is a diagram illustrating the use of serialized parameters to indicate how parameters of a neural network are serialized, according to an embodiment. [Diagram 5] FIG. 5 illustrates an example of a single output channel convolutional layer. [Figure 6] FIG. 6 is a diagram showing an example of a fully connected layer. [Figure 7a] FIG. 7a is a diagram illustrating a set of n coding orders in which neural network parameters may be coded, according to an embodiment. [Figure 7b] FIG. 7b is a diagram illustrating a set of n coding orders in which neural network parameters may be coded, according to an embodiment. [Figure 7c] FIG. 7c illustrates a set of n coding orders in which neural network parameters may be coded, according to an embodiment. [Figure 7d] FIG. 7d illustrates a set of n coding orders in which neural network parameters may be coded, according to an embodiment. [Figure 8] FIG. 8 illustrates a diagram of context-adaptive arithmetic coding of individually accessible parts or sub-parts according to an embodiment. [Figure 9] FIG. 9 is a diagram illustrating the use of numerical calculation expression parameters according to an embodiment. [Figure 10] FIG. 10 is a diagram illustrating the use of a neural network layer type parameter to indicate a neural network layer type of a neural network layer of a neural network, according to an embodiment. [Figure 11] FIG. 11 illustrates a general embodiment of a data stream having pointers to the beginning of individually accessible portions, according to an embodiment. [Figure 12] FIG. 12 illustrates a detailed embodiment of a data stream having pointers to the beginning of individually accessible portions according to an embodiment. [Figure 13] FIG. 13 illustrates the use of start codes and / or pointers and / or data stream length parameters to enable access to individually accessible sub-portions according to an embodiment. [Figure 14a] FIG. 14a illustrates sub-layer access using pointers according to an embodiment. [Figure 14b] FIG. 14b illustrates sub-layer access using start codes according to an embodiment. [Figure 15] FIG. 15 illustrates an exemplary type of random access as a possible processing option for individually accessible portions, according to an embodiment. [Figure 16] FIG. 16 is a diagram illustrating the use of processing option parameters, according to an embodiment. [Figure 17] FIG. 17 is a diagram illustrating the use of neural network part-dependent reconstruction rules, according to an embodiment. [Figure 18] FIG. 18 illustrates the determination of a reconstruction rule based on quantization indexes representing quantized neural network parameters according to an embodiment. [Figure 19] FIG. 19 is a diagram illustrating the use of discrimination parameters, according to an embodiment. [Figure 20] FIG. 20 is a diagram illustrating encoding / decoding of different versions of neural networks according to an embodiment. [Figure 21] FIG. 21 illustrates delta encoding of two versions of a neural network, where the two versions differ in their weights and / or biases, according to an embodiment. [Figure 22] FIG. 22 illustrates alternative delta encoding for two versions of a neural network, the two versions differing in the number of neurons or neuronal interconnections, according to an embodiment. [Diagram 23] FIG. 23 is a diagram illustrating encoding of different versions of a neural network using a compensation neural network portion according to an embodiment. [Figure 24a] FIG. 24a illustrates an embodiment of a data stream with supplemental data, according to an embodiment. [Figure 24b] FIG. 24b illustrates an alternative embodiment of a data stream with supplemental data, according to an embodiment. [Diagram 25] FIG. 25 is a diagram illustrating one embodiment of a data stream having a sequence of control data portions. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Furthermore, in the following description, elements having equivalent or equivalent functions are given the same or equivalent reference numerals even if they are depicted in different figures.

[0031] In the following description, a number of details are set forth to provide a more thorough description of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail in order not to obscure the embodiments of the present invention. Furthermore, features of different embodiments described below in this specification may be combined with each other unless otherwise specified.

[0032] The following description of the embodiments of the present application begins with a brief introduction and overview of the embodiments of the present application to describe its advantages and how the same achieves these advantages.

[0033] In current work on coded representations of NNs, such as those developed in the ongoing MPEG work on NN compression, it has been found to be beneficial to split model bitstreams that represent parameter tensors in a multi-layer fashion into smaller sub-bitstreams, or layer bitstreams, that contain coded representations of the parameter tensors of the individual layers. This can be useful when such model bitstreams need to be saved / loaded in the context of a container format, or in application scenarios that feature parallel decoding / execution of NN layers in general.

[0034] Below, various examples are described that may help achieve effective compression of neural networks, NNs, and / or improve access to data representing the NNs, thus resulting in efficient transmission / updating of the NNs.

[0035] To facilitate understanding of the following embodiments of the present application, the description begins with a presentation of possible compatible encoders and decoders into which the following schematic embodiments of the present application may be incorporated.

[0036] FIG. 1 shows a simple sketch of an example of a DeepCABAC encoding / decoding pipeline to explain the inner workings of such a compression scheme. First, between neurons 14, 20 and / or 18, e.g., between leading neurons 141-143 and intermediate neuron 201, 202, the weights 32 of the connections 22, e.g., the connections 221 to 226, e.g., the weights 321 to 326 are formed into a tensor (step 1 in FIG. 1), shown in this example as matrix 30. In step 1 in FIG. 1, for example, weights 32 associated with a first layer of neural network 10, NN, are formed into matrix 30. According to the embodiment shown in FIG. 1, the columns of matrix 30 are associated with leading neurons 141-143 and the rows of matrix 30 are associated with hidden neurons 201, 202, although it will be apparent that the formed matrix could alternatively represent the inverse of the illustrated matrix 30.

[0037] Then, each NN parameter, e.g., weights 32, is coded, e.g., quantized and entropy coded, using a context-adaptive arithmetic coding 600 according to a particular scan order, e.g., row-major order (left to right, top to bottom), as shown in steps 2 and 3. It is also possible to use a different scan order, i.e., coding order, as outlined in more detail below. Steps 2 and 3 are performed by an encoder 40, i.e., an apparatus for encoding. A decoder 50, i.e., an apparatus for decoding, follows the same process with steps in the reverse processing order: first, it decodes the list of integer representations of the coded values, as shown in step 4, and then it reforms the list into its tensor representation 30', as shown in step 5. Finally, it loads the tensors 30' into the network architecture 10', i.e., the reconstructed NN, as shown in step 6. The reconstructed tensors 30' constitute the reconstructed NN parameters, i.e., the decoded NN parameters 32'.

[0038] The NN 10 shown in Fig. 1 is just a simple NN with a small number of neurons 14, 20, 18. A neuron is also understood in the following as a node, an element, a model element or a dimension. Furthermore, the reference number 10 denotes a machine learning (ML) predictor, in other words a machine learning model such as a neural network.

[0039] The neural network will be described in more detail with reference to FIG. 2. In particular, FIG. 2 shows an ML predictor 10 comprising an input interface 12 with input nodes or elements 14 and an output interface 16 with output nodes or elements 18. The input nodes / elements 14 receive input data; in other words, the input data are applied thereto. For example, they may receive, for example, a picture, where each element 14 is associated with a pixel of the picture. Alternatively, the input data applied on the elements 14 may be signals, such as one-dimensional signals, such as audio signals, sensor signals, etc. Still alternatively, the input data may represent a particular data set, such as medical file data. The number of input elements 14 may be any number, depending for example on the type of input data. The number of output nodes 18 may be one, as shown in FIG. 1, or more than one, as shown in FIG. 2. Each output node or element 18 may be associated with a particular inference or prediction task. In particular, when the ML predictor 10 is applied on a certain input applied on the input interface 12 of the ML predictor 10, the ML predictor 10 generates at each output node 18 a prediction task. The resulting activations, i.e. activation values, are output at the output interface 16 as inference or prediction results which may indicate answers to certain questions about the input data, such as whether or how likely it is that an input image contains a certain feature, e.g. whether it contains a certain object, such as a car, a person, a face, etc.

[0040] In this regard, the input applied to the input interface may also be interpreted as an activation, ie an activation applied to each input node or element 14 .

[0041] Between the input nodes 14 and the output node(s) 18, the ML predictor 10 includes further elements or nodes 20 that are connected via connections 22 to predecessor nodes to receive activations from these predecessor nodes, and via one or more further connections 24 to successor nodes to transfer the activations, i.e. activation values, of the nodes 20 to the successor nodes.

[0042] The predecessor node may be another internal node 20 of the ML predictor 10, through which the intermediate node 20 exemplarily shown in FIG. 2 is indirectly connected to the input node 14, or may be directly connected to the input node 14 as shown in FIG. 1, and the successor node may be another intermediate node of the ML predictor 10, through which the intermediate node 20 exemplarily shown is connected to an output interface or output node, or may be directly connected to the output node 28 as shown in FIG. 1.

[0043] Although the input nodes 14, output nodes 18 and internal nodes 20 of the ML predictor 10 may be associated or attributed to a particular layer of the ML predictor 10, the layered structuring of the ML predictor 10 is arbitrary and the ML predictor to which the embodiments of the present application are applied is not limited to such layered networks. As far as the exemplary illustrated intermediate node 20 of the ML predictor 10 is concerned, the same contributes to the inference or prediction task of the ML predictor 10 by transferring activations, i.e. activation values, received from the input interface 12 via connections 22 to the successor nodes via connections 24 towards the output interface 16. In so doing, the node or element 20 calculates its activations, i.e. activation values, transferred towards the successor nodes via connections 24 based on the activations, i.e. activation values, at the input nodes 22, which calculation includes calculating a sum having a weighted sum or addition for each connection 22 which is the product of the inputs, i.e. activations, received from the respective predecessor nodes and the weights associated with the connections 22 connecting the respective predecessor nodes and the intermediate node 20. Alternatively, or more generally, the activation x can be derived from a node or element i, 20, via a connection 24, as a function of a mapping function m ij Note that the nodes are forwarded to the successor node j by (x). Thus, each connection 22, as well as the connections 24, has some sort of weight associated with it, or, alternatively, a mapping function m ij, which may have the result of: Optionally, further parameters may be involved in the calculation of the activation that the node 20 outputs towards a particular successor node. To determine the relevance score for a portion of the ML predictor 10, the activation occurring at the output node 18 when finishing a certain prediction or inference task for a certain input at the input interface 12, or a predefined or interesting output activation of interest, may be used. This activation at each output node 18 is used as the starting point for the relevance score determination, and the relevance is back propagated towards the input interface 12. In particular, at each node of the ML predictor 10, such as node 20, the relevance score is distributed towards the predecessor nodes, such as via the connection 22 in the case of node 20, in a manner proportional to the aforementioned product associated with each predecessor node and contribution, via a weighted sum, to the activation of the current node, which is the back propagated activation, such as node 20. That is, the relevance ratio backpropagated from a node, such as node 20, to a predecessor node is calculated by multiplying the relevance of the node by a factor that depends on the ratio of the activations received from the predecessor node to the weights that those activations contributed to the sum of the respective nodes, and then multiplying the relevance of the predecessor node by the activations of those predecessors. may be calculated by dividing by a value according to the sum of all products of weights that contributed to the weighted sum of the current node whose error is to be backpropagated.

[0044] In the above method, relevance scores for portions of the ML predictor 10 are determined based on, for example, the activation of these portions as they manifest themselves in one or more inferences performed by the ML predictor. The "portions" for which such relevance scores are determined may be nodes or elements of the predictor 10, as described above, where it is again noted that the ML predictor 10 is not limited to any layered ML network, as, for example, an element 20 may be any calculation of an intermediate value, such as calculated during an inference or prediction performed by the predictor 10. For example, in the method discussed above, the relevance score of an element or node 20 is calculated by aggregating or summing the inbound relevance messages that this node or element 20 receives from its successor nodes / elements, which in turn distribute their relevance scores in the manner representatively outlined above for the node 20.

[0045] The ML predictor 10, i.e., the NN, described with reference to FIG. 2 is encoded into a data stream 45 using the encoder 40 described with reference to FIG. 1 and reconstructed / decoded from the data stream 45 using the decoder 50 described with reference to FIG. 1.

[0046] The features and / or functionality described below may be implemented in the compression scheme described with respect to FIG. 1 and relate to the NN described with respect to FIGS.

[0047] Serialization of one-parameter tensors There are applications where sub-layer processing of bitstreams is useful, for example NNs where layers are composed into independent subsets, e.g., baseline and advanced portions that are trained separately, and the NN adapts to the available computing power of the client in a way that the client can decide to run only a subset of the baseline layers or additionally a subset of the advanced layers (Tao, 2018). Another example is a NN that features data channel specific operations, e.g., layers in an image processing NN where operations can be performed separately for each color channel in parallel (Chollet, 2016).

[0048] For the above purposes, referring to FIG. 3, the serialization 1001 or 1002 of the parameter tensor 30 of a layer requires, for example, a bit string 421 or 422 that can be easily split into consecutive subsets 431 to 433, or 441 and 442, that are meaningful from the application point of view, before entropy coding. All NN parameters per sample 1002, e.g. grouping with weight 32, or It may include grouping of neurons in the baseline versus advanced portion. Such bit strings can then be entropy coded to form sub-layer bit streams with functional relationships.

[0049] 4, the serialization parameters 102 may be encoded / decoded to / from the data stream 45. The serialization parameters may indicate how the NN parameters 32 are grouped prior to or during the encoding of the NN parameters 32. The serialization parameters 102 indicate how the NN parameters 32 of the parameter tensor 30 are serialized into a bit stream to enable encoding of the NN parameters into the data stream 45.

[0050] In one embodiment, the serialization information, i.e., the serialization parameters 102, are stored in a bitstream. , i.e. in the parameter set portion 110 of the data stream 45, with a range of layers, see for example Figs. 12, 14a, 14b or 24b.

[0051] Another embodiment is to use the parameter tensor 30 (see encoding order 1061 in FIG. 1 and FIG. 7 ). ) as serialized parameters 102. This information can be useful, for example, in cases where the decoding list of parameters should be grouped / organized in a different way, e.g., in memory, to enable efficient execution, as shown in Fig. 3 for an exemplary image processing NN with a clear association between the entries, i.e., weights 32, of the parameter matrix, i.e., parameter tensor 30, and the samples 1002 and color channels 1001. Fig. 3 shows two different serialization modes 1001 and 1002. 002 and an exemplary illustration of the resulting sub-layers 43 and 44.

[0052] In a further embodiment, as shown in Fig. 4, the bit stream, i.e., data stream 45, specifies the order 104 in which the encoder 40 traverses the NN parameters 32, e.g., layers, neurons, tensors, so that the decoder 50 can reconstruct the NN parameters 32 accordingly during decoding. See Fig. 1 for a description of the encoder 40 and the decoder 50. That is, different traversal orders 301, 302 of the NN parameters 32 can be applied in different application scenarios.

[0053] For example, encoding parameters along different dimensions may benefit the resulting compression performance since the entropy coder may be able to better capture dependencies between them. In another example, it may be desirable to group parameters according to some application-specific criteria, i.e., which parts of the input data they relate to or whether they can be performed jointly, so that they are decoded / input in parallel. A further example is encoding parameters according to the General Matrix Matrix (GEMM) product scan order, which supports efficient memory allocation of decoded parameters when performing dot product operations (Andrew Kerr, 2017).

[0054] A further example is, for example, a data set as shown by the encoding order 1064 in FIG. 7 to achieve energy compaction of the NN parameters 32 to be encoded and then process / serialize / encode the resulting reordered data according to the resulting order 104. The reordering may therefore involve reordering the NN parameters 32 to ensure consistent, uniform increases or decreases along the coding order 104.

[0055] Figure 5 shows an example for a single output channel convolutional layer, e.g., for image and / or video analysis applications. A color image has multiple channels, typically one for each color channel, red, green, blue, etc. From a data perspective, this means that one image provided as input to the model is actually three images.

[0056] The tensor 30a is applied to the input data 12 and can scan over the input like a window with a constant step size. The tensor 30a can be understood as a filter. The tensor 30a moves over the input data 12 from left to right, jumping to the next subrow after each pass. An optional so-called padding determines how the tensor 30a should behave when it hits an edge of the input matrix. For each point in its field of view, the tensor 30a has NN parameters 32, e.g. fixed weights, and calculates a result matrix from these weights, e.g. pixel values ​​in the current field of view. The size of this result matrix depends on the size of the tensor 30a (kernel size), the padding and in particular the step size. The depth of the tensor 30a depends on the input 12 depth. An input image may have three channels (e.g., depth 3), and then a tensor 30a applied to that image may also have three channels (e.g., depth 3). Regardless of the depth of the input 12 and the depth of the tensor 30a, the tensor 30a is applied to the input 12 using a dot product operation that results in a single value.

[0057] By default, DeepCABAC converts any given tensor 30a into its respective matrix 30b form, as shown in Figure 5, in row-length order 1041, i.e. That is, we encode 3 the NN parameters 32 into the data stream 45 from left to right and top to bottom. However, as will be explained with respect to Figure 7, other encoding orders 104 / 106 may be advantageous to achieve high compression.

[0058] FIG. 6 is an example for the case of a fully connected layer. A fully connected layer or dense layer is a typical neural network structure, where every neuron is connected to every input 12, i.e., predecessor node, and every output 16', i.e., successor node. A tensor 30 represents the corresponding NN layer, and the tensor 30 includes NN parameters 32. The NN parameters 32 are coded into the data stream according to a coding order 104. As will be explained with respect to FIG. 7, a particular coding order 104 / 106 may be advantageous to achieve high compression.

[0059] The discussion now turns back to Fig. 4 to allow a general discussion of the serialization of NN parameters 32. The concepts described with respect to Fig. 4 are applicable to both single output channel convolutional layers, see Fig. 5, and fully connected layers, see Fig. 6.

[0060] As shown in Fig. 4, embodiment A1 of the present application relates to a data stream 45 (DS) in which a representation of a neural network (NN) is encoded. The data stream includes serialization parameters 102 indicating the encoding order 104 in which the NN parameters 32 defining the neuron interconnections of the neural network are encoded in the data stream 45.

[0061] According to embodiment ZA1, the apparatus for encoding a representation of a neural network into a DS45 is configured to provide to the data stream 45 serialization parameters 102 indicating an encoding order 104 in which the NN parameters 32 defining the neuron interconnections of the neural network are encoded into the data stream 45.

[0062] According to embodiment XA1, an apparatus for decoding a representation of a neural network from a DS45 is configured, for example, to decode from a data stream 45 serialized parameters 102 indicating an encoding order 104 in which NN parameters 32 defining the neuronal interconnections of the neural network are encoded in the data stream 45, and to assign the NN parameters 32 serially decoded from the DS45 using the encoding order 104 to the neuronal interconnections.

[0063] 4 shows different representations of a NN layer with associated NN parameters 32. According to an embodiment, a two-dimensional tensor 301, i.e. a matrix or a 3D A dimension tensor 302 can represent the corresponding NN layer.

[0064] In the following, different features and / or functionalities are described in the context of data stream 45, but the features and / or functionalities may also be the features and / or functionalities of an apparatus according to embodiment ZA1 or of an apparatus according to embodiment XA1 in the same or similar manner.

[0065] According to embodiment A2, the NN parameters 32 of the DS 45 of embodiment A1 are encoded into the DS 45 using a context-adaptive arithmetic coding 600, for example, as shown in FIG. 1 and FIG. 8. Thus, an apparatus according to embodiment ZA1 may be configured to encode the NN parameters 32 using context adaptive arithmetic coding 600, and an apparatus according to embodiment XA1 may be configured to decode the NN parameters 32 using context adaptive arithmetic decoding.

[0066] According to embodiment A3, in the DS 45 of embodiment A1 or A2, the data stream 45 is structured into one or more individually accessible portions 200, each individually accessible portion 200 representing a corresponding NN layer 210 of the neural network, as shown in FIG. 8 or one of the following figures, and the serialization parameters 102 indicate the encoding order 104 in which the NN parameters 32 defining the neuron interconnections of the neural network in a given NN layer 210 are encoded into the data stream 45.

[0067] According to embodiment A4, in the DS 45 of any of the previous embodiments A1 to A3, the serialization parameter 102 is an n-ary parameter indicating an encoding order 104 in a set 108 of n encoding orders, as shown, for example, in FIG.

[0068] According to embodiment A4a, in the DS 45 of embodiment A4, the set 108 of n encoding orders includes: a first predefined encoding order 1061 that differs in the order in which the predefined encoding instructions 104 traverse a dimension, e.g., the x-dimension, the y-dimension and / or the z-dimension, of a tensor 30 describing a predefined NN layer of a NN; and / or a second predefined coding order 1062 for scalable coding of the NN, the predefined coding order 104 differing in the number of times 107 the predefined NN layer of the NN is scanned; and / or a third predefined encoding order 1063 that differs from the predefined encoding order 104 in the order in which the NN layers 210 of the NN are traversed; and / or A fourth predetermined encoding order 1064 which differs in the order in which the neurons 20 in the NN layer of the NN are traversed.

[0069] The first predetermined encoding order 1061 is, for example, a tensor For example, encoding order 1041 differs from encoding order 1042 in that encoding order 1041 is line-length ordered, i.e. That is, the tensor 30 is scanned row-wise, from left to right and top to bottom, and the predetermined encoding order 1042 is column-length order, that is, the tensor 30 is scanned column-wise, from top to bottom and left to right. Similarly, the first predetermined coding order 1061 is a coding order that is similar to the coding order 104 for three-dimensional tiles. The order in which the dimensions of the console 30 are traversed can vary.

[0070] The second predetermined encoding sequence 1062 may be represented, for example, by a tensor / matrix 30. The NN layers may differ in the frequency with which the NN layers are scanned. The NN layers may, for example, be scanned twice through a predefined encoding order 104, so that a baseline portion and an advanced portion of the NN layers may be encoded / decoded from the data stream 45. The number of times 107 the NN layers are scanned through a predefined encoding order defines the number of versions of the NN layers that are encoded into the data stream. Thus, for a serialization parameter 102 indicating an encoding order that scans the NN layers at least twice, the decoder is configured to determine which versions of the NN layers it can decode based on its processing capabilities, and decode the NN parameters 32 corresponding to the selected NN layer version.

[0071] The third predetermined encoding order 1063 is related to the different NN layers 2101 and 2102 of the NN 10. It defines whether the associated NN parameters are coded into the data stream 45 using a different predetermined coding order or the same coding order as one or more other NN layers 210 of the NN 10 .

[0072] The fourth predefined encoding sequence 1064 is a tensor / matrix representing the corresponding NN layer. 30 from the upper left NN parameter 321 to the lower right NN parameter 32 12 Run diagonally in a zigzag pattern until The encoding sequence includes a predetermined encoding order 1043 to be checked.

[0073] According to embodiment A4a, in the DS 45 of any of the previous embodiments A1-A4a, the serialization parameter 102 indicates the permutation that the encoding order 104 uses to permute the neurons of the NN layer with respect to a default order. In other words, the serialization parameter 102 indicates the permutation that, when used, the encoding order 104 permutes the neurons of the NN layer with respect to the default order. As shown in FIG. 7 for a fourth predetermined encoding order 1064, the data The line length ordering as illustrated for stream 450 represents the default ordering. The data stream 45 contains the NN parameters encoded therein using a permutation relative to the default order.

[0074] According to embodiment A4b, of the DS 45 of embodiment A4a, the permutation orders the neurons of the NN layer 210 such that the NN parameters 32 are monotonically increasing along the encoding order 104 or monotonically decreasing along the encoding order 104.

[0075] According to embodiment A4c, in the DS 45 of embodiment A4a, the permutation permutes the neurons of the NN layer 210 such that, among the predefined encoding orders 104 signalable by the serialization parameters 102, the bit rate for encoding the NN parameters 32 into the data stream 45 is the lowest for the permutation indicated by the serialization parameters 102.

[0076] According to embodiment A5, in the DS 45 of any of the previous embodiments A1-A4c, the NN parameters 32 include weights and biases.

[0077] According to embodiment A6, the data stream 45 of the DS 45 of any of the previous embodiments A1-A5 is structured into individually accessible sub-portions 43 / 44, each sub-portion 43 / 44 representing a corresponding NN portion of the neural network 10, e.g., a portion of the NN layer 210, such that each sub-portion 43 / 44 is completely scanned by the coding order 104 before a subsequent sub-portion 43 / 44 is scanned by the coding order 104. A row, column, or channel of the tensor 30 representing a NN layer may be coded into an individually accessible sub-portion 43 / 44. Different individually accessible sub-portions 43 / 44 associated with the same NN layer include different neurons 14 / 18 / 20 or neuronal interconnections 22 / 24 associated with the same NN layer. The individually accessible sub-portions 43 / 44 represent rows, columns, or channels of the tensor 30. The individually accessible sub-portions 43 / 44 are, for example, shown in FIG. 3. Alternatively, as shown in Figures 21 to 23, the separately accessible sub-portions 43 / 44 represent different versions of the NN layer, such as a baseline portion of the NN layer and an advanced portion of the NN layer.

[0078] According to embodiment A7, of the DS 45 of any of embodiments A3 and A6, the NN parameters 32 are coded into the DS 45 using context-adaptive arithmetic coding 600 and using context initialization at the beginning 202 of any individually accessible portion 200 or sub-portion 43 / 44. See, for example, FIG. 8.

[0079] According to embodiment A8, the data stream 45 of the DS 45 of any of embodiments A3 and A6 includes a start code 242 at which each individually accessible portion 200 or sub-portion 240 begins, and / or a pointer 220 / 244 pointing to the beginning of each individually accessible portion 200 or sub-portion 240, and / or a pointer data stream length, i.e., a pointer 220 / 244 pointing to the beginning of each individually accessible portion 200 or sub-portion 240, as shown in Figures 11 to 14, when parsing the DS 45. A parameter indicating the data stream length 246 of each individually accessible portion 200 or sub-portion 240 for skipping each individually accessible portion 200 or sub-portion 240.

[0080] Another embodiment specifies the bit size and numeric representation of the decoded parameters 32' in the bitstream, i.e., data stream 45. For example, an embodiment may specify that the decoded parameters 32' may be represented in an 8-bit signed fixed-point format. This specification is very useful, for example, in applications where the activation values ​​may also be represented, for example, in an 8-bit fixed-point representation, since inference can then be performed more efficiently with fixed-point arithmetic.

[0081] According to embodiment A9, the DS 45 of any of the previous embodiments A1 to A8 further includes a numerical calculation expression parameter 120 indicating a numerical expression and a bit size for expressing the NN parameter 32 when the NN is used for inference. For example, see FIG. 9.

[0082] FIG. 9 shows an embodiment B1 of a data stream 45 in which a representation of a neural network is encoded, where the NN parameters 32 of the NN encoded in the DS 45 include numerical computation representation parameters 120 indicating the numerical representation, e.g., floating point, fixed point representation, and bit size, in which numbers should be represented when using the NN for inference.

[0083] A corresponding embodiment ZB1 relates to an apparatus for encoding a representation of a neural network into a DS45, the apparatus being configured to provide a data stream 45 with numerical calculation representation parameters 120 indicating the numerical representation, e.g., floating point, fixed point representation and bit size, in which the NN parameters 32 of the NN encoded into the DS45 are represented when the NN is used for inference.

[0084] The corresponding embodiment XB1 relates to an apparatus for decoding a representation of a neural network from a DS 45. The apparatus is configured to decode from a data stream 45 a numerical representation, e.g., a numerical computation representation parameter 120 indicating a floating point, fixed point representation and a bit size in which the NN parameters 32 of the NN encoded in the DS 45 are represented when the NN is used for inference, and is configured to optionally use the numerical representation and the bit size to represent the NN parameters 32 decoded from the DS 45.

[0085] In the following, different features and / or functionalities are described in the context of data stream 45, but the features and / or functionalities may also be, in the same or similar way, features and / or functionalities of an apparatus according to embodiment ZB1 or features and / or functionalities of an apparatus according to embodiment XB1.

[0086] Further embodiments signal the type of parameters in a layer. In most cases, a layer is composed of two types of parameters 32: weights and biases. Distinguishing between these two types of parameters can be useful before decoding, for example, if different types of dependencies are used during encoding or if parallel decoding is desired.

[0087] According to embodiment A10, the data stream 45 of the DS 45 of any of the previous embodiments A1-B1 is structured into individually accessible sub-portions 43 / 44. Each sub-portion 43 / 44 represents a portion of a corresponding NN portion, e.g., a NN layer, of the neural network, and each sub-portion 43 / 44 is completely scanned by the coding order 104 before a subsequent sub-portion 43 / 44 is scanned by the coding order 104, and the data stream 45 is structured into individually accessible sub-portions 43 / 44. The stream 45 includes, for a given sub-portion, a type parameter that indicates the parameter type of the NN parameters 32 that are encoded in the given sub-portion.

[0088] According to embodiment A10a, in the DS of embodiment A10, the type parameters identify at least the NN weights and the NN biases.

[0089] Finally, further embodiments signal the type of layer 210 in which the NN parameters 32 are included, e.g., convolutional or fully connected. This information may be useful, for example, to understand the meaning of the dimensions of the parameter tensor 30. For example, the weight parameters of a 2d convolutional layer may be expressed as a 4d tensor 30, where the first dimension specifies the number of filters, the second dimension specifies the number of channels, and the rest specifies the 2d spatial dimensions of the filters. Furthermore, different layers 210 may be treated differently during encoding (e.g., by using different sets or modes of context models) to better capture dependencies in the data and lead to higher coding efficiency, information that may be important for the decoder to know before decoding.

[0090] According to embodiment A11, of the DS 45 of any of the previous embodiments A1-A10a, the data stream 45 is structured into one or more separately accessible portions 200, each portion 200 representing a corresponding NN layer 210 of the neural network 10, and the data stream 45 further includes, for a given NN layer, an NN layer type parameter 130 indicating the NN layer type of the given NN layer of the NN. See, e.g., FIG. 10.

[0091] 10 shows an embodiment C1 of a data stream 45 in which a representation of a neural network is encoded. The data stream 45 is structured into one or more separately accessible portions 200, each portion representing a corresponding NN layer 210 of the neural network, and the data stream 45 further includes, for a given NN layer, an NN layer type parameter 130 indicating the NN layer type of the given NN layer of the NN.

[0092] A corresponding embodiment ZC1 relates to an apparatus for encoding a representation of a neural network into a DS 45 such that the data stream 45 is structured into one or more individually accessible portions 200. Each portion 200 represents a corresponding NN layer 210 of the neural network, and the apparatus is configured to provide, for a given NN layer 210, to the data stream 45 an NN layer type parameter 130 indicating the NN layer type of the given NN layer 210 of the NN.

[0093] The corresponding embodiment XC1 relates to an apparatus for decoding a representation of a neural network from a DS 45. The data stream 45 is structured into one or more separately accessible portions 200, each portion 200 representing a corresponding NN layer 210 of the neural network. The apparatus is configured to decode from the data stream 45, for a given NN layer 210, an NN layer type parameter indicating the NN layer type of the given NN layer 210 of the NN.

[0094] According to embodiment A12, in the DS 45 of any of embodiments A11 and C1, the NN layer type parameter 130 is at least: fully connected, NN layer 2101 reference; and Convolutional layer type, NN layer 210 N Therefore, according to embodiment ZC1, An apparatus according to embodiment XB1 can encode the NN layer type parameter 130 to distinguish between the two layer types, and an apparatus according to embodiment XB1 can decode the NN layer type parameter 130 to distinguish between the two layer types.

[0095] 2 Bitstream Random Access 2.1 Layer Bitstream Random Access It is important to access subsets of the bitstream for many applications, such as parallelizing layer processing or packaging bitstreams in respective container formats. One state-of-the-art approach to enable such access is, for example, to break the coding dependency after the parameter tensors 30 of each layer 210 and insert start codes into the model bitstream, i.e., data stream 45, before each layer bitstream, e.g., an individually accessible portion 200. In particular, start codes in the model bitstream are not a suitable way to separate layer bitstreams, since detection of the start code would require parsing through the entire model bitstream from the beginning across a potentially very large number of start codes.

[0096] This aspect of the invention is concerned with structuring the encoded model bitstream of parameter tensors 30 in a way that goes beyond the state of the art, and is concerned with further techniques that allow easier, faster and better access to bitstream portions, e.g. layer bitstreams, to facilitate applications that require parallel or partial decoding and execution of NNs.

[0097] In one embodiment of the present invention, the model bitstream, i.e., individual layer bitstreams within the data stream 45, e.g., individually accessible portions 200, are indicated through their bitstream location in bytes or offsets (e.g., byte offsets relative to the beginning of a coding unit) within the parameter set / header portion 47 of the bitstream having the model scope. Figures 11 and 12 are diagrams for illustrating this embodiment. Figure 12 shows layer access via bitstream location or offsets indicated by pointers 220. Furthermore, each individually accessible portion 200 may optionally comprise a layer parameter set 110 into which one or more of the aforementioned parameters may be encoded and decoded.

[0098] According to embodiment A13, the data stream 45 of the DS 45 of any of the previous embodiments A1 to A12 is structured into individually accessible portions 200. Each portion 200 represents a corresponding NN portion, for example one or more NN layers or a part of an NN layer of a neural network. The data stream 45 comprises, for each of one or more predefined individually accessible portions 200, a pointer 220, for example pointing to the beginning of the respective individually accessible portion 200, see FIG. 11 or FIG. 12 if the individually accessible portion represents a corresponding NN layer. See FIG. 13 to FIG. 15 if the individually accessible portion represents a part of a predefined NN layer, for example an individually accessible sub-portion 240. In the following, the pointer 220 may also be designated with the reference sign 244.

[0099] For each NN layer, the individually accessible portions 200 associated with the respective NN layer represent the corresponding NN portion of the respective NN layer, whereby, here and in the following description, such individually accessible portions 200 are understood as individually accessible sub-portions 240.

[0100] 11 shows a more general embodiment D1 in which a representation of a neural network is encoded in a data stream 45, the data stream 45 being structured into individually accessible portions 200. Each individually accessible portion 200 represents a corresponding NN portion of the neural network, e.g., one or more NN layers or portions of an NN layer. For each of the one or more predefined individually accessible portions 200, the data stream 45 includes a pointer 220 pointing to the beginning of the respective predefined individually accessible portion 200.

[0101] According to an embodiment, the pointer 220 points to the beginning of the first individually accessible portion 2001. The first address pointing to the first individually accessible portion 2001 indicates an offset relative to the first address. Pointer 2201 indicates no offset. It is therefore possible to omit the first pointer 2201. Alternatively, pointer 220 indicates an offset, for example relative to the end of the parameter set in which pointer 220 is encoded.

[0102] A corresponding embodiment ZD1, relates to an apparatus for encoding a representation of a neural network into a DS 45 such that the data stream 45 is structured into one or more individually accessible portions 200. Each portion 200 represents a corresponding NN portion of the neural network, for example one or more NN layers or a part of a NN layer. The apparatus is configured to provide the data stream 45 with a pointer 220 for each of the one or more predefined individually accessible portions 200, pointing to the beginning of the respective predefined individually accessible portion 200.

[0103] The corresponding embodiment XD1 concerns an apparatus for decoding a representation of a neural network from a DS 45. The data stream 45 is structured into one or more individually accessible portions 200, each portion 200 representing a corresponding NN portion of the neural network, for example one or more NN layers or a part of a NN layer. The apparatus is configured to decode, for each of one or more predetermined individually accessible portions 200, a pointer 220 pointing to the beginning of the respective predetermined individually accessible portion 200 from the data stream 45 and to use one or more of the pointers 220 to access, for example, the DS 45.

[0104] According to embodiment A14, each individually accessible portion 200 of a DS 45 according to any of the previous embodiments A13 and D1 represents: a corresponding NN layer 210 of the neural network, or NN portion of NN layer 210 of a NN. See, for example, FIG. 3 or any of FIGS. 21-23.

[0105] 2.2 Sub-layer Bitstream Random Access As mentioned in Section 1, there are applications that rely on grouping parameter tensors 30 in layers 210 in a certain configurable way, because it is beneficial to have them decoded / processed / inferred partially or in parallel. Thus, sub-layer-wise access to layer bitstreams, e.g., individually accessible portions 200, can be useful to access desired data in parallel or to omit unnecessary data portions.

[0106] In one embodiment, the coding dependencies within the layer bitstream are reset at sub-layer granularity, i.e., resetting the DeepCABAC probability states.

[0107] In another embodiment of the invention, the individual sub-layer bitstreams, i.e. individually accessible sub-portions 240, within a layer bitstream, i.e. individually accessible portion 200, are indicated together with the layer or model range through a bytewise bitstream position, e.g. pointer 244, or an offset, e.g. pointer 244, within the parameter set portion 110 of the bitstream, i.e. data stream 45. Figures 13, 14a and 15 illustrate the embodiment. Figure 14a illustrates the sub-layer access, i.e. access to the individually accessible sub-portions 240, by relative bitstream position or offset. Furthermore, for example, the individually accessible portions 200 can also be accessed by a layer level pointer 220. The layer level pointer 220 is, for example, encoded in the model parameter set 47, i.e. header, of the DS 45. The pointer 220 points to the individually accessible portion 200 representing the corresponding NN portion including the NN layer of the NN. The sub-layer level pointer 244 points to the corresponding NN portion including the NN layer of the NN. The NN layer parameter set 110 of the individually accessible portion 200 representing the NN portion is encoded in the layer parameter set 110. A pointer 244 points to the beginning of an individually accessible sub-portion 240 representing a corresponding NN portion that includes a portion of the NN layer of the NN.

[0108] According to an embodiment, the layer level pointer 220 indicates an offset relative to the beginning of the first individually accessible portion 2001. The sub-layer level pointer 244 indicates an offset relative to the beginning of the first individually accessible portion 2001. 2 shows an offset relative to the beginning of the first individually accessible sub-portion 240 of the individually accessible portion 200.

[0109] According to an embodiment, the pointer 220 / 244 indicates a byte offset into an aggregation unit that contains multiple units. The pointer 220 / 244 indicates a byte offset from the beginning of the aggregation unit to the beginning of a unit within the payload of the aggregation unit.

[0110] In another embodiment of the invention, individual sub-layer bitstreams or individually accessible sub-portions 240 within a layer bitstream or individually accessible portion 200 are indicated via detectable start codes 242 in the bitstream or data stream 45, which may be sufficient since the amount of data per layer is usually less than if the layer were detected by start codes 242 in the entire model bitstream or data stream 45. Figures 13 and 14b illustrate this embodiment. Figure 14b illustrates the use of start codes 242 at the sub-layer level, i.e. for each individually accessible sub-portion 240, and bitstream positions or pointers 220 at the layer level, i.e. for each individually accessible portion 200.

[0111] In another embodiment, the run length, i.e. data stream length 246, of a (sub)layer bitstream portion, individually accessible sub-portion 240, is indicated in the parameter set / header portion 47 of the bitstream 45 or in the parameter set portion 110 of the individually accessible portion 200, to facilitate carving out said portion, i.e. individually accessible sub-portion 240, for packaging purposes in a suitable container. As shown in Figure 13, the data stream length 246 of an individually accessible sub-portion 240 is indicated by a data stream length parameter.

[0112] 13 shows an embodiment E1 of a data stream 45 in which a representation of the neural network is encoded. The data stream 45 is structured into one or more individually accessible portions 200, each individually accessible portion 200 representing a corresponding NN layer of the neural network. The data stream 45 is further structured within a given portion, e.g. within an individually accessible portion 200, into individually accessible sub-portions 240, each sub-portion 240 representing a corresponding NN portion of a respective NN layer of the neural network. The data stream 45 comprises, for each of the one or more predetermined individually accessible sub-portions 240: a start code 242 where each given individually accessible sub-portion 240 begins; and / or a pointer 244 pointing to the beginning of each predetermined individually accessible sub-portion 240; and / or A data stream length parameter indicating the data stream length 246 of each predetermined individually accessible sub-portion 240 for skipping each predetermined individually accessible sub-portion 240 when parsing the DS45.

[0113] The individually accessible sub-portions 240 described herein may have the same or similar features and / or functionality as described with respect to the individually accessible sub-portions 43 / 44. There is a possibility that.

[0114] The individually accessible sub-portions 240 within the same predetermined portion all have the same data stream length 246, whereby the data stream length parameter indicates one data stream length 246 that is applicable for each individually accessible sub-portion 240 within the same predetermined portion. The data stream length parameter indicates the data stream length 246 of all individually accessible sub-portions 240 of the entire data stream 45, or the data stream length parameter indicates for each individually accessible portion 200 the data stream length 246 of all individually accessible sub-portions 240 of each individually accessible portion 200. The one or more data stream length parameters are encoded in the header portion 47 of the data stream 45 or in the parameter set portion 110 of each individually accessible portion 200.

[0115] The corresponding embodiment ZE1 relates to an apparatus for encoding a representation of a neural network into a DS 45. The data stream 45 is structured into one or more individually accessible portions 200, each individually accessible portion 200 representing a corresponding NN layer of the neural network. The data stream 45 is further structured within a given portion, e.g. an individually accessible portion 200, into individually accessible sub-portions 240, each sub-portion 240 representing a corresponding NN portion of a respective NN layer of the neural network. The apparatus is configured to provide the data stream 45, for each of the one or more predetermined individually accessible sub-portions 240: a start code 242 where each given individually accessible sub-portion 240 begins; and / or a pointer 244 pointing to the beginning of each predetermined individually accessible sub-portion 240; and / or A data stream length parameter indicating the data stream length 246 of each predetermined individually accessible sub-portion 240 for skipping each predetermined individually accessible sub-portion 240 when parsing the DS45.

[0116] Another corresponding embodiment XE1 relates to an apparatus for decoding a representation of a neural network from a DS 45. The data stream 45 is structured into one or more individually accessible portions 200, each individually accessible portion 200 representing a corresponding NN layer of the neural network. The data stream 45 is further structured within a given portion, e.g., the individually accessible portion 200, into individually accessible sub-portions 240, each sub-portion 240 representing a corresponding NN portion of a respective NN layer of the neural network. The apparatus is configured to decode from the data stream 45, for each of the one or more given individually accessible sub-portions 240: a start code 242 where each given individually accessible sub-portion 240 begins; and / or a pointer 244 pointing to the beginning of each predetermined individually accessible sub-portion 240; and / or A data stream length parameter indicating the data stream length 246 of each predetermined individually accessible sub-portion 240 for skipping each predetermined individually accessible sub-portion 240 when parsing the DS45. It is then configured to use this information, eg, start code 242, pointer 244 and / or data stream length parameters, to access the DS 45, eg, for one or more given individually accessible sub-portions 240.

[0117] According to embodiment E2, the data stream 45 of embodiment E1 of the DS 45 is encoded with a representation of a neural network using context-adaptive arithmetic coding and using context initialization at the beginning of each individually accessible portion 200 and each individually accessible sub-portion 240. See, for example, FIG.

[0118] According to embodiment E3, the data stream 45 of embodiment E1 or embodiment E2 is according to any other embodiment herein, and it is clear that the devices of embodiments ZE1 and XE1 may also be completed with any other features and / or functions described herein.

[0119] 2.3 Bitstream Random Access Types Depending on the type of (sub-)layer 240 resulting from the selected serialization type, e.g., serialization types 1001 and 1002 shown in Figure 3, various processing options are also available that determine if and how the client will access the (sub-)layer bitstream 240. For example, if the selected serialization 1001 indicates that the sub-layer 240 is image color channel specific, If this allows for per-data-channel parallelization of decoding / inference, this should be indicated in the bitstream 45 to the client. Another example is the derivation of preliminary results from a baseline NN subset that can be decoded / inferred independently of an advanced NN subset of a particular layer / model, as described with respect to Figures 20-23.

[0120] In one embodiment, the parameter set / header 47 in the bitstream 45 with scope to the whole model, one or more layers, indicates the type of (sub)layer random access to allow the client to make an appropriate decision. Figure 15 shows two exemplary types 2521 and 2522 of random access determined by serialization. The illustrated types of random access 2521 and 2522 represent possible processing options for the individually accessible portions 200 representing the corresponding NN layers. The first processing option 2521 indicates data channel-wise access to the NN parameters in the individually accessible portions 2001, the second processing option 2522 indicates data channel-wise access to the NN parameters in the individually accessible portions 2002, the third processing option 2523 indicates data channel-wise access to the NN parameters in the individually accessible portions 2003, the fourth processing option 2524 indicates data channel-wise access to the NN parameters in the individually accessible portions 2004, the fifth processing option 2525 indicates data channel-wise access to the NN parameters in the individually accessible portions 2005, the sixth processing option 2526 indicates data channel-wise access to the NN parameters in the individually accessible portions 2006, the seventh processing option 2527 indicates data channel-wise access to the NN parameters in the individually accessible portions 2007, the eighth processing option 2528 indicates data channel-wise access to the NN parameters in the individually accessible portions 2008, the eighth processing option 2529 indicates data channel-wise access to the NN parameters in the individually accessible portions 2009, the eighth processing option 2530 indicates data channel-wise access to the NN parameters in the individually accessible portions 2009, the eighth processing option 2531 indicates data channel-wise access to the NN parameters in the individually accessible portions 2001, the eighth processing option 2532 indicates data channel-wise access to the NN parameters in the individually accessible portions 2001, the eighth processing option 2533 indicates data channel-wise access to Sample-by-sample access to the NN parameters in portion 2002 is shown.

[0121] 16 shows a general embodiment F1 of a data stream 45 in which a representation of a neural network is encoded. The data stream 45 is structured into individually accessible portions 200, each individually accessible portion 200 representing a corresponding NN portion, e.g., including one or more NN layers or including a portion of a NN layer of a neural network. The data stream 45 includes, for each of one or more given individually accessible portions 200, a processing option parameter 250 indicating one or more processing options 252 that must or can optionally be used when using the NN for inference.

[0122] The corresponding embodiment ZF1 relates to an apparatus for encoding a representation of a neural network into a DS 45. The data stream 45 is structured into individually accessible portions 200. Each individually accessible portion 200 represents a corresponding NN portion, for example including one or more NN layers or including a portion of a NN layer of a neural network. The apparatus is configured to provide the data stream 45 with a processing option parameter 250 indicating, for each of one or more predetermined individually accessible portions 200, one or more processing options 252 that must or can optionally be used when using the NN for inference.

[0123] Another corresponding embodiment XF1 decodes the representation of the neural network from DS45. The present invention relates to an apparatus for performing inference processing on a neural network. A data stream 45 is structured into individually accessible portions 200. Each individually accessible portion 200 represents a corresponding NN portion, e.g. including one or more NN layers of a neural network or including a portion of a NN layer. The apparatus is configured to decode from the data stream 45, for each of one or more predetermined individually accessible portions 200, a processing option parameter 250 indicating one or more processing options 252 that must or can be used when using the NN for inference. For example, the apparatus is configured to decode, based on the latter, which of the one or more predetermined individually accessible portions to access, skip and / or decode. Based on the one or more processing options 252, the apparatus is configured to determine how and / or which individually accessible portions or individually accessible sub-portions can be accessed, skipped and / or decoded.

[0124] According to embodiment F2 of DS 45 of embodiment F1, processing option parameter 250 indicates one or more available processing options 252 from a predefined set of processing options including: the parallel processing capabilities of each given individually accessible portion 200; and / or The sample-by-sample parallel processing capability 2521 of each given individually accessible portion 200 and / or the per-channel parallel processing capability 2522 of each given individually accessible portion 200; and / or the classification category-based parallel processing capacity of each predetermined individually accessible portion 200; and / or The dependency of a NN part, e.g. a NN layer, expressed by a respective given individually accessible part on the computation results obtained from another individually accessible part of the DS related to the same neural network part but belonging to another version of the NN version coded in the DS in a layer manner, as shown in Figures 20 to 23.

[0125] The apparatus according to embodiment ZF1 is configured to encode the processing options parameters 250 such that the processing options parameters 250 indicate one or more processing options of a set of predefined processing options. The apparatus according to embodiment XF1 is configured to decode the processing options parameters 250 indicating one or more processing options of a set of predefined processing options.

[0126] 3 Signaling quantization parameters The NN parameters 32 encoded in a layer payload, e.g., an individual accessible portion 200, or the NN parameters 32 encoded in a sub-layer payload, e.g., an individual accessible sub-portion 240, may include different types of parameters 32 that represent rational numbers, e.g., weights, biases, etc.

[0127] In a preferred embodiment shown in Fig. 18, one such type of parameter is signaled as an integer value in the bitstream such that the reconstructed values, i.e. reconstructed NN parameters 32', are derived by applying reconstruction rules 270 to these values, i.e. quantization indexes 32'', which comprise the reconstructed parameters. For example, such reconstruction rules 270 may consist of multiplying each integer value, i.e. quantization index 32'', by the associated quantization step size 263. The quantization step size 263 is the reconstructed parameter in this case.

[0128] In a preferred embodiment, the reconstruction parameters are signaled either in the model parameter set 47 , in the layer parameter set 110 , or in the sub-layer header 300 .

[0129] In another preferred embodiment, a first set of reconfiguration parameters is signaled in the model parameter set, optionally a second set of reconfiguration parameters is signaled in the layer parameter set, and optionally a third set of reconfiguration parameters is signaled in the sublayer header. If present, the second set of reconfiguration parameters depends on the first set of reconfiguration parameters. If present, the third set of reconfiguration parameters may depend on the first and / or second set of reconfiguration parameters. This embodiment is described in more detail with respect to FIG. 17.

[0130] TIFF2025063087000002.tif45170

[0131] In a preferred embodiment, s=2 -0.5 It is.

[0132] The rational number s may, for example, be encoded as a floating-point value. The first and second integer values ​​x1 and x2 may be signaled using a fixed or variable number of bits to minimize the overall signaling cost. For example, if the quantization step sizes of the sub-layers of a layer are similar, the associated value x2 will be a rather small integer and only a few bits will be needed to signal them. In some cases, it may be effective to allow or not allow

[0133] In a preferred embodiment, the reconstruction parameters consist of a codebook, i.e., a quantization index-to-reconstruction level mapping, which is a list of mappings between integers and rational numbers, as shown in Figure 18. n Layers or or the associated parameters of the sublayer payload are reconstructed using the following reconstruction rule 270: n is looked up in the codebook. The associated integer is W n One mapping that matches is selected and the associated rational number becomes the reconstructed value, ie, the reconstructed NN parameter 32'.

[0134] In another preferred embodiment, the first and / or second and / or third sets of reconstructed parameters are each composed of a codebook according to the previous preferred embodiment. However, to apply the reconstruction rule, one joint codebook is derived by forming a union of the mappings of the codebooks of the first and / or second and / or third sets of reconstructed parameters. If there are mappings with the same integers, the mappings of the codebook of the third set of reconstructed parameters are preferred over the mappings of the codebook of the second set of reconstructed parameters, which are preferred over the mappings of the codebook of the first set of reconstructed parameters.

[0135] 17 shows an embodiment G1 of a data stream 45 having NN parameters 32 encoded therein, representing a neural network 10. The NN parameters 32 are encoded into the DS 45 in a manner that they are quantized 260 into quantization indexes. The NN parameters 32 are encoded into the DS 45 such that the NN parameters 32 of different NN portions of the NN 10 are quantized 260 differently. The DS 45 stores, for each of the NN portions, the NN parameters 32 associated with the respective NN portion. A reconstruction rule 270 for dequantizing parameters is shown.

[0136] For example, each NN part of a NN includes interconnections between the nodes of the NN, and different NN parts include different interconnections between the nodes of the NN.

[0137] According to an embodiment, the NN portion includes NN layers 210 of the NN 10 and / or layer sub-portions 43 into which a given NN layer of the NN is subdivided. As shown in FIG. 17, all NN parameters 32 in one layer 210 of the NN are calculated based on the NN parameters 32 in a first layer 2101 of the NN 10. are quantized 260 to be different from the NN parameters 32 in the second layer 2102 of the NN 10. The NN parameter 32 in the NN layer 2101 represents the NN part of the NN. The NN layer 2101 may also be grouped into different layer subportions 43, i.e., separately accessible subportions, with each group potentially representing a NN portion. Thus, different layer subportions 43 of the NN layer 2101 are quantized 260 differently.

[0138] A corresponding embodiment ZG1 relates to an apparatus for encoding NN parameters 32 representing a neural network 10 into a DS 45. The NN parameters 32 are encoded into the DS 45 in a quantized manner 260 into quantization indexes. The NN parameters 32 are encoded into the DS 45 such that the NN parameters 32 of different NN parts of the NN 10 are quantized 260 differently. The apparatus is configured to provide, for each of the NN parts, a DS 45 indicating a reconstruction rule for dequantizing the NN parameters 32 associated with the respective NN part. Optionally, the apparatus can also perform the quantization 260.

[0139] Another corresponding embodiment XG1 relates to an apparatus for decoding NN parameters 32 representing a neural network 10 from a DS 45. The NN parameters 32 are coded in the DS 45 in a quantized manner 260 into quantization indexes. The NN parameters 32 are coded in the DS 45 such that the NN parameters 32 of different NN parts of the NN 10 are quantized 260 differently. The apparatus is configured to decode, for each of the NN parts, a reconstruction rule 270 for dequantizing the NN parameters 32 associated with the respective NN part from the data stream 45. Optionally, the apparatus can also perform dequantization using the reconstruction rule 270, i.e. the one associated with the NN part to which the currently dequantized NN parameters 32 belongs. The apparatus can also be configured to dequantize, for each of the NN parts, the NN parameters of the respective NN part using the decoded reconstruction rule 270 associated with the respective NN part.

[0140] In the following, different features and / or functionalities are described in the context of data stream 45, but the features and / or functionalities can also be, in the same or similar way, features and / or functionalities of an apparatus according to embodiment ZG1 or an apparatus according to embodiment XG1.

[0141] As already described above, according to embodiment G2 of the DS 45 of embodiment G1, the NN portion includes the NN layer 210 of the NN 10 and / or a layer portion into which a given NN layer 210 of the NN 10 is subdivided.

[0142] According to embodiment G3, the DS 45 of embodiment G1 or G2 is delta-coded with respect to a second reconstruction rule 2702 for dequantizing 260 the NN parameters 32 associated with the second NN part, and the NN parameters 32 associated with the first NN part are dequantized. The first reconstruction rule 2701 for dequantization is encoded therein. Alternatively, as shown in FIG. 17, the NN associated with the first NN portion, i.e., layer subportion 431 The first reconstruction rule 270a1 for dequantizing the parameters 32 is , i.e., the layer sub-portion 432, is delta-coded and coded in the DS 45. Also, the first NN portion, i.e., the layer sub-portion 43 A first reconstruction rule 270a1 for dequantizing the NN parameters 32 associated with , it is also possible that the second reconstruction rule 2702 associated with the second NN part, i.e., the NN layer 2102, is delta-coded and coded into the DS45.

[0143] In the following embodiments, in order to avoid ambiguity of the embodiments, the first reconstruction rule is denoted as 2701 and the second reconstruction rule is denoted as 2702. However, even in the following embodiments, it is clear that the first reconstruction rule and / or the second reconstruction rule may correspond to the NN portion representing the layer sub-portion 43 of the NN layer 210, as described above.

[0144] According to embodiment G4, of the DS45 of embodiment G3, the DS45 includes a first exponent value for indicating the first reconstruction rule 2701 and a second exponent value for indicating the second reconstruction rule 2702. The first reconstruction rule 2701 is defined by a predetermined base index and a first index value. a first quantization step size defined by a first exponent, The second reconstruction rule 2702 is a sum of the exponent of a given basis and the first and second exponent values. a second exponent defined by a second quantization step size defined by ...

[0145] According to embodiment G4a, among the DSs of embodiment G4, DS45 further indicates a predetermined basis.

[0146] According to embodiment G4', among the DSs of any of the previous embodiments G1 to G3, the DS 45 includes a first exponent value for indicating a first reconstruction rule 2701 for inverse quantizing the NN parameters 32 associated with the first NN part, and a second exponent value for indicating a second reconstruction rule 2702 for inverse quantizing the NN parameters 32 associated with the second NN part. and a second exponent value for indicating a second reconstruction rule 2702 for inverse quantizing 2. . The first reconstruction rule 2701 is a rule for determining a first exponent value and a second exponent value of a predetermined base. a first exponent defined by the sum of The second reconstruction rule is defined by an exponent of a predetermined basis and a second quantization step size defined by the sum of the second exponent value and the predetermined exponent value.

[0147] According to embodiment G4'a of the DS of embodiment G4', the DS further indicates a predefined basis.

[0148] According to embodiment G4'b, in the DS of embodiment G4'a, the DS indicates a predetermined basis related to the range of the NN, i.e. the entire NN.

[0149] According to embodiment G4'c, of the DS of any of the previous embodiments G4' to G4'b, the DS 45 further indicates a predefined exponent value.

[0150] According to embodiment G4'd, of the DS45 of embodiment G4'c, the DS45 indicates a predetermined exponent value for the NN layer range, ie, for a given NN layer 210 to which the first NN portion 431 and the second NN portion 432 belong.

[0151] According to embodiment G4'e, of the DS of any of the previous embodiments G4'c and G4'd, DS45 further indicates a predetermined base, and DS45 indicates a predetermined exponent value in a finer range than the range in which the predetermined base is indicated by DS45.

[0152] According to embodiment G4f, the DS45 of any of the previous embodiments G4 to G4a or G4' to G4'e has the predetermined base encoded therein in a non-integer format, e.g., floating point or rational or fixed point, and the first and second exponent values ​​in an integer format, e.g., signed integers. Optionally, the predetermined exponent values ​​may also be encoded in the DS45 in integer format.

[0153] According to embodiment G5, the DS 45 of any of embodiments G3 to G4f includes a first quantization index-reconstruction level to indicate the first reconstruction rule 2701. A first set of parameters defining the mapping and a second reconstruction rule 2702 are shown. and a second set of parameters defining a second quantization index-to-reconstruction level mapping for the first quantization index-to-reconstruction level mapping. The first reconstruction rule 2701 is a first quantization index-reconstruction level mapping is defined as follows: The second reconstruction rule 2702 is a first quantization index-reconstruction level mapping in a predetermined manner with the second quantization index-to-reconstruction level mapping.

[0154] According to an embodiment G5', the DS45 according to any one of the embodiments G3 to G5 is configured to include a first quantization index-reconstruction rule 2701. The first parameter set defines a bell mapping, and the second reconstruction rule 2702 is specified. and a second set of parameters defining a second quantization index-to-reconstruction level mapping for indicating. The first reconstruction rule 2701 defines a predetermined quantization index-to-reconstruction level mapping in a predetermined manner by a first quantization index-to-reconstruction level mapping, and The second reconstruction rule 2702 defines a predetermined quantization index-to-reconstruction level mapping is defined by extending in a predetermined manner by the second quantization index-reconstruction level mapping.

[0155] According to embodiment G5'a, of the DS45 of embodiment G5', the DS45 further indicates a predefined quantization index-reconstruction level mapping.

[0156] According to embodiment G5'b, of the DS45 of embodiment G5'a, the DS45 is in the NN range, i.e. for the entire NN, or in the NN layer range, i.e. for the first NN part 431 and and a predetermined quantization index for a predetermined NN layer 210 to which the second NN portion 432 belongs. -reconstruction level mapping. The predetermined quantization index-reconstruction level mapping may be indicated in NN scope in case of NN parts representing NN layers. For example, for each of the NN parts, the respective NN part represents a corresponding NN layer, e.g. a first NN part represents a different NN layer than the second NN part. However, it is also possible to indicate the predetermined quantization index-reconstruction level mapping in NN scope in case of at least some of the NN parts representing layer subportions 43. Additionally or alternatively, the predetermined quantization index-reconstruction level mapping may be indicated in NN layer scope in case of NN parts representing layer subportions 43.

[0157] According to embodiment G5c, according to a certain aspect of the DS45 of any of the previous embodiments G5 or G5' to G5'b, The mapping of each index value, i.e., quantization index 32″, to a first reconstruction level according to the extended quantization index-reconstruction level mapping is a mapping of each index value to a second reconstruction level according to a quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping. If a mapping to a bell exists, it is given priority; and / or According to the extended quantization index-reconstruction level mapping, the reconstruction level to which each index value should be mapped is not defined, and according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, for any index value that is mapped to a corresponding reconstruction level, a mapping from each index value to a corresponding reconstruction level is adopted, and / or According to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, the reconstruction level to which each index value should be mapped is not defined, and according to the extended quantization index-reconstruction level mapping, for any index value that is mapped onto a corresponding reconstruction level, a mapping from each index value to a corresponding reconstruction level is adopted.

[0158] According to embodiment G6 shown in FIG. 18, of the DS45 of any of the previous embodiments G1 to G5c, the DS45 includes the following to indicate the reconstruction rule 270 of a given NN portion, for example to represent an NN layer or to indicate a layer sub-portion of an NN layer: a quantization step size parameter 262 indicating the quantization step size 263; and A parameter set 264 that defines a quantization index to reconstruction level mapping 265 . Here, the reconstruction rule 270 for a given NN part is: quantization step size 263 for quantization index 32'' within a given index interval 268; and quantization index to reconstruction level mapping 265 for quantization indexes 32'' outside a predetermined index interval 268; It is defined by:

[0159] FIG. 18 illustrates an embodiment H1 of a data stream 45 having encoded therein the NN parameters 32 representing a neural network. Here, the NN parameters 32 are encoded into the DS 45 in a form quantized 260 on the quantization indexes 32''. Here, the DS 45 includes the following to indicate the NN parameters 280, ie, the reconstruction rules 270 for dequantizing the quantization indexes 32″: A quantization step size parameter 262 indicating a quantization step size 263; and A parameter set 264 that defines a quantization index to reconstruction level mapping 265 . Here, the reconstruction rule 270 for a given NN part is: quantization step size 263 for quantization index 32'' within a given index interval 268; and a quantization index to reconstruction level mapping 265 for quantization indexes 32'' outside a predetermined index interval 268.

[0160] The corresponding embodiment ZH1 relates to an apparatus for encoding NN parameters 32 representing a neural network into a DS 45. The NN parameters 32 are quantized 260 into quantization indexes 32″ and encoded into the DS 45. The apparatus is configured to provide the DS 45 with the following to indicate the reconstruction rules 270 for dequantizing the NN parameters 32: A quantization step size parameter 262 indicating a quantization step size 263; and A parameter set 264 that defines a quantization index to reconstruction level mapping 265 . Here, the reconstruction rule 270 for a given NN part is: quantization step size 263 for quantization index 32'' within a given index interval 268; and a quantization index to reconstruction level mapping 265 for quantization indexes 32'' outside a predetermined index interval 268.

[0161] Another corresponding embodiment XH1 relates to an apparatus for decoding NN parameters 32 representing a neural network from a DS 45. The NN parameters 32 are quantized to quantization indexes 32'' and encoded in the DS 45. The apparatus comprises: A quantization step size parameter 262 indicating a quantization step size 263; and By decoding the parameter set 264 that defines the quantization index-reconstruction level mapping 265 from the DS 45, it is possible to derive the reconstruction rule 270 for dequantizing the NN parameters, i.e., the quantization index 32''. Here, the reconstruction rule 270 for a given NN part is: quantization step size 263 for quantization index 32'' within a given index interval 268; and a quantization index to reconstruction level mapping 265 for quantization indexes 32'' outside a predetermined index interval 268.

[0162] In the following, different features and / or functionalities are described in the context of data stream 45, but the features and / or functionalities may be the same or similar to the features and / or functionalities of the device according to embodiment ZH1 or the device according to embodiment XH1.

[0163] According to embodiment G7, the predefined index interval 268 of the DS 45 of any of the previous embodiments G6 or H1 includes zero.

[0164] According to embodiment G8, in the DS 45 of embodiment G7, a predetermined index interval 268 extends up to a predetermined magnitude threshold y, and a quantization index 32'' that exceeds the predetermined magnitude threshold y represents an escape code that signals that the quantization index-reconstruction level mapping 265 is used for inverse quantization 280.

[0165] According to embodiment G9, the parameter set 264 of the DS45 of any of the previous embodiments G6 to G8 defines a quantization index-reconstruction level mapping 265 by a list of reconstruction levels associated with quantization indexes 32'' outside a predetermined index interval 268.

[0166] According to embodiment G10, the NN portion of the DS 45 of any of the previous embodiments G1 to G9 includes one or more sub-portions of an NN layer of a NN and / or one or more NN layers of a NN. Figure 18 is an example of an NN portion including one NN layer of a NN. The NN parameter tensor 30 including the NN parameters 32 represents the corresponding NN layer.

[0167] According to embodiment G11, the data stream 45 of the DS 45 of any of the previous embodiments G1-G10 is structured into separately accessible portions, each of which has, for a corresponding NN portion, NN parameters 32 encoded therein. See, for example, FIG. 8 or one of FIGs. 10-17.

[0168] According to embodiment G12, the individually accessible portions of DS45 of G11 are encoded using context-adaptive arithmetic coding and using context initialization at the start of each individually accessible portion, for example as shown in FIG. 8.

[0169] According to embodiment G13, the DS45 of any of the previous embodiments G11 or G12 may be The data stream 45 may be, for each individually accessible portion, as shown, for example, in one of Figs. a start code 242 where each individually accessible portion begins, and / or a pointer 220 / 244 to the beginning of each individually accessible portion, and / or and a data stream length parameter 246 indicating the data stream length of each individually accessible portion for skipping each individually accessible portion when parsing the DS45.

[0170] According to embodiment G14, the data stream 45 of the DS 45 of any of the previous embodiments G11 to G13 further comprises, for each of the NN parts, a reconstruction rule 270 for dequantizing 280 the NN parameters 32 associated with the respective NN part, DS45 main header section 47 for the entire NN, an NN layer-related header portion 110 of the DS 45 associated with the NN layer 210 to which the respective NN portion belongs; or NN-part specific header portion 300 of the DS 45 associated with each NN part, e.g., part of the NN part if it represents a layer sub-part of the NN layer 210, i.e., an individually accessible sub-part 43 / 44 / 240; Shown within.

[0171] According to embodiment G15, of DS45 of any of the previous embodiments G11-G14, DS45 is according to any of the previous embodiments A1-F2.

[0172] 4 Identifiers that depend on the parameter hash In a scenario such as distributed training, where many clients independently further train the network and send relative NN updates back to a central entity, it is important to identify the network through a versioning scheme, so that the central entity can identify the NN on which a NN update is being built.

[0173] In other use cases, such as scalable NNs, a baseline portion of a NN may be run, e.g., to generate preliminary results, before running a full or enhanced NN to obtain full results. The enhanced NN may use a slightly different version of the baseline NN, e.g., an updated parameter tensor. If such an updated parameter tensor is differentially encoded, i.e., as an update to a previously encoded parameter tensor, it is necessary to identify the parameter tensor on which the differentially encoded update is constructed, e.g., using an identification parameter 310, as shown in FIG. 19.

[0174] Furthermore, there are use cases where the integrity of the NN is paramount, i.e., transmission errors or inadvertent changes in the parameter tensors need to be easily discernible. The identifier, i.e., the identification parameter 310, makes the operation more error-robust if it can be verified based on the properties of the NN.

[0175] However, state-of-the-art versioning is done via checksums or hashes of the data format of the entire container, making it difficult to match equivalent NNs in different containers, but also because the clients involved may use different frameworks / containers, and it is impossible to identify / verify only a subset of the NN (a layer, a sub-layer) without a complete reconfiguration of the NN.

[0176] Thus, as part of the present invention, in one embodiment, an identifier, i.e. an identification parameter 310 is carried with each entity, i.e. model, layer, sub-layer, and enables each entity to: -Verification of identity, and / or -referring to or being referred to, and / or -Check consistency.

[0177] In another embodiment, the identifier is derived from the parameter tensor using a hashing algorithm such as MD5 or SHA5, or an error detection code such as a CRC or checksum.

[0178] In another embodiment, such an identifier for an entity may be derived using identifiers of its subordinate entities, e.g., a layer identifier may be derived from the identifiers of its constituent sublayers, and a model identifier may be derived from the identifiers of its constituent layers.

[0179] 19 shows an embodiment I1 of a data stream 45 in which a representation of a neural network is encoded. The data stream 45 is structured into individually accessible portions 200. Each portion 200 represents a corresponding NN portion of the neural network, for example comprising one or more NN layers or comprising a portion of a NN layer. The data stream 45 comprises, for each of the one or more predefined individually accessible portions 200, an identification parameter 310 for identifying the respective predefined individually accessible portion 200.

[0180] The corresponding embodiment ZI1 relates to an apparatus for encoding a representation of a neural network in a DS 45. The data stream 45 is structured into individually accessible portions 200. Each portion 200, for example including one or more NN layers or portions of NN layers, represents a corresponding NN portion of the neural network. The apparatus is configured to provide the data stream 45 with an identification parameter 310 for each of the one or more predetermined individually accessible portions 200 for identifying the respective predetermined individually accessible portion 200.

[0181] Another corresponding embodiment XI1 relates to an apparatus for decoding a representation of a neural network from a DS 45. The data stream 45 is structured into individually accessible portions 200. Each portion 200 represents a corresponding NN portion of the neural network, for example including one or more NN layers or including a portion of a NN layer. The apparatus is configured to decode from the data stream 45, for each of one or more predetermined individually accessible portions 200, an identification parameter 310 for identifying the respective predetermined individually accessible portion 200.

[0182] In the following, different features and / or functionalities are described in the context of data stream 45, but the features and / or functionalities can also be, in the same or similar way, features and / or functionalities of an apparatus according to embodiment ZI1 or features and / or functionalities of an apparatus according to embodiment XI1.

[0183] According to embodiment I2, of the DS 45 of embodiment I1, the identification parameter 310 is associated with each given individually accessible portion 200 via a hash function or an error detection or correction code.

[0184] According to embodiment I3, the DS 45 of any of the previous embodiments I1 and I2 further includes a higher level identification parameter for identifying the collection of one or more predetermined individually accessible portions 200 .

[0185] According to embodiment I4, the higher-level identification parameters of the DS 45 of embodiment I3 are associated with the identification parameters 310 of one or more given individually accessible portions 200 via a hash function or an error detection or correction code.

[0186] According to embodiment I5, an individually accessible portion 200 of a DS45 of any of the previous embodiments I1 to I4 is encoded using context-adaptive arithmetic coding and using context initialization at the start of each individually accessible portion, as shown, for example, in FIG.

[0187] According to embodiment I6, in the DS 45 of any of the previous embodiments I1 to I5, the data stream 45 is, for each of the individually accessible portions 200, as shown in any of Figs. 11 to 15, a start code 242 where each individually accessible portion 200 begins, and / or a pointer 220 / 244 to the beginning of each individually accessible portion 200; and / or and a data stream length parameter 246 indicating the data stream length of each individually accessible portion 200 for skipping each individually accessible portion 200 when parsing the DS45.

[0188] According to embodiment I7, in DS 45 of any of the previous embodiments I1-I6, the NN portion includes one or more subportions of an NN layer of the NN and / or one or more NN layers of the NN.

[0189] According to embodiment I8, of the DS45 of any of the previous embodiments I1 to I7, the DS45 is according to any of the previous embodiments A1 to G15.

[0190] 5 Scalable NN bitstream As mentioned above, some applications may further structure the NN 10 by splitting the layer 210 or a group thereof, i.e., sub-layer 43 / 44 / 240, into a baseline, e.g., second version 3301 of the NN 10, and an advanced portion 3302, e.g., first version 3302 of the NN 10, as shown in Figs. 20 to 23. It depends. The client may be able to match its processing power or may be able to make a guess at the baseline first prior to processing the more complex advanced NN. In such cases, it is beneficial to sort and code the parameter tensors30 for each sub-portion of the NN layer independently so that they can be accessed in an informed way, as described in Sections 1 through 4.

[0191] In addition, in some cases, the NN10 may Reducing the number of neurons in a layer, e.g. requiring fewer operations, as shown in FIG. 22, and / or - a coarser quantization of the weights, as shown in FIG. 21, e.g. allowing faster reconstruction, and / or - Different training (learning) such as general baseline NN and personalized advanced NN, as shown in Figure 23. - etc., can be divided into baseline and advanced variants.

[0192] 21 is a diagram showing a variation of the NN and the differential delta signal 342. A baseline version, e.g., the second version 3301 of the NN, and an advanced version, e.g., A first version 3302 of the NN is shown. FIG. 21 shows the first version 3302 of the NN with two quantization settings. We show one of the above cases where we create two variants of the original NN from a single layer, e.g., parameter tensor 30 representing the corresponding layers, and create the respective delta signals 342. The baseline version 3301 is associated with coarse quantization, while the advanced version The advanced version 3302 is associated with finer quantization. The advanced version 3302 may be delta coded relative to the baseline version 3301.

[0193] FIG. 22 shows a further variation of the separation of the original NN. In FIG. 22, a further variation of the NN separation is shown. For example, on the left side, a separation of a layer, e.g., a parameter tensor 30 representing a corresponding layer, into a baseline portion 30a and an advanced portion 30b is shown, i.e., the advanced portion 30b extends the baseline portion 30a. The inference of the advanced portion 30b requires the inference of the baseline portion 30a. On the right side of FIG. 22, the central portion of the advanced portion 30b is shown to consist of an update of the baseline portion 30a, which can also be delta coded as shown in FIG. 21.

[0194] In these cases, the NN parameters 32, e.g., weights, of the baseline 3301 and advanced 3302 NN versions have an explicit dependency and / or the baseline version 3301 of the NN becomes part of the advanced version 3302 of the NN in some way.

[0195] Therefore, the parameters of the advanced NN part, i.e., the first version 3302 of the NN Encoding the metric tensor 30b at the NN scale or layer scale, or even at the sub-layer scale, as a delta relative to the parameter tensor 30b of the baseline NN version, i.e., the second part of the NN 3301, is beneficial in terms of coding efficiency, processing overhead, parallelization, etc.

[0196] A further variation is depicted in FIG. 23, where an advanced version of a NN is created to compensate for the compression effects on the original NN by training in the presence of a lossy-compressed baseline NN variant. The advanced NN is inferred in parallel with the baseline NN, and its NN parameters, e.g., weights, are connected to the same neurons as the baseline NN. FIG. 23 illustrates training of an augmented NN based, for example, on a lossy-coded baseline NN variant.

[0197] In one embodiment, a (sub-)layer bitstream, i.e. an individually accessible portion 200 or an individually accessible sub-portion 34 / 44 / 220, is split into two or more (sub-)layer bitstreams, the first one being the baseline version 3301 of the (sub-)layer the second one is the advanced version 3302 of the first (sub)layer, and so on. In bitstream order, baseline version 3301 is followed by advanced version 3302. It precedes Model 3302.

[0198] In another embodiment, the (sub)layer bitstream is shown to include incremental updates of parameter tensors 30 of another (sub)layer in the bitstream, such as delta parameter tensors, i.e., delta signal 342, and / or incremental updates including parameter tensors.

[0199] In another embodiment, the (sub-)layer bitstream carries a reference identifier that references a (sub-)layer bitstream with a matching identifier that contains an incremental update of the parameter tensor 30 .

[0200] 20 shows an embodiment J1 of a data stream 45 in which a representation of a neural network 10 is encoded in a layered manner such that different versions 330 of the NN 10 are encoded in the data stream 45. The data stream 45 may include one or more individually accessible 2. The neural network 10 is structured into portions 200. Each portion 200 is associated with a corresponding version 330 of the neural network 10. Data stream 45 indicates that a first version 3302 of NN 10 and / or To perform inference based on the first version of the neural network 3302 , each in the form of one or more compensation NN portions 332 executed in addition to the execution of a corresponding NN portion 334 of the second version 3301 of the NN 10 encoded in the second portion 2001; and The output 336 of each compensation NN section 332 and the corresponding NN section 334 are summed 338 in the form of a compensation NN section 332. The first portion 2002 is encoded.

[0201] According to an embodiment, the compensation NN portion 332 may include a delta signal 342 as shown in FIG. 21, or additional tensor and delta signals as shown in FIG. 22, or NN parameters that are trained differently than the NN parameters in the corresponding NN portion 334, e.g., as shown in FIG. 23.

[0202] According to an embodiment as shown in FIG. 23 , the compensation NN portion 332 includes quantized NN parameters of the NN portion of a second neural network, which is associated with NN10, i.e., the corresponding NN portion 334 of the first NN. The second neural network may be trained such that the compensation NN portion 332 may be used to compensate for compression effects, e.g., quantization errors, on the corresponding NN portion 334 of the first NN. The output of each compensation NN portion 332 and corresponding NN portion 334 may be used to reconstruct the NN parameters corresponding to the first version 3302 of NN10. The configuration is summed up to enable inference based on the first version of NN10, 3302. can be.

[0203] Although the above-described embodiment mainly focuses on providing different versions 330 of the NN 10 in one data stream, it is also possible to provide different versions 330 in different data streams. The different versions 330 are, for example, delta-coded into different data streams compared to a simpler version. Thus, separate data streams (DS) may be used. For example, first a DS containing initial NN data is transmitted, and later a DS containing updated NN data is transmitted.

[0204] A corresponding embodiment ZJ1 relates to an apparatus for hierarchically encoding a representation of a neural network into a DS 45 such that different versions 330 of the NN 10 are encoded into a data stream 45, the data stream 45 being structured into one or more individually accessible portions 200, each portion 200 being associated with a corresponding version 330 of the neural network 10. The apparatus comprises: The first version of NN10, 3302, and / or To perform inference based on the first version 3302 of the NN 10, each in the form of one or more compensation NN portions 332 executed in addition to the execution of a corresponding NN portion 334 of the second version 3301 of the NN 10 encoded in the portion 2001 of The output 336 of each compensation NN section 332 and the corresponding NN section 334 are summed 338 in the form of a compensation NN section 332. The first portion 2002 is adapted to be encoded.

[0205] Another corresponding embodiment XJ1 relates to an apparatus for decoding a representation of a neural network 10 from a layer-wise encoded DS 45. Different versions 330 of the neural network 10 are encoded in a data stream 45, the data stream 45 is structured into one or more separately accessible portions 200, each portion 200 being associated with a corresponding version 330 of the neural network 10, and the apparatus: by using delta decoding 340 on the second version 3301 of the NN 10 encoded in the second portion 2001; and / or From DS45, each performs inference based on the first version 3302 of NN10 NN portion 332 to obtain configured to decode a first version 3302 of the NN 10 encoded from the first portion 2002; 2001 in addition to executing the corresponding NN portion 334 of the second version 3301 of the NN 10 encoded in the second portion 2001; The outputs 336 of each compensation NN section 332 and the corresponding NN section 334 are summed 338 .

[0206] In the following, different features and / or functionalities are described in the context of data stream 45, but the features and / or functionalities may be, in the same or similar manner, features and / or functionalities of the apparatus according to embodiment ZJ1 or features and functionalities of the apparatus according to embodiment XJ1.

[0207] According to embodiment J2, among the data streams 45 of embodiment J1, the data streams 45 are the first version 3301 of the NN 10: Differences in weights and / or biases, i.e., differences between the NN parameters associated with a first version 3301 of NN 10 and the NN parameters associated with a second version 3302 of NN 10, as shown, for example, in FIG. 21 ; and / or Additional neurons or interconnections of neurons, e.g. as shown in FIG. NN 10 is delta coded 340 to a second version 3302 of NN 10 that was coded in second portion 2002 in and coded into first portion 2001 .

[0208] According to embodiment J3, the individually accessible portions 200 of a DS of any of the previous embodiments J1 and J2 are coded using context-adaptive arithmetic coding with context initialization at the start of each individually accessible portion 200, for example as shown in FIG. 8.

[0209] According to embodiment J4, the data stream 45 of the DS of any of the previous embodiments J1 to J3 is, for example, as shown in one of Figs. 11 to 15, each individually accessible portion 200: a start code 242 where each individually accessible portion 200 begins, and / or a pointer 220 / 244 to the beginning of each individually accessible portion 200; and / or A data stream length parameter indicating the data stream length 246 of each individually accessible portion 200 is included to skip each individually accessible portion 200 when parsing the DS 45 .

[0210] According to embodiment J5, in the DS45 of any of the previous embodiments J1 to J4, the data stream 45 includes, for each of one or more predetermined individually accessible portions 200, an identification parameter 310 for identifying each predetermined individually accessible portion 200, as shown, for example, in FIG. 19.

[0211] According to embodiment J6, of the DS45 of any of the previous embodiments J1 to J5, the DS45 is according to any of the previous embodiments A1 to I8.

[0212] 6 Supplementary Data As shown in Figures 24a and 24b, there are application scenarios in which the parameter tensor 30 is accompanied by additional augmentation (or auxiliary / supplementary) data 350. This supplementary data 350 is usually not necessary for the decoding / reconstruction / inference of the NN, but may be mandatory from the application point of view. Examples may be, for example, information on the relevance of each parameter 32 (Sebastian Lapuschkin, 2019) or information on sufficient statistics of the parameters 32, such as intervals or variances, indicating the robustness of each parameter 32 against perturbations (Christos Louizos, 2017).

[0213] Such augmentation information, i.e., supplemental data 350, may introduce a significant amount of data with respect to the parameter tensor 30 of the NN, such that it may be desirable to similarly encode the augmentation data 350 using a scheme such as DeepCABAC. However, it is important to mark this data as irrelevant to the decoding of the NN, for inference purposes only, so that clients that do not require the augmentation can skip this portion of the data.

[0214] In one embodiment, the enhancement data 350 is coded independently of the (sub-)layer bitstream data, e.g., independently of the separately accessible portions 200 and / or the separately accessible sub-portions 240, but is carried in an additional (sub-)layer enhancement bitstream, i.e., a further separately accessible portion 352, interspersed with the respective (sub-)layer bitstreams to form the model bitstream, i.e., data stream 45. Figures 24a and 24b illustrate this embodiment. Figure 24b illustrates the enhancement bitstream 352.

[0215] Figures 24a and 24b show embodiment K1, a data stream 45 in which a representation of a neural network is encoded. The data stream 45 is structured into individually accessible portions 200, each portion 200 representing a corresponding NN portion of the neural network. The data stream 45 includes, for each of one or more predefined individually accessible portions 200, supplementary data 350 for supplementing the representation of the NN. Alternatively, as shown in Figure 24b, the data stream 45 includes, for each of one or more predefined individually accessible portions 200, supplementary data 350 for supplementing the representation of the NN.

[0216] A corresponding embodiment ZK1 relates to an apparatus for encoding a representation of a neural network in a DS 45. The data stream 45 is structured into individually accessible portions 200, each portion 200 representing a corresponding NN portion of the neural network. The apparatus is configured to provide the data stream 45 with supplementary data 350 for each of one or more predetermined individually accessible portions 200 to supplement the representation of the NN. Alternatively, the apparatus is configured to provide the data stream 45 with supplementary data 350 for each of one or more predetermined individually accessible portions 200 to supplement the representation of the NN.

[0217] Another corresponding embodiment XK1 relates to an apparatus for decoding a representation of a neural network from a DS 45. The data stream 45 is structured into individually accessible portions 200, each portion 200 representing a corresponding NN portion of the neural network. The apparatus is configured to decode, from the data stream 45, supplementary data 350 for supplementing the representation of the NN for each of one or more predetermined individually accessible portions 200. Alternatively, the apparatus may decode, from the data stream 45, supplementary data 350 for supplementing the representation of the NN for each of one or more predetermined NN for each individually accessible portion 200 of the NN, the NN is configured to decode supplemental data 350 for supplementing the representation of the NN.

[0218] In the following, different features and / or functionalities are described in the context of data stream 45, but the features and / or functionalities may also be, in the same or similar way, features and / or functionalities of an apparatus according to embodiment ZK1 or features and / or functionalities of an apparatus according to embodiment XK1.

[0219] According to embodiment K2, among the data streams 45 of embodiment K1, the DS 45 indicates that the supplementary data 350 is not essential for inference based on a NN.

[0220] According to embodiment K3, the data stream 45 of any of the previous embodiments K1 and K2, the data stream 45 comprises supplementary data 350 for supplementing the representation of the NN for one or more predefined individually accessible portions 200 encoded in further individually accessible portions 352, as shown in Fig. 24b. The DS 45 comprises for one or more predefined individually accessible portions 200, e.g. for each of the one or more predefined individually accessible portions 200, a corresponding further predefined individually accessible portion 352 associated with the NN portion to which each predefined individually accessible portion 200 corresponds.

[0221] According to embodiment K4, the NN portion of the DS 45 of any of the previous embodiments K1 to K3 includes one or more NN layers of the NN and / or layer portions into which a certain NN layer of the NN is subdivided. For example, according to FIG. 24b, the separately accessible portion 2002 and corresponding further certain The individually accessible portion 352 of the NN relates to a NN portion that includes one or more NN layers.

[0222] According to embodiment K5, the individually accessible portions 200 of a DS45 of any previous embodiment K1 to K4 are coded using context-adaptive arithmetic coding with context initialization at the start of each individually accessible portion 200, for example as shown in FIG. 8.

[0223] According to the embodiment K6, in the DS 45 of any of the previous embodiments K1 to K5, the data stream 45 is, for each individually accessible portion 200, as shown in any of Figs. 11 to 15, a start code 242 where each individually accessible portion 200 begins, and / or a pointer 220 / 244 to the beginning of each individually accessible portion 200; and / or A data stream length parameter indicating the data stream length 246 of each individually accessible portion 200 for skipping each individually accessible portion 200 when parsing the DS45.

[0224] According to embodiment K7, the supplementary data 350 of the DS45 of any of the previous embodiments K1 to K6 is Relevance scores of the NN parameters, and / or This is about the robustness to perturbations of NN parameters.

[0225] According to embodiment K8, of the DS45 of any of the previous embodiments K1 to K7, the DS45 is according to any of the previous embodiments A1 to J6.

[0226] 7. Extended Control Data Besides the described functionality of the different access functions, an extended hierarchical control data structure, 25. That is, a sequence 410 of control data portions 420 may be required for different applications and usage scenarios. On the one hand, the compressed NN representation (or bitstream) may be used from within a particular framework, such as TensorFlow or Pytorch, in which case only minimal control data 400 is needed to decode, for example, the deepCABAC encoded parameter tensors. On the other hand, the particular type of framework may be unknown to the decoder, in which case additional control data 400 is required. Thus, depending on the use case and knowledge of the environment, different levels of control data 400 may be required, as shown in FIG. 25.

[0227] Figure 25 shows a hierarchical control data (CD) structure for a compressed neural network, i.e. a sequence 410 of control data portions 420. Depending on the usage environment, different CD levels, i.e. control data portions 420, e.g. dotted boxes, may or may not be present. In Figure 25, the compressed bitstream containing, e.g., a representation 500 of a neural network, may be any of the model bitstream types mentioned above, e.g., containing all the compressed data of the network with or without subdivision into sub-bitstreams.

[0228] Thus, if a particular network (e.g., TensorFlow, Pytorch, Keras, etc.) with a type and architecture known to the decoder and encoder includes a compressed NN technique, only the Compressed NN Bitstream is needed. However, if the decoder does not know any encoder settings, a full set of control data, i.e., a complete sequence 410 of the control data portion 420, is needed in addition to allowing a complete reconstruction of the network.

[0229] Examples of different hierarchical control data layers, ie, control data sections 420, are as follows: -CD Level 1: Compressed Data Decoder Control Information. -CD Level 2: Specific syntax elements from each framework (Tensor Flow, Pytorch, Keras) -CD Level 3: Cross-framework format elements, e.g. ONNX (ONNX = Open Neural Network Exchange) for use in different frameworks -CD Level 4: Information about the topology of the network -CD Level 5: Full network parameter information (for full reconstruction without knowledge of the network topology)

[0230] This embodiment thus describes a hierarchical control data structure of N levels, i.e., N control data portions 420, where 0 to N levels may exist to enable different modes of use ranging from specific compression-only core data usage to fully self-contained network reconfiguration. The levels, i.e., control data portions 420, may even include syntax from existing network architectures and frameworks.

[0231] In another embodiment, different levels, i.e., control data portion 420, may contain information about the neural network at different granularities. For example, the level structure may be structured as follows: -CD Level 1: Contains information about the network parameters, e.g. type, dimensions, etc. -CD Level 2: Contains information about the layers of the network. For example: type, identification, etc. -CD Level 3: Contains information about the topology of the network, e.g. connectivity between layers. -CD Level 4: Contains information about the model of the neural network. Example: training, learning parameters, performance, etc. - CD Level 5: Contains information about the training and validation dataset. Example: input natural images with 227x227 resolution with 1000 label categories.

[0232] Fig. 25 illustrates embodiment L1. A data stream 45 is encoded with a representation 500 of a neural network. The data stream 45 includes hierarchical control data 400 structured in a sequence 410 of control data portions 420. The control data portions 420 provide information about the NN with increasing detail along the sequence 410 of the control data portions 420. The second hierarchical control data 4002 of the second control data portion 4202 includes more detailed information than the first hierarchical control data 4001 of the first control data portion 4201.

[0233] According to an embodiment, the control data portion 420 may represent different units that may contain additional topology information.

[0234] The corresponding embodiment ZL1 relates to an apparatus for encoding a representation 500 of a neural network into a DS 45. The apparatus is arranged to provide to a data stream 45 hierarchical control data 400 structured in a sequence 410 of control data portions 420. The control data portions 420 provide information about the NN with increasing detail along the sequence 410 of control data portions 420.

[0235] Another corresponding embodiment XL1 relates to an apparatus for decoding a representation 500 of a neural network from a DS 45. The apparatus is configured to decode hierarchical control data 400 structured in a sequence 410 of control data portions 420 from a data stream 45. The control data portions 420 provide information about the NN with increasing detail along the sequence 410 of control data portions 420.

[0236] In the following, different features and / or functionalities are described in the context of data stream 45, but the features and / or functionalities can also be features and functionality of an apparatus according to embodiment ZL1 or according to embodiment XL1 in the same or similar manner.

[0237] According to embodiment L2, at least some of the control data portions 420 of the data stream 45 of embodiment L1 provide partially redundant information regarding the NN.

[0238] According to embodiment L3, the first control data portion 4201 of the data stream 45 of embodiment L1 or L2 indicates a default NN type, which means a default setting. The second control data part 4202 provides information about the NN. It includes parameters that indicate each.

[0239] According to embodiment L4, of the DS45 according to any one of the previous embodiments L1 to L3, the DS45 according to any one of the previous embodiments A1 to K8.

[0240] The embodiment X1 relates to an apparatus for decoding a data stream 45 according to any previous embodiment. The apparatus is configured to derive a NN 10, for example according to any of the above embodiments XA1 to XL1, from the data stream 45. For example, the apparatus is further configured to encode / decode a DS 45 according to any previous embodiment.

[0241] The present device includes, for example, Search for the start code 242, and / or The Data Stream Length 45 parameter is used to stream 200 individually accessible parts. KIP and / or Using the pointer 220 / 244 to resume parsing the data stream 45 at the beginning of the individually accessible portion 200; and / or Associating the decoded NN parameters 32' with the neurons 14, 18, 20 or the neuronal interconnections 22 / 24 according to the encoding order 104; and / or performing context adaptive arithmetic decoding and context initialization; and / or Perform inverse quantization / value reconstruction 280; and / or Performing a sum of exponents to calculate the quantization step size 263, and / or performing a lookup of the quantization index to reconstruction level mapping 265 in response to the quantization index 32'' leaving a predetermined index interval 268, such as to estimate an escape code; and / or performing a hash or applying an error detection / correction code to an individually accessible portion 200 and comparing the result with a corresponding identification parameter 310 to check the correctness of the individually accessible portion 200; and / or reconstructing a version 330 of the NN 10 by adding weights and / or bias differences to the underlying NN version 330, and / or adding additional neurons 14, 18, 20 or neuronal interconnects 22 / 24 to the underlying NN version 330, or by performing a joint execution of one or more compensation NN portions and corresponding NN portions and summing their outputs; and / or Reading stops when the currently read control data portion 420 assumes a parameter state known to the device, and control data portions 420 are read sequentially while providing information, i.e., hierarchical control data 400, in a manner consistent with a predetermined degree of conformance.

[0242] Embodiment Y1 relates to an apparatus for performing inference using NN10, and includes an apparatus for decoding a data stream 45 according to embodiment X1 to derive NN10 from the data stream 45, and a processor configured to perform inference based on NN10.

[0243] Embodiment Z1 relates to an apparatus for encoding a data stream 45 according to any previous embodiment, e.g. according to any of the above embodiments ZA1 to ZL1, for example further configured to encode / decode such that DS45 follows any previous embodiment.

[0244] The device selects the encoding order 104, for example, to find the optimum one for optimal compression efficiency.

[0245] Embodiment U relates to a method performed by the apparatus of any of embodiments XA1 to XL1 or ZA1 to ZL1.

[0246] Embodiment W relates to a computer program which, when executed by a computer, causes the computer to carry out the method of embodiment U.

[0247] A modified embodiment.

[0248] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

[0249] Depending on specific implementation requirements, embodiments of the present invention can be implemented in hardware or software. Implementation can be performed using a digital storage medium, such as a floppy disk, DVD, Blu-Ray, CD, ROM, PROM, EPROM, EEPROM or FLASH memory having electronically readable control signals stored thereon and cooperating (or capable of cooperating) with a programmable computer system such that the respective method is performed. Thus, the digital storage medium may be computer readable.

[0250] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, capable of cooperating with a programmable computer system to carry out one of the methods described herein.

[0251] In general, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product is run on a computer. The program code may for example be stored on a machine readable carrier.

[0252] A further embodiment comprises the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0253] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0254] A further embodiment of the inventive method is therefore a data carrier (or digital storage medium, or computer readable medium) having recorded thereon a computer program for performing one of the methods described herein. The data carrier, digital storage medium, or recording medium is typically tangible and / or non-transitory.

[0255] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, The data stream or the sequence of signals may for example be adapted to be transmitted via a data communication connection, for example the Internet.

[0256] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0257] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0258] A further embodiment according to the invention consists of an apparatus or system configured to transfer (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may for example be a computer, a mobile device, a memory device, etc. The apparatus or system may for example comprise a file server for transferring the computer program to the receiver.

[0259] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0260] The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0261] The apparatus described herein, or any components of the apparatus described herein, may be implemented at least in part in hardware and / or software.

[0262] The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0263] Any of the methods described herein, or elements of the apparatus described herein, may be implemented at least in part by hardware and / or software.

[0264] The above-described embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. It is therefore intended to be limited only by the scope of the appended claims, and not by the specific details presented in the description and explanation of the embodiments herein.

[0265] 8. References Andrew Kerr, DM (2017, 5). Retrieved from https: / / devblogs.nvidia.com / cutlass-linear-algebra-cuda / Chollet, F. (2016). Xception: Deep Learning with Depthwise Separable Convolutions. Retrieved from https: / / arxiv.org / abs / 1610.02357 Christos Louizos, KU (2017). Bayesian Compression for Deep Learning. NI PSSebastian Lapuschkin, SW-R. (2019). Unmasking Clever Hans predictors and asse ssing what machines really learn. Nature Communications. Tao, KC (2018). Once for All: A Two-Flow Convolutional Neural Network for Vis ual Tracking. IEEE Transactions on Circuits and Systems for Video Techno logy, 3377-3386

Claims

1. A data stream (45) in which a representation of a neural network (10) is encoded, A data stream (45) including serialization parameters (102) indicating an encoding order (104) in which neural network parameters (32) defining neuronal interconnections (22, 24) of said neural network (10) are encoded into said data stream (45).

2. The data stream (45) of claim 1, wherein the neural network parameters (32) are encoded into the data stream (45) using context-adaptive arithmetic coding (600).

3. 3. The data stream (45) of claim 1 or 2, wherein the data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion (200) representing a corresponding neural network layer (210, 30) of the neural network (10), and the serialization parameters (102) indicate the encoding order (104) in which neural network parameters defining the neuron interconnections (22, 24) of the neural network within a given neural network layer (210, 30) are encoded into the data stream (45).

4. The data stream (45) of any of claims 1 to 3, wherein the serialization parameter (102) is an n-ary parameter indicating the encoding order (104) out of a set (108) of n encoding orders (104).

5. The set (108) of n encoding orders (104) comprises: a first predetermined encoding order (106) in which the predetermined encoding order differs in the order in which the dimensions (34) of the tensors (30) describing the predetermined neural network layers (210, 30) of the neural network (10) are traversed; 1 ), and / or For scalable coding of the neural network, a second predefined coding order (106) is selected, which differs in the number of times (107) the predefined coding order scans a predefined neural network layer (210, 30) of the neural network. 2 ), and / or A third predetermined encoding order (106) in which the predetermined encoding order traverses the neural network layers (210, 30) of the neural network in a different order. 3 ), and / or 、 a fourth predetermined coding order (106) which differs in the order in which the neurons (14, 18, 20) of the neural network layers (210, 30) of the neural network are scanned; 4 ) The data stream (45) of claim 4, comprising:

6. 6. The data stream (45) of claim 1, wherein the serialization parameter (102) indicates a permutation that the encoding order (104) uses to permute neurons (14, 18, 20) of a neural network layer (210, 30) relative to a default order.

7. 7. The data stream (45) of claim 6, wherein the permutation orders the neurons (14, 18, 20) of the neural network layers (210, 30) such that the neural network parameters (32) are monotonically increasing along the coding order (104) or monotonically decreasing along the coding order (104).

8. The permutation is a predefined coding sequence signallable by the serialization parameters (102).

7. The data stream (45) of claim 6, wherein the neurons (14, 18, 20) of the neural network layers (210, 30) are ordered such that, for the permutation indicated by the serialization parameter (102), a bit rate for encoding the neural network parameters (32) into the data stream (45) is minimized.

9. The data stream (45) of any of claims 1 to 8, wherein the neural network parameters (32) include weights and biases.

10. 10. A data stream (45) as claimed in any one of claims 1 to 9, wherein the data stream (45) is structured into individually accessible sub-portions (43, 44, 240), each sub-portion (43, 44, 240) representing a corresponding neural network portion of the neural network (10), and each sub-portion (43, 44, 240) is completely scanned by the coding order (104) before a subsequent sub-portion is scanned by the coding order (104).

11. 11. A data stream (45) according to any of claims 3 to 10, wherein the neural network parameters (32) are coded into the data stream (45) using context-adaptive arithmetic coding (600) and using context initialization at the start of any individually accessible portion (200) or sub-portion (43, 44, 240).

12. 12. A data stream (45) as claimed in any one of claims 3 to 11, wherein the data stream (45) comprises a start code (242) at which each individually accessible portion (200) or sub-portion (43, 44, 240) begins, and / or a pointer (220, 244) to the start of each individually accessible portion or sub-portion, and / or a pointer to a data stream length (246) of each individually accessible portion or sub-portion for skipping said each individually accessible portion or sub-portion when parsing the data stream (45).

13. 13. The data stream (45) of claim 1, further comprising numerical computation expression parameters (120) indicating a numerical representation and bit size for expressing the neural network parameters (32) when the neural network (10) is used for inference.

14. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) includes a numerical computation expression parameter (120) indicating a numerical expression and a bit size for expressing neural network parameters (32) of the neural network that are encoded in the data stream when the neural network (10) is used for inference.

15. 15. A data stream (45) according to any one of claims 1 to 14, wherein the data stream (45) is structured into individually accessible sub-portions (43, 44, 240), each representing a corresponding neural network portion of the neural network, such that each individually accessible sub-portion is completely scanned by the coding order (104) before a subsequent individually accessible sub-portion is scanned by the coding order (104), and the data stream (45) includes, for a given individually accessible sub-portion, a type parameter indicating a parameter type of the neural network parameters (32) encoded in the given individually accessible sub-portion.

16. 16. The data stream (45) of claim 15, wherein the type parameter distinguishes at least between a neural network weight and a neural network bias.

17. 17. The data stream (45) of claim 1, wherein the data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, and the data stream (45) further includes, for a given neural network layer, a neural network layer type parameter (130) indicating a neural network layer type of the given neural network layer of the neural network.

18. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, the data stream (45) further including, for a given neural network layer, a neural network layer type parameter (130) indicating a neural network layer type of the given neural network layer of the neural network.

19. 19. The data stream (45) of claim 17 or 18, wherein the neural network layer type parameter (130) distinguishes at least between fully connected layer types and convolutional layer types.

20. 20. A data stream (45) as claimed in any preceding claim, wherein the data stream (45) is structured into individually accessible portions (200), each individually accessible portion representing a corresponding neural network portion of the neural network, and the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), a pointer (220, 244) to the beginning of the respective individually accessible portion.

21. A data stream (45) in which a representation of the neural network (10) is encoded, 1. A data stream (45) comprising: a data stream (45) structured into individually accessible portions (200), each individually accessible portion representing a corresponding neural network portion of the neural network; and wherein the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), a pointer (220, 244) pointing to the beginning of the respective predetermined individually accessible portion.

22. Each individually accessible part is a corresponding neural network layer (210) of said neural network; or 22. A data stream (45) according to claim 20 or 21, representing a neural network portion (43, 44, 240) of a neural network layer (210) of said neural network.

23. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, the data stream (45) is further structured within a given portion into individually accessible sub-portions (43, 44, 240), each sub-portion (43, 44, 240) representing a corresponding neural network portion of the respective neural network layer (210, 30) of the neural network, and the data stream (45) is structured into one or more individually accessible portions (43, 44, 240), each individually accessible portion representing a corresponding neural network portion of the respective neural network layer (210, 30) of the neural network, and for each of the one or more predetermined individually accessible sub-portions (43, 44, 240): a start code (242) at which each of said predetermined individually accessible sub-portions begins; and / or a pointer (244) to the beginning of each of said predetermined individually accessible sub-portions; and / or 23. The data stream (45) of claim 1, further comprising a data stream length parameter indicating a data stream length (246) of each of the predetermined individually accessible sub-portions for skipping each of the predetermined individually accessible sub-portions when parsing the data stream (45).

24. 24. The data stream (45) of claim 23, wherein the representation of the neural network is encoded using context-adaptive arithmetic coding (600) and using context initialization at the beginning of each individually accessible portion and each individually accessible sub-portion.

25. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, the data stream (45) is further structured within a given portion into individually accessible sub-portions (43, 44, 240), each sub-portion (43, 44, 240) representing a corresponding neural network portion of the respective neural network layer (210, 30) of the neural network, and the data stream (45) is structured such that, for each of the one or more predetermined individually accessible sub-portions (43, 44, 240): a start code (242) at which each of said predetermined individually accessible sub-portions begins; and / or a pointer (244) to the beginning of each of said predetermined individually accessible sub-portions; and / or a data stream (45) including a data stream length parameter indicating a data stream length (246) of each of the predetermined individually accessible sub-portions for skipping each of the predetermined individually accessible sub-portions when parsing the data stream (45).

26. 26. The data stream (45) of claim 25, wherein the representation of the neural network is encoded using context-adaptive arithmetic coding (600) and using context initialization at the beginning of each individually accessible portion and each individually accessible sub-portion.

27. The data stream (45) is structured into individually accessible portions (200), each individually accessible portion representing a corresponding neural network portion of the neural network, and the data stream (45) comprises, for each of one or more predetermined individually accessible portions (200), 27. A data stream (45) as claimed in any preceding claim, comprising processing option parameters (250) indicating one or more processing options (252) that must or may optionally be used when using (10) for inference.

28. the processing option parameters (250) indicate the one or more available processing options (252) from a set of predefined processing options (252); The predetermined processing options (252) are: the parallel processing capabilities of each of said predetermined individually accessible portions; and / or A sample-by-sample parallel processing capability of each of said predetermined individually accessible portions (252 2 ), and / or The per-channel parallel processing capability of each of said predetermined individually accessible portions (252 1 ), and / or a classification category-based parallel processing capability of each of the predetermined individually accessible portions; and / or 28. The data stream (45) of claim 27, comprising a dependency of the neural network portion represented by the respective given individually accessible portion on computation results obtained from another individually accessible portion of the data stream (45) related to the same neural network portion but belonging to another version of the neural network version (330) encoded in a layer manner in the data stream (45).

29. A data stream (45) in which a representation of a neural network (10) is encoded, The data stream (45) is structured into individually accessible portions (200), each individually accessible portion representing a corresponding neural network portion of the neural network, and the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), processing option parameters (250) indicating one or more processing options (252) that must be or can optionally be used when using the neural network (10) for inference.

30. the processing option parameters (250) indicate the one or more available processing options (252) from a set of predefined processing options (252); The predetermined processing options (252) are: the parallel processing capabilities of each of said predetermined individually accessible portions; and / or A sample-by-sample parallel processing capability of each of said predetermined individually accessible portions (252 2 ), and / or The per-channel parallel processing capability of each of said predetermined individually accessible portions (252 1 ), and / or a classification category-based parallel processing capability of each of the predetermined individually accessible portions; and / or 30. The data stream (45) of claim 29, comprising a dependency of the neural network portion represented by the respective given individually accessible portion on computation results obtained from another individually accessible portion of the data stream (45) related to the same neural network portion but belonging to another version of the neural network version (330) encoded in a layer manner in the data stream (45).

31. Neural network parameters (32) representing the neural network are encoded; the neural network parameters (32) are quantized (260) into quantization indexes (32'') and encoded into the data stream (45); the neural network parameters (32) are encoded in the data stream (45) such that neural network parameters (32) in different neural network portions of the neural network are quantized (260) differently, the data stream (45) indicating, for each of the neural network portions, a reconstruction rule (270) for dequantizing the neural network parameters (32) associated with the respective neural network portion. A data stream (45) according to any of the preceding claims.

32. a data stream (45) having encoded therein neural network parameters (32) representative of a neural network, the neural network parameters (32) are quantized (260) into quantization indexes (32'') and encoded into the data stream (45); The neural network parameters (32) are encoded into the data stream (45) such that neural network parameters (32) in different neural network portions of the neural network are quantized (260) differently, the data stream (45) indicating, for each of the neural network portions, a reconstruction rule (270) for dequantizing the neural network parameters (32) associated with the respective neural network portion.

33. The data stream (45) according to claim 31 or 32, wherein the neural network portion comprises a neural network layer (210, 30) of the neural network and / or a layer portion into which a given neural network layer (210, 30) of the neural network is subdivided.

34. The data stream (45) is coupled to a second reconstruction rule (270) for dequantizing the neural network parameters (32) associated with a second neural network portion. 2 , 270a 2 a first reconstruction rule (270) for dequantizing the neural network parameters (32) associated with the first neural network portion that has been delta-coded with respect to 1 , 270a 1 34. A data stream (45) according to any of claims 31 to 33, comprising:

35. The data stream (45) is based on the first reconstruction rule (270 1 , 270a 1 a first index value for indicating the second reconstruction rule (270) 2 , 270a 2 a second exponent value to indicate The first reconstruction rule (270 1 , 270a 1 ) is defined by a first quantization step size defined by a predetermined base exponent and a first exponent defined by the first exponent value; and The second reconstruction rule (270 2 , 270a 2 35. The data stream (45) of claim 34, wherein a first quantization step size is defined by an exponent of the predetermined base and a second exponent is defined by a sum of the first and second exponent values.

36. 36. The data stream (45) of claim 35, wherein said data stream (45) further exhibits said predetermined basis.

37. The data stream (45) is coupled to a first reconstruction rule (270) for dequantizing neural network parameters (32) associated with a first neural network portion. 1 , 270a 1 ) for the second neural network portion. A second reconstruction rule (270) for dequantizing the associated neural network parameters (32). 2 , 270a 2 and a second exponent value to indicate The first reconstruction rule (270 1 , 270a 1 ) is defined by a first quantization step size defined by a predetermined base exponent and a first exponent defined by the sum of the first exponent value and a predetermined exponent value; and The second reconstruction rule (270 2 , 270a 2 35. A data stream (45) according to any of claims 31 to 34, wherein a first quantization step size is defined by an exponent of the predetermined basis and a second exponent is defined by a sum of the second exponent value and the predetermined exponent value.

38. 38. The data stream (45) of claim 37, wherein said data stream (45) further exhibits said predetermined basis.

39. 39. The data stream (45) of claim 38, wherein said data stream (45) represents said predetermined basis within a neural network.

40. 40. The data stream (45) of any of claims 37 to 39, wherein said data stream (45) further indicates said predetermined exponent value.

41. 41. The data stream (45) of claim 40, wherein the data stream (45) exhibits the predetermined index value within a neural network layer (210, 30).

42. 42. The data stream (45) of claim 40 or 41, wherein the data stream (45) further indicates the predetermined basis, the data stream (45) indicating the predetermined exponent values ​​in a finer range than the range of the predetermined basis indicated by the data stream (45).

43. 43. The data stream (45) of any of claims 35 to 42, wherein the predetermined base is encoded in a non-integer format and the first and second exponent values ​​are encoded in an integer format.

44. The data stream (45) is based on the first reconstruction rule (270 1 , 270a 1 a first parameter set (264) defining a first quantization index-reconstruction level mapping (265) for indicating said second reconstruction rule (270); 2 , 270a 2 a second set of parameters (264) defining a second quantization index to reconstruction level mapping (265) for indicating The first reconstruction rule (270 1 , 270a 1 ) is defined by the first quantization index-to-reconstruction level mapping (265), and The second reconstruction rule (270 2 , 270a 2 44. A data stream (45) according to any of claims 34 to 43, wherein the first quantization index - reconstruction level mapping (265) is defined by extending the first quantization index - reconstruction level mapping (265) in a predetermined manner by the second quantization index - reconstruction level mapping (265).

45. The data stream (45) is based on the first reconstruction rule (270 1 , 270a 1 ) to indicate the second reconstruction rule (270), 2 , 270 a 2 ), the second quantization index-to-reconstruction level mapping (265) is a second set of parameters (264) defining The first reconstruction rule (270 1 , 270a 1 ) converts a predetermined quantization index-reconstruction level mapping (265) into the first quantization index-reconstruction level mapping 265, by expanding it in a predetermined manner, The second reconstruction rule (270 2 , 270a 2 45. A data stream (45) according to any of claims 34 to 44, wherein the first quantization index - reconstruction level mapping (265) is defined in the predetermined manner by extending the first quantization index - reconstruction level mapping (265) by the second quantization index - reconstruction level mapping (265).

46. The data stream (45) of claim 45, further indicating the predetermined quantization index to reconstruction level mapping (265).

47. 47. The data stream (45) of claim 46, wherein the data stream (45) exhibits the predetermined quantization index-reconstruction level mapping (265) at a neural network extent or at a neural network layer extent.

48. According to the above-mentioned predetermined method, a mapping of each index value (32'') to a first reconstruction level according to the extended quantization index-reconstruction level mapping is replaced by a mapping of each index value (32'') to a second reconstruction level according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, if present; and / or According to the extended quantization index-reconstruction level mapping, a reconstruction level to which each of the index values ​​(32'') should be mapped is not defined, and for any of the index values ​​(32'') that are mapped to a corresponding reconstruction level according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, a mapping from the respective index value (32'') to the corresponding reconstruction level is adopted; and / or A data stream (45) according to any one of claims 44 to 47, wherein a reconstruction level to which each of the index values ​​(32'') should be mapped according to the quantization index to reconstruction level mapping that extends the extended quantization index to reconstruction level mapping is not defined, and for any index value (32'') that is mapped to a corresponding reconstruction level according to the extended quantization index to reconstruction level mapping, a mapping from the respective index value (32'') to the corresponding reconstruction level is adopted.

49. The data stream (45) is further adapted to represent the reconstruction rules (270) of a given neural network portion by: a quantization step size parameter (262) indicating the quantization step size (263); a parameter set (264) defining a quantization index to reconstruction level mapping (265); The reconstruction rule (270) for the given neural network portion comprises: a quantization step size (263) for the quantization index (32'') within a given index interval (268); and A data stream (45) according to any of claims 31 to 48, defined by a quantization index to reconstruction level mapping (265) for quantization indexes (32'') outside said predetermined index interval (268).

50. A data stream (45) in which neural network parameters (32) representing a neural network are encoded, said neural network parameters (32) being quantized (260) into quantization indexes (32'') and encoded into said data stream (45); The data stream (45) is further processed to indicate a reconstruction rule (270) for dequantizing (280) the neural network parameters (32): a quantization step size parameter (262) indicating the quantization step size (263); a parameter set (264) defining a quantization index to reconstruction level mapping (265); The reconstruction rule (270) for the given neural network portion comprises: a quantization step size (263) for the quantization indexes (32'') within a given index interval (268); and The data stream (45) is defined by a quantization index-to-reconstruction level mapping (265) for quantization indexes (32'') outside said predetermined index interval (268).

51. 51. A data stream (45) according to claim 49 or 50, wherein the predetermined index interval (268) comprises zero.

52. 52. The data stream (45) of claim 51, wherein the predetermined index interval (268) extends up to a predetermined magnitude threshold, and a quantization index (32'') exceeding the predetermined magnitude threshold represents an escape code indicating that the quantization index to reconstruction level mapping (265) is to be used for inverse quantization (280).

53. A data stream (45) according to any of claims 49 to 52, wherein the parameter set (264) defines the quantization index - reconstruction level mapping (265) by a list of reconstruction levels associated with quantization indexes (32'') outside the predetermined index interval (268).

54. 54. The data stream (45) of any of claims 31 to 53, wherein the neural network portion comprises one or more sub-portions of a neural network layer (210, 30) of the neural network and / or one or more neural network layers of the neural network.

55. 55. A data stream (45) as claimed in any one of claims 31 to 54, wherein the data stream (45) is structured into individually accessible portions (200), each individually accessible portion having the neural network parameters (32) for a corresponding neural network portion encoded therein.

56. 56. The data stream (45) of claim 55, wherein the individually accessible portions (200) are encoded using context-adaptive arithmetic coding (600) and using context initialization at the start of each individually accessible portion.

57. The data stream (45) includes, for each individually accessible portion: a start code (242) at which each of said individually accessible portions begins; and / or a pointer (220, 244) to the beginning of each of said individually accessible portions; and / or 57. The data stream (45) of claim 55 or 56, further comprising: a data stream length parameter indicating a data stream length (246) of each said individually accessible portion to skip over each said individually accessible portion when parsing said data stream (45).

58. The data stream (45) comprises, for each of the neural network portions, the reconstruction rule (270) for dequantizing (280) the neural network parameters (32) associated with the respective neural network portion; a main header portion (47) of said data stream (45) relating to said entire neural network; a neural network layer (210, 30) associated with a header portion (110) of the data stream (45) associated with the neural network layer (210) of which each said neural network portion is a part; or 58. The data stream (45) of any of claims 55 to 57, indicated in a neural network portion specific header portion of the data stream (45) associated with the respective neural network portion.

59. A data stream (45) in which a representation of the neural network (10) is encoded, 59. A data stream (45) as described in any one of claims 1 to 58, wherein the data stream (45) is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), an identification parameter (310) for identifying the respective predetermined individually accessible portion.

60. A data stream (45) in which a representation of the neural network (10) is encoded, 1. A data stream (45) comprising: a neural network portion (200) that is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network; and wherein the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), an identification parameter (310) for identifying the respective predetermined individually accessible portion.

61. 61. A data stream (45) according to claim 59 or 60, wherein said identification parameter (310) is associated with said respective predetermined individually accessible portion via a hash function or an error detection or correction code.

62. 62. A data stream (45) according to any of claims 59 to 61, further comprising a high level identification parameter (310) for identifying a set of two or more predetermined individually accessible portions.

63. 63. A data stream (45) as described in claim 62, wherein the higher level identification parameter (310) is related to the identification parameters (310) of the two or more predetermined individually accessible portions via a hash function or an error detection or correction code.

64. 64. A data stream (45) according to any of claims 59 to 63, wherein the individually accessible portions (200) are encoded using context-adaptive arithmetic coding (600) with context initialization at the start of each individually accessible portion.

65. The data stream (45) comprises, for each individually accessible portion: a start code (242) at which each of said individually accessible portions begins; and / or a pointer (220, 244) to the beginning of each of said individually accessible portions; and / or 65. A data stream (45) as claimed in any one of claims 59 to 64, comprising a data stream length parameter indicating a data stream length (246) of each said individually accessible portion for skipping each said individually accessible portion when parsing the data stream.

66. 66. A data stream (45) according to any of claims 59 to 65, wherein the neural network portion comprises one or more sub-portions of a neural network layer (210, 30) of the neural network and / or one or more neural network layers of the neural network.

67. the data stream (45) is structured such that a representation of a neural network (10) is coded in a layered manner such that different versions (330) of the neural network are coded into the data stream (45), the data stream (45) being structured into one or more separately accessible portions (200), each portion being associated with a corresponding version (330) of the neural network; The data stream (45) is fed to a first version (330) of the neural network. 2 )but, A second version of the neural network (330) encoded in a second portion. 1 ) and / or The first version of the neural network (330 2 ) based on Each of the second versions of the neural network (330) encoded in a second portion for performing the 1 ) to the execution of the corresponding neural network portion (334) In the form of one or more compensation neural network portions (332) implemented as in the form of a compensation neural network portion, in which the outputs of the respective compensation neural network portion (332) and the corresponding neural network portion (334) are summed; A data stream (45) according to any of claims 1 to 66, encoded into a first portion.

68. A data stream (45) in which a representation of a neural network (10) is coded in a layered manner, the different versions (330) of the neural network are encoded into a data stream (45), the data stream (45) being structured into one or more separately accessible portions (200), each portion being associated with a corresponding version of the neural network; The data stream (45) is fed to a first version (330) of the neural network. 2 )but, A second version of the neural network (330) encoded in a second portion. 1 ) and / or The first version of the neural network (330 2 ) based on Each of the second versions of the neural network (330) encoded in a second portion for performing the 1 ) to the execution of the corresponding neural network portion (334) In the form of one or more compensation neural network portions (332) implemented as in the form of a compensation neural network portion (332) in which the outputs of the respective compensation neural network portion (332) and the corresponding neural network portion (334) are summed; A data stream (45) is encoded into a first portion.

69. The data stream (45) is fed to the first version (330) of the neural network. 2 )but, Differences in weights and / or biases, and / or Additional neurons (14, 18, 20) or neuronal interconnections (22, 24) The second version of the neural network (330) encoded in the second portion at 1 68. The method of claim 67, wherein the first portion is delta coded relative to the first portion. Or a data stream (45) according to claim 68.

70. 70. A data stream (45) according to any of claims 67 to 69, wherein the individually accessible portions (200) are encoded using context-adaptive arithmetic coding (600) and using context initialization at the start of each individually accessible portion.

71. The data stream (45) includes, for each individually accessible portion: a start code (242) at which each of said individually accessible portions begins; and / or a pointer (220, 244) to the beginning of each of said individually accessible portions; and / or 71. A data stream (45) as claimed in any one of claims 67 to 70, comprising a data stream length parameter indicating a data stream length (246) of each said individually accessible portion in order to skip each said individually accessible portion when parsing said data stream (45).

72. 72. A data stream (45) as described in any one of claims 67 to 71, wherein the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), an identification parameter (310) for identifying the respective predetermined individually accessible portion.

73. A data stream (45) in which a representation of the neural network (10) is encoded, 73. A data stream (45) as claimed in any preceding claim, wherein the data stream (45) is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), supplemental data (350) for supplementing the representation of the neural network.

74. A data stream (45) in which a representation of the neural network (10) is encoded, A data stream (45) structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, the data stream (45) including, for each of one or more predetermined individually accessible portions (200), supplemental data (350) for supplementing the representation of the neural network.

75. 75. The data stream (45) of claim 73 or 74, wherein the data stream (45) indicates that the supplemental data (350) is not essential to the neural network based inference.

76. The data stream (45) is encoded with supplementary data (350) for supplementing the representation of the neural network for the one or more predetermined individually accessible portions (200) in another individually accessible portion (200), such that the data stream (45) is 76. A data stream (45) as claimed in any one of claims 73 to 75, wherein each said predetermined individually accessible portion includes a corresponding another predetermined individually accessible portion associated with a corresponding said neural network portion.

77. 77. The data stream (45) of any of claims 73 to 76, wherein the neural network portions include neural network layers (210, 30) of the neural network and / or layer portions into which certain neural network layers of the neural network are subdivided.

78. 78. A data stream (45) according to any of claims 73 to 77, wherein the individually accessible portions (200) are encoded using context-adaptive arithmetic coding (600) and using context initialization at the start of each individually accessible portion.

79. The data stream (45) comprises, for each individually accessible portion: a start code (242) at which each of said individually accessible portions begins; and / or a pointer (220, 244) indicating the beginning of each of said individually accessible portions; and / or 80. A data stream (45) as claimed in any one of claims 73 to 78, comprising a data stream length parameter indicating a data stream length (246) of each said individually accessible portion, for skipping each said individually accessible portion when parsing said data stream (45).

80. The supplementary data (350) may include: the relevance scores of the neural network parameters (32), and / or A data stream (45) according to any of claims 73 to 79, related to perturbation robustness of neural network parameters (32).

81. A data stream (45) in which a representation of the neural network (10) is encoded, 81. A data stream (45) as described in any preceding claim, wherein the data stream (45) comprises hierarchical control data (400) structured into a sequence (410) of control data portions (420), the control data portions (420) providing information about the neural network with increasing detail along the sequence of the control data portions (420).

82. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) includes hierarchical control data (400) structured into a sequence (410) of control data portions (420), the control data portions (420) providing information of the neural network with increasing detail along the sequence of control data portions (420).

83. 83. The data stream (45) of claim 81 or 82, wherein at least some of the control data portions (420) provide partially redundant information regarding the neural network.

84. 82. The method of claim 81, wherein a first control data portion provides the information about the neural network by indicating a default neural network type with default settings, and a second control data portion includes parameters indicative of each of the default settings. A data stream (45) according to any of claims 1 to 83.

85. An apparatus for encoding a representation of a neural network (10) into a data stream (45), comprising: The apparatus is configured to provide the data stream (45) with serialization parameters (102) indicating an encoding order (104) in which neural network parameters (32) defining neuronal interconnections (22, 24) of the neural network are encoded into the data stream (45).

86. 86. The apparatus of claim 85, wherein the apparatus is configured to encode the neural network parameters (32) into the data stream (45) using context-adaptive arithmetic coding.

87. The apparatus comprises: structuring said data stream (45) into one or more separately accessible portions (200), each separately accessible portion representing a corresponding neural network layer (210, 30) of said neural network; and 87. The apparatus of claim 85 or 86, configured to encode, in the data stream (45), neural network parameters defining neuronal interconnections (22, 24) of the neural network in a given neural network layer according to the encoding order (104) indicated by the serialization parameter (102).

88. Apparatus according to any of claims 85 to 87, wherein the serialization parameter (102) is an n-ary parameter indicating the encoding order (104) among a set (108) of n encoding orders (104).

89. The set (108) of n encoding orders (104) comprises: a first predetermined encoding order (106) in which the predetermined encoding order differs in the order in which the dimensions (34) of the tensors (30) describing the predetermined neural network layers (210, 30) of the neural network (10) are traversed; 1 ), and / or For scalable coding of the neural network, a second predefined coding order (106) is selected, which differs in the number of times (107) the predefined coding order scans a predefined neural network layer of the neural network. 2 ), and / or a third predetermined coding order (106) in which the predetermined coding order traverses the neural network layers of the neural network in a different order; 3 ), and / or a fourth predetermined coding order (106) which differs in the order in which the neurons (14, 18, 20) of the neural network layers (210, 30) of the neural network are scanned; 4 ) 89. The apparatus of claim 88, comprising:

90. 90. The apparatus of claim 85, wherein the serialization parameter (102) indicates a permutation that the encoding order (104) uses to permute neurons (14, 18, 20) of a neural network layer (210, 30) relative to a default order.

91. 91. The apparatus of claim 90, wherein the permutation orders the neurons (14, 18, 20) of the neural network layer (210, 30) such that the neural network parameters (32) are monotonically increasing along the encoding order (104) or monotonically decreasing along the encoding order (104).

92. The permutation is a permutation of the neural network that is signaled by the serialization parameter (102) among a predetermined encoding order that can be signaled by the serialization parameter (102).

91. The apparatus of claim 90, further comprising: ordering the neurons (14, 18, 20) of the neural network layers (210, 30) such that a bit rate for encoding network parameters (32) into the data stream (45) is minimized.

93. 93. Apparatus according to any of claims 85 to 92, wherein the neural network parameters (32) include weights and biases.

94. 94. The apparatus of claim 85, wherein the data stream (45) is structured into individually accessible sub-portions (43, 44, 240), each sub-portion (43, 44, 240) representing a corresponding neural network portion of the neural network, and wherein each sub-portion (43, 44, 240) is configured to be completely scanned by the coding order (104) before a subsequent sub-portion is scanned by the coding order (104).

95. 95. An apparatus according to any of claims 87 to 94, wherein the neural network parameters (32) are coded into the data stream using context-adaptive arithmetic coding and using context initialization at the start of any individually accessible portion (200) or sub-portion (43, 44, 240).

96. The apparatus, 96. An apparatus as claimed in any one of claims 87 to 95, configured to encode into the data stream a start code (242) at which each individually accessible portion (200) or sub-portion (43, 44, 240) begins and / or a pointer (220, 244) to the start of each individually accessible portion or sub-portion and / or a data stream length (246) pointer for each individually accessible portion or sub-portion to skip over said each individually accessible portion or sub-portion when parsing the data stream.

97. 97. The apparatus of any of claims 85 to 96, wherein the apparatus is configured to encode into the data stream numerical computation representation parameters (120) indicating numerical representations and bit sizes for representing the neural network parameters (32) when the neural network (10) is used for inference.

98. An apparatus for encoding a representation of a neural network (10) into a data stream (45), comprising: The apparatus is configured to provide, to the data stream (45), numerical computation representation parameters (120) indicating numerical representations and bit sizes for representing neural network parameters (32) of the neural network encoded in the data stream when the neural network (10) is used for inference.

99. 99. An apparatus according to any of claims 85 to 98, wherein the apparatus is configured to structure the data stream (45) into individually accessible sub-portions (43, 44, 240), each individually accessible sub-portion representing a corresponding neural network portion of the neural network, such that each individually accessible sub-portion is completely scanned by the coding order (104) before a subsequent individually accessible sub-portion is scanned by the coding order (104), and the apparatus is configured to encode in the data stream (45), for a given individually accessible sub-portion, the neural network parameters and a type parameter indicating a parameter type of the neural network parameters encoded in the given individually accessible sub-portion.

100. 100. The apparatus of claim 99, wherein the type parameters distinguish at least between a weight of a neural network and a bias of a neural network.

101. the apparatus is configured to structure the data stream (45) into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network; 101. The apparatus of any of claims 85 to 100, configured to encode in the data stream (45) for a given neural network layer a neural network layer type parameter (130) indicating the given neural network layer type of the neural network.

102. An apparatus for encoding a representation of a neural network (10) into a data stream (45), comprising:

1. An apparatus, comprising: the data stream (45) structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network; and the apparatus configured to provide, for a given neural network layer, in the data stream (45), a neural network layer type parameter (130) indicating a neural network layer type of the given neural network layer of the neural network.

103. 103. The apparatus of claim 101 or 102, wherein the neural network layer type parameter (130) distinguishes at least between a fully connected layer type and a convolutional layer type.

104. the apparatus is configured to structure the data stream (45) into separately accessible portions (200), each separately accessible portion representing a corresponding neural network portion of the neural network; 104. An apparatus as claimed in any of claims 85 to 103, configured to encode in the data stream (45), for each of one or more predetermined individually accessible portions, a pointer (220, 244) to the start of the respective individually accessible portion.

105. An apparatus for encoding a representation of a neural network (10) into a data stream (45), comprising:

1. An apparatus, comprising: the data stream (45) structured into one or more individually accessible portions (200), each portion representing a corresponding neural network layer (210, 30) of the neural network, the apparatus being configured to provide, for each of one or more predetermined individually accessible portions, a pointer (220, 244) to the data stream (45) that points to the beginning of the respective predetermined individually accessible portion.

106. Each individually accessible part is a corresponding neural network layer (210) of said neural network; or 106. Apparatus according to claim 104 or 105, representing a neural network portion (43, 44, 240) of a neural network layer (210) of the neural network.

107. The apparatus is configured to encode a representation of a neural network (10) into the data stream (45), the data stream (45) being structured into one or more individually accessible portions (200), each of the individually accessible portions being a portion representing a corresponding neural network layer (210, 30) of the neural network, the data stream (45) being further structured within a given portion into individually accessible sub-portions (43, 44, 240) each representing a corresponding neural network portion of the respective neural network layer of the neural network, and the apparatus transmitting, to the data stream (45), for each of one or more predetermined individually accessible sub-portions (43, 44, 240): a start code (242) at which each of said predetermined individually accessible sub-portions begins; and / or a pointer (244) to the beginning of each of said predetermined individually accessible sub-portions; and / or 107. An apparatus as claimed in any one of claims 85 to 106, configured to provide a data stream length parameter indicating a data stream length (246) of each of the predetermined individually accessible sub-portions for skipping over each of the predetermined individually accessible sub-portions when parsing the data stream.

108. 108. The apparatus of claim 107, wherein the apparatus is configured to encode the representation of the neural network in the data stream (45) at the beginning of each individually accessible portion and each individually accessible sub-portion using context-adaptive arithmetic coding and using context initialization.

109. An apparatus for encoding a representation of a neural network (10) into a data stream (45), comprising: The data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, the data stream (45) is further structured within a given portion into individually accessible sub-portions (43, 44, 240), each sub-portion (43, 44, 240) representing a corresponding neural network portion of the respective neural network layer of the neural network, and the apparatus transmits to the data stream (45) for each of the one or more predetermined individually accessible sub-portions (43, 44, 240): a start code (242) at which each of said predetermined individually accessible sub-portions begins; and / or a pointer (244) to the beginning of each of said predetermined individually accessible sub-portions; and / or and providing a data stream length parameter indicating a data stream length (246) of each of the predetermined individually accessible sub-portions for skipping over each of the predetermined individually accessible sub-portions when parsing the data stream (45).

110. 110. The apparatus of claim 109, wherein the apparatus is configured to encode the representation of the neural network in the data stream (45) at the beginning of each individually accessible portion and each individually accessible sub-portion using context-adaptive arithmetic coding and using context initialization.

111. The apparatus is configured to encode a representation of a neural network (10) into a data stream, the data stream (45) being structured into individually accessible portions (200), each individually accessible portion representing a corresponding neural network portion of the neural network, and the apparatus encodes the data stream (45) with, for each of one or more predetermined individually accessible portions, 111. An apparatus as claimed in any one of claims 85 to 110, configured to provide processing option parameters (250) indicating one or more processing options (252) that must or may optionally be used when using the network (10) for inference.

112. the processing option parameters (250) indicate the one or more available processing options (252) from a set of predefined processing options (252); The predetermined processing options (252) are: the parallel processing capabilities of each of said predetermined individually accessible portions; and / or A sample-by-sample parallel processing capability of each of said predetermined individually accessible portions (252 2 ), and / or The per-channel parallel processing capability of each of said predetermined individually accessible portions (252 1 ), and / or a classification category-based parallel processing capability of each of the predetermined individually accessible portions; and / or 112. The apparatus of claim 111, comprising a dependency of the neural network portion represented by each of the respective predetermined individually accessible portions on computational results obtained from another individually accessible portion of the data stream (45) that is associated with the same neural network portion but belongs to another version of the neural network version (330) that is coded in a layered manner in the data stream (45).

113. An apparatus for encoding a representation of a neural network (10) into a data stream (45), comprising:

1. An apparatus for encoding a data stream, the data stream (45) being structured into individually accessible portions (200), each individually accessible portion representing a corresponding neural network portion of the neural network, the apparatus being configured to provide the data stream (45) with processing option parameters (250) indicating, for each of one or more predetermined individually accessible portions, one or more processing options (252) that must be or may optionally be used when using the neural network (10) for inference.

114. the processing option parameters (250) indicate the one or more available processing options (252) from a set of predefined processing options (252); The predetermined processing options (252) are: the parallel processing capabilities of each of said predetermined individually accessible portions; and / or A sample-by-sample parallel processing capability of each of said predetermined individually accessible portions (252 2 ), and / or The per-channel parallel processing capability of each of said predetermined individually accessible portions (252 1 ), and / or a classification category-based parallel processing capability of each of the predetermined individually accessible portions; and / or 114. The apparatus of claim 113, including a dependency of the neural network portion represented by each given individually accessible portion on computational results obtained from another individually accessible portion of the data stream (45) that is associated with the same neural network portion but belongs to another version of the neural network version (330) that is coded in a layered manner in the data stream (45).

115. The apparatus is configured to encode neural network parameters (32) representing a neural network into a data stream (45), the neural network parameters (32) being quantized (260) into quantization indexes (32'') and encoded into the data stream (45), and differences in the neural network are encoded into the data stream (45).

115. An apparatus according to any of claims 85 to 114, wherein the neural network parameters (32) are encoded in the data stream (45) such that neural network parameters (32) in neural network portions are differently quantized (260), and the apparatus is configured to provide, for each of the neural network portions, the data stream (45) indicating a reconstruction rule (270) for dequantizing (280) the neural network parameters (32) associated with the respective neural network portion.

116. 1. An apparatus for encoding neural network parameters (32) representing a neural network into a data stream (45), comprising:

1. An apparatus for encoding, in a data stream, said neural network parameters (32) being quantized (260) into quantization indices (32'') and encoded into said data stream (45), said neural network parameters (32) being encoded into said data stream (45) such that neural network parameters (32) in different neural network portions of the neural network are quantized (260) differently, said apparatus being configured to provide, for each of said neural network portions, said data stream (45) indicating a reconstruction rule (270) for dequantizing (280) the neural network parameters (32) associated with the respective neural network portion.

117. 117. The apparatus of claim 115 or 116, wherein the neural network portion comprises a neural network layer (210, 30) of the neural network and / or a layer portion into which a given neural network layer of the neural network is subdivided.

118. The apparatus further comprises: a first reconstruction rule (270) for dequantizing neural network parameters (280) associated with a first neural network portion in the data stream (45); 1 , 270a 1 ) to a second reconstruction rule (270) for dequantizing (280) the neural network parameters (32) associated with the second neural network portion. 2 , 270a 2 118. An apparatus as claimed in any of claims 115 to 117, configured to encode the signal with delta encoding on the signal.

119. The device applies the first reconstruction rule (270) to the data stream (45). 1 , 270a 1 a first index value for indicating the second reconstruction rule (270) 2 , 270a 2 and a second exponent value to indicate The first reconstruction rule (270 1 , 270a 1 ) is defined by a first quantization step size (263) defined by an exponent of a predetermined base and a first exponent defined by said first exponent value; and The second reconstruction rule (270 2 , 270a 2 119. The apparatus of claim 118, wherein the first quantization step size (263) is defined by an exponent of the predetermined basis and a second exponent defined by the sum of the first and second exponent values.

120. 120. The apparatus of claim 119, wherein the data stream further exhibits a predetermined basis.

121. The apparatus further comprises: a first reconstruction rule (270) for dequantizing (280) neural network parameters (32) associated with a first neural network portion in the data stream; 1 , 270a 1 a first exponent value for indicating the first neural network portion; and a second reconstruction rule for dequantizing the second neural network portion. 2 , 270a 2 and a second exponent value to indicate The first reconstruction rule (270 1 , 270a 1 ) is defined by the exponents of a given base a first quantization step size (263) that is determined by the first quantization step size and a first exponent that is defined by the sum of the first exponent value and a predetermined exponent value; The second reconstruction rule (270 2 , 270a 2 119. The apparatus of claim 115, wherein the first quantization step size (263) is defined by an exponent of the predetermined basis and a second exponent defined by a sum of the second exponent value and the predetermined exponent value.

122. 122. The apparatus of claim 121, wherein the data stream further exhibits a predetermined basis.

123. 123. The apparatus of claim 122, wherein the data stream represents the predetermined basis within a neural network.

124. 124. Apparatus according to any of claims 121 to 123, wherein the data stream is further indicative of the predetermined exponent value.

125. 126. The apparatus of claim 125, wherein the data stream indicates the predetermined index value within a neural network layer (210, 30).

126. 126. Apparatus according to claim 124 or 125, wherein the data stream further indicates the predetermined basis, the data stream indicating the predetermined exponent values ​​in a finer range than a range in which the predetermined basis is indicated by the data stream.

127. 127. An apparatus according to any of claims 119 to 126, wherein the apparatus is configured to encode in the data stream the predetermined base in a non-integer format and the first and second exponent values ​​in an integer format.

128. The device includes a first reconstruction rule (270) for the data stream. 1 , 270a 1 a first parameter set (264) defining a first quantization index-reconstruction level mapping (265) and a second reconstruction rule (270) for indicating the first quantization index-reconstruction level mapping (265) 2 , 270 a 2 ), the second quantization index-to-reconstruction level mapping (265) is and a second set of parameters (264) defining the The first reconstruction rule (270 1 , 270a 1 ) is defined by the first quantization index-to-reconstruction level mapping (265), and The second reconstruction rule (270 2 , 270a 2 128. The apparatus according to claim 118, wherein the first quantization index - reconstruction level mapping (265) is defined by extending the first quantization index - reconstruction level mapping (265) in a predetermined manner by the second quantization index - reconstruction level mapping (265).

129. The device includes a first reconstruction rule (270) for the data stream. 1 , 270a 1 a first parameter set (264) defining a first quantization index-reconstruction level mapping (265) and a second reconstruction rule (270) for indicating the first quantization index-reconstruction level mapping (265) 2 , 270 a 2 ), the second quantization index-to-reconstruction level mapping (265) is and a second set of parameters (264) defining the The first reconstruction rule (270 1 , 270a 1 ) is defined by the first quantization index-to-reconstruction level mapping (265), The second reconstruction rule (270 2 , 270a 2 129. The apparatus according to claim 118, wherein the first quantization index - reconstruction level mapping (265) is defined by extending the first quantization index - reconstruction level mapping (265) in a predetermined manner with the second quantization index - reconstruction level mapping (265).

130. 130. The apparatus of claim 129, wherein the data stream further indicates the predetermined quantization index to reconstruction level mapping (265).

131. The apparatus of claim 130, wherein the data stream indicates the predetermined quantization index-to-reconstruction level mapping (265) at a neural network extent or at a neural network layer extent (210, 30).

132. According to the above-mentioned predetermined method, a mapping of each index value (32'') to a first reconstruction level according to the extended quantization index-reconstruction level mapping is replaced by a mapping of each index value (32'') to a second reconstruction level according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, if present; and / or According to the extended quantization index-reconstruction level mapping, a reconstruction level to which each index value (32'') should be mapped is not defined, and for any index value (32'') that is mapped to a corresponding reconstruction level according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, a mapping from the each index value (32'') to the corresponding reconstruction level is adopted; and / or An apparatus according to any one of claims 128 to 131, wherein the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping does not define a reconstruction level to which each of the index values ​​(32'') should be mapped, and for any index value (32'') that is mapped to a corresponding reconstruction level according to the extended quantization index-reconstruction level mapping, a mapping from the respective index value (32'') to the corresponding reconstruction level is adopted.

133. The apparatus further comprises: a quantization step size parameter (262) indicating the quantization step size (263); a parameter set (264) defining a quantization index to reconstruction level mapping (265); The reconstruction rule (270) for the given neural network portion comprises: a quantization step size (263) for the quantization index (32'') within a given index interval (268); and The apparatus of any of claims 115 to 132, wherein the predetermined index interval is defined by a quantization index to reconstruction level mapping (265) for quantization indexes (32'') outside the predetermined index interval (268).

134. 1. An apparatus for encoding neural network parameters (32) representing a neural network into a data stream (45), comprising: the neural network parameters (32) are quantized (260) into quantization indexes (32'') and encoded into the data stream (45); said device providing to said data stream (45) a reconstruction rule (270) for dequantizing (280) said neural network parameters (32); a quantization step size parameter (262) indicating the quantization step size (263); a parameter set (264) defining a quantization index to reconstruction level mapping (265); The reconstruction rule (270) for the given neural network portion comprises: a quantization step size (263) for the quantization indexes (32'') within a given index interval (268); and a quantization index to reconstruction level mapping (265) for quantization indexes (32'') outside said predetermined index interval (268).

135. 135. The apparatus of claim 133 or 134, wherein the predetermined index interval (268) includes zero.

136. 136. The apparatus of claim 135, wherein the predetermined index interval (268) extends up to a predetermined magnitude threshold, and a quantization index (32'') exceeding the predetermined magnitude threshold represents an escape code indicating that the quantization index-to-reconstruction level mapping (265) is to be used for inverse quantization (280).

137. The apparatus of any of claims 133 to 136, wherein the parameter set (264) defines a quantization index - reconstruction level mapping (265) by a list of reconstruction levels associated with quantization indexes (32'') outside the predetermined index interval (268).

138. 138. Apparatus according to any of claims 115 to 137, wherein the neural network portion comprises one or more sub-portions of a neural network layer (210, 30) of the neural network and / or one or more neural network layers of the neural network.

139. 139. An apparatus according to any of claims 115 to 138, wherein the apparatus is configured to structure the data stream (45) into individually accessible portions (200) and to encode the neural network parameters (32) for a corresponding neural network portion into each individually accessible portion.

140. 140. The apparatus of claim 139, wherein the apparatus is configured to encode the individually accessible portions (200) using context-adaptive arithmetic coding, using context initialization at the start of each individually accessible portion, and into a data stream.

141. The apparatus further comprises, for each individually accessible portion of the data stream: a start code (242) at which each of said individually accessible portions begins; and / or a pointer (220, 244) to the beginning of each of said individually accessible portions; and / or 141. The apparatus of claim 139 or 140, configured to encode a data stream length parameter indicating a data stream length (246) of each individually accessible portion for skipping each individually accessible portion when parsing the data stream.

142. The apparatus further comprises: for each of the neural network portions, instructions for a reconstruction rule (270) for dequantizing (280) the neural network parameters (32) associated with the respective neural network portion; A main header portion (47) of the data stream relating to the entire neural network; The neural network layer to which each of the neural network parts belongs ( a neural network layer (210, 30) associated with a header portion (110) of the data stream associated with the in a neural network portion specific header portion of the data stream associated with each said neural network portion, 142. Apparatus according to any of claims 139 to 141, configured for encoding into the data stream.

143. 143. An apparatus according to any one of claims 85 to 142, wherein the apparatus is configured to encode a representation of a neural network (10) into a data stream (45), the data stream (45) being structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the apparatus is configured to provide the data stream (45) with, for each of one or more predetermined individually accessible portions, an identification parameter (310) for identifying the respective predetermined individually accessible portion.

144. An apparatus for encoding a representation of a neural network (10) into a data stream (45), comprising:

1. An apparatus, comprising: the data stream (45) structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, the apparatus being configured to, for each of one or more predetermined individually accessible portions, provide the data stream (45) with an identification parameter (310) for identifying the respective predetermined individually accessible portion.

145. 145. Apparatus according to claim 143 or 144, wherein the identification parameter (310) is related to the respective predetermined individually accessible portion via a hash function or an error detection or correction code.

146. 146. An apparatus according to any of claims 143 to 145, wherein the apparatus is configured to encode in the data stream (45) a high-level identification parameter (310) for identifying a set of two or more predetermined individually accessible portions.

147. 147. The apparatus of claim 146, wherein the higher level identification parameters (310) are related to the identification parameters (310) of the one or more predetermined individually accessible portions via a hash function or an error detection or correction code.

148. 148. An apparatus according to any of claims 143 to 147, wherein the apparatus is configured to encode the individually accessible portions (200) into the data stream using context-adaptive arithmetic coding and using context initialization at the start of each individually accessible portion.

149. The device, for each individually accessible portion, adds to the data stream: a start code (242) at which each of said individually accessible portions begins; and / or a pointer (220, 244) to the beginning of each of said individually accessible portions; and / or 149. An apparatus according to any of claims 143 to 148, configured to encode a data stream length parameter indicating a data stream length (246) of each of the individually accessible portions for skipping each of the individually accessible portions when parsing the data stream.

150. 150. Apparatus according to any of claims 143 to 149, wherein the neural network portion comprises a neural network layer (210, 30) of the neural network and / or one or more sub-portions of one or more neural network layers (210, 30) of the neural network.

151. the apparatus is configured to encode a representation of a neural network (10) into a data stream (45) in a layered manner, different versions (330) of the neural network being encoded into the data stream (45), and the data stream (45) being structured into one or more individually accessible portions (200), each portion being associated with a corresponding version of the neural network; The apparatus comprises: A first version of the neural network (330 2 )of, A second version of the neural network (330) encoded in a second portion. 1 ) and / or The first version of the neural network (330 2 ) based on and a second version of the neural network (330), each encoded in a second portion, for performing 1 ) in addition to the execution of the corresponding neural network portion (334) of in the form of one or more compensation neural network portions (332) implemented by the outputs of the respective compensation neural network portion (332) and the corresponding neural network portion (334) are summed, in the form of a compensation neural network portion; 151. Apparatus according to any of claims 85 to 150 configured for encoding into a first portion.

152. An apparatus for encoding a representation of a neural network (10) into a data stream (45) in a layered manner, comprising: different versions (330) of the neural network are encoded into the data stream (45) and the data stream (45) is structured into one or more separately accessible portions (200), each portion being associated with a corresponding version of the neural network; The apparatus comprises: A first version of the neural network (330 2 )of, A second version of the neural network (330) encoded in a second portion. 1 ) by delta coding, and / or The first version of the neural network (330 2 ) based on and a second version of the neural network (330), each encoded in a second portion, for performing 1 ) in addition to the execution of the corresponding neural network portion (334) of in the form of one or more compensation neural network portions (332) implemented by in the form of a compensation neural network portion (332) in which the outputs of the respective compensation neural network portion (332) and the corresponding neural network portion (334) are summed; The apparatus is configured to encode the first portion.

153. The device further comprises: a second version of the neural network (330) in a second portion of the data stream; 1 ) ; and The apparatus comprises: The first version of the neural network (330 2 )of, Differences in weights and / or biases, and / or Additional neurons (14, 18, 20) or neuronal interconnections (22, 24) the second version of the neural network (330) encoded in the second part by 1 ) and encoding it into a first portion of the data stream.

153. An apparatus as described in claim 151 or 152, configured so as to

154. 154. An apparatus according to any of claims 151 to 153, wherein the apparatus is configured to encode the individually accessible portions (200) into the data stream using context-adaptive arithmetic coding (600) and using context initialization at the start of each individually accessible portion.

155. The apparatus, for each individually accessible portion, adds to the data stream: a start code (242) at which each of said individually accessible portions begins; and / or a pointer (220, 244) to the beginning of each of said individually accessible portions; and / or 155. An apparatus as claimed in any one of claims 151 to 154, configured to encode a data stream length parameter indicating a data stream length (246) of each of the individually accessible portions for skipping each of the individually accessible portions when parsing the data stream.

156. 156. An apparatus according to any of claims 151 to 155, wherein the apparatus is configured to encode, for each of one or more predetermined individually accessible portions (200), an identification parameter (310) into the data stream for identifying the respective predetermined individually accessible portion.

157. 157. An apparatus according to any one of claims 85 to 156, wherein the apparatus is configured to encode a representation of a neural network (10) into a data stream (45), the data stream (45) being structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the apparatus is configured to provide to the data stream (45), for each of one or more predetermined individually accessible portions (200), supplementary data (350) for supplementing the representation of the neural network.

158. An apparatus for encoding a representation of a neural network (10) into a data stream (45), comprising:

1. An apparatus, comprising: the data stream (45) structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network; and the apparatus configured to provide to the data stream (45) supplemental data (350) for supplementing the representation of the neural network for each of one or more predetermined individually accessible portions (200).

159. 159. The apparatus of claim 157 or 158, wherein the data stream (45) indicates that the supplemental data (350) is not essential to the neural network based inference.

160. The apparatus is configured to encode the supplemental data (350) for supplementing the representation of the neural network for the one or more predetermined individually accessible portions (200) into another individually accessible portion (200), the data stream comprising, for the one or more predetermined individually accessible portions (200), a representation of the neural network associated with a portion of the neural network to which each of the predetermined individually accessible portions corresponds.

160. Apparatus according to any one of claims 157 to 159, adapted to include corresponding separate predetermined individually accessible portions.

161. 161. An apparatus according to any one of claims 157 to 160, wherein the neural network portion comprises a neural network layer (210, 30) of the neural network and / or a layer portion into which a given neural network layer (210, 30) of the neural network is subdivided.

162. 162. An apparatus according to any of claims 157 to 161, wherein the apparatus is configured to encode the individually accessible portions (200) using context-adaptive arithmetic coding and using context initialization at the start of each individually accessible portion.

163. said device for each individually accessible portion: a start code (242) at which each of said individually accessible portions begins; and / or a pointer (220, 244) indicating the beginning of each of said individually accessible portions; and / or 163. An apparatus as claimed in any one of claims 157 to 162, configured to encode into the data stream a data stream length parameter indicating a data stream length (246) of each of the individually accessible portions for skipping each of the individually accessible portions when parsing the data stream.

164. The supplementary data (350) may include: the relevance scores of the neural network parameters (32), and / or 164. An apparatus according to any of claims 157 to 163, relating to perturbation robustness of neural network parameters (32).

165. An apparatus for encoding a representation of a neural network (10) into a data stream (45), comprising:

165. An apparatus according to any of claims 85 to 164, wherein the apparatus is configured to provide to the data stream (45) hierarchical control data (400) structured into a sequence (410) of control data portions (420), the control data portions providing information about the neural network with increasing levels of detail in the control data portions.

166. An apparatus for encoding a representation of a neural network (10) into a data stream (45), comprising: The apparatus is configured to provide hierarchical control data (400) structured into a sequence (410) of control data portions (420) to the data stream (45), the control data portions providing information of the neural network with increasing detail along the sequence of control data portions.

167. 167. Apparatus according to claim 165 or 166, wherein at least some of the control data portions (420) provide partially redundant information regarding the neural network.

168. 168. Apparatus according to any of claims 165 to 167, wherein a first control data portion provides the information regarding the neural network by indicating a default neural network type with default settings, and a second control data portion includes parameters indicating each of the default settings.

169. To decode a representation of a neural network (10) from a data stream (45). An apparatus comprising: The apparatus is configured to decode from the data stream (45) serialization parameters (102) indicating an encoding order (104) in which neural network parameters (32) defining neuronal interconnections (22, 24) of the neural network are encoded into the data stream (45).

170. 170. The apparatus of claim 169, wherein the apparatus is configured to decode the neural network parameters (32) from the data stream (45) using context-adaptive arithmetic decoding.

171. the data stream is structured into one or more separately accessible portions (200), each separately accessible portion representing a corresponding neural network layer (210, 30) of the neural network; the apparatus is configured to successively decode, from the data stream (45), neural network parameters defining neuronal interconnections (22, 24) of the neural network within a given neural network layer; 171. The apparatus of claim 169 or 170, further comprising: an encoding order (104) for assigning neural network parameters successively decoded from the data stream (45) to the neuronal interconnections (22, 24).

172. Apparatus according to any of claims 169 to 171, wherein the serialization parameter (102) is an n-ary parameter indicating the encoding order (104) out of a set (108) of n encoding orders (104).

173. The set (108) of n encoding orders (104) comprises: a first predetermined encoding order (106) in which the predetermined encoding order differs in the order in which the dimensions (34) of the tensors (30) describing the predetermined neural network layers (210, 30) of the neural network (10) are traversed; 1 ), and / or For scalable coding of the neural network, a second predefined coding order (106) is selected, which differs in the number of times (107) the predefined coding order scans a predefined neural network layer (210, 30) of the neural network. 2 ), and / or a third predetermined coding order (106) in which the predetermined coding order traverses the neural network layers of the neural network in a different order; 3 ), and / or a fourth predetermined coding order (106) which differs in the order in which the neurons (14, 18, 20) of the neural network layers of the neural network are scanned; 4 ) 72. The apparatus described in claim 72.

174. 174. The apparatus of any of claims 169 to 173, wherein the serialization parameter (102) indicates a permutation that the encoding order (104) uses to permute neurons (14, 18, 20) of a neural network layer (210, 30) relative to a default order.

175. 175. The apparatus of claim 174, wherein the permutation orders the neurons (14, 18, 20) of the neural network layer (210, 30) such that the neural network parameters (32) are monotonically increasing along the encoding order (104) or monotonically decreasing along the encoding order (104).

176. The permutation is a permutation of the neural network parameters (32) of the neural network layers (210, 30) from among a predetermined encoding order signalable by the serialization parameter (102) such that, for the permutation indicated by the serialization parameter (102), a bit rate for encoding the neural network parameters (32) into the data stream (45) is minimized.

175. The apparatus of claim 174, further comprising: a coil (14, 18, 20) for sequencing the coils.

177. 177. Apparatus according to any of claims 169 to 176, wherein the neural network parameters (32) include weights and biases.

178. 178. An apparatus as claimed in any one of claims 169 to 177, wherein the apparatus is configured to decode from the data stream individually accessible sub-portions (43, 44, 240) into which the data stream is structured, each sub-portion (43, 44, 240) representing a corresponding neural network portion of the neural network, such that each sub-portion (43, 44, 240) is completely scanned by the coding order (104) before a subsequent sub-portion is scanned by the coding order (104).

179. An apparatus as described in any of claims 171 to 178, wherein the neural network parameters (32) are decoded from the data stream at the start of any individually accessible portion (200) or sub-portion (43, 44, 240) using context-adaptive arithmetic decoding and using context initialization.

180. 180. An apparatus according to any one of claims 171 to 179, wherein the apparatus is configured to decode from the data stream a start code (242) starting each individually accessible portion (200) or sub-portion (43, 44, 240) and / or a pointer (220, 244) pointing to the start of each individually accessible portion or sub-portion and / or a data stream length (246) pointer for each individually accessible portion or sub-portion in order to skip the each individually accessible portion or sub-portion when parsing the data stream.

181. 181. The apparatus of any of claims 169 to 180, wherein the apparatus is configured to decode from the data stream numerical computation representation parameters (120) indicating numerical representations and bit sizes for representing the neural network parameters (32) when using the neural network (10) for inference.

182. An apparatus for decoding a representation of a neural network (10) from a data stream (45), comprising: The apparatus is configured to decode, from the data stream (45), a numerical calculation representation parameter (120) indicating a numerical representation and a bit size for representing neural network parameters (32) of the neural network encoded in the data stream when the neural network (10) is used for inference, and is configured to use the numerical representation and the bit size for representing the neural network parameters (32) decoded from the data stream (45).

183. 183. The method according to any of claims 169 to 182, wherein the data stream (45) is structured into individually accessible sub-portions (43, 44, 240), each individually accessible sub-portion representing a corresponding neural network portion of the neural network, such that each individually accessible sub-portion is completely scanned by the coding order (104) before a subsequent individually accessible sub-portion is scanned by the coding order (104), and the apparatus is configured to decode from the data stream (45), for a given individually accessible sub-portion, the neural network parameters and a type parameter indicative of a parameter type of the neural network parameters decoded from the given individually accessible sub-portion. The apparatus described.

184. 184. The apparatus of claim 183, wherein the type parameters distinguish at least between a weight of a neural network and a bias of a neural network.

185. a data stream (45) structured into one or more individually accessible portions (200), each of the one or more individually accessible portions representing a corresponding neural network layer (210, 30) of said neural network; 185. The apparatus of any of claims 169 to 184, wherein the apparatus is configured to decode, for a given neural network layer, from the data stream (45), a neural network layer type parameter (130) indicative of a neural network layer type of the given neural network layer of the neural network.

186. An apparatus for decoding a representation of a neural network (10) from a data stream (45), comprising:

1. An apparatus, comprising: the data stream (45) structured into one or more individually accessible portions (200), each portion representing a corresponding neural network layer (210, 30) of the neural network, the apparatus being configured to decode from the data stream (45), for a given neural network layer (210, 30), a neural network layer type parameter (130) indicative of a neural network layer type of the given neural network layer of the neural network.

187. 187. The apparatus of claim 185 or 186, wherein the neural network layer type parameter (130) distinguishes at least between a fully connected layer type and a convolutional layer type.

188. said data stream (45) being structured into separately accessible portions (200), each separately accessible portion representing a corresponding neural network portion of said neural network; 188. An apparatus as claimed in any one of claims 169 to 187, wherein the apparatus is configured to decode, for each of one or more predetermined individually accessible portions (200), a pointer (220, 244) to the start of the respective individually accessible portion from the data stream (45).

189. An apparatus for decoding a representation of a neural network (10) from a data stream (45), comprising:

1. An apparatus, comprising: the data stream (45) structured into one or more individually accessible portions (200), each portion representing a corresponding neural network layer (210, 30) of the neural network, the apparatus being configured to decode from the data stream (45), for each of one or more predetermined individually accessible portions, a pointer (220, 244) to the beginning of the respective predetermined individually accessible portion.

190. Each individually accessible part is a corresponding neural network layer (210) of said neural network; or 190. Apparatus according to claim 188 or 189, representing a neural network portion (43, 44, 240) of a neural network layer (210) of the neural network.

191. The apparatus is configured to decode a representation of the neural network (10) from the data stream (45), the data stream (45) being one or more individual wherein the data stream (45) is structured into individually accessible portions (200) each representing a corresponding neural network layer (210, 30) of the neural network, the data stream (45) being further structured within a given portion into individually accessible sub-portions (43, 44, 240) each representing a corresponding neural network portion of the respective neural network layer (210, 30) of the neural network, and the apparatus extracts from the data stream (45) for each of one or more predetermined individually accessible sub-portions (43, 44, 240): a start code (242) at which each of said predetermined individually accessible sub-portions begins; and / or a pointer (244) to the beginning of each of said predetermined individually accessible sub-portions; and / or 191. An apparatus according to any of claims 169 to 190, configured to decode a data stream length parameter indicating a data stream length (246) of each of the predetermined individually accessible sub-portions for skipping each of the predetermined individually accessible sub-portions when parsing the data stream (45).

192. 192. The apparatus of claim 191, wherein the apparatus is configured to decode the representation of the neural network from the data stream (45) at the start of each individually accessible portion and each individually accessible sub-portion using context-adaptive arithmetic decoding and using context initialization.

193. An apparatus for decoding a representation of a neural network (10) from a data stream (45), comprising: The data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, the data stream (45) being further structured within a given portion into individually accessible sub-portions (43, 44, 240), each sub-portion (43, 44, 240) representing a corresponding neural network portion of the respective neural network layer (210, 30) of the neural network, and the apparatus extracts from the data stream (45), for each of the one or more predetermined individually accessible sub-portions (43, 44, 240): a start code (242) at which each of said predetermined individually accessible sub-portions begins; and / or a pointer (244) to the beginning of each of said predetermined individually accessible sub-portions; and / or an apparatus configured to decode a data stream length parameter indicating a data stream length (246) of each of the predetermined individually accessible sub-portions for skipping each of the predetermined individually accessible sub-portions when parsing the data stream (45).

194. 200. The apparatus of claim 193, wherein the apparatus is configured to decode the representation of the neural network from the data stream (45) at the start of each individually accessible portion and each individually accessible sub-portion using context-adaptive arithmetic decoding and using context initialization.

195. The apparatus is configured to decode a representation of a neural network (10) from a data stream (45), the data stream (45) being structured into individually accessible portions (200), each individually accessible portion being a representation of the neural network (10).

200. An apparatus as claimed in any one of claims 169 to 194, further comprising: an apparatus configured to decode, for each of one or more predetermined individually accessible portions (200) from the data stream (45), processing option parameters (250) indicative of one or more processing options (252) that must be or can optionally be used when using the neural network (10) for inference.

196. the processing option parameters (250) indicate the one or more available processing options (252) from a set of predefined processing options (252); The predetermined processing options (252) are: the parallel processing capabilities of each of said predetermined individually accessible portions; and / or A sample-by-sample parallel processing capability of each of said predetermined individually accessible portions (252 2 ), and / or The per-channel parallel processing capability of each of said predetermined individually accessible portions (252 1 ), and / or a classification category-based parallel processing capability of each of the predetermined individually accessible portions; and / or 200. The apparatus of claim 195, including a dependency of the neural network portion represented by each of the predetermined individually accessible portions on computational results obtained from another individually accessible portion of the data stream (45) that is associated with the same neural network portion but belongs to another version of the neural network version (330) that is coded in a layered manner in the data stream (45).

197. An apparatus for decoding a representation of a neural network (10) from a data stream (45), comprising:

1. An apparatus, wherein the data stream (45) is structured into individually accessible portions (200), each individually accessible portion representing a corresponding neural network portion of the neural network, and the apparatus is configured to decode, for each of one or more predetermined individually accessible portions, from the data stream (45) a processing option parameter (250) indicative of one or more processing options (252) that must or may optionally be used when using the neural network (10) for inference.

198. the processing option parameters (250) indicate the one or more available processing options (252) from a set of predefined processing options (252); The predetermined processing options (252) are: the parallel processing capabilities of each of said predetermined individually accessible portions; and / or A sample-by-sample parallel processing capability of each of said predetermined individually accessible portions (252 2 ), and / or The per-channel parallel processing capability of each of said predetermined individually accessible portions (252 1 ), and / or a classification category-based parallel processing capability of each of the predetermined individually accessible portions; and / or 200. The apparatus of claim 197, including a dependency of the neural network portion represented by each of the predetermined individually accessible portions on computational results obtained from another individually accessible portion of the data stream (45) that is associated with the same neural network portion but belongs to another version of the neural network version (330) that is coded in a layered manner in the data stream (45).

199. The device generates neural network parameters ( 200. An apparatus according to any of claims 169 to 198, further comprising: an apparatus for decoding (decoding) neural network parameters (32) from a data stream (45), the neural network parameters (32) being quantized (260) into quantization indexes (32'') and encoded into the data stream (45), the neural network parameters (32) being encoded into the data stream (45) such that neural network parameters (32) in different neural network portions of the neural network are quantized (260) differently, the apparatus being configured to decode (270) for each of the neural network portions from the data stream (45) a reconstruction rule for dequantizing (280) the neural network parameters (32) associated with the respective neural network portion.

200. 1. An apparatus for decoding neural network parameters (32) representing a neural network from a data stream (45), comprising:

1. An apparatus for decoding, for each of the neural network portions, a reconstruction rule for dequantizing (280) the neural network parameters (32) associated with the respective neural network portion from the data stream (45).

201. 201. The apparatus of claim 199 or 200, wherein the neural network portion comprises a neural network layer (210, 30) of the neural network and / or a layer portion into which a given neural network layer of the neural network is subdivided.

202. The apparatus derives from the data stream (45) a first reconstruction rule (270) for dequantizing neural network parameters (280) associated with a first neural network portion. 1 , 270a 1 ) to a second reconstruction rule (270) for dequantizing the neural network parameters (280) associated with the second neural network portion. 2 , 270a 2 202. An apparatus according to any one of claims 199 to 201, configured to delta-decode and decode on

203. The device extracts the first reconstruction rule (270) from the data stream (45). 1 , 270a 1 a first index value for indicating the second reconstruction rule (270) 2 , 270a 2 and a second exponent value to indicate The first reconstruction rule (270 1 , 270a 1 ) is defined by a first quantization step size (263) defined in terms of a predetermined base of exponents and a first exponent defined in terms of said first exponent value; and The second reconstruction rule (270 2 , 270a 2 203. The apparatus of claim 202, wherein the first quantization step size (263) is defined by an exponent of the predetermined basis and a second exponent defined by the sum of the first and second exponent values.

204. 204. The apparatus of claim 203, wherein the data stream (45) further indicates a predetermined basis.

205. The apparatus derives from the data stream (45) a first reconstruction rule (270) for dequantizing (280) neural network parameters (32) associated with a first neural network portion. 1 , 270a 1 a first exponent value for indicating the first neural network parameter (32) associated with the second neural network portion; (280) the second reconstruction rule (270 2 , 270a 2 ) , The first reconstruction rule (270 1 , 270a 1 ) is defined by a first quantization step size (263) defined by a predetermined base exponent and a first exponent defined by the sum of the first exponent value and a predetermined exponent value; and The second reconstruction rule (270 2 , 270a 2 203. The apparatus of any of claims 199 to 202, wherein a quantization step size (263) is defined by an exponent of the predetermined basis and a second exponent defined by the sum of the second exponent value and the predetermined exponent value.

206. 206. The apparatus of claim 205, wherein the data stream further exhibits a predetermined basis.

207. 207. The apparatus of claim 206, wherein the data stream represents the predetermined basis within a neural network.

208. 208. Apparatus according to any of claims 205 to 207, wherein the data stream further indicates the predetermined exponent value.

209. 209. The apparatus of claim 208, wherein the data stream (45) indicates the predetermined index value within a neural network layer (210, 30).

210. 210. Apparatus according to claim 208 or 209, wherein the data stream further indicates the predetermined basis, the data stream indicating the predetermined exponent values ​​in a finer range than the range in which the predetermined basis is indicated by the data stream.

211. 211. An apparatus according to any of claims 203 to 210, wherein the apparatus is configured to decode, from the data stream, the predetermined base in non-integer form and the first and second exponent values ​​in integer form.

212. The device extracts the first reconstruction rule (270) from the data stream. 1 , 270 a 1 ), the first quantization index-to-reconstruction level mapping (265) is a first set of parameters (264) defining said second reconstruction rule (270) 2 , 2 70a 2 ), a second quantization index-reconstruction level mapping (265 and a second set of parameters (264) defining the The first reconstruction rule (270 1 , 270a 1 ) is defined by the first quantization index-to-reconstruction level mapping (265), The second reconstruction rule (270 2 , 270a 2 212. The apparatus according to claim 202, wherein the first quantization index - reconstruction level mapping (265) is defined by extending the first quantization index - reconstruction level mapping (265) in a predetermined manner with the second quantization index - reconstruction level mapping (265).

213. The device extracts the first reconstruction rule (270) from the data stream. 1 , 270 a 1 ), the first quantization index-to-reconstruction level mapping (265) is a first set of parameters (264) defining said second reconstruction rule (270) 2 , 2 70a 2 ), a second quantization index-reconstruction level mapping (265 and a second set of parameters (264) defining the The first reconstruction rule (270 1 , 270a 1 ) is defined by the first quantization index-to-reconstruction level mapping (265), The second reconstruction rule (270 2 , 270a 2 ) is the first quantization index - The apparatus according to any of claims 202 to 212, wherein a reconstruction level mapping (265) is defined by extending the second quantization index to reconstruction level mapping (265) in a predetermined manner.

214. 214. The apparatus of claim 213, wherein the data stream further indicates the predetermined quantization index to reconstruction level mapping (265).

215. 215. The apparatus of claim 214, wherein the data stream indicates the predetermined quantization index-to-reconstruction level mapping (265) at a neural network extent or at a neural network layer extent (210, 30).

216. According to the above-mentioned predetermined method, a mapping of each index value (32'') to a first reconstruction level according to the extended quantization index-reconstruction level mapping is replaced by a mapping of each index value (32'') to a second reconstruction level according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, if present; and / or According to the extended quantization index-reconstruction level mapping, a reconstruction level to which each of the index values ​​(32'') should be mapped is not defined, and for any of the index values ​​(32'') that are mapped to a corresponding reconstruction level according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, a mapping from the respective index value (32'') to the corresponding reconstruction level is adopted; and / or An apparatus as described in any of claims 212 to 215, wherein the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping does not define a reconstruction level to which each of the index values ​​(32'') should be mapped, and for any index value (32'') that is mapped to a corresponding reconstruction level according to the extended quantization index-reconstruction level mapping, a mapping from the respective index value (32'') to the corresponding reconstruction level is adopted.

217. the apparatus is configured to decode from the data stream to indicate a reconstruction rule (270) for a given neural network portion; a quantization step size parameter (262) indicating the quantization step size (263); a parameter set (264) defining a quantization index to reconstruction level mapping (265); The reconstruction rule (270) for the given neural network portion comprises: a quantization step size (263) for the quantization index (32'') within a given index interval (268); and The apparatus of any of claims 199 to 216, wherein the predetermined index interval is defined by a quantization index-to-reconstruction level mapping (265) for quantization indexes (32'') outside the predetermined index interval (268).

218. 1. An apparatus for decoding neural network parameters (32) representing a neural network from a data stream (45), comprising: the neural network parameters (32) are quantized (260) into quantization indexes (32'') and encoded into the data stream (45); The apparatus, a quantization step size parameter (262) indicating the quantization step size (263); 、 a parameter set (264) defining a quantization index to reconstruction level mapping (265); from the data stream (45), thereby being able to derive a reconstruction rule (270) for dequantizing (280) the neural network parameters (32); The reconstruction rule (270) for the given neural network portion comprises: a quantization step size (263) for the quantization indexes (32'') within a given index interval (268); and a quantization index to reconstruction level mapping (265) for quantization indexes (32'') outside said predetermined index interval (268).

219. 219. The apparatus of claim 217 or 218, wherein the predetermined index interval (268) includes zero.

220. The apparatus of claim 219, wherein the predetermined index interval (268) extends up to a predetermined magnitude threshold, and a quantization index (32'') exceeding the predetermined magnitude threshold represents an escape code indicating that the quantization index-to-reconstruction level mapping (265) is to be used for inverse quantization (280).

221. The apparatus of any of claims 217 to 220, wherein the parameter set (264) defines a quantization index to reconstruction level mapping (265) by a list of reconstruction levels associated with quantization indexes (32'') outside the predetermined index interval (268).

222. 222. Apparatus according to any of claims 199 to 221, wherein the neural network portion comprises one or more sub-portions of a neural network layer (210, 30) of the neural network and / or one or more neural network layers of the neural network.

223. 223. An apparatus according to any of claims 199 to 222, wherein the data stream (45) is structured into individually accessible portions (200), and the apparatus is configured to decode the neural network parameters (32) for a corresponding neural network portion from each individually accessible portion.

224. The apparatus of claim 223, wherein the apparatus is configured to decode the individually accessible portions (200) from the data stream (45) using context-adaptive arithmetic decoding at the start of each individually accessible portion and using context initialization.

225. The device extracts from the data stream (45) for each individually accessible portion: a start code (242) at which each of said individually accessible portions begins; and / or a pointer (220, 244) to the beginning of each of said individually accessible portions; and / or 25. The apparatus of claim 223 or 224, configured to read a data stream length parameter indicating a data stream length (246) of each of the individually accessible portions to skip when parsing the data stream (45).

226. The apparatus further comprises: for each of the neural network portions, instructions for a reconstruction rule (270) for dequantizing (280) the neural network parameters (32) associated with the respective neural network portion; A main header portion (47) of the data stream (45) relating to the entire neural network; a neural network layer (210, 30) associated with a header portion (110) of the data stream (45) associated with the neural network layer to which the respective neural network portion belongs; or 226. Apparatus according to any of claims 223 to 225, configured to read from the data stream (45) at a neural network portion specific header portion of the data stream associated with the respective neural network portion.

227. 227. An apparatus as claimed in any one of claims 169 to 226, wherein the apparatus is configured to decode a representation of a neural network (10) from a data stream (45), the data stream (45) being structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the apparatus is configured to decode from the data stream (45), for each of one or more predetermined individually accessible portions, an identification parameter (310) for identifying the respective predetermined individually accessible portion.

228. An apparatus for decoding a representation of a neural network (10) from a data stream (45), comprising:

13. An apparatus according to claim 12, wherein the data stream (45) is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the apparatus is configured to decode, from the data stream (45), for each of one or more predetermined individually accessible portions, an identification parameter (310) for identifying the respective predetermined individually accessible portion.

229. 229. Apparatus according to claim 227 or 228, wherein the identification parameter (310) is associated with the respective predetermined individually accessible portion via a hash function or an error detection or correction code.

230. 230. An apparatus according to any of claims 227 to 229, wherein the apparatus is configured to decode a high-level identification parameter (310) for identifying a set of two or more predetermined individually accessible portions from the data stream (45).

231. 231. The apparatus of claim 230, wherein the higher level identification parameter (310) is related to the identification parameter (310) of the one or more predetermined individually accessible portions via a hash function or an error detection or correction code.

232. 232. An apparatus according to any of claims 227 to 231, wherein the apparatus is configured to decode the individually accessible portions (200) from the data stream (45) using context-adaptive arithmetic decoding and using context initialization at the start of each individually accessible portion.

233. The device extracts from the data stream, for each individually accessible portion: a start code (242) at which each of said individually accessible portions begins; and / or 、 a pointer (220, 244) to the beginning of each of said individually accessible portions; and / or 233. An apparatus as claimed in any one of claims 227 to 232, configured to read a data stream length parameter indicating a data stream length (246) of each of the individually accessible portions to skip when parsing the data stream.

234. 234. Apparatus according to any of claims 227 to 233, wherein the neural network portion comprises a neural network layer (210, 30) of the neural network and / or one or more sub-portions of one or more neural network layers of the neural network.

235. the apparatus is configured to decode a representation of a neural network (10) from a data stream (45) coded in a layered manner, different versions (330) of the neural network being coded into the data stream (45) and the data stream (45) being structured into one or more individually accessible portions (200), each portion being associated with a corresponding version of the neural network; The apparatus comprises: by using delta decoding on a second version of the neural network (3301) encoded in a second part; and / or From the data stream (45), each of the first versions (330) of the neural network is 2 ) to perform inference based on one or more compensation neural By decoding the loop network portion (332), A first version of the neural network encoded from the first portion (330 2 ) a second version of said neural network (3301) encoded in said second portion is executed in addition to the execution of the corresponding neural network portion (334); 235. Apparatus according to any of claims 169 to 234, wherein the outputs of each compensation neural network portion (332) and the corresponding neural network portion (334) are summed.

236. the apparatus is configured to decode a representation of a neural network (10) from a data stream (45) coded in a layered manner, different versions (330) of the neural network being coded into the data stream (45) and the data stream (45) being structured into one or more individually accessible portions (200), each portion being associated with a corresponding version of the neural network; The apparatus comprises: by using delta decoding on a second version of the neural network (3301) encoded in a second part; and / or from said data stream (45), said first version (330) of said neural network; 2 ) to perform inference based on one or more complements, each of which is By decoding the compensation neural network portion (332), A first version of the neural network (330) from the first portion 2 ) is decrypted It is configured to in addition to executing the corresponding neural network portion (334) of the second version (3301) of the neural network encoded in the second portion, The outputs of the respective compensation neural network portions (332) and the corresponding neural network portions (334) are summed.

237. The device extracts the second version of the neural network from a second portion of the data stream. 1 ) ; and The apparatus comprises: The first version of the neural network (330 2 ) with weights and / or or bias differences, and / or Additional neurons (14, 18, 20) or neuronal interconnections (22, 24) the second version of the neural network (330) encoded in the second part by 1 ) from the first portion of the data stream (45) 237. Apparatus according to claim 235 or 236 configured for decoding.

238. 238. An apparatus according to any preceding claim 235 to 237, wherein the apparatus is configured to decode the individually accessible portions (200) from the data stream (45) using context-adaptive arithmetic decoding (600) using context initialization at the start of each individually accessible portion.

239. The device extracts from the data stream (45) for each individually accessible portion: a start code (242) at which each of said individually accessible portions begins; and / or a pointer (220, 244) to the beginning of each of said individually accessible portions; and / or 239. An apparatus as claimed in any one of claims 235 to 238, configured to decode a data stream length parameter indicating a data stream length (246) of each of the individually accessible portions for skipping each of the individually accessible portions when parsing the data stream.

240. 240. An apparatus as claimed in any one of claims 235 to 239, wherein the apparatus is configured to decode, for each of one or more predetermined individually accessible portions (200) from the data stream, an identification parameter (310) for identifying the respective predetermined individually accessible portion.

241. 241. An apparatus according to any one of claims 169 to 240, wherein the apparatus is configured to decode a representation of a neural network (10) from a data stream (45), the data stream (45) being organized into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the apparatus is configured to decode from the data stream (45), for each of one or more predetermined individually accessible portions, supplementary data (350) for supplementing the representation of the neural network.

242. An apparatus for decoding a representation of a neural network (10) from a data stream (45), comprising:

1. An apparatus according to claim 1, wherein the data stream (45) is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the apparatus is configured to decode, from the data stream (45), supplemental data (350) for each of one or more predetermined individually accessible portions (200) to supplement the representation of the neural network.

243. 243. The apparatus of claim 241 or 242, wherein the data stream (45) indicates that the supplemental data (350) is not essential to the neural network based inference.

244. 244. An apparatus as described in any of claims 241 to 243, wherein the apparatus is configured to decode supplementary data (350) for supplementing the representation of the neural network for the one or more predetermined individually accessible portions (200) from another individually accessible portion, such that the data stream (45) includes, for each of the one or more predetermined individually accessible portions, a corresponding another predetermined individually accessible portion associated with the portion of the neural network to which the respective predetermined individually accessible portion corresponds.

245. 245. An apparatus according to any one of claims 241 to 244, wherein the neural network portion comprises a neural network layer (210, 30) of the neural network and / or a layer portion into which a given neural network layer of the neural network is subdivided.

246. 246. An apparatus according to any of claims 241 to 245, wherein the apparatus is configured to decode the individually accessible portions (200) using context-adaptive arithmetic decoding and using context initialization at the start of each individually accessible portion.

247. The device extracts from the data stream, for each individually accessible portion: a start code (242) at which each of said individually accessible portions begins; and / or a pointer (220, 244) indicating the beginning of each of said individually accessible portions; and / or 247. An apparatus as claimed in any one of claims 241 to 246, configured to read a data stream length parameter indicating a data stream length (246) of each of the individually accessible portions for skipping each of the individually accessible portions when parsing the data stream (45).

248. The supplementary data (350) may include: the relevance scores of the neural network parameters (32), and / or 248. An apparatus according to any of claims 241 to 247, relating to perturbation robustness of neural network parameters (32).

249. An apparatus for decoding a representation of a neural network (10) from a data stream (45), comprising:

249. An apparatus as described in any of claims 169 to 248, wherein the apparatus is configured to decode from the data stream (45) hierarchical control data (400) structured into a sequence (410) of control data portions (420), the control data portions providing information about the neural network with increasing detail along the sequence of control data portions.

250. An apparatus for decoding a representation of a neural network (10) from a data stream (45), comprising: The apparatus is configured to decode, from the data stream (45), hierarchical control data (400) structured into a sequence (410) of control data portions (420), the control data portions providing information about the neural network with increasing detail along the sequence of control data portions.

251. 251. The apparatus of claim 249 or 250, wherein at least some of the control data portions (420) provide partially redundant information regarding the neural network.

252. 252. Apparatus as claimed in any one of claims 249 to 251, wherein a first control data portion provides the information regarding the neural network by indicating a default neural network type with default settings, and a second control data portion includes parameters indicating each of the default settings.

253. An apparatus for performing inference using a neural network, comprising: An apparatus for decoding a data stream (45) according to any one of claims 169 to 252 and deriving said neural network from said data stream (45); and a processor configured to perform the inference based on the neural network.

254. A method for encoding a representation of a neural network into a data stream (45), comprising: The method includes providing a serialization parameter to the data stream that indicates a coding order in which neural network parameters defining neuronal interconnections of the neural network are coded into the data stream.

255. 1. A method for encoding a representation of a neural network into a data stream, comprising: providing, to the data stream, numerical computation representation parameters indicating numerical representations and bit sizes in which neural network parameters of the neural network are expressed that are encoded into the data stream when the neural network is used for inference.

256. 1. A method for encoding a representation of a neural network into a data stream, comprising the steps of:

11. A method according to claim 10, wherein the data stream is structured into one or more separately accessible portions, each separately accessible portion representing a corresponding neural network layer of the neural network, the method comprising, for a given neural network layer, providing a neural network layer type parameter to the data stream indicating a neural network layer type of the given neural network layer of the neural network.

257. 1. A method for encoding a representation of a neural network into a data stream, comprising the steps of:

11. A method according to claim 1, wherein the data stream is structured into one or more individually accessible portions, each portion representing a corresponding neural network layer of the neural network, the method comprising, for each of one or more predetermined individually accessible portions, providing in the data stream a pointer to the beginning of the respective predetermined individually accessible portion.

258. 1. A method for encoding a representation of a neural network into a data stream, comprising the steps of: The method includes the steps of: the data stream is structured into one or more individually accessible portions, each individually accessible portion representing a corresponding neural network layer of the neural network; the data stream is further structured into individually accessible sub-portions within a given portion, each sub-portion representing a corresponding neural network portion of a respective neural network layer of the neural network; and the method further comprises: a start code where each of said predetermined individually accessible sub-portions begins; and / or a pointer to the beginning of each of said predetermined individually accessible sub-portions; and / or Or, and a data stream length parameter indicating a data stream length of the predetermined individually accessible sub-portion for skipping the predetermined individually accessible sub-portion when parsing the data stream.

259. 1. A method for encoding a representation of a neural network into a data stream, comprising the steps of:

11. A method according to claim 10, wherein the data stream is structured into individually accessible portions, each individually accessible portion representing a corresponding neural network portion of the neural network, the method comprising the step of providing, for each of one or more predetermined individually accessible portions, processing option parameters to the data stream indicating one or more processing options that must be used or can optionally be used when using the neural network for inference.

260. 1. A method for encoding neural network parameters representing a neural network into a data stream, comprising: The neural network parameters are quantized into quantization indices and encoded into the data stream, and neural network parameters of different neural network portions of the neural network are encoded into the data stream such that they are quantized differently, the method including providing the data stream indicating, for each of the neural network portions, a reconstruction rule for dequantizing the neural network parameters associated with the respective neural network portion.

261. 1. A method for encoding neural network parameters representing a neural network into a data stream, comprising: The neural network parameters are quantized into quantization indexes and encoded into the data stream, and the method further comprises: A quantization step size parameter indicating a quantization step size; a parameter set defining a quantization index to reconstruction level mapping to said data stream; The reconstruction rule for the given neural network portion is the quantization step size for a quantization index within a predetermined index interval; the quantization index to reconstruction level mapping for quantization indexes outside the predetermined index interval.

262. 1. A method for encoding a representation of a neural network into a data stream, comprising the steps of:

11. A method, wherein the data stream is structured into individually accessible portions, each portion representing a corresponding neural network portion of the neural network, the method comprising, for each of one or more predetermined individually accessible portions, providing the data stream with an identification parameter for identifying the respective predetermined individually accessible portion.

263. 1. A method for encoding a representation of a neural network into a data stream in a layer-wise manner, comprising: Different versions of the neural network are encoded into the data stream, and the data stream is structured into one or more separately accessible portions, each portion associated with a corresponding version of the neural network, the method comprising: delta-encoding a second version of the neural network encoded in a second portion; and / or in the form of one or more compensating neural network portions, each of which is executed in addition to the execution of a corresponding neural network portion of a second version of the neural network encoded in a second portion, to perform inference based on the first version of the neural network, in the form of a compensation neural network portion, the outputs of each said compensation neural network portion and said corresponding neural network portion being summed; encoding said first portion.

264. 1. A method for encoding a representation of a neural network into a data stream, comprising the steps of:

11. A method according to claim 1, wherein the data stream is structured into separately accessible portions, each portion representing a corresponding neural network portion of the neural network, the method comprising, for each of one or more predetermined separately accessible portions, providing supplemental data to the data stream for supplementing the representation of the neural network.

265. 1. A method for encoding a representation of a neural network into a data stream, comprising the steps of: The method includes providing hierarchical control data to the data stream structured into a sequence of control data portions, the control data portions providing information about the neural network with increasing levels of detail along the sequence of control data portions.

266. 1. A method for decoding a representation of a neural network from a data stream, comprising:

23. The method of claim 22, further comprising: decoding from the data stream serialized parameters indicating an encoding order in which neural network parameters defining neuronal interconnections of the neural network are encoded into the data stream.

267. 1. A method for decoding a representation of a neural network from a data stream, comprising: The method includes the steps of: decoding from the data stream numerical computation representation parameters indicating a numerical representation and a bit size in which neural network parameters of the neural network encoded in the data stream should be expressed when the neural network is used for inference; and using the numerical representation and bit size decoded from the data stream to express the neural network parameters.

268. 1. A method for decoding a representation of a neural network from a data stream, comprising:

11. A method according to claim 1, wherein the data stream is structured into one or more separately accessible portions, each portion representing a corresponding neural network layer of the neural network, the method comprising: decoding, from the data stream, for a given neural network layer, a neural network layer type parameter indicative of a neural network layer type of the given neural network layer of the neural network.

269. 1. A method for decoding a representation of a neural network from a data stream, comprising:

11. A method, wherein the data stream is structured into one or more individually accessible portions, each portion representing a corresponding neural network layer of the neural network, the method comprising the step of decoding, from the data stream, for each of one or more predetermined individually accessible portions, a pointer to the beginning of the respective predetermined individually accessible portion.

270. 1. A method for decoding a representation of a neural network from a data stream, comprising: wherein the data stream is structured into one or more individually accessible portions, each individually accessible portion representing a corresponding neural network layer of the neural network, and the data stream is further structured into individually accessible sub-portions within a given portion, each sub-portion representing a corresponding neural network portion of a respective neural network layer of the neural network, and the method comprises: extracting from the data stream, for each of one or more predetermined individually accessible sub-portions: a start code where each of said predetermined individually accessible sub-portions begins; and / or a pointer to the beginning of each of said predetermined individually accessible sub-portions; and / or and a data stream length parameter indicating a data stream length of each of the predetermined individually accessible sub-portions for skipping the predetermined individually accessible sub-portions when parsing the data stream.

271. 1. A method for decoding a representation of a neural network from a data stream, comprising:

11. A method, wherein the data stream is structured into individually accessible portions, each individually accessible portion representing a corresponding neural network portion of the neural network, the method comprising the step of decoding from the data stream, for each of one or more predetermined individually accessible portions, processing option parameters indicating one or more processing options that must or may optionally be used by the neural network for inference.

272. 1. A method for decoding neural network parameters representing a neural network from a data stream, comprising: The neural network parameters are quantized into quantization indices and encoded into the data stream, and neural network parameters of different neural network portions of the neural network are encoded into the data stream so as to be different, the method including the step of decoding, for each of the neural network portions, from the data stream a reconstruction rule for dequantizing the neural network parameters associated with the respective neural network portion.

273. 1. A method for decoding neural network parameters representing a neural network from a data stream, comprising: The neural network parameters are quantized into quantization indexes and encoded into the data stream, the method comprising: A quantization step size parameter indicating a quantization step size; a parameter set defining a quantization index to reconstruction level mapping; from the data stream, deriving a reconstruction rule for dequantizing the neural network parameters; The reconstruction rule for the given neural network portion is the quantization step size for a quantization index within a predetermined index interval; the quantization index to reconstruction level mapping for quantization indexes outside the predetermined index interval.

274. 1. A method for decoding a representation of a neural network from a data stream, comprising:

11. A method, wherein the data stream is structured into individually accessible portions, each portion representing a corresponding neural network portion of the neural network, the method comprising the step of decoding, from the data stream, for each of one or more predetermined individually accessible portions, an identification parameter for identifying the respective predetermined individually accessible portion.

275. 1. A method for decoding from a data stream a representation of a neural network that has been encoded in the data stream in a layer-wise manner, the method comprising: Different versions of the neural network are encoded into the data stream, and the data stream is structured into one or more separately accessible portions, each portion associated with a corresponding version of the neural network, the method comprising: by using delta decoding on a second version of the neural network encoded in a second portion; and / or by decoding from the data stream one or more compensating neural network portions, each of which is executed in addition to execution of a corresponding neural network portion of a second version of the neural network encoded in a second portion, to perform inference based on the first version of the neural network; Decoding from a first portion, A method in which the outputs of the respective compensation neural network portions and the corresponding neural network portions are summed.

276. 1. A method for decoding a representation of a neural network from a data stream, comprising:

11. A method according to claim 1, wherein the data stream is structured into individually accessible portions, each portion representing a corresponding neural network portion of the neural network, the method comprising the step of decoding, from the data stream, supplemental data for supplementing the representation of the neural network for each of one or more predetermined individually accessible portions.

277. 1. A method for decoding a representation of a neural network from a data stream, comprising: The method includes decoding hierarchical control data from the data stream that is structured into a sequence of control data portions, the control data portions providing information about the neural network with increasing detail along the sequence of control data portions.

278. A computer program product which, when executed by a computer, causes the computer to carry out a method according to any of claims 254 to 277.

Images