Representation formats of neural networks

Abstract

It provides a data stream for the efficient use of neural networks and / or for the efficient transmission and / or updating of neural networks. [Solution] A data stream (45) in which a representation of a neural network (10) is encoded, and which includes serialization parameters (102) indicating the encoding order (104) in which neural network parameters (32) defining the neuron interconnections (22, 24) of the neural network (10) are encoded in the data stream (45), the NN parameters decoded from the data stream (45) are assigned to the neuron interconnections.

Description

Technical Field

[0001] The present invention relates to a concept related to the representation form of neural networks.

Background Art

[0002] Neural networks (NNs) are currently making breakthroughs in many applications. · Detection and classification of objects in image / video data · Speech recognition and keyword recognition of speech · Speech synthesis · Optical character recognition · Language translation · etc.

[0003] However, since the amount of data required for NN representation is enormous, it may be difficult to apply depending on the usage scenario. In many cases, this data is composed of two types of parameters, "weights" and "biases", which represent the connections between neurons. Weights are usually parameters that perform some linear transformation (e.g., dot product or convolution) on the input values, in other words, parameters that assign weights to the inputs of neurons, and biases are offsets added after linear calculation, in other words, something that cancels out the aggregation of weighted messages received by neurons. More specifically, these weights, biases, and additional parameters that characterize each connection between potentially very many neurons (up to tens of millions) in each layer (up to hundreds) of the NN occupy the main part of the data related to a specific NN. Also, these parameters are typically composed of large floating-point date types. These parameters are usually represented as large tensors containing all the parameters of each layer. In an application, when frequent transmission or update of the relevant NN is required, the required data rate becomes a serious bottleneck. Therefore, performing non-invertible compression of these matrices is a promising approach to reduce the encoding size of NN representation.

[0004] Typically, parameter tensors are stored in a container format (such as ONNX (Open Neural Network Exchange), PyTorch, or TensorFlow) that carries all the data necessary to completely reconstruct and run the neural network (such as the parameter matrix mentioned above) and further properties (such as the dimension of the parameter tensor, the type of layers, and the operations).

[0005] It would be advantageous to have at hand a concept that makes sending / updating machine learning predictors, or alternatively, machine learning models such as neural networks, more efficient in terms of preserving inference quality while simultaneously reducing the coded size of the NN representation, the complexity of computational inference, and the complexity of describing or storing the NN representation; or to enable more frequent sending / updating of NNs than currently; or to improve the inference quality for a particular task and / or specific local input data statistics. Furthermore, it would be advantageous to provide neural network representations for making predictions based on neural networks, derivations of such neural network representations, and methods for using such neural network representations, so that the use of neural networks becomes more effective than currently. [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] Accordingly, an object of the present invention is to provide a concept for the efficient use of neural networks and / or for the efficient transmission and / or updating of neural networks. This object is achieved by the subject matter of the independent claims of this application. [Means for solving the problem]

[0007] Further embodiments of the present invention are defined by the subject matter of the dependent claims of this application.

[0008] The fundamental idea of ​​the first aspect of the present invention is that the efficiency of using a neural network (NN) can be increased by encoding / decoding serialization parameters into a data stream (DS) that encodes the representation of the NN. The serialization parameters indicate the encoding order in which NN parameters defining the neuronal interconnections of the NN are encoded into the data stream. Neuronal interconnections may represent connections between neurons in different NN layers of the NN. In other words, NN parameters may define connections between a first neuron associated with the first layer of the NN and a second neuron associated with the second layer of the NN. A decoder can use the encoding order to assign the NN parameters, which are sequentially decoded from the data stream, to the neuronal interconnections.

[0009] In particular, it has been found that using serialization parameters allows for the efficient partitioning of bit sequences into meaningful, consecutive subsets of NN parameters. Serialization parameters may indicate a grouping of NN parameters that enables efficient execution of the NN. This may depend on the application scenario of the NN. For different application scenarios, the encoder may scan the NN parameters using different coding orders. Therefore, NN parameters can be coded using individual coding orders dependent on the application scenario of the NN, and the decoder can reconstruct the NN parameters accordingly during decoding based on the information provided by the serialization parameters. NN parameters may represent entries in one or more parameter matrices or tensors, which may be used in the inference procedure. It has been found that one or more parameter matrices or tensors of the NN can be efficiently reconstructed by the decoder based on the decoded NN parameters and serialization parameters.

[0010] Thus, serialization parameters can use different application-specific coding orders, enabling flexible coding and decoding with improved efficiency. For example, coding parameters along different dimensions can benefit compression performance because the entropy encoder can better capture the dependencies between them. In another example, it may be desirable to group parameters according to some application-specific criterion, namely which part of the input data they relate to or whether they are co-executable, so that they can be coded / inferred in parallel. A further example is coding parameters according to a GEMM (General Matrix Matrix) product scan order, which supports efficient memory allocation of coded parameters when performing dot product operations (Andrew Kerr, 2017).

[0011] Further embodiments are directed towards selective sorting on the data encoder side to achieve energy compression of the NN parameters to be encoded, and then to process / serialize / encode the resulting sorted data according to the order of the results. Thus, the sorting can be done so that the parameters are sorted so that they steadily increase or decrease along the encoding order.

[0012] According to a second aspect of the present application, the inventors have found that encoding / decoding numerical computational representation parameters to / from a data stream in which the representation of the NN is encoded improves the efficiency of using a neural network (NN). The numerical computational representation parameters indicate the numerical representation, e.g., floating-point or fixed-point representation, and bit size, of the NN parameters of the NN encoded in the data stream, which should be represented when the NN is used for inference. An encoder is configured to encode the NN parameters. A decoder is configured to decode the NN parameters and may be configured to use the numerical representation and bit size to represent the NN parameters decoded from the data stream DS.

[0013] This embodiment is based on the idea that it is advantageous to represent NN parameters and activation values ​​obtained as a result of using NN parameters in inference using the NN with the same numerical representation and bit size. Based on the numerical computation representation parameter, it is possible to efficiently compare the possible numerical representations and bit sizes of the NN parameters and the activation values. This is considered particularly advantageous when the numerical computation representation parameter shows a fixed-point representation as its numerical representation. This is because if both the NN parameters and activation values ​​can be represented in a fixed-point representation, inference can be performed efficiently using fixed-point arithmetic.

[0014] According to a third aspect of the present application, the inventors have achieved increased efficiency in using neural networks by encoding / decoding NN layer type parameters to / from a data stream in which the representation of the NN is encoded. The NN layer type parameters indicate the NN layer type of a given NN layer of the NN, e.g., a convolutional layer type or a fully connected layer type. The data stream is structured into one or more individually accessible parts, each individually accessible part representing a corresponding NN layer of the NN. A given NN layer represents one of the NN layers of the neural network. Optionally, for each of two or more given NN layers of the NN, the NN layer type parameters are encoded / decoded to / from the data stream, and the NN layer type parameters can differ among at least several given NN layers.

[0015] This embodiment is based on the idea that, for example, it may be useful for the data stream to contain the NN layer type parameters of the NN layer in order to understand the meaning of the dimensions of the parameter tensor / matrix. Furthermore, different layers may be treated differently when encoding important information that the decoder should know before decoding, for example by using different sets or modes of the context model, in order to better capture data dependencies and lead to higher encoding efficiency.

[0016] Similarly, it may be advantageous to encode / decode type parameters into the data stream, indicating the parameter types of NN parameters. Type parameters can indicate whether NN parameters represent weights or biases. The data stream is structured into one or more individually accessible parts, each representing a corresponding NN layer of the NN. An individually accessible part representing a given NN layer may be further structured into individually accessible subparts. Each individually accessible subpart is fully traversed by the encoding order before subsequent individually accessible subparts are traversed by the encoding order. Each individually accessible subpart can, for example, encode and decode NN parameters and type parameters. The NN parameters in the first individually accessible subpart may be of the same parameter type as those in the second individually accessible subpart, or they may be of different parameter types. Different types of NN parameters associated with the same NN layer may be encoded / decoded into different individually accessible subparts associated with the same individually accessible part. Parameter type distinctions may be beneficial for encoding / decoding, for example, when different type dependencies can be used for each parameter type, or when parallel decoding is desired. For example, it is possible to encode / decode different types of NN parameters associated in parallel with the same NN layer. This can improve the efficiency of encoding / decoding NN parameters and allow the entropy encoder to better capture the dependencies between NN parameters, which can result in advantages in compression performance.

[0017] According to a fourth aspect of the present application, the inventors have discovered that transmitting / updating a neural network becomes highly efficient when a pointer is encoded in a data stream in which the representation of the NN is encoded, and the pointer is decoded from the data stream. This is because the data stream is composed of individually accessible portions, and for each of one or more predetermined individually accessible portions, a pointer points to the beginning of each predetermined individually accessible portion. Not all individually accessible portions need to be predetermined individually accessible portions, but all individually accessible portions may represent predetermined individually accessible portions. One or more predetermined individually accessible portions may be set by default, or they may be set depending on the application of the NN encoded in the data stream. The pointer indicates the beginning of each predetermined individually accessible portion, for example, as a data stream position in bytes, or as an offset, for example, relative to the beginning of the data stream, or as a byte offset relative to the beginning of the portion corresponding to the NN layer to which each predetermined individually accessible portion belongs. This pointer may be encoded in the header portion of the data stream and decoded from the header portion. According to the embodiment, for each of one or more predetermined individually accessible portions, the pointer is encoded / decoded into the header portion of the data stream if each predetermined individually accessible portion represents the corresponding NN layer of the neural network, and the pointer is encoded / decoded into the parameter set portion of the corresponding NN layer if each predetermined individually accessible portion represents the NN portion of the NN layer of the NN. The NN portion of the NN layer of the NN may represent the baseline portion of each NN layer, or it may represent the advanced section of each layer. By using pointers, predetermined individually accessible portions of the data stream can be accessed efficiently, making it possible, for example, to parallelize layer processing or package the data stream into its respective container format.By using pointers, it becomes easier, faster, and more appropriate to access specific, individually accessible parts, thus facilitating applications that require parallel or partial decoding and execution of neural networks.

[0018] According to a fifth aspect of the present application, the inventors have achieved highly efficient transmission / update of a neural network when a start code, a pointer, and / or a data stream length parameter are encoded into / decoded from the data stream into individually accessible sub-parts of the data stream in which the representation of the NN is encoded. The data stream is structured into one or more individually accessible parts, each individually accessible part representing a corresponding NN layer of the neural network. Furthermore, within one or more predetermined individually accessible parts, the data stream is further structured into individually accessible sub-parts, each individually accessible sub-part representing a corresponding NN part of each NN layer of the neural network. The device is configured to encode into / decode from the data stream, for each of the one or more predetermined individually accessible sub-parts, a start code in which each predetermined individually accessible sub-part begins, and / or a pointer to the beginning of each predetermined individually accessible sub-part, and / or a data stream length parameter indicating the data stream length of each predetermined individually accessible sub-part in order to skip each predetermined individually accessible sub-part when parsing the DS. The start code, pointer, and / or data stream length parameter enable efficient access to a given individually accessible sub-part. This is particularly useful for applications that may rely on grouping NN parameters within NN layers in a specific configurable way, as it is beneficial to decode / process / input NN parameters partially or in parallel. Thus, access to individually accessible sub-parts of individually accessible portions helps in accessing necessary data in parallel or filtering out unnecessary data portions. It has also been found that it is sufficient to indicate individually accessible sub-parts using the start code. This is based on the observation that the amount of data per NN layer, i.e., the individually accessible portion, is usually less than the amount of data in the entire data stream that the NN layer is found by the start code.Nevertheless, it is also advantageous to use pointers and / or datastream length parameters to improve access to individually accessible sub-parts. According to embodiments, one or more individually accessible sub-parts within an individually accessible part of a datastream are indicated by pointers that indicate the datastream position in bytes within the parameter set portion of the individually accessible part. A datastream length parameter may indicate the run length of an individually accessible sub-part. The datastream length parameter may be encoded / decoded to / from the header portion of the datastream, or to / from the parameter set portion of the individually accessible part. The datastream length parameter may be used to facilitate the extraction of each individually accessible sub-part for the purpose of packaging one or more individually accessible sub-parts into appropriate containers. According to embodiments, a device for decoding a datastream is configured to use a start code and / or pointers and / or datastream length parameters to access the datastream for one or more predetermined individually accessible sub-parts.

[0019] According to a sixth aspect of the present application, the inventors have found that the efficiency of using a neural network is increased when processing option parameters are encoded in a data stream in which a representation of the NN is encoded / decoded from the data stream. The data stream is structured into individually accessible parts, and for each of one or more predetermined individually accessible parts, the processing option parameters indicate one or more processing options that must be used, or can be optionally used, when using the neural network for inference. The processing option parameters may indicate one of several processing options that determine whether and how a client accesses an individually accessible part (P) and / or an individually accessible subpart (SP). For example, for each of P and / or SP, such as the parallel processing capability of each P or SP and / or the sample-level parallel processing capability of each P or SP and / or the channel-level parallel processing capability of each P or SP and / or the classification-level parallel processing capability of each P or SP and / or other processing options. The processing option parameters enable the client to make appropriate decisions, which in turn enables the highly efficient use of the NN.

[0020] According to a seventh aspect of the present application, the inventors have discovered that the transmission / update of a neural network is made more efficient when the reconstruction rule for dequantizing NN parameters depends on the NN portion to which the NN parameters belong. The NN parameters representing the neural network are quantized into quantization indices and encoded into a data stream. The decoding device is configured, for example, to dequantize the quantization indices to reconstruct the NN parameters using a reconstruction rule. The NN parameters are encoded into a data stream such that the NN parameters of different NN portions of the NN are quantized differently, and the data stream indicates a reconstruction rule for dequantizing the NN parameters associated with each NN portion. For each NN portion, the decoding device is configured to dequantize the NN parameters in each NN portion using the reconstruction rule indicated by the data stream for each NN portion. An NN portion includes, for example, one or more NN layers of an NN and / or a portion of NN layers into which a given NN layer of an NN is subdivided.

[0021] According to one embodiment, a first reconstruction rule for dequantizing NN parameters associated with a first NN portion is delta-coded with respect to a second reconstruction rule for dequantizing NN parameters associated with a second NN portion and encoded into a data stream. The first NN portion may consist of a first NN layer, and the second NN portion may consist of a second layer, where the first NN layer is different from the second NN layer. Alternatively, the first NN portion may consist of a first NN layer, and the second NN portion may consist of one of the first NN layers. In this alternative example, a reconstruction rule associated with NN parameters in a given portion of an NN layer, e.g., a second reconstruction rule, is delta-coded with respect to a reconstruction rule associated with a given NN layer, e.g., a first reconstruction rule. This special delta coding of the reconstruction rule may allow for the use of only a small number of bits to signal the reconstruction rule, potentially resulting in efficient transmission / update of the neural network.

[0022] According to an eighth aspect of the present application, the inventors have found that the transmission / update of a neural network is greatly streamlined if the reconstruction rule for dequantizing NN parameters depends on the size of the quantization index associated with the NN parameters. The NN parameters representing the neural network are quantized into quantization indices and encoded into a data stream. The decoding device is configured, for example, to dequantize the quantization indices to reconstruct the NN parameters using a reconstruction rule. The data stream comprises a quantization step size parameter indicating the quantization step size and a set of parameters defining a quantization index-reconstruction level mapping to indicate the reconstruction rule for dequantizing the NN parameters. The reconstruction rule for NN parameters in a given portion of the NN is defined by the quantization step size for quantization indices within a given index interval and by a quantization index-reconstruction level mapping for quantization indices outside the given index interval. For each NN parameter, for example, each NN parameter related to a quantization index within a given index interval is reconstructed by multiplying each quantization index by the quantization step size, and for example, each NN parameter corresponding to a quantization index outside a given index interval is reconstructed by mapping each quantization index onto a reconstruction level using a quantization index-reconstruction level mapping. The decoder may be configured to determine the quantization index-reconstruction level mapping based on a parameter set in the data stream. According to one embodiment, the parameter set defines the quantization index-reconstruction level mapping by pointing to one of the set of quantization index-reconstruction level mappings, and the set of quantization index-reconstruction level mappings may not be part of the data stream and may be stored, for example, on the encoder side and the decoder side.Defining a reconstruction rule based on the magnitude of the quantization index enables signaling of the reconstruction rule with a small number of bits.

[0023] According to a ninth aspect of the present application, the inventor of the present application has noticed that if the representation of the NN encodes / decodes the identification parameter into / from an individually accessible part of the encoded data stream, the transmission / update of the neural network becomes very efficient. The data stream is structured into individually accessible parts, and for each of one or more predetermined individually accessible parts, an identification parameter for identifying each of the predetermined individually accessible parts is encoded into / decoded from the data stream. The identification parameter may indicate the version of a predetermined individually accessible part. This is particularly advantageous in scenarios such as distributed learning where many clients further learn the NN individually and send back relative NN updates to a central entity. The identification parameter can be used to identify the NN of an individual client by a versioning (version management) scheme. Thereby, the central entity can identify the NN for which the NN update is constructed. Additionally, or alternatively, the identification parameter may indicate whether a predetermined individually accessible part is related to a baseline part of the NN or an advanced / enhanced / complete part of the NN. This is advantageous in use cases such as, for example, a scalable NN, and before a complete or enhanced NN is executed to receive a complete result, for example, the baseline part of the NN can be executed to generate preliminary results. Further, a transmission error or an involuntary change of the parameter tensor that can be reconstructed based on the NN parameters representing the NN is easily recognizable using the identification parameter. With the identification parameter, each predetermined individually accessible part determined in advance can check for consistency and make the operation more error-robust when it can be verified based on the NN characteristics.

[0024] According to the tenth aspect of the present application, the inventor of the present application noticed that by encoding / demodulating different versions of NN into / from a data stream using delta coding or using a compensation scheme, the transmission / update of the neural network can be made very efficient. The data stream has the representation of the NN encoded in a layer-by-layer manner such that different versions of the NN are encoded into the data stream. The data stream is structured into one or more individually accessible parts, and each individually accessible part is associated with a corresponding version of the NN. For example, the data stream has the first version of the NN encoded in the first part being delta-coded with respect to the second version of the NN encoded in the second part. Additionally or alternatively, the data stream has, for example, the first version of the NN encoded in the first part in the form of one or more compensated NN parts, each of which is executed in addition to the execution of the corresponding NN part of the second version of the NN encoded in the second part for making inferences based on the first version of the NN, and the outputs of each compensated NN part and the corresponding NN part are summed. These encoded versions of the NN in the data stream allow a client, for example, a decoder, to combine its processing capabilities or perhaps first make inferences with the first version, for example, a baseline, before processing the second version, for example, a more complex advanced NN. Further, by applying / using the delta coding and / or compensation scheme, different versions of the NN can be encoded into the DS with fewer bits.

[0025] According to an eleventh aspect of the present application, the inventors have found that encoding / decoding supplemental data into / from individually accessible portions of a data stream in which a representation of a neural network (NN) is encoded improves the efficiency of the neural network. The data stream is structured into individually accessible portions, and for each of one or more predetermined individually accessible portions, the data stream contains supplemental data to supplement the representation of the NN. This supplemental data is not typically required for decoding / reconstructing / inferring the NN, but may be essential from an application standpoint. Therefore, it is advantageous to mark this supplemental data as irrelevant to decoding the NN solely for inference purposes, so that clients that do not require the supplemental data, such as a decoder, can skip this portion of the data.

[0026] According to a twelfth aspect of the present application, the inventors have found that the use of neural networks becomes highly efficient if hierarchical control data is encoded / decoded into a data stream in which the representation of the neural network is encoded. The data stream consists of hierarchical control data structured into a sequence of control data portions, where the control data portions provide information about the neural network with increasing detail along the sequence of control data portions. Hierarchically structuring the control data is advantageous because the decoder may only require control data up to a certain level, and therefore can skip control data that provides further aspects. Thus, different levels of control data may be required depending on knowledge of the use case and environment, and the aforementioned method of presenting such control data enables efficient access to the control data required for different use cases.

[0027] While several embodiments have been described in the context of apparatus, it is clear that these embodiments also represent a description of a method in which a block or apparatus corresponds to a method step or a feature of a method step. Embodiments relate to computer programs having program code for performing such methods when executed on a computer.

[0028] Embodiments of the present invention are the subject of the dependent claims. Preferred embodiments of the present invention are described below with reference to the drawings. The drawings are not necessarily to scale and instead focus on illustrating the general principles of the present invention. Various embodiments of the present invention are described below with reference to the following drawings. [Brief explanation of the drawing]

[0029] [Figure 1] Figure 1 shows an example of an encoding / decoding pipeline for encoding / decoding a neural network. [Figure 2] Figure 2 shows a neural network that can be encoded / decoded according to one embodiment. [Figure 3] Figure 3 shows the serialization of the parameter tensors of the layers of a neural network according to an embodiment. [Figure 4] Figure 4 shows the use of serialization parameters to illustrate how neural network parameters are serialized according to an embodiment. [Figure 5] Figure 5 shows an example of a single-output channel convolutional layer. [Figure 6] Figure 6 shows an example of a fully connected layer. [Figure 7a] Figure 7a shows a set of n coding orders in which neural network parameters can be coded, according to an embodiment. [Figure 7b] Figure 7b shows a set of n coding sequences in which neural network parameters can be coded, according to an embodiment. [Figure 7c] Figure 7c shows a set of n coding orders in which neural network parameters can be coded, according to an embodiment. [Figure 7d] Figure 7d shows a set of n coding sequences in which neural network parameters can be coded, according to an embodiment. [Figure 8] Figure 8 shows a context-adaptive arithmetic coding of individually accessible parts or subparts according to an embodiment. [Figure 9] Figure 9 shows how numerical calculation representation parameters are used according to an embodiment. [Figure 10] Figure 10 shows how to use the neural network layer type parameter to indicate the neural network layer type of the neural network layer of a neural network according to an embodiment. [Figure 11] Figure 11 shows a typical embodiment of a data stream having pointers that point to the beginning of individually accessible portions, according to the embodiment. [Figure 12] Figure 12 shows a detailed embodiment of a data stream having pointers that point to the beginning of individually accessible portions, according to the embodiment. [Figure 13] Figure 13 illustrates the use of start code and / or pointers and / or data stream length parameters to enable access to individually accessible sub-parts, according to an embodiment. [Figure 14a] Figure 14a shows a sub-layer access using a pointer according to an embodiment. [Figure 14b] Figure 14b shows a sub-tier access using a start code according to an embodiment. [Figure 15] Figure 15 shows an exemplary type of random access as a possible processing option for individually accessible parts, according to an embodiment. [Figure 16] Figure 16 shows how to use processing option parameters according to an embodiment. [Figure 17] Figure 17 shows how to use the neural network partial dependency reconstruction rule according to an embodiment. [Figure 18] Figure 18 shows the determination of a reconstruction rule based on a quantization index that represents quantized neural network parameters, according to an embodiment. [Figure 19] Figure 19 shows how to use the identification parameter according to an embodiment. [Figure 20] Figure 20 shows different versions of neural network encoding / decoding according to the embodiment. [Figure 21] Figure 21 shows two versions of delta coding of a neural network according to an embodiment, where the two versions differ in their weights and / or biases. [Figure 22] Figure 22 shows two versions of alternative delta coding for neural networks according to an embodiment, the two versions differing in the number of neurons or interconnections of neurons. [Figure 23] Figure 23 shows the coding of different versions of the neural network that use the compensating neural network portion according to the embodiment. [Figure 24a] Figure 24a shows one embodiment of a data stream having supplementary data according to an embodiment. [Figure 24b] Figure 24b shows an alternative embodiment of a data stream with supplementary data according to the embodiment. [Figure 25] Figure 25 shows one embodiment of a data stream having a sequence of control data portions. [Modes for carrying out the invention]

[0030] Furthermore, in the following explanation, elements with similar or equivalent functions are given the same or equivalent reference numbers, even if they are depicted in different diagrams.

[0031] The following description includes several details to provide a more overall description of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention can be carried out without these specific details. In other examples, well-known structures and devices are shown in block diagrams rather than in detail, so as not to obscure embodiments of the present invention. Furthermore, features of different embodiments described herein may be combined with each other unless otherwise specified.

[0032] The following description of embodiments of the present invention begins with a brief introduction and overview of the embodiments to explain their advantages and how they achieve these advantages.

[0033] Current work on coded representations of neural networks, such as those developed in the ongoing MPEG activities for NN compression, has found it beneficial to split a model bitstream representing the parameter tensors of multiple layers into smaller sub-bitstreams, i.e., layer bitstreams, each containing the coded representation of the parameter tensors of an individual layer. This can be generally useful when such model bitstreams need to be stored / loaded in a containerized context, or in application scenarios featuring parallel decoding / execution of NN layers.

[0034] The following sections describe various examples that can help achieve effective compression of neural networks (NNs) and / or improve access to the data representing the NN, thus leading to more effective transmission / updates of the NN.

[0035] To facilitate understanding of the following embodiments of the present application, the description begins with the presentation of possible encoders and decoders that can be adapted thereto, into which subsequent schematic embodiments of the present application may be incorporated.

[0036] Figure 1 shows a simple sketch example of an encoding / decoding pipeline using DeepCABAC, illustrating the internal workings of such a compression scheme. First, the weights 32, e.g., weights 321-326 of the connections 22, e.g., connections 221-226, between neurons 14, 20 and / or 18, e.g., between predecessor neurons 141-143 and intermediate neurons 201-202, e.g., weights 321-326, are formed into a tensor shown as matrix 30 in this example (step 1 in Figure 1). In step 1 in Figure 1, for example, the weights 32 associated with the first layer of the neural network 10, NN, are formed into matrix 30. According to the embodiment shown in Figure 1, the columns of matrix 30 are associated with predecessor neurons 141-143 and the rows of matrix 30 are associated with intermediate neurons 201-202, although it is clear that the formed matrix can alternatively represent the inversion of matrix 30 shown.

[0037] Next, each NN parameter, for example, weight 32, is encoded, e.g., quantized and entropy encoded, according to a specific scan order, e.g., row-major (left to right, top to bottom), using context adaptive arithmetic coding 600, as shown in steps 2 and 3. It is also possible to use a different scan order, i.e., a coding order, as outlined in more detail below. Steps 2 and 3 are performed by the encoder 40, i.e., the device for coding. The decoder 50, i.e., the device for decoding, follows the same process in the reverse order of steps. That is, first, as shown in step 4, it decodes a list of integer representations of the encoded values, and then, as shown in step 5, it reshapes that list into its tensor representation 30'. Finally, as shown in step 6, the tensor 30' is loaded into the network architecture 10', i.e., the reconstructed NN. The reconstructed tensor 30' constitutes the reconstructed NN parameters, i.e., the decoded NN parameters 32'.

[0038] The NN10 shown in Figure 1 is merely a simple neural network with a small number of neurons: 14, 20, and 18. Neurons are also understood below as nodes, elements, model elements, or dimensions. Furthermore, reference numeral 10 indicates a machine learning (ML) predictor, in other words, a machine learning model such as a neural network.

[0039] The neural network will be described in more detail with reference to Figure 2. In particular, Figure 2 shows an ML predictor 10 comprising an input interface 12 having input nodes or elements 14 and an output interface 16 having output nodes or elements 18. The input nodes / elements 14 receive input data. In other words, input data is applied thereto. For example, they can receive a picture in which each element 14 is associated with a pixel of the picture. Alternatively, the input data applied to the elements 14 may be a one-dimensional signal such as an audio signal or a sensor signal. Even more alternatively, the input data may represent a specific dataset, such as medical file data. The number of input elements 14 may be any number, for example, depending on the type of input data. The number of output nodes 18 may be one, as shown in Figure 1, or more than one, as shown in Figure 2. Each output node or element 18 may be associated with a specific inference or prediction task. In particular, when the ML predictor 10 is applied to an input applied to the input interface 12 of the ML predictor 10, the ML predictor 10 outputs an inference or prediction result at the output interface 16 that can indicate an answer to a question about the input data, such as whether or not the input image contains certain features such as a car, a person, or a phase, or to what extent such features are likely to be present.

[0040] In this regard, the inputs applied to the input interface can also be interpreted as activations, i.e., activations applied to each input node or element 14.

[0041] Between the input node 14 and the output node(s) 18, the ML predictor 10 includes further elements or nodes 20, which are connected to the predecessor nodes via connections 22 to receive activations from these predecessor nodes, and to the successor nodes via one or more further connections 24 to transfer the activations, i.e., activation values, of the node 20 to the successor nodes.

[0042] The precursor node may be another internal node 20 of the ML predictor 10, through which an intermediate node 20, as illustrated in Figure 2, may be indirectly connected to the input node 14, or directly connected to the input node 14, as shown in Figure 1. The successor node may be another intermediate node of the ML predictor 10, through which an intermediate node 20, as illustrated in Figure 2, may be connected to an output interface or output node, or directly connected to an output node 28, as shown in Figure 1.

[0043] The input node 14, output node 18, and internal node 20 of the ML predictor 10 may be associated with or assigned to a specific layer of the ML predictor 10, but the layered structure of the ML predictor 10 is optional, and the ML predictor to which embodiments of the present application apply is not limited to such a layered network. With respect to the exemplary intermediate node 20 of the ML predictor 10, it contributes to the inference or prediction task of the ML predictor 10 by transferring activations, i.e., activation values, from a precursor node received from the input interface 12 via connection 22 to a successor node via connection 24 toward the output interface 16. In doing so, the node or element 20 calculates its activation, i.e., activation value, which is transferred to the successor node via connection 24 based on the activation, i.e., activation value at the input node 22, and this calculation includes calculating a weighted sum, i.e., an addendum, for each connection 22 which is the product of the input received from each precursor node, i.e., its activation, and the weight associated with the connection 22 connecting each precursor node and the intermediate node 20. Alternatively or more generally, activation x is performed via a node or element i, 20 through connection 24, and a mapping function m ij Note that (x) is forwarded to the subsequent node j. Therefore, each connection 22, like connection 24, is also associated with some kind of weight, or alternatively, the mapping function m ijThe results may be obtained. Optionally, additional parameters may be involved in the calculation of the activation that node 20 outputs toward a particular subsequent node. To determine the relevance score for a portion of the ML predictor 10, an activation that occurs at the output node 18 when a certain prediction or inference task is completed for a certain input at the input interface 12, or a predefined or interesting output activation of interest, may be used. This activation at each output node 18 is used as a starting point for determining the relevance score, and the relevance is backpropagated toward the input interface 12. In particular, at each node of the ML predictor 10, such as node 20, the relevance score is distributed toward precursor nodes such as via connection 22 in the case of node 20, distributed in a manner proportional to the aforementioned product associated with each precursor node, and contributes to the activation of the current node, which is the backpropagated activation such as node 20, via a weighted sum. In other words, the relevance rate for backpropagation from a node such as node 20 to a certain precursor node can be calculated by multiplying the relevance of that node by a coefficient corresponding to the ratio of the activation received from the precursor node and the weight to which that activation contributed to the sum of each node mentioned above, and then dividing by a value corresponding to the sum of all products of the activations of the precursor node and the weights to which these activations contributed to the weighted sum of the current node to which the backpropagation is to be performed.

[0044] In the method described above, the relevance score for a part of the ML predictor 10 is determined based on the activation of these parts, for example, such that they appear in one or more inferences performed by the ML predictor. The “parts” on which such relevance scores are determined may be nodes or elements of the predictor 10, as described above, and again, it should be noted that the ML predictor 10 is not limited to any layered ML network, for example, element 20 may be any calculation of intermediate values, for example, that is calculated during inference or prediction performed by the predictor 10. For example, in the method discussed above, the relevance score of an element or node 20 is calculated by aggregating or summing the inbound relevance messages that this node or element 20 receives from subsequent nodes / elements, which then, in turn, distribute their relevance scores with respect to node 20 in the manner outlined above.

[0045] The ML predictor 10, i.e., the neural network (NN), as described in Figure 2, is encoded into a data stream 45 using the encoder 40 described in Figure 1, and then reconstructed / decoded from the data stream 45 using the decoder 50 described in Figure 1.

[0046] The features and / or functions described below can be implemented with respect to the compression scheme described in Figure 1 and are related to the neural networks described in respect to Figures 1 and 2.

[0047] 1. Serialization of a parameter tensor There are applications where bitstream sub-layer processing is effective. For example, there are neural networks (NNs) that adapt to the available computing power of the client, where the layers consist of independent subsets, such as separately trained baseline and advanced portions, and the client can decide whether to run only the baseline layer subset or additionally the advanced layer subset (Tao, 2018). Another example is a NN featuring data channel-specific operations, such as layers in an image processing NN that can perform operations separately and in parallel for each color channel (Chollet, 2016).

[0048] For the purposes described above, referring to Figure 3, the serialization 1001 or 1002 of the layer parameter tensor 30 requires a bit string 421 or 422 that can be easily divided into a contiguous subset 431 to 433, or 441 and 442, which are meaningful from an application standpoint, before entropy coding. This could include all NN parameters per channel 1001 or per sample 1002, for example, a grouping of weights 32, or a grouping of neurons in baseline versus advanced portion. Such a bit string can then be entropy coded to form sublayer bitstreams with functional relationships.

[0049] As shown in Figure 4, the serialization parameter 102 can be encoded into / decoded from the data stream 45. The serialization parameter can indicate how the NN parameters 32 are grouped before or during encoding. The serialization parameter 102 can indicate how the NN parameters 32 of the parameter tensor 30 are serialized into a bitstream to enable encoding of the NN parameters into the data stream 45.

[0050] In one embodiment, serialization information, i.e., serialization parameters 102, is shown in layers in the parameter set portion 110 of the bitstream, i.e., datastream 45; see, for example, Figures 12, 14a, 14b, or 24b.

[0051] Another embodiment signals the dimensions 341 and 342 of the parameter tensor 30 (see coding order 1061 in Figures 1 and 7) as serialization parameters 102. This information may be useful, for example, in an exemplary image processing NN where there is a clear relationship between the parameter matrix, i.e., the entries of the parameter tensor 30, i.e., the weights 32, and the samples 1002 and color channels 1001, as shown in Figure 3, when the decoded list of parameters should be grouped / organized in their respective ways, for example, in memory, to enable efficient execution. Figure 3 is an illustrative diagram of two different serialization modes 1001 and 1002 and the resulting sublayers 43 and 44.

[0052] In a further embodiment, as shown in Figure 4, the bitstream, or data stream 45, specifies during encoding the order in which the encoder 40 scanned the NN parameters 32, e.g., layers, neurons, and tensors, so that the decoder 50 can reconstruct the NN parameters 32 accordingly during decoding. See Figure 1 for a description of the encoder 40 and decoder 50. That is, different scanning orders 301 and 302 of the NN parameters 32 may be applied in different application scenarios.

[0053] For example, encoding parameters along different dimensions may benefit the resulting compression performance because the entropy encoder might be able to better capture the dependencies between them. In another example, it may be desirable to group parameters according to application-specific criteria, namely which parts of the input data they relate to or whether they can be performed jointly, so that they are decoded / input in parallel. A further example is encoding parameters according to a GEMM (General Matrix Matrix) product scan order that supports efficient memory allocation of decoded parameters when performing an inner product operation (Andrew Kerr, 2017).

[0054] Further examples relate to the encoder-side selective sorting of data, for example, shown by the coding order 1064 in Figure 7, in order to achieve energy compression of the NN parameters 32 to be encoded, and then process / serialize / encode the resulting sorted data according to the resulting order 104. The sorting may therefore also involve rearranging the NN parameters 32 so that they steadily increase or decrease along the coding order 104.

[0055] Figure 5 shows an example of a single-output channel convolutional layer, for example, for image and / or video analysis applications. A color image has multiple channels, typically one for each color channel such as red, green, and blue. From a data perspective, this means that one image provided as input to the model is actually three images.

[0056] Tensor 30a is applied to the input data 12 and can scan the input like a window with a constant step size. Tensor 30a is understood as a filter. Tensor 30a moves through the input data 12 from left to right, jumping to the next lower row after passing through each. Optional so-called padding determines how tensor 30a should behave when it encounters an edge of the input matrix. For each point in its field of view, tensor 30a has NN parameters 32, e.g., fixed weights, and calculates a result matrix from, for example, the pixel value in the current field of view and these weights. The size of this result matrix depends on the size of tensor 30a (kernel size), padding, and especially the step size. The input image has 3 channels (e.g., depth 3), and the tensor 30a applied to that image also has, for example, 3 channels (e.g., depth 3). Regardless of the depth of input 12 and tensor 30a, tensor 30a is applied to input 12 using an inner product operation that yields a single value.

[0057] By default, DeepCABAC transforms any given tensor 30a into the form of its respective matrix 30b and encodes the NN parameters 32 into the data stream 45 in row-major 1041, i.e., from left to right and top to bottom, as shown in Figure 5. However, other encoding orders 104 / 106 may be advantageous for achieving higher compression, as will be discussed with respect to Figure 7.

[0058] Figure 6 shows an example for a fully connected layer. A fully connected layer is a typical neural network structure where all neurons are connected to all inputs 12, i.e., precursor nodes, and all outputs 16', i.e., successor nodes. Tensor 30 represents the corresponding NN layer, and tensor 30 contains NN parameters 32. The NN parameters 32 are encoded into a data stream according to coding order 104. As will be explained with respect to Figure 7, a particular coding order 104 / 106 would be advantageous for achieving high compression.

[0059] Now, we return to Figure 4 to allow for a general explanation of the serialization of NN parameters 32. The concepts described in Figure 4 are applicable to both single-output channel convolutional layers (see Figure 5) and fully connected layers (see Figure 6).

[0060] As shown in Figure 4, Embodiment A1 of the present invention relates to a data stream 45 (DS) on which a representation of a neural network (NN) is encoded. The data stream includes serialization parameters 102 that indicate an encoding order 104 in which NN parameters 32 defining the neuronal interconnections of the neural network are encoded in the data stream 45.

[0061] According to Embodiment ZA1, the device for encoding a representation of a neural network into a data stream 45 is configured to provide the data stream 45 with serialization parameters 102 indicating an encoding order 104 in which NN parameters 32 defining the neuronal interconnections of the neural network are encoded into the data stream 45.

[0062] According to Embodiment XA1, the device for decoding a representation of a neural network from DS45 is configured to decode from the data stream 45 a serialization parameter 102 indicating an encoding order 104 in which NN parameters 32 defining the neuronal interconnections of the neural network are encoded in the data stream 45, and to assign the NN parameters 32 decoded in series from DS45 using the encoding order 104 to the neuronal interconnections.

[0063] Figure 4 shows different representations of an NN layer having NN parameters 32 associated with the NN layer. According to the embodiment, a two-dimensional tensor 301, i.e., a matrix, or a three-dimensional tensor 302 can represent the corresponding NN layer.

[0064] In the following, different features and / or functionalities are described in the context of data stream 45, but features and / or functionalities may also be features and / or functionalities of the apparatus according to embodiment ZA1 or the apparatus according to embodiment XA1.

[0065] According to Embodiment A2, the NN parameter 32 in the DS45 of the previous Embodiment A1 is encoded in the DS45 using context-adaptive arithmetic coding 600, as shown in Figures 1 and 8, for example. Therefore, the device according to Embodiment ZA1 can be configured to encode the NN parameter 32 using context-adaptive arithmetic coding 600, and the device according to Embodiment XA1 can be configured to decode the NN parameter 32 using context-adaptive arithmetic decoding.

[0066] According to Embodiment A3, in Embodiment A1 or A2, the data stream 45 of the DS45 is structured into one or more individually accessible parts 200, as shown in Figure 8 or one of the following figures, where each individually accessible part 200 represents a corresponding NN layer 210 of a neural network, and the serialization parameter 102 indicates an encoding order 104 in which NN parameters 32 defining the neuronal interconnections of the neural network within a given NN layer 210 are encoded in the data stream 45.

[0067] According to Embodiment A4, in any of the above embodiments A1 to A3, the serialization parameter 102 in the DS45 is an n-ary parameter that indicates coding order 104 from a set of n coding orders 108, as shown in Figure 7, for example.

[0068] According to Embodiment A4a, the set of n coding sequences 108 in the DS45 of Embodiment A4 includes the following: A predetermined coding order 1061 that differs in the order in which the dimensions of the tensor 30 describing a predetermined NN layer of the NN, for example, the x, y and / or z dimensions are traversed; and / or For scalable coding of NN, a second predetermined coding order 1062 in which a predetermined coding order 104 scans a predetermined NN layer of the NN a different number of times 107; and / or, A third predetermined coding order 1063 in which a predetermined coding order 104 scans the NN layer 210 of the NN in a different order; and / or, A fourth predetermined coding order 1064 in which the order in which the neurons 20 of the NN layer of the NN are scanned is different.

[0069] The first predetermined coding order 1061 differs from each other in how the individual dimensions of the tensor 30 are scanned when coding the NN parameters 32, for example. For example, coding order 1041 differs from coding order 1042. That is, the predetermined coding order 1041 is row-major, i.e., the tensor 30 is scanned row by row from left to right and top to bottom, while the predetermined coding order 1042 is column-major, i.e., the tensor 30 is scanned column by column from top to bottom and left to right. Similarly, the first predetermined coding order 1061 can differ from the predetermined coding order 104 in the order in which the dimensions of the 3-dimensional tensor 30 are scanned.

[0070] The second predetermined coding order 1062 differs in the frequency with which the NN layer, represented, for example by a tensor / matrix 30, is scanned. The NN layer can be scanned twice, for example, through the predetermined coding order 104, thereby allowing the baseline and advanced portions of the NN layer to be encoded into / decoded from the data stream 45. The number of times the NN layer is scanned by the predetermined coding order 107 defines the number of versions of the NN layer encoded into the data stream. Thus, in the case of a serialization parameter 102 indicating a coding order that scans the NN layer at least twice, the decoder is configured to determine, based on its processing capacity, which versions of the NN layer it can decode and to decode the NN parameter 32 corresponding to the selected NN layer version.

[0071] The third predetermined coding order 1063 defines whether NN parameters associated with different NN layers 2101 and 2102 of NN10 are coded into the data stream 45 using a different predetermined coding order or the same coding order as one or more other NN layers 210 of NN10.

[0072] The fourth predetermined coding order 1064 is used to represent the tensor / matrix 30 that represents the corresponding NN layer, from the upper left NN parameter 321 to the lower right NN parameter 32 12 It includes a predetermined encoding order 1043 that scans diagonally in a staggered pattern up to [a certain point].

[0073] According to Embodiment A4a, in any of the earlier Embodiments A1 to A4a, the serialization parameter 102 in the DS45 represents a permutation used by the coding order 104 to permutate the neurons of the NN layer relative to the default order. In other words, the serialization parameter 102 represents a permutation, and when using this permutation, the coding order 104 permutates the neurons of the NN layer relative to the default order. For a fourth predetermined coding order 1064, as shown in Figure 7, a row-major order as illustrated for the data stream 450 represents the default order. Other data streams 45 contain NN parameters encoded within them using a permutation relative to the default order.

[0074] According to Embodiment A4b, among the DS45 of Embodiment A4a, the permutation orders the neurons of the NN layer 210 such that the NN parameter 32 increases monotonically along the coding order 104, or decreases monotonically along the coding order 104.

[0075] According to Embodiment A4c, among the DS45 of Embodiment A4a, the permutation is such that the neurons of the NN layer 210 are permutated among a predetermined coding sequence 104 that can be signaled by the serialization parameter 102, such that the bit rate for encoding the NN parameter 32 into the data stream 45 is the lowest for the permutation indicated by the serialization parameter 102.

[0076] According to Embodiment A5, in any of the previous embodiments A1 to A4c, the NN parameters 32 include weights and biases.

[0077] According to Embodiment A6, in any of the earlier embodiments A1 to A5, the data stream 45 of the DS45 is structured into individually accessible subparts 43 / 44, each subpart 43 / 44 representing a corresponding NN portion of the neural network 10, for example, a portion of the NN layer 210, so that each subpart 43 / 44 is fully scanned by the coding order 104 before subsequent subparts 43 / 44 are scanned by the coding order 104. Rows, columns, or channels of the tensor 30 representing the NN layer may be encoded into individually accessible subparts 43 / 44. Different individually accessible subparts 43 / 44 associated with the same NN layer include different neurons 14 / 18 / 20 or neuron interconnections 22 / 24 associated with the same NN layer. The individually accessible subparts 43 / 44 represent rows, columns, or channels of the tensor 30. Individually accessible subparts 43 / 44 are shown, for example, in Figure 3. Alternatively, as shown in Figures 21 to 23, individually accessible sub-parts 43 / 44 represent different versions of the NN layer, such as the baseline and advanced parts of the NN layer.

[0078] According to Embodiment A7, in any of the DS45s of Embodiments A3 and A6, the NN parameter 32 is encoded into the DS45 using context-adaptive arithmetic coding 600 and context initialization at the beginning 202 of any individually accessible portion 200 or sub-part 43 / 44. See, for example, Figure 8.

[0079] According to Embodiment A8, in any of the DS45s of Embodiments A3 and A6, the data stream 45 is a parameter that indicates the start code 242 at which each individually accessible portion 200 or sub-portion 240 begins, and / or a pointer 220 / 244 pointing to the beginning of each individually accessible portion 200 or sub-portion 240, and / or a pointer data stream length, i.e., the data stream length 246 of each individually accessible portion 200 or sub-portion 240 for skipping each individually accessible portion 200 or sub-portion 240 when parsing the DS45, as shown in Figures 11 to 14.

[0080] Another embodiment specifies the bit size and numerical representation of the decoded parameter 32' in the bitstream, i.e., the data stream 45. For example, an embodiment may specify that the decoded parameter 32' can be represented in an 8-bit signed fixed-point format. This specification is very useful, for example, in applications where the activation value can also be represented in an 8-bit fixed-point representation, for example, as it allows for more efficient inference using fixed-point arithmetic.

[0081] Embodiment A9 further includes a numerical computation representation parameter 120 that indicates the numerical representation and bit size of the NN parameters 32 when the NN is used for inference, as in any of the previous embodiments A1 to A8. See, for example, Figure 9.

[0082] Figure 9 shows an embodiment B1 of a data stream 45 in which a representation of a neural network is encoded, and the data stream 45 includes numerical computation representation parameters 120 that indicate the numerical representations to be represented when the NN parameters 32 of the NN encoded in DS45 are used for inference, such as floating-point, fixed-point, and bit sizes.

[0083] The corresponding embodiment ZB1 relates to a device for encoding a representation of a neural network into a DS45, the device being configured such that the NN parameters 32 of the NN encoded into the DS45 provide a data stream 45 with numerical computation representation parameters 120 indicating the numerical representation to be represented, such as floating-point, fixed-point, and bit size, when the NN is used for inference.

[0084] A corresponding embodiment XB1 relates to a device for decoding a neural network representation from DS45. The device is configured to decode numerical computation representation parameters 120 from the data stream 45, which indicate the floating-point, fixed-point, and bit sizes of the numerical representation, for example, the NN parameters 32 of the NN encoded in DS45, when the NN is used for inference, and is configured to optionally use the numerical representation and bit size to represent the NN parameters 32 decoded from DS45.

[0085] In the following, different features and / or functionalities are described in the context of data stream 45, but features and / or functionalities may also be, in a similar or analogous way, features and / or functionalities of the apparatus according to embodiment ZB1 or the apparatus according to embodiment XB1.

[0086] Further embodiments signal the parameter types within a layer. In most cases, a layer consists of two types of parameters 32: weights and biases. Distinguishing between these two types of parameters can be beneficial before decoding, for example, when different types of dependencies are used during encoding or when parallel decoding is desired.

[0087] According to Embodiment A10, in any of the DS45s of the previous embodiments A1 to B1, the data stream 45 is structured into individually accessible sub-parts 43 / 44. Each sub-part 43 / 44 represents a corresponding NN part of a neural network, for example, a part of an NN layer, and each sub-part 43 / 44 is fully scanned by the coding sequence 104 before subsequent sub-parts 43 / 44 are scanned by the coding sequence 104. For a given sub-part, the data stream 45 includes a type parameter indicating the parameter type of the NN parameters 32 encoded in the given sub-part.

[0088] According to Embodiment A10a, the type parameter in the DS of Embodiment A10 identifies at least the NN weights and the NN bias.

[0089] Finally, in further embodiments, the type of layer 210 containing the NN parameters 32 is signaled, for example, convolutional or fully connected. This information may be useful, for example, to understand the meaning of the dimensions of the parameter tensor 30. For example, the weight parameters of a 2d convolutional layer may be expressed as a 4d tensor 30, where the first dimension is the number of filters, the second dimension is the number of channels, and the remainder specifies the 2d spatial dimension of the filters. Furthermore, different layers 210 may be treated differently during encoding (for example, by using different sets or modes of the context model) to better capture dependencies in the data and lead to higher encoding efficiency, and this information may be important for the decoder to know before decoding.

[0090] According to Embodiment A11, in any of the DS45s of the earlier embodiments A1 to A10a, the data stream 45 is structured into one or more individually accessible parts 200, each part 200 representing a corresponding NN layer 210 of the neural network 10, and the data stream 45 further includes an NN layer type parameter 130 indicating the NN layer type of a given NN layer for a given NN layer. See, for example, Figure 10.

[0091] Figure 10 shows an embodiment C1 of a data stream 45 in which a representation of a neural network is encoded, the data stream 45 being structured into one or more individually accessible parts 200, each part representing a corresponding NN layer 210 of the neural network, and the data stream 45 further includes an NN layer type parameter 130 indicating the NN layer type of a given NN layer of the NN for a given NN layer.

[0092] A corresponding embodiment ZC1 relates to a device for encoding a representation of a neural network into a DS45 such that the data stream 45 is structured into one or more individually accessible parts 200, each part 200 representing a corresponding NN layer 210 of the neural network, and the device is configured to provide the data stream 45 with an NN layer type parameter 130 indicating the NN layer type of the given NN layer 210 of the NN for a given NN layer 210.

[0093] A corresponding embodiment XC1 relates to a device for decoding a representation of a neural network from DS45, wherein the data stream 45 is structured into one or more individually accessible parts 200, each part 200 representing a corresponding NN layer 210 of the neural network. The device is configured to decode from the data stream 45, for a given NN layer 210, an NN layer type parameter indicating the NN layer type of a given NN layer 210 of the NN.

[0094] According to Embodiment A12, in any of the DS45s of Embodiments A11 and C1, the NN layer type parameter 130 is at least fully connected, NN layer 2101 reference, and convolutional layer type, NN layer 210 N Reference, to identify. Therefore, an apparatus according to embodiment ZC1 can encode the NN layer type parameter 130 to identify two layer types, and an apparatus according to embodiment XB1 can decode the NN layer type parameter 130 to identify two layer types.

[0095] 2. Bitstream Random Access 2.1 Layer Bitstream Random Access Accessing subsets of a bitstream is crucial in many applications, such as parallelizing layer processing or packaging bitstreams in various container formats. One state-of-the-art method for enabling such access is, for example, to break the coding dependencies after the parameter tensor 30 of each layer 210 and insert start codes into the model bitstream, i.e., the data stream 45, before each layer bitstream, e.g., individually accessible portions 200. In particular, inserting start codes within the model bitstream is not a good way to isolate layer bitstreams, as start code detection would potentially require analyzing the entire model bitstream from the beginning across a very large number of start codes.

[0096] This aspect of the present invention relates to further techniques for structuring the encoded model bitstream of the parameter tensor 30 in a state-of-the-art and superior manner, enabling easier, faster, and more appropriate access to bitstream portions, such as layer bitstreams, to facilitate applications requiring parallel or partial decoding and execution of neural networks.

[0097] In one embodiment of the present invention, the model bitstream, i.e., individual layer bitstreams within the data stream 45, for example, individually accessible portions 200, are indicated through the bitstream position in bytes or offsets (e.g., byte offsets relative to the beginning of the encoding unit) within the parameter set / header portion 47 of the bitstream having a model range. Figures 11 and 12 illustrate this embodiment. Figure 12 shows layer access via the bitstream position or offset indicated by the pointer 220. Furthermore, each individually accessible portion 200 can optionally constitute a layer parameter set 110, to which one or more of the aforementioned parameters can be encoded and decoded.

[0098] According to Embodiment A13, the data stream 45 of the DS45 in any of the previous embodiments A1 to A12 is structured into individually accessible portions 200. Each portion 200 represents a corresponding NN portion, for example, one or more NN layers or a portion of an NN layer of a neural network, and the data stream 45 includes, for each of the one or more predetermined individually accessible portions 200, a pointer 220 that points to, for example, the beginning of each individually accessible portion 200. Refer to Figure 11 or Figure 12 when the individually accessible portion represents a corresponding NN layer. Refer to Figures 13 to 15 when the individually accessible portion represents a portion of a predetermined NN layer, for example, an individually accessible sub-portion 240. Hereinafter, the pointer 220 may also be denoted by reference numeral 244.

[0099] For each NN layer, the individually accessible portion 200 associated with that NN layer represents the corresponding NN portion of that NN layer. In this case, in this section and the following description, such individually accessible portion 200 is understood as an individually accessible sub-part 240.

[0100] Figure 11 shows a more general embodiment D1, in which a data stream 45 on which a representation of a neural network is encoded is structured into individually accessible portions 200, each individually accessible portion 200 representing a corresponding NN portion of the neural network, for example, one or more NN layers or portions of NN layers, and for each of the one or more predetermined individually accessible portions 200, the data stream 45 includes a pointer 220 pointing to the beginning of each predetermined individually accessible portion 200.

[0101] According to one embodiment, pointer 220 indicates an offset relative to the beginning of the first individually accessible portion 2001. A first pointer 2201 pointing to the first individually accessible portion 2001 indicates no offset. Therefore, the first pointer 2201 can be omitted. Alternatively, pointer 220 indicates an offset relative to, for example, the end of the parameter set to which pointer 220 is encoded.

[0102] A corresponding embodiment ZD1 relates to a device for encoding a representation of a neural network into a DS45 such that the data stream 45 is structured into one or more individually accessible portions 200. Each portion 200 represents a corresponding NN portion of the neural network, for example, one or more NN layers or a portion of NN layers. The device is configured to provide the data stream 45 with a pointer 220 that points to the beginning of each of the one or more predetermined individually accessible portions 200.

[0103] Corresponding embodiment XD1 relates to a device for decoding a representation of a neural network from DS45, wherein the data stream 45 is structured into one or more individually accessible portions 200, each portion 200 representing a corresponding NN portion of a neural network, for example, one or more NN layers or a portion of NN layers, and the device decodes from the data stream 45, for each of the one or more predetermined individually accessible portions 200, a pointer 220 pointing to the beginning of each predetermined individually accessible portion 200, and is configured to use one or more of the pointers 220 to access DS45, for example.

[0104] According to Embodiment A14, in either Embodiment A13 or D1, the individually accessible portion 200 of the DS45 represents the following: The corresponding NN layer 210 of the neural network, or This is the neural network portion of the NN layer 210 of the NN. For example, please refer to Figure 3 or any one of Figures 21 to 23.

[0105] 2.2 Sublayer Bitstream Random Access As discussed in Section 1, there are applications that rely on grouping the parameter tensor 30 within layer 210 in a specific configurable way, since it is beneficial to decode / process / infer partially or in parallel. Thus, sub-layer unit access to the layer bitstream, e.g., individually accessible portions 200, can be useful for accessing desired data in parallel or for omitting unnecessary data portions.

[0106] In one embodiment, the coding dependencies within the layer bitstream are reset with sublayer precision, i.e., the DeepCABAC probabilistic state is reset.

[0107] In another embodiment of the present invention, individual sub-layer bitstreams within a layer bitstream, i.e., an individually accessible portion 200, i.e., individually accessible sub-portions 240, are indicated together with the range of a layer or model through a byte-level bitstream position, e.g., pointer 244, or offset, e.g., pointer 244, within the parameter set portion 110 of the bitstream, i.e., the data stream 45. Figures 13, 14a, and 15 illustrate embodiments. Figure 14a shows sub-layer access by relative bitstream position or offset, i.e., access to an individually accessible sub-portion 240. Furthermore, for example, an individually accessible portion 200 can also be accessed by a layer-level pointer 220. The layer-level pointer 220 is encoded, for example, in the model parameter set 47 of DS45, i.e., the header. The pointer 220 points to an individually accessible portion 200 representing the corresponding NN portion, including the NN layers of the NN. The sub-layer-level pointer 244 is encoded, for example, in the layer parameter set 110 of an individually accessible portion 200 representing the corresponding NN portion, including the NN layers of the NN. Pointer 244 points to the beginning of an individually accessible sub-part 240 that represents the corresponding NN portion, which includes a part of the NN layer of the NN.

[0108] According to one embodiment, a layer-level pointer 220 indicates an offset relative to the beginning of a first individually accessible portion 2001. A sub-layer-level pointer 244 indicates an offset of an individually accessible sub-part 240 of an individually accessible portion 200 relative to the beginning of a first individually accessible sub-part 240 of that individually accessible portion 200.

[0109] According to one embodiment, pointers 220 / 244 indicate a byte offset relative to a set unit containing multiple units. Pointers 220 / 244 indicate a byte offset from the beginning of the set unit to the beginning of a unit within the payload of the set unit.

[0110] In another embodiment of the present invention, individual sub-layer bitstreams within a layer bitstream, i.e., individually accessible portion 200, i.e., individually accessible sub-parts 240, are indicated via a discoverable start code 242 within a bitstream, i.e., a data stream 45, which would be sufficient since the amount of data per layer is typically less than if the layer were discovered by the start code 242 within the entire model bitstream, i.e., data stream 45. Figures 13 and 14b illustrate this embodiment. Figure 14b illustrates the use of the start code 242 at the sub-layer level, i.e., for each individually accessible sub-part 240, and the bitstream location, i.e., pointer 220, at the layer level, i.e., for each individually accessible portion 200.

[0111] In another embodiment, the run length of the (sub)layer bitstream portion, the individually accessible sub-portion 240, i.e., the data stream length 246, is indicated in the parameter set / header portion 47 of the bitstream 45 or the parameter set portion 110 of the individually accessible portion 200, in order to facilitate the extraction of the portion, i.e., the individually accessible sub-portion 240, for the purpose of packaging it into a suitable container. As shown in Figure 13, the data stream length 246 of the individually accessible sub-portion 240 is indicated by the data stream length parameter.

[0112] Figure 13 shows an embodiment E1 of a data stream 45 in which a representation of a neural network is encoded, the data stream 45 being structured into one or more individually accessible portions 200, each individually accessible portion 200 representing a corresponding NN layer of the neural network, the data stream 45 being further structured into individually accessible sub-parts 240 within a given portion, for example within an individually accessible portion 200, each sub-part 240 representing a corresponding NN portion of each NN layer of the neural network, and for each of the one or more predetermined individually accessible sub-parts 240, the data stream 45 includes the following: A start code 242 from which each predetermined individually accessible sub-part 240 begins, and / or, A pointer 244 that points to the beginning of each predetermined individually accessible sub-part 240, and / or A data stream length parameter indicating the data stream length 246 of each predetermined individually accessible sub-part 240, for skipping each predetermined individually accessible sub-part 240 when analyzing DS45.

[0113] The individually accessible sub-parts 240 described herein may have the same or similar features and / or functionality as those described with respect to the individually accessible sub-parts 43 / 44.

[0114] All individually accessible sub-parts 240 within the same predetermined portion have the same data stream length 246, so that a data stream length parameter indicates a single data stream length 246, and that data stream length 246 is applicable to each individually accessible sub-part 240 within the same predetermined portion. The data stream length parameter indicates the data stream length 246 of all individually accessible sub-parts 240 of the entire data stream 45, or the data stream length parameter indicates the data stream length 246 of all individually accessible sub-parts 240 of each individually accessible portion 200. One or more data stream length parameters are encoded in the header portion 47 of the data stream 45, or in the parameter set portion 110 of each individually accessible portion 200.

[0115] The corresponding embodiment ZE1 relates to a device for encoding a representation of a neural network into a DS45, wherein the data stream 45 is structured into one or more individually accessible portions 200. Each individually accessible portion 200 represents a corresponding NN layer of the neural network, and the data stream 45 is further structured into a predetermined portion, for example, an individually accessible portion 200, into individually accessible sub-parts 240, each sub-part 240 representing a corresponding NN portion of each NN layer of the neural network, and the device is configured to supply the following to each of the one or more predetermined individually accessible sub-parts 240 of the data stream 45: A start code 242 from which each predetermined individually accessible sub-part 240 begins, and / or, A pointer 244 that points to the beginning of each predetermined individually accessible sub-part 240, and / or A data stream length parameter indicating the data stream length 246 of each predetermined individually accessible sub-part 240, for skipping each predetermined individually accessible sub-part 240 when analyzing DS45.

[0116] Another corresponding embodiment XE1 relates to a device for decoding a representation of a neural network from DS45, wherein the data stream 45 is structured into one or more individually accessible portions 200, each individually accessible portion 200 representing a corresponding NN layer of the neural network. The data stream 45 is further structured into individually accessible sub-parts 240 within a predetermined portion, for example, an individually accessible portion 200, each sub-part 240 representing a corresponding NN portion of each NN layer of the neural network, and the device is configured to decode from the data stream 45 for each of the one or more predetermined individually accessible sub-parts 240 as follows: A start code 242 from which each predetermined individually accessible sub-part 240 begins, and / or, A pointer 244 that points to the beginning of each predetermined individually accessible sub-part 240, and / or A data stream length parameter indicating the data stream length 246 of each predetermined individually accessible sub-part 240, for skipping each predetermined individually accessible sub-part 240 when analyzing DS45. Then, for example, one or more predetermined individually accessible sub-parts 240 are configured to use this information, such as a start code 242, a pointer 244, and / or a data stream length parameter, to access the DS45.

[0117] According to Embodiment E2, in the DS45 of Embodiment E1, the data stream 45 encodes the representation of the neural network using context-adaptive arithmetic coding and context initialization at the starting point of each individually accessible portion 200 and each individually accessible sub-portion 240. See, for example, Figure 8.

[0118] According to Embodiment E3, the data stream 45 of Embodiment E1 or Embodiment E2 is by any other embodiment described herein. It is also clear that the apparatus of Embodiments ZE1 and XE1 can be completed by any other features and / or functions described herein.

[0119] 2.3 Types of Bitstream Random Access Depending on the type of (sub)layer 240 resulting from the selected serialization type, for example, serialization types 1001 and 1002 shown in Figure 3, various processing options are available to determine whether and how the client will access the (sub)layer bitstream 240. For example, if the selected serialization 1001 makes the sublayer 240 image color channel specific, thereby enabling parallelization of decoding / inference on a data channel basis, this should be shown in the bitstream 45 to the client. Another example is the derivation of preliminary results from a baseline NN subset that can be decoded / inferred independently of an advanced NN subset of a particular layer / model, as described with respect to Figures 20 to 23.

[0120] In one embodiment, the parameter set / header 47 within the bitstream 45, which comprises the entire model and a range of one or more layers, indicates the type of (sub)layer random access to enable the client to make appropriate decisions. Figure 15 shows two exemplary types of random access 2521 and 2522 determined by serialization. The exemplary types of random access 2521 and 2522 represent possible processing options for individually accessible portions 200 that represent the corresponding NN layers. The first processing option 2521 indicates data channel-level access to the NN parameters within the individually accessible portion 2001, and the second processing option 2522 indicates sample-level access to the NN parameters within the individually accessible portion 2002.

[0121] Figure 16 shows a typical embodiment F1 of a data stream 45 in which a representation of a neural network is encoded, the data stream 45 being structured into individually accessible portions 200, each individually accessible portion 200 representing a corresponding NN portion, for example, containing one or more NN layers or a portion of the NN layers of a neural network, and for each of the one or more predetermined individually accessible portions 200, the data stream 45 includes processing option parameters 250 indicating one or more processing options 252 that must be used or optionally used when using the NN for inference.

[0122] A corresponding embodiment ZF1 relates to a device for encoding a representation of a neural network into a DS45, wherein the data stream 45 is structured into individually accessible portions 200, each individually accessible portion 200 representing a corresponding NN portion, for example, containing one or more NN layers or a portion of the NN layers of a neural network, and the device is configured to supply to the data stream 45, for each of one or more predetermined individually accessible portions 200, processing option parameters 250 indicating one or more processing options 252 that must be used when using NN for inference, or that can be optionally used.

[0123] Another corresponding embodiment XF1 relates to a device for decoding a representation of a neural network from DS45, wherein the data stream 45 is structured into individually accessible portions 200, each individually accessible portion 200 representing a corresponding NN portion, for example, containing one or more NN layers of a neural network, or a portion of an NN layer, and the device is configured to decode from the data stream 45 a processing option parameter 250 indicating one or more processing options 252 that must be used or optionally used when using the NN for inference, for each of one or more predetermined individually accessible portions 200, for example, the device is configured to decode, based on the latter, which portions of one or more predetermined individually accessible portions to access, skip, and / or decode. Based on one or more processing options 252, the device is configured to determine how and / or which individually accessible portions or individually accessible sub-parts can be accessed, skip, and / or decoded.

[0124] According to Embodiment F2 of Embodiment F1 of DS45, the processing option parameter 250 indicates one or more available processing options 252 from a predetermined set of processing options, which include the following: The parallel processing capability of each predetermined individually accessible portion 200, and / or Each predetermined individually accessible portion 200 has a sample-level parallel processing capability of 2521, and / or Each predetermined individually accessible portion 200 has a channel-level parallel processing capability 2522, and / or Each predetermined individually accessible portion 200 has a classification category unit parallel processing capability, and / or As shown in Figures 20 to 23, the dependency of a neural network portion, such as an NN layer, on computation results obtained from another individually accessible portion of DS belonging to a different version of the NN that is encoded in a layered manner in DS, although related to the same neural network portion.

[0125] The device according to embodiment ZF1 is configured to encode the processing option parameter 250 such that the processing option parameter 250 indicates one or more processing options from a predetermined set of processing options. The device according to embodiment XF1 is configured to decode the processing option parameter 250 which indicates one or more processing options from a predetermined set of processing options.

[0126] 3. Signaling of Quantization Parameters The layer payload, for example, NN parameters 32 encoded in individual accessible parts 200, or the sublayer payload, for example, NN parameters 32 encoded in individual accessible subparts 240, may include different types of parameters 32 that represent rational numbers, such as weights and biases.

[0127] In a preferred embodiment shown in Figure 18, one such type of parameter is signaled as an integer value in the bitstream so that the reconstructed value, i.e., the reconstructed NN parameter 32', is derived by applying a reconstruction rule 270 to these values, i.e., the quantization index 32'', which includes the reconstruction parameter. For example, such a reconstruction rule 270 may consist of multiplying each integer value, i.e., the quantization index 32'', by the associated quantization step size 263, where the quantization step size 263 is the reconstruction parameter in this case.

[0128] In a preferred embodiment, the reconstruction parameters are signaled either in the model parameter set 47, in the layer parameter set 110, or in the sublayer header 300.

[0129] In another preferred embodiment, a first set of reconstruction parameters is signaled within a model parameter set, optionally a second set of reconstruction parameters is signaled within a layer parameter set, and optionally a third set of reconstruction parameters is signaled within a sublayer header. If present, the second set of reconstruction parameters depends on the first set of reconstruction parameters. If present, the third set of reconstruction parameters may depend on the first and / or second set of reconstruction parameters. This embodiment is described in more detail with respect to Figure 17.

[0130] TIFF2026113502000002.tif38167

[0131] In a preferred embodiment, s=2 -0.5 That is the case.

[0132] A rational number s may be encoded, for example, as a floating-point value. The first and second integer values ​​x1 and x2 may be signaled using a fixed or variable number of bits to minimize the overall signal cost. For example, if the quantization step sizes of the sublayers of the layer are similar, the associated values ​​x2 may be rather small integers, and it may be efficient to allow only a few bits to signal them.

[0133] In a preferred embodiment, as shown in Figure 18, the reconstruction parameters consist of a codebook, i.e., a quantization index-reconstruction level mapping which is a list of mappings between integers and rational numbers. The integer value W is entered into bitstream 45. n The relevant parameters of the layer or sublayer payload, encoded as follows, are reconstructed using the following reconstruction rule 270: Each integer value W n This is searched in the codebook. The related integer is W n One matching mapping is selected, and the associated rational number becomes the reconstructed value, i.e., the reconstructed NN parameter 32'.

[0134] In another preferred embodiment, the first and / or second and / or third sets of reconfiguration parameters each consist of codebooks according to the previously preferred embodiment. However, in order to apply the reconfiguration rules, a single joint codebook is derived by creating a set union of the codebook mappings for the first and / or second and / or third sets of reconfiguration parameters. Where mappings with the same integers exist, the codebook mappings for the third set of reconfiguration parameters take precedence over the codebook mappings for the second set of reconfiguration parameters, and the codebook mappings for the second set of reconfiguration parameters take precedence over the codebook mappings for the first set of reconfiguration parameters.

[0135] Figure 17 shows an embodiment G1 of a data stream 45 having encoded NN parameters 32 within it, which represents a neural network 10, where the NN parameters 32 are encoded in DS45 in a method 260 which is quantized to a quantization index, and the NN parameters 32 of different NN parts of NN10 are encoded in DS45 in a different way 260, where for each NN part, DS45 shows a reconstruction rule 270 for dequantizing the NN parameters associated with each NN part.

[0136] For example, each part of a neural network (NN) includes interconnections between nodes of the NN, while different parts of the NN include different interconnections between nodes of the NN.

[0137] According to the embodiment, the NN portion includes NN layers 210 of NN10 and / or layer sub-parts 43 into which a given NN layer of NN is subdivided. As shown in Figure 17, all NN parameters 32 in one layer 210 of NN represent an NN portion of NN that is quantized 260 such as the NN parameters 32 in the first layer 2101 of NN10. It is also possible that the NN parameters 32 in NN layer 2101 are grouped into different layer sub-parts 43, i.e., individually accessible sub-parts, each group potentially representing an NN portion. Thus, different layer sub-parts 43 of NN layer 2101 are quantized 260 differently.

[0138] A corresponding embodiment ZG1 relates to a device for encoding NN parameters 32 representing a neural network 10 into a DS45. The NN parameters 32 are encoded into the DS45 in a method 260 quantized to a quantization index. The NN parameters 32 are encoded into the DS45 such that the NN parameters 32 of different NN parts of NN10 are quantized differently 260. The device is configured to provide, for each NN part, a DS45 indicating a reconstruction rule for dequantizing the NN parameters 32 associated with each NN part. Optionally, the device may also perform the quantization 260.

[0139] Another corresponding embodiment XG1 relates to a device for decoding NN parameters 32 representing a neural network 10 from a data stream 45. The NN parameters 32 are encoded in the DS45 in a method 260 quantized to a quantization index. The NN parameters 32 are encoded in the DS45 such that the NN parameters 32 of different NN parts of NN10 are quantized differently 260. The device is configured to decode a reconstruction rule 270 from the data stream 45 for dequantizing the NN parameters 32 associated with each NN part for each NN part. Optionally, the device may also perform dequantization using the reconstruction rule 270 associated with the NN part to which the currently dequantized NN parameters 32 belong. The device may also be configured to dequantize the NN parameters of each NN part for each NN part using the decoded reconstruction rule 270 associated with each NN part.

[0140] The following describes different features and / or functionalities in the context of data stream 45, but features and / or functionalities can also be features and / or functionalities of the apparatus according to embodiment ZG1 or embodiment XG1, similarly or in a similar manner.

[0141] As already described above, in the DS45 of embodiment G1, according to embodiment G2, the NN portion includes the NN layer 210 of NN10 and / or a subdivided layer portion of a predetermined NN layer 210 of NN10.

[0142] According to Embodiment G3, in Embodiment G1 or G2, the DS45 has a first reconstruction rule 2701 encoded therein, which is delta-coded with respect to a second reconstruction rule 2702 for inversely quantizing the NN parameters 32 related to the second NN portion 260, and then inversely quantized with respect to the first reconstruction rule 2701 for inversely quantizing the NN parameters 32 related to the first NN portion. Alternatively, as shown in Figure 17, the first reconstruction rule 270a1 for inversely quantizing the NN parameters 32 related to the first NN portion, i.e., the layer sub-part 431, is delta-coded with respect to a second reconstruction rule 270a2 related to the second NN portion, i.e., the layer sub-part 432, and then encoded into the DS45. It is also possible that the first reconstruction rule 270a1 for inversely quantizing the NN parameters 32 related to the first NN portion, i.e., the layer sub-part 431, is delta-coded with respect to a second reconstruction rule 2702 related to the second NN portion, i.e., the NN layer 2102, and then encoded into the DS45.

[0143] In the following embodiments, to avoid ambiguity of the embodiments, the first reconstruction rule will be denoted as 2701 and the second reconstruction rule as 2702. However, in the following embodiments as well, it is clear that the first reconstruction rule and / or the second reconstruction rule can correspond to the NN portion representing the layer sub-part 43 of the NN layer 210, as described above.

[0144] According to Embodiment G4, in Embodiment G3, DS45 includes a first index value for representing the first reconstruction rule 2701 and a second index value for representing the second reconstruction rule 2702. The first reconstruction rule 2701 is defined by a first quantization step size defined by a predetermined basis exponent and a first exponent defined by the first exponent value, and The second reconstruction rule 2702 is defined by a second quantization step size defined by a given basis exponent and a second exponent defined by the sum of the first and second exponent values.

[0145] According to Embodiment G4a, among the DS of Embodiment G4, DS45 further represents a predetermined base.

[0146] According to Embodiment G4', among the DS of any of the above embodiments G1 to G3, DS45 includes a first exponential value for indicating a first reconstruction rule 2701 for inverse quantizing the NN parameters 32 related to the first NN portion, and a second exponential value for indicating a second reconstruction rule 2702 for inverse quantizing the NN parameters 32 related to the second NN portion. The first reconstruction rule 2701 is defined by a first quantization step size defined by a predetermined basis exponent and a first exponent defined by the sum of the first exponent value and the predetermined exponent value, The second reconstruction rule is defined by a second quantization step size, which is defined by a given basis exponent and a second quantization step size defined by the sum of the second exponent value and the given exponent value.

[0147] According to Embodiment G4', in Embodiment G4'a, the DS further represents a predetermined base.

[0148] According to Embodiment G4'b, in Embodiment G4'a, DS represents a predetermined basis related to the range of NN, i.e., the entire NN.

[0149] According to embodiment G4'c, among the DS of any of the earlier embodiments from embodiment G4' to G4'b, DS45 further exhibits a predetermined index value.

[0150] According to Embodiment G4'd, in Embodiment G4'c, DS45 shows a predetermined index value for a predetermined NN layer 210 to which the NN layer range, i.e., the first NN portion 431 and the second NN portion 432, belong.

[0151] According to embodiment G4'e, in any of the earlier embodiments G4'c and G4'd, DS45 further indicates a predetermined base, and DS45 indicates a predetermined exponential value in a range finer than the range indicated by DS45.

[0152] According to embodiment G4f, among the DS45s of the preceding embodiments G4 to G4a or G4' to G4'e, the DS45 has a predetermined base encoded therein in a non-integer format, such as a floating-point, rational, or fixed-point number, and first and second exponent values ​​in an integer format, such as signed integers. Optionally, the predetermined exponent values ​​may also be encoded in the DS45 in integer format.

[0153] According to Embodiment G5, among the DS of any of Embodiments G3 to G4f, DS45 includes a first parameter set that defines a first quantization index-reconstruction level mapping to represent a first reconstruction rule 2701, and a second parameter set that defines a second quantization index-reconstruction level mapping to represent a second reconstruction rule 2702. The first reconstruction rule 2701 is defined by the first quantization index-reconstruction level mapping, The second reconstruction rule 2702 is defined by extending the first quantization index-reconstruction level mapping in a predetermined manner with the second quantization index-reconstruction level mapping.

[0154] According to Embodiment G5', the DS45 described in any of Embodiments G3 to G5 includes a first parameter set that defines a first quantization index-reconstruction level mapping to represent a first reconstruction rule 2701, and a second parameter set that defines a second quantization index-reconstruction level mapping to represent a second reconstruction rule 2702. The first reconstruction rule 2701 is defined by extending a predetermined quantization index-reconstruction level mapping in a predetermined manner by the first quantization index-reconstruction level mapping, and The second reconstruction rule 2702 is defined by extending a given quantization index-reconstruction level mapping in a predetermined manner by a second quantization index-reconstruction level mapping.

[0155] According to Embodiment G5'a, in Embodiment G5', DS45 further indicates a predetermined quantization index-reconstruction level mapping.

[0156] According to Embodiment G5'b, in Embodiment G5'a, DS45 indicates a predetermined quantization index-reconstruction level mapping in the NN range, i.e., relating to the entire NN, or in the NN layer range, i.e., relating to a predetermined NN layer 210 to which the first NN portion 431 and the second NN portion 432 belong. The predetermined quantization index-reconstruction level mapping may be indicated in the NN scope if it is an NN portion representing an NN layer. For example, for each NN portion, each NN portion represents a corresponding NN layer, and for example, the first NN portion represents a different NN layer from the second NN portion. However, it is also possible to indicate the predetermined quantization index-reconstruction level mapping in the NN range if at least some of the NN portions represent a layer sub-part 43. Furthermore, or alternatively, in the case of an NN portion representing a layer sub-part 43, the predetermined quantization index-reconstruction level mapping may be indicated in the NN layer range.

[0157] According to embodiment G5c, according to a predetermined configuration of the DS45 from any of the previous embodiments G5 or G5' to G5'b, The mapping of each index value, i.e., quantization index 32'', to the first reconstruction level, according to the extended quantization index-reconstruction level mapping, is superseded by the mapping of each index value to the second reconstruction level, if such a mapping exists, according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, and / or According to the extended quantization index-reconstruction level mapping, the reconstruction level to which each index value should be mapped is not defined, and according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, for any index value that maps to the corresponding reconstruction level, the mapping from each index value to the corresponding reconstruction level is adopted, and / or, According to the extended quantization index-reconstruction level mapping, the reconstruction level to which each index value should be mapped is not defined, and according to the extended quantization index-reconstruction level mapping, for any index value that is mapped on the corresponding reconstruction level, the mapping from each index value to the corresponding reconstruction level is adopted.

[0158] According to Embodiment G6 shown in Figure 18, among the DS45s of any of the above embodiments G1 to G5c, the DS45 includes, for example, the following to represent an NN layer or to include a layer sub-part of an NN layer in order to show a predetermined NN portion reconstruction rule 270. The quantization step size parameter 262 indicates a quantization step size of 263, and A parameter set 264 defines the quantization index-reconstruction level mapping 265. Here, the reconstruction rule 270 for a given NN portion is: A quantization step size 263 for a quantization index 32'' within a predetermined index interval 268, and Quantization index-reconstruction level mapping 265 for quantization index 32'' outside a predetermined index interval 268, Defined by:

[0159] Figure 18 shows an embodiment H1 of a data stream 45 in which the NN parameters 32 representing the neural network are encoded. Here, the NN parameter 32 is encoded in DS45 in a quantized form on the quantization index 32'', Here, DS45 includes the following to show the reconstruction rule 270 for dequantizing the NN parameter 280, i.e., the quantization index 32'': Quantization step size parameter 262, which indicates a quantization step size of 263, A parameter set 264 defines the quantization index-reconstruction level mapping 265. Here, the reconstruction rule 270 for a given NN portion is: A quantization step size 263 for a quantization index 32'' within a predetermined index interval 268, Defined by a quantization index-reconstruction level mapping 265 for quantization indices 32'' outside a predetermined index interval 268.

[0160] A corresponding embodiment ZH1 relates to a device for encoding NN parameters 32 representing a neural network into a DS45, wherein the NN parameters 32 are quantized to a quantization index 32'' and encoded into the DS45, and the device is configured to provide the DS45 with the following to indicate a reconstruction rule 270 for dequantizing the NN parameters 32: Quantization step size parameter 262, which indicates a quantization step size of 263, A parameter set 264 defines the quantization index-reconstruction level mapping 265. Here, the reconstruction rule 270 for a given NN portion is: A quantization step size 263 for a quantization index 32'' within a predetermined index interval 268, This is defined by a quantization index-reconstruction level mapping 265 for quantization indices 32'' outside a predetermined index interval 268.

[0161] Another corresponding embodiment XH1 relates to an apparatus for decoding NN parameters 32 representing a neural network from DS45, wherein the NN parameters 32 are quantized to a quantization index 32'' and encoded in DS45, and the apparatus Quantization step size parameter 262, which indicates a quantization step size of 263, By decoding the parameter set 264 that defines the quantization index-reconstruction level mapping 265 from DS45, the reconstruction rule 270 for dequantizing the NN parameters, i.e., the quantization index 32'', can be derived from DS45. Here, the reconstruction rule 270 for a given NN portion is: A quantization step size 263 for a quantization index 32'' within a predetermined index interval 268, This is defined by a quantization index-reconstruction level mapping 265 for quantization indices 32'' outside a predetermined index interval 268.

[0162] The following describes different features and / or functionalities in the context of data stream 45, but the features and / or functionalities may be the features and / or functionalities of the apparatus according to embodiment ZH1 or embodiment XH1, similarly or in a similar manner.

[0163] According to Embodiment G7, the predetermined index interval 268 of the DS45 in either Embodiment G6 or H1 includes zero.

[0164] According to Embodiment G8, in the DS45 of Embodiment G7, a predetermined index interval 268 extends up to a predetermined size threshold y, and quantization indices 32'' exceeding the predetermined size threshold y represent an escape code that signals that the quantization index-reconstruction level mapping 265 is used for inverse quantization 280.

[0165] According to Embodiment G9, the parameter set 264 of the DS45 in any of the previous embodiments G6 to G8 defines a quantization index-reconstruction level mapping 265 by a list of reconstruction levels associated with quantization indices 32'' outside a predetermined index interval 268.

[0166] According to Embodiment G10, in any of the DS45s from Embodiments G1 to G9, the NN portion includes one or more sub-parts of the NN layers of the NN and / or one or more NN layers of the NN. Figure 18 shows an example of an NN portion that includes one NN layer of the NN. The NN parameter tensor 30, which includes the NN parameters 32, represents the corresponding NN layer.

[0167] According to Embodiment G11, in any of the DS45s of the previous embodiments G1 to G10, the data stream 45 is structured into individually accessible parts, and each individually accessible part has NN parameters 32 encoded therein for the corresponding NN part. See, for example, Figure 8 or one of Figures 10 to 17.

[0168] According to embodiment G12, the individually accessible portions of DS45 in G11 are encoded at the starting point of each individually accessible portion using context-adaptive arithmetic coding and context initialization, as shown in Figure 8, for example.

[0169] According to Embodiment G13, in any of the previous Embodiments G11 or G12, the data stream 45 is, for example, shown in one of Figures 11 to 15, with respect to each individually accessible portion, Each individually accessible section begins with starting code 242, and / or Pointers 220 / 244 that point to the beginning of each individually accessible portion, and / or This includes a data stream length parameter 246 that indicates the data stream length of each individually accessible portion, for skipping each individually accessible portion when analyzing DS45.

[0170] According to Embodiment G14, in any of the DS45s of Embodiments G11 to G13 above, the data stream 45 has a reconstruction rule 270 for inverse quantization 280 of the NN parameters 32 associated with each NN portion for each NN portion. 47, the main header section of DS45 concerning the entire NN, The NN layer-related header portion 110 of DS45 associated with the NN layer 210 to which each NN portion belongs, or The NN portion-specific header portion 300 of DS45 related to each NN portion, for example, if the NN portion represents a layer sub-part of the NN layer 210, i.e., individually accessible sub-parts 43 / 44 / 240, then it is a part of that. It will be shown inside.

[0171] According to Embodiment G15, among the DS45s of any prior embodiments G11 to G14, the DS45 is from any prior embodiment A1 to F2.

[0172] 4. Identifiers that depend on parameter hashing In scenarios like distributed learning, where many clients individually further train a network and send relative NN updates back to a central entity, it is crucial to identify the network using a versioning scheme. This allows the central entity to identify the NN on which the NN update is being built.

[0173] In other use cases, such as scalable neural networks (NNs), a baseline portion of the NN can be run to generate preliminary results, for example, before running the full or enhanced NN to obtain the complete results. The enhanced NN may use a slightly different version of the baseline NN, such as an updated parameter tensor. If such an updated parameter tensor is differentially encoded, i.e., the parameter tensor on which the differentially encoded update is constructed as an update to a previously encoded parameter tensor must be identified using the identification parameter 310, for example, as shown in Figure 19.

[0174] Furthermore, there are use cases where the integrity of the NN is paramount, namely, where transmission errors or unintended changes in the parameter tensor must be easily detectable. Identifiers, i.e., identification parameters 310, make the operation more error-robust if they can be verified based on the properties of the NN.

[0175] However, state-of-the-art version control is done via checksums or hashes of the data format across the entire container, making it difficult to match equivalent neural networks (NNs) in different containers. Moreover, the clients involved may be using different frameworks / containers. Furthermore, it is impossible to identify / verify only a subset (layers, sublayers) of an NN without completely reconstructing it.

[0176] Therefore, as part of the present invention, in one embodiment, an identifier, i.e., an identification parameter 310, is carried with each entity, i.e., a model, layer, sublayer, enabling each entity to do the following: - Identity verification, and / or, - To refer to, or be referred to, and / or, - Verify consistency.

[0177] In another embodiment, the identifier is derived from a parameter tensor using a hash algorithm such as MD5 or SHA5, or an error detection code such as CRC or checksum.

[0178] In another embodiment, such identifiers for an entity are derived using identifiers of subordinate entities. For example, layer identifiers might be derived from the identifiers of the constituent sub-layers, and model identifiers from the identifiers of the constituent layers.

[0179] Figure 19 shows Embodiment I1 of a data stream 45 in which a representation of a neural network is encoded. The data stream 45 is structured into individually accessible portions 200. Each portion 200 represents a corresponding NN portion of the neural network, for example, containing one or more NN layers or a portion of an NN layer, and the data stream 45 includes, for each of the one or more predetermined individually accessible portions 200, an identification parameter 310 for identifying each predetermined individually accessible portion 200.

[0180] Corresponding embodiment ZI1 relates to a device for encoding a representation of a neural network into a DS45, wherein the data stream 45 is structured into individually accessible portions 200, each portion 200 containing, for example, one or more NN layers or a portion of an NN layer, representing a corresponding NN portion of the neural network, and the device is configured to provide the data stream 45 with an identification parameter 310 for each of the one or more predetermined individually accessible portions 200, for each predetermined individually accessible portion 200.

[0181] Another corresponding embodiment XI1 relates to a device for decoding a representation of a neural network from DS45, wherein the data stream 45 is structured into individually accessible portions 200, each portion 200 representing a corresponding NN portion of a neural network, for example, containing one or more NN layers or a portion of NN layers, and the device is configured to decode from the data stream 45, for each of one or more predetermined individually accessible portions 200, an identification parameter 310 for identifying each predetermined individually accessible portion 200.

[0182] The following describes different features and / or functionalities in the context of data stream 45, but features and / or functionalities can also be features and / or functionalities of the apparatus according to embodiment ZI1 or according to embodiment XI1, in the same or similar manner.

[0183] According to Embodiment I2, in the DS45 of Embodiment I1, the identification parameter 310 is associated with each predetermined individually accessible portion 200 via a hash function or an error detection code or an error correction code.

[0184] Embodiment I3 further includes a higher-level identification parameter for identifying one or more predetermined sets of individually accessible portions 200 of the DS45 from any of the previous embodiments I1 and I2.

[0185] According to Embodiment I4, the higher-level identification parameter of the DS45 in Embodiment I3 is associated with one or more predetermined individually accessible parts 200 identification parameters 310 via a hash function or an error detection code or an error correction code.

[0186] According to Embodiment I5, in any of the DS45s from Embodiments I1 to I4, the individually accessible portion 200 is encoded at the starting point of each individually accessible portion using context-adaptive arithmetic coding and context initialization, as shown in Figure 8, for example.

[0187] According to Embodiment I6, in any of the DS45s of the previous embodiments I1 to I5, the data stream 45 is, for example, as shown in any of Figures 11 to 15, individually accessible portions 200, Each individually accessible portion 200 begins with starting code 242, and / or, Pointers 220 / 244 that point to the beginning of each individually accessible portion 200, and / or Includes a data stream length parameter 246 that indicates the data stream length of each individually accessible portion 200, for skipping each individually accessible portion 200 when analyzing DS45.

[0188] According to Embodiment I7, in any of the DS45s of the previous embodiments I1 to I6, the NN portion includes one or more sub-parts of the NN layer and / or one or more NN layers of the NN.

[0189] According to Embodiment I8, among the DS45s in any of the previous embodiments I1 to I7, the DS45 is one of the previous embodiments A1 to G15.

[0190] 5. Scalable NN Bitstream As mentioned above, some applications rely on further structuring the NN10 by dividing layer 210 or a group of layers, i.e., sublayers 43 / 44 / 240, into a baseline, e.g., a second version 3301 of NN10, and an advanced part, e.g., a first version 3302 of NN10, as shown in Figures 20 to 23, so that the client can match its processing capabilities or make inferences on the baseline before processing the more complex advanced NN. In such cases, it is beneficial to sort, encode, and access the parameter tensor 30 of each subpart of the NN layer independently, as described in Sections 1 to 4.

[0191] Furthermore, in some cases, NN10, - As shown in Figure 22, reducing the number of neurons in a layer, for example, requiring fewer operations, and / or, - As shown in Figure 21, making the weight quantization coarser, for example, to allow for faster reconstruction, and / or, - As shown in Figure 23, different training (learning) methods are used, for example, a general baseline NN versus a personalized advanced NN. -These factors allow for the division into a basic model (baseline) and a higher-level model (advanced variant).

[0192] Figure 21 shows a modified version of the NN and the differential delta signal 342. A baseline version, e.g., a second version 3301 of the NN, and an advanced version, e.g., a first version 3302 of the NN, are shown. Figure 21 shows one of the above cases where two variants of the layer are created from a single layer of the original NN with two quantization settings, e.g., a parameter tensor 30 representing the corresponding layers, and their respective delta signals 342 are created. The baseline version 3301 is associated with coarse quantization, and the advanced version 3302 is associated with fine quantization. The advanced version 3302 can be delta coded relative to the baseline version 3301.

[0193] Figure 22 shows a further variation of the original NN separation. In Figure 22, a further variation of the NN separation is shown. For example, on the left side, the separation of a layer, e.g., a parameter tensor 30 representing a corresponding layer, into a baseline portion 30a and an advanced portion 30b is shown, i.e., the advanced portion 30b extends the baseline portion 30a. Inference of the advanced portion 30b requires inference of the baseline portion 30a. On the right side of Figure 22, it is shown that the central portion of the advanced portion 30b consists of an update of the baseline portion 30a, which can also be delta coded as shown in Figure 21.

[0194] In these cases, the NN parameters 32, e.g., weights, of the baseline 3301 and advanced 3302 NN versions have a clear dependency, and / or the baseline version 3301 of the NN is in some way part of the advanced version 3302 of the NN.

[0195] Therefore, encoding the parameter tensor 30b of the advanced NN portion, i.e., the first version 3302 of the NN, as the delta of the parameter tensor 30b of the baseline NN version, i.e., the second portion 3301 of the NN, at the NN scale, layer scale, or sub-layer scale is beneficial in terms of encoding efficiency, processing overhead, and parallelization.

[0196] A further variation is shown in Figure 23, where an advanced version of the NN is created to compensate for the compression effect on the original NN by training in the presence of an irreversibly compressed baseline NN variant. The advanced NN is inferred in parallel with the baseline NN, and its NN parameters, e.g., weights, are connected to the same neurons as the baseline NN. Figure 23 shows, for example, the training of an augmented NN based on an irreversibly encoded baseline NN variant.

[0197] In one embodiment, a (sub)layer bitstream, i.e., individually accessible portion 200 or individually accessible sub-parts 34 / 44 / 220, is divided into two or more (sub)layer bitstreams, the first of which is represented as a baseline version 3301 of the (sub)layer, the second as an advanced version 3302 of the first (sub)layer, and so on, with the baseline version 3301 preceding the advanced version 3302 in bitstream order.

[0198] In another embodiment, the (sub)layer bitstream is shown to include incremental updates of another (sub)layer parameter tensor 30 in the bitstream, for example, an incremental update including a delta parameter tensor, i.e., a delta signal 342, and / or a parameter tensor.

[0199] In another embodiment, the (sub)layer bitstream carries a reference identifier that refers to the (sub)layer bitstream having a matching identifier that includes incremental updates of the parameter tensor 30.

[0200] Figure 20 shows an embodiment J1 of the data stream 45 in which representations of the neural network 10 are encoded in a layered manner so that different versions 330 of NN10 are encoded in the data stream 45, the data stream 45 is structured into one or more individually accessible parts 200, each part 200 relating to a corresponding version 330 of the neural network 10, Data stream 45 is version 3302 of NN10, Delta-coded 340 for the second version 3301 of NN10 encoded in the second part 2001, and / or, To perform inference based on the first version 3302 of NN10, one or more compensating NN parts 332 are performed in addition to the execution of the corresponding NN part 334 of the second version 3301 of NN10, each encoded in the second part 2001, and In the form of a 338-compensated NN portion 332, the outputs 336 of each compensated NN portion 332 and the corresponding NN portion 334 are summed up. The first part is encoded in 2002.

[0201] According to the embodiment, the compensating NN portion 332 may include a delta signal 342 as shown in Figure 21, or an additional tensor and delta signal as shown in Figure 22, or NN parameters learned to be different from the NN parameters in the corresponding NN portion 334 as shown in Figure 23, for example.

[0202] As shown in Figure 23, according to the embodiment, the compensated NN portion 332 includes the quantized NN parameters of the NN portion of the second neural network, and the NN portion of the second neural network is associated with NN10, i.e., the corresponding NN portion 334 of the first NN. The second neural network may be trained so that the compensated NN portion 332 can be used to compensate for compression effects, such as quantization errors, on the corresponding NN portion 334 of the first NN. The outputs of each compensated NN portion 332 and the corresponding NN portion 334 are summed to reconstruct the NN parameters corresponding to the first version 3302 of NN10, enabling inference based on the first version 3302 of NN10.

[0203] While the embodiments described above primarily focus on providing different versions 330 of NN10 in a single data stream, it is also possible to provide multiple different versions 330 in multiple different data streams. Different versions 330 are delta-coded into different data streams, for example, compared to a simpler version. Therefore, separate data streams (DS) may be used. For example, a DS containing the initial NN data might be sent first, followed later by a DS containing the updated NN data.

[0204] Corresponding embodiment ZJ1 relates to an apparatus for hierarchically encoding a representation of a neural network into a DS45 such that different versions 330 of NN10 are encoded into a data stream 45, and the data stream 45 is structured into one or more individually accessible parts 200, each part 200 relating to a corresponding version 330 of the neural network 10, and the apparatus, The first version 3302 of NN10, Delta-coded 340 for the second version 3301 of NN10 encoded in the second part 2001, and / or, To perform inference based on the first version 3302 of NN10, one or more compensating NN parts 332 are performed in addition to the execution of the corresponding NN part 334 of the second version 3301 of NN10, each encoded in the second part 2001, and In the form of a 338-compensated NN portion 332, the outputs 336 of each compensated NN portion 332 and the corresponding NN portion 334 are summed up. It is configured to encode into the first part 2002.

[0205] Another corresponding embodiment XJ1 relates to an apparatus for decoding a representation of a neural network 10 from a layered encoded DS45, wherein different versions 330 of NN10 are encoded in a data stream 45, the data stream 45 is structured into one or more individually accessible parts 200, each part 200 relating to a corresponding version 330 of the neural network 10, and the apparatus By using delta decoding 340 for the second version 3301 of NN10 encoded in the second part 2001, and / or, From DS45, each performs inference based on the first version 3302 of NN10 by decoding one or more compensating NN portions 332, It is configured to decode the first version 3302 of NN10 encoded from the first part 2002, The second part 2001 is encoded in addition to the execution of the corresponding NN part 334 of the second version 3301 of NN10, The outputs 336 of each compensated NN section 332 and the corresponding NN section 334 are summed up to 338.

[0206] The following describes different features and / or functionalities in the context of data stream 45, but the features and / or functionalities may be the same or similar features and functionalities of the apparatus according to embodiment ZJ1 or embodiment XJ1.

[0207] According to Embodiment J2, among the data streams 45 of Embodiment J1, data stream 45 is the first version 3301 of NN10, The difference in weights and / or biases, i.e., the difference between the NN parameters associated with the first version 3301 of NN10 and the NN parameters associated with the second version 3302 of NN10, as shown in Figure 21, and / or For example, as shown in Figure 22, additional neurons or interconnections of neurons, The second version of NN10, 3302, which is encoded in the second part 2002, is delta-coded 340 and then encoded in the first part 2001.

[0208] According to Embodiment J3, any individually accessible portion 200 of the DS in any of the previous embodiments J1 and J2 is encoded at the starting point of each individually accessible portion 200 using context-adaptive arithmetic coding and context initialization, as shown in Figure 8, for example.

[0209] According to Embodiment J4, in any of the previous embodiments J1 to J3, the data stream 45, as shown in one of Figures 11 to 15, has each individually accessible portion 200, Each individually accessible portion 200 begins with starting code 242, and / or, Pointers 220 / 244 that point to the beginning of each individually accessible portion 200, and / or When analyzing DS45, a data stream length parameter is included that indicates the data stream length 246 of each individually accessible portion 200, in order to skip each individually accessible portion 200.

[0210] According to Embodiment J5, in any of the earlier embodiments J1 to J4, the data stream 45 includes, for each of the one or more predetermined individually accessible portions 200, an identification parameter 310 for identifying each predetermined individually accessible portion 200, as shown in Figure 19, for example.

[0211] According to Embodiment J6, among the DS45s of any of the previous embodiments J1 to J5, the DS45 is one of the previous embodiments A1 to I8.

[0212] 6. Reinforcement Data As shown in Figures 24a and 24b, there are application scenarios in which the parameter tensor 30 is accompanied by additional augmentative (or supplementary / adjunctive) data 350. This supplemental data 350 is not usually necessary for decoding / reconstructing / inference of the neural network, but can be essential from an application perspective. For example, it may be information about the relationships between each parameter 32 (Sebastian Lapuschkin, 2019), or information about sufficient statistics of the parameters 32, such as intervals or variances that show the robustness of each parameter 32 to perturbations (Christos Louizos, 2017).

[0213] Such augmentation information, i.e., supplemental data 350, can introduce a considerable amount of data with respect to the NN parameter tensor 30, so that it is desirable to encode the augmentation data 350 similarly using a scheme such as DeepCABAC. However, it is important to mark this data as irrelevant to the NN's decoding, solely for inference purposes, so that clients that do not require augmentation can skip this part of the data.

[0214] In one embodiment, the augmented data 350 is encoded independently of the (sub)layer bitstream data, for example, independently of the individually accessible portions 200 and / or individually accessible sub-portions 240, but is carried in additional (sub)layer augmented bitstreams, i.e., further individually accessible portions 352, which are scattered with each (sub)layer bitstream to form a model bitstream, i.e., a data stream 45. Figures 24a and 24b illustrate this embodiment. Figure 24b is a diagram illustrating the augmented bitstream 352.

[0215] Figures 24a and 24b show Embodiment K1, which shows a data stream 45 on which a representation of a neural network is encoded, the data stream 45 is structured into individually accessible portions 200, each portion 200 representing a corresponding NN portion of the neural network, and for each of one or more predetermined individually accessible portions 200, the data stream 45 includes augmentation data 350 to alternatively supplement the representation of the NN, and alternatively, as shown in Figure 24b, the data stream 45 includes supplemental data 350 to supplement the representation of the NN for one or more predetermined individually accessible portions 200.

[0216] Corresponding embodiment ZK1 relates to a device for encoding a representation of a neural network into DS45, wherein the data stream 45 is structured into individually accessible portions 200, each portion 200 representing a corresponding NN portion of the neural network, and the device is configured to provide supplemental data 350 to supplement the representation of the NN for each of one or more predetermined individually accessible portions 200 of the data stream 45. Alternatively, the device is configured to provide supplemental data 350 to supplement the representation of the NN for one or more predetermined individually accessible portions 200 of the data stream 45.

[0217] Another corresponding embodiment XK1 relates to a device for decoding a representation of a neural network from DS45, wherein the data stream 45 is structured into individually accessible portions 200, each portion 200 representing a corresponding NN portion of a neural network, and the device is configured to decode from the data stream 45 supplemental data 350 to supplement the representation of the NN for each of one or more predetermined individually accessible portions 200. Alternatively, the device is configured to decode from the data stream 45 supplemental data 350 to supplement the representation of the NN for one or more predetermined individually accessible portions 200.

[0218] In the following, different features and / or functionalities are described in the context of data stream 45, but features and / or functionalities may also be features and / or functionalities of the apparatus according to embodiment ZK1 or according to embodiment XK1, in the same or similar manner.

[0219] According to Embodiment K2, in the data stream 45 of Embodiment K1, DS45 indicates that the supplemental data 350 is not essential for NN-based inference.

[0220] According to Embodiment K3, in any of the data streams 45 of the previous embodiments K1 and K2, the data stream 45 has supplemental data 350 to supplement the representation of the NN for one or more predetermined individually accessible portions 200 encoded into further individually accessible portions 352, as shown in Figure 24b. The DS45 includes, for one or more predetermined individually accessible portions 200, for each of the one or more predetermined individually accessible portions 200, a corresponding further predetermined individually accessible portion 352 related to the NN portion that each predetermined individually accessible portion 200 corresponds to.

[0221] According to Embodiment K4, in any of the preceding embodiments K1 to K3, the NN portion of the DS45 includes one or more NN layers of the NN and / or a subdivided layer portion of a predetermined NN layer of the NN. According to Figure 24b, for example, the individually accessible portion 2002 and the corresponding further predetermined individually accessible portion 352 relate to an NN portion containing one or more NN layers.

[0222] According to Embodiment K5, any individually accessible portion 200 of the DS45 from any previous Embodiments K1 to K4 is encoded at the starting point of each individually accessible portion 200 using context-adaptive arithmetic coding and context initialization, as shown in Figure 8, for example.

[0223] According to Embodiment K6, among the DS45 of any of the above embodiments K1 to K5, the data stream 45 is, for example, as shown in any of Figures 11 to 15, for each individually accessible portion 200, Each individually accessible portion 200 begins with starting code 242, and / or, Pointers 220 / 244 that point to the beginning of each individually accessible portion 200, and / or The DS45 includes a data stream length parameter that indicates the data stream length 246 of each individually accessible portion 200, for skipping each individually accessible portion 200 when analyzing the DS45.

[0224] According to Embodiment K7, among the DS45 of any of the above embodiments K1 to K6, the supplementary data 350 is: Relevance score of NN parameters, and / or, This concerns the perturbation robustness of NN parameters.

[0225] According to Embodiment K8, among the DS45s of any of the above embodiments K1 to K7, the DS45 is one of the above embodiments A1 to J6.

[0226] 7. Extended control data In addition to the described functionality of different access functions, an extended hierarchical control data structure, i.e., a sequence of control data portions 420 410, may be required for different applications and usage scenarios. On the one hand, a compressed NN representation (or bitstream) may be used from within a specific framework such as TensorFlow or PyTorch, in which case only a minimum amount of control data 400 is required, for example, to decode a deepCABAC encoded parameter tensor. On the other hand, a specific type of framework may not be known to the decoder, in which case additional control data 400 is required. Thus, depending on the use case and knowledge of the environment, different levels of control data 400 may be required, as shown in Figure 25.

[0227] Figure 25 shows a hierarchical control data (CD) structure for a compressed neural network, i.e., a sequence 410 of control data portions 420. Depending on the environment, different CD levels, i.e., control data portions 420, may or may not exist, for example, the dotted boxes. In Figure 25, for example, the compressed bitstream containing the representation 500 of the neural network can be any of the above model bitstream types, for example, containing all the compressed data of a network with or without subdivision into subbitstreams.

[0228] Therefore, if a particular network (e.g., TensorFlow, PyTorch, Keras, etc.) has known types and architectures for its decoder and encoder, and includes compressed NN techniques, then only a compressed NN bitstream is required. However, if the decoder does not know the configuration of any encoder, then a full set of control data, i.e., a complete sequence 410 of the control data portion 420, is required, in addition to enabling a complete reconstruction of the network.

[0229] Examples of different hierarchical control data layers, i.e., control data units 420, are as follows: -CD Level 1: Compressed data decoder control information. -CD Level 2: Specific syntax elements from each framework (Tensor Flow, PyTorch, Keras) -CD Level 3: Inter-framework formatting elements, such as ONNX (Open Neural Network Exchange) for use in different frameworks. -CD Level 4: Information about the network topology -CD Level 5: Complete network parameter information (for complete reconstruction without knowledge of network topology)

[0230] Therefore, this embodiment describes a hierarchical control data structure of N levels, i.e., N control data portions 420, where levels 0 to N may exist to enable different usage modes ranging from the use of core data with specific compression only to fully self-contained network reconstruction. The levels, i.e., control data portions 420, may even include syntax from existing network architectures and frameworks.

[0231] In another embodiment, different levels, i.e., control data portions 420, may contain information about the neural network at different levels of precision. For example, the level structure may be configured as follows: -CD Level 1: Contains information about network parameters, such as type and dimensions. -CD Level 2: Includes information about the network layer. Examples: type, identification, etc. -CD Level 3: Includes information about the network topology. Example: connectivity between layers. -CD Level 4: Includes information about the neural network model. Examples: version, training parameters, performance, etc. - CD Level 5: Includes information on the learned and verified dataset. Example: Input natural images with a resolution of 227x227 having 1000 label categories, etc.

[0232] FIG. 25 shows Embodiment L1, where data stream 45 has the representation 500 of the neural network encoded, and data stream 45 includes hierarchical control data 400 structured in a sequence 410 of control data portions 420. The control data portion 420 provides information regarding the NN with increasing detail along the sequence 410 of the control data portion 420. The second hierarchical control data 4002 of the second control data portion 4202 includes more detailed information than the first hierarchical control data 4001 of the first control data portion 4201.

[0233] According to an embodiment, the control data portion 420 may represent different units that can include additional topology information.

[0234] The corresponding Embodiment ZL1 relates to an apparatus for encoding the representation 500 of the neural network into DS45. The apparatus is configured to provide hierarchical control data 400 structured in a sequence 410 of control data portions 420 to data stream 45. The control data portion 420 provides information regarding the NN with increasing detail along the sequence 410 of the control data portion 420.

[0235] Another corresponding Embodiment XL1 relates to an apparatus for decoding the representation 500 of the neural network from DS45. The apparatus is configured to decode hierarchical control data 400 structured in a sequence 410 of control data portions 420 from data stream 45. The control data portion 420 provides information regarding the NN with increasing detail along the sequence 410 of the control data portion 420.

[0236] In the following, different features and / or functionalities are described in the context of data stream 45, but features and / or functionalities may also be features and functionalities of the device according to embodiment ZL1 or embodiment XL1, similarly or in a similar manner.

[0237] According to Embodiment L2, at least some of the control data portion 420 of the data stream 45 in Embodiment L1 provides partially redundant information about the NN.

[0238] According to Embodiment L3, in the data stream 45 of Embodiment L1 or L2, the first control data portion 4201 provides information about the NN in a manner indicating a default NN type that signifies a default setting, and the second control data portion 4202 includes parameters that indicate each of the default settings.

[0239] According to Embodiment L4, among the DS45s of any of the previous embodiments L1 to L3, the DS45 is one of the previous embodiments A1 to K8.

[0240] Embodiment X1 relates to a device for decoding a data stream 45 according to any previous embodiment. The device is configured to derive an NN10 from the data stream 45, for example, according to any of the embodiments XA1 to XL1 described above. For example, the device is further configured to encode / decode DS45 according to any previous embodiment.

[0241] This device is, for example, Search for start code 242, and / or, The data stream length parameter 45 is used to skip the individually accessible portion 200, and / or, Using pointers 220 / 244, restart the parsing of data stream 45 at the beginning of the individually accessible portion 200, and / or Associate the decoded NN parameter 32' with neurons 14, 18, 20 or neuron interconnection 22 / 24 according to coding order 104, and / or Perform context-adaptive arithmetic decoding and context initialization, and / or, Perform inverse quantization / value reconstruction 280, and / or To compute a quantization step size of 263, perform an exponential sum and / or, In response to a quantized index 32'' that deviates from a predetermined index interval 268, such as estimating an escape code, a lookup of the quantized index-reconstruction level mapping 265 is performed, and / or A hash is performed on a certain individually accessible portion 200, or an error detection / correction code is applied, and the result is compared with the corresponding identification parameter 310 to check the correctness of the individually accessible portion 200, and / or Reconstruct a version 330 of NN10 by adding weights and / or bias differences to the underlying NN version 330, and / or by adding additional neurons 14, 18, 20 or neuron interconnections 22 / 24 to the underlying NN version 330, or by performing joint execution of one or more compensated NN parts and corresponding NN parts and summing their outputs, and / or When the currently read control data unit 420 assumes a known parameter state for the device, reading stops, and the control data unit 420 is sequentially read while providing information, i.e., hierarchical control data 400, in a manner that conforms to a predetermined degree of fineness.

[0242] Embodiment Y1 relates to an apparatus for performing inference using NN10, and includes an apparatus for decoding a data stream 45 and deriving NN10 from the data stream 45 according to Embodiment X1, and a processor configured to perform inference based on NN10.

[0243] Embodiment Z1 relates to a device for encoding a data stream 45 according to any previous embodiment, for example, a device according to any of the above embodiments ZA1 to ZL1, further configured to encode / decode DS45 according to any previous embodiment.

[0244] This device, for example, selects encoding order 104 to find the optimal one for optimal compression efficiency.

[0245] Embodiment U relates to a method that is carried out by any of the apparatuses of Embodiments XA1 to XL1 or ZA1 to ZL1.

[0246] Embodiment W relates to a computer program that, when executed by a computer, causes a computer to perform the method of Embodiment U.

[0247] Modifications of the embodiment:

[0248] While some embodiments have been described in the context of apparatus, it is clear that these embodiments also represent a description of the corresponding method, where a block or apparatus corresponds to a method step or a feature of a method step. Similarly, embodiments described in the context of a method step also represent a description of the corresponding block or item or a feature of the corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware device, such as a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such a device.

[0249] Depending on specific implementation requirements, embodiments of the present invention can be implemented in hardware or software. The implementation can use a digital storage medium, such as a floppy (registered trademark) disk, DVD, Blu-Ray (registered trademark), CD, ROM, PROM, EPROM, EEPROM or flash memory, that has electronically readable control signals stored thereon and that can cooperate (or be capable of cooperating) with a programmable computer system so that each method is executed. Thus, the digital storage medium may be computer-readable.

[0250] Some embodiments according to the present invention consist of a data carrier having electronically readable control signals and are capable of cooperating with a programmable computer system so that one of the methods described herein is executed.

[0251] Generally, embodiments of the present invention can be implemented as a computer program product having program code, the program code being operative to execute one of the methods when the computer program product is executed on a computer. The program code may be stored, for example, on a machine-readable carrier.

[0252] Other embodiments are constituted by storing a computer program for executing one of the methods described herein on a machine-readable carrier.

[0253] In other words, embodiments of the method of the present invention are thus computer programs having program code for executing one of the methods described herein when the computer program is executed on a computer.

[0254] A further embodiment of the method of the present invention is a data carrier (or digital storage medium, or computer-readable medium) having a computer program for performing one of the methods described herein recorded thereon. The data carrier, digital storage medium, or recording medium is typically tangible and / or non-transitional.

[0255] Therefore, a further embodiment of the method of the present invention is a data stream or sequence of signals representing a computer program for performing one of the methods described herein. The data stream or sequence of signals may be configured to be transmitted, for example, over a data communication connection, such as the Internet.

[0256] Further embodiments include processing means configured or adapted to perform one of the methods described herein, such as a computer or a programmable logic device.

[0257] Further embodiments include a computer on which a computer program for performing one of the methods described herein is installed.

[0258] Further embodiments of the present invention comprise an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may include, for example, a file server for transferring the computer program to the receiver.

[0259] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) can be used to perform some or all of the functionality of the method described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, the method is preferably performed by any hardware device.

[0260] The devices described in this book can be implemented using hardware devices, computers, or a combination of hardware devices and computers.

[0261] The apparatus described herein, or any component of the apparatus described herein, may be implemented at least partially in hardware and / or software.

[0262] The methods described herein may be performed using hardware devices, or using a computer, or using a combination of hardware devices and a computer.

[0263] Any method or apparatus described herein may be performed at least partially by hardware and / or software.

[0264] The embodiments described above are for illustrative purposes only. Modifications and variations of the arrangements and details described herein will be obvious to those skilled in the art. Therefore, it is intended that the invention is limited only by the scope of the imminent claims and not by any specific details presented in the description and commentary of the embodiments herein.

[0265] 8.References Andrew Kerr, D. M. (2017, 5). Retrieved from https: / / devblogs.nvidia.com / cutlass-linear-algebra-cuda / Chollet, F. (2016). Xception: Deep Learning with Depthwise Separable Convolutions. Retrieved from https: / / arxiv.org / abs / 1610.02357 Christos Louizos, K. U. (2017). Bayesian Compression for Deep Learning. NI PSSebastian Lapuschkin, S. W.-R. (2019). Unmasking Clever Hans predictors and asse ssing what machines really learn. Nature Comminications. Tao, K. C. (2018). Once for All: A Two-Flow Convolutional Neural Network for Vis ual Tracking. IEEE Transactions on Circuits and Systems for Video Techno logy, 3377-3386

Claims

1. A data stream (45) in which a representation of the neural network (10) is encoded, A data stream (45) including a serialization parameter (102) that indicates the coding order (104) in which the neural network parameters (32) defining the neuronal interconnections (22, 24) of the neural network (10) are encoded in the data stream (45).

2. The data stream (45) according to claim 1, wherein the neural network parameters (32) are encoded into the data stream (45) using context-adaptive arithmetic coding (600).

3. The data stream (45) according to claim 1 or 2, wherein the data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion (200) representing a corresponding neural network layer (210, 30) of the neural network (10), and the serialization parameter (102) indicates the encoding order (104) in which the neural network parameters are encoded in the data stream (45), defining the neuronal interconnections (22, 24) of the neural network within a given neural network layer (210, 30).

4. The serialization parameter (102) is an n-ary parameter that indicates the coding order (104) from a set (108) of n coding orders (104), the data stream (45) according to any one of claims 1 to 3.

5. The set (108) of the n coding order (104) is, The predetermined coding order is a first predetermined coding order (106) which differs in the order in which it traverses the dimensions (34) of the tensor (30) that describes the predetermined neural network layers (210, 30) of the neural network (10). 1 ), and / or, For scalable coding of the neural network, a second predetermined coding sequence (106) is used, where the predetermined coding sequence differs in the number of times (107) that it scans the predetermined neural network layers (210, 30) of the neural network. 2 ), and / or, A third predetermined coding sequence (106) is a third predetermined coding sequence in which the order in which the neural network layers (210, 30) of the neural network are scanned is different from the aforementioned predetermined coding sequence. 3 ), and / or, A fourth predetermined coding order (106) in which the neurons (14, 18, 20) of the neural network layers (210, 30) of the aforementioned neural network are scanned in a different order. 4 The data stream (45) according to claim 4, including ).

6. The data stream (45) according to any one of claims 1 to 5, wherein the serialization parameter (102) indicates a permutation used by the coding order (104) to permutate the neurons (14, 18, 20) of the neural network layers (210, 30) relative to the default order.

7. The data stream (45) according to claim 6, wherein the permutation orders the neurons (14, 18, 20) of the neural network layers (210, 30) such that the neural network parameter (32) increases monotonically along the coding order (104) or decreases monotonically along the coding order (104).

8. The data stream (45) according to claim 6, wherein the permutation is one of a predetermined coding order that can be signaled by the serialization parameter (102), and the neurons (14, 18, 20) of the neural network layer (210, 30) are ordered such that the bit rate for encoding the neural network parameter (32) into the data stream (45) is minimized for the permutation indicated by the serialization parameter (102).

9. The neural network parameters (32) include weights and biases, and the data stream (45) is as described in any one of claims 1 to 8.

10. The data stream (45) according to any one of claims 1 to 9, wherein the data stream (45) is structured into individually accessible sub-parts (43, 44, 240), each sub-part (43, 44, 240) representing a corresponding neural network part of the neural network (10), and each sub-part (43, 44, 240) is fully scanned by the coding sequence (104) before subsequent sub-parts are scanned by the coding sequence (104).

11. The data stream (45) according to any one of claims 3 to 10, wherein the neural network parameters (32) are encoded into the data stream (45) at the starting point of any individually accessible portion (200) or sub-part (43, 44, 240) using context-adaptive arithmetic coding (600) and context initialization.

12. The data stream (45) according to any one of claims 3 to 11, wherein the data stream (45) includes a start code (242) in which each individually accessible portion (200) or sub-portion (43, 44, 240) begins, and / or pointers (220, 244) pointing to the beginning of each individually accessible portion or sub-portion, and / or a pointer (246) to the data stream length (246) of each individually accessible portion or sub-portion for skipping each individually accessible portion or sub-portion when parsing the data stream (45).

13. The data stream (45) according to any one of claims 1 to 12, further comprising a numerical representation of the neural network parameters (32) and a numerical computation representation parameter (120) indicating the bit size when the neural network (10) is used for inference.

14. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) includes numerical representations of the neural network parameters (32) of the neural network encoded in the data stream and numerical computation representation parameters (120) indicating the bit size, when the neural network (10) is used for inference.

15. The data stream (45) is structured into individually accessible sub-parts (43, 44, 240), and each individually accessible sub-part represents a corresponding neural network part of the neural network, such that each individually accessible sub-part is fully scanned by the coding sequence (104) before a subsequent individually accessible sub-part is scanned by the coding sequence (104), and the data stream (45) includes, for a given individually accessible sub-part, a type parameter indicating the parameter type of the neural network parameter (32) encoded in the given individually accessible sub-part, according to any one of claims 1 to 14.

16. The data stream (45) according to claim 15, wherein the type parameter distinguishes at least the weights of the neural network from the biases of the neural network.

17. The data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, and the data stream (45) further includes, for a given neural network layer, a neural network layer type parameter (130) indicating the neural network layer type of the given neural network layer of the neural network, according to any one of claims 1 to 16.

18. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, and the data stream (45) further includes, for a given neural network layer, a neural network layer type parameter (130) indicating the neural network layer type of the given neural network layer of the neural network.

19. The data stream (45) according to claim 17 or 18, wherein the neural network layer type parameter (130) distinguishes at least between fully connected layer types and convolutional layer types.

20. The data stream (45) according to any one of claims 1 to 19, wherein the data stream (45) is structured into individually accessible portions (200), each individually accessible portion representing a corresponding neural network portion of the neural network, and the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), pointers (220, 244) pointing to the beginning of each individually accessible portion.

21. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) is structured into individually accessible portions (200), each of which represents a corresponding neural network portion of the neural network, and the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), a pointer (220, 244) pointing to the beginning of each predetermined individually accessible portion.

22. Each individually accessible part is, The corresponding neural network layer (210) of the aforementioned neural network, or A data stream (45) according to claim 20 or 21, representing the neural network portion (43, 44, 240) of the neural network layer (210) of the aforementioned neural network.

23. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, and within a predetermined portion, the data stream (45) is further structured into individually accessible sub-parts (43, 44, 240), each sub-part (43, 44, 240) representing a corresponding neural network portion of each of the neural network layers (210, 30) of the neural network, and for each of the one or more predetermined individually accessible sub-parts (43, 44, 240), A start code (242) in which each of the aforementioned predetermined individually accessible sub-parts begins, and / or, A pointer (244) that points to the beginning of each of the predetermined individually accessible sub-parts, and / or The data stream (45) according to any one of claims 1 to 22, wherein when analyzing the data stream (45), the data stream (45) includes a data stream length parameter indicating the data stream length (246) of each predetermined individually accessible sub-part for skipping each predetermined individually accessible sub-part.

24. The data stream (45) according to claim 23, wherein the representation of the neural network is encoded at the starting point of each individually accessible portion and each individually accessible sub-portion using context adaptive arithmetic coding (600) and context initialization.

25. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, and within a predetermined portion, the data stream (45) is further structured into individually accessible sub-parts (43, 44, 240), each sub-part (43, 44, 240) representing a corresponding neural network portion of each of the neural network layers (210, 30) of the neural network, and for each of the one or more predetermined individually accessible sub-parts (43, 44, 240), A start code (242) in which each of the aforementioned predetermined individually accessible sub-parts begins, and / or, A pointer (244) that points to the beginning of each of the predetermined individually accessible sub-parts, and / or A data stream (45) that includes a data stream length parameter indicating the data stream length (246) of each predetermined individually accessible sub-part in order to skip each predetermined individually accessible sub-part when analyzing the data stream (45).

26. The data stream (45) according to claim 25, wherein the representation of the neural network is encoded using context-adaptive arithmetic coding (600) and context initialization at the starting point of each individually accessible portion and each individually accessible sub-portion of the data stream (45).

27. The data stream (45) according to any one of claims 1 to 26, wherein the data stream (45) is structured into individually accessible portions (200), each individually accessible portion representing a corresponding neural network portion of the neural network, and the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), processing option parameters (250) indicating one or more processing options (252) that must be used or may be used when using the neural network (10) for inference.

28. The processing option parameter (250) indicates one or more available processing options (252) from a predetermined set of processing options (252). The predetermined processing option (252) is, The parallel processing capability of each of the aforementioned predetermined individually accessible portions, and / or, The sample-unit parallel processing capability of each of the aforementioned predetermined individually accessible portions (252 2 ), and / or, The channel-unit parallel processing capability of each of the predetermined individually accessible portions (252 1 ), and / or, The parallel processing capability per classification category for each of the aforementioned predetermined individually accessible parts, and / or, The data stream (45) according to claim 27, wherein the data stream (45) is related to the same neural network portion, but includes a dependency of the neural network portion represented by each predetermined individually accessible portion to computation results obtained from another individually accessible portion of the data stream (45) that belongs to another version of the version (330) of the neural network which is encoded in layers in the data stream (45).

29. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) is structured into individually accessible portions (200), each of which represents a corresponding neural network portion of the neural network, and the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), processing option parameters (250) indicating one or more processing options (252) that must be used or may be used when using the neural network (10) for inference.

30. The processing option parameter (250) indicates one or more available processing options (252) from a predetermined set of processing options (252). The predetermined processing option (252) is, The parallel processing capability of each of the aforementioned predetermined individually accessible portions, and / or, The sample-unit parallel processing capability of each of the aforementioned predetermined individually accessible portions (252 2 ), and / or, The channel-unit parallel processing capability of each of the predetermined individually accessible portions (252 1 ), and / or, The parallel processing capability per classification category for each of the aforementioned predetermined individually accessible parts, and / or, The data stream (45) according to claim 29, wherein the data stream (45) is related to the same neural network portion, but includes a dependency of the neural network portion represented by each predetermined individually accessible portion to computation results obtained from another individually accessible portion of the data stream (45) that belongs to another version of the version (330) of the neural network which is encoded in layers in the data stream (45).

31. The neural network parameters (32) representing the neural network are encoded. The neural network parameters (32) are quantized (260) to a quantization index (32'') and encoded into the data stream (45). The data stream (45) according to any one of claims 1 to 30, wherein the neural network parameters (32) are encoded in the data stream (45) such that the neural network parameters (32) in different neural network portions of the neural network are quantized (260) in different ways, and the data stream (45) provides for each of the neural network portions a reconstruction rule (270) for dequantizing the neural network parameters (32) associated with each of the neural network portions.

32. A data stream (45) in which neural network parameters (32) representing a neural network are encoded, The neural network parameters (32) are quantized (260) to a quantization index (32'') and encoded into the data stream (45). The neural network parameters (32) are encoded in the data stream (45) such that the neural network parameters (32) in different neural network portions of the neural network are quantized (260) differently, and the data stream (45) represents a reconstruction rule (270) for dequantizing the neural network parameters (32) associated with each of the neural network portions.

33. The data stream (45) according to claim 31 or 32, wherein the neural network portion includes a neural network layer (210, 30) of the neural network and / or a subdivided layer portion of a predetermined neural network layer (210, 30) of the neural network.

34. The data stream (45) is delta-encoded with respect to a second reconstruction rule (270 2 , 270a 2 ) for inverse quantizing neural network parameters (32) related to a second neural network portion, and has a first reconstruction rule (270 1 , 270a 1 ) for inverse quantizing neural network parameters (32) related to a first neural network portion that has been encoded by being delta-encoded. The data stream (45) according to any one of claims 31 to 33.

35. The data stream (45) is subject to the first reconstruction rule (270 1 ,270a 1 A first index value to indicate ) and the second reconstruction rule (270 2 ,270a 2 A second exponential value to show ) and, The first reconstruction rule (270 1 ,270a 1 ) is defined by a first quantization step size defined by a predetermined basis exponent and a first exponent defined by the first exponent value, and, Second Reconstruction Rule (270 2 ,270a 2 The data stream (45) according to claim 34, wherein the second quantization step size is defined by the exponent of the predetermined basis, and the second exponent is defined by the sum of the first and second exponent values.

36. The data stream (45) according to claim 35, wherein the data stream (45) further represents the predetermined base.

37. The data stream (45) is used to dequantize the neural network parameters (32) associated with the first neural network portion using a first reconstruction rule (270 1 ,270a 1 A first exponential value to show ) and a second reconstruction rule (270) to inverse quantize the neural network parameters (32) related to the second neural network portion. 2 ,270a 2 This includes a second exponential value to show ) The first reconstruction rule (270 1 ,270a 1 ) is defined by a first quantization step size defined by a predetermined basis exponent, and a first exponent defined by the sum of the first exponent value and the predetermined exponent value, The second reconstruction rule (270 2 ,270a 2 The data stream (45) according to any one of claims 31 to 34, wherein the second quantization step size is defined by the exponent of the predetermined basis, and the second exponent is defined by the sum of the second exponent value and the predetermined exponent value.

38. The data stream (45) according to claim 37, wherein the data stream (45) further represents the predetermined base.

39. The data stream (45) according to claim 38, wherein the data stream (45) represents the predetermined basis within the neural network range.

40. The data stream (45) further indicates the predetermined exponential value, according to any one of claims 37 to 39.

41. The data stream (45) according to claim 40, wherein the data stream (45) exhibits the predetermined exponential value in the neural network layer (210, 30) range.

42. The data stream (45) according to claim 40 or 41, wherein the data stream (45) further indicates the predetermined basis, and the data stream (45) indicates the predetermined exponential value in a range finer than the range indicated by the data stream (45).

43. The data stream (45) according to any one of claims 35 to 42, wherein the predetermined basis is encoded in a non-integer form and the first and second exponential values ​​are encoded in an integer form.

44. The data stream (45) is subject to the first reconstruction rule (270 1 ,270a 1 A first parameter set (264) that defines a first quantization index-reconstruction level mapping (265) to show the second reconstruction rule (270 2 ,270a 2 This includes a second set of parameters (264) that defines a second quantization index-reconstruction level mapping (265) to show, The first reconstruction rule (270 1 ,270a 1 ) is defined by the first quantization index-reconstruction level mapping (265), and The second reconstruction rule (270 2 ,270a 2 The data stream (45) according to any one of claims 34 to 43, defined by extending the first quantization index-reconstruction level mapping (265) in a predetermined manner with the second quantization index-reconstruction level mapping (265).

45. The data stream (45) is subject to the first reconstruction rule (270 1 ,270a 1 To demonstrate this, a first parameter set (264) that defines a first quantization index-reconstruction level mapping (265) is included, and the second reconstruction rule (270 2 ,270a 2 To demonstrate this, a second set of parameters (264) is included which defines a second quantization index-reconstruction level mapping (265), The first reconstruction rule (270 1 ,270a 1 ) is defined by extending a predetermined quantization index-reconstruction level mapping (265) in a predetermined manner by the first quantization index-reconstruction level mapping (265), The second reconstruction rule (270 2 ,270a 2 The data stream (45) according to any one of claims 34 to 44, defined by extending the predetermined quantization index-reconstruction level mapping (265) with the second quantization index-reconstruction level mapping (265) in the predetermined manner.

46. The data stream (45) according to claim 45, wherein the data stream (45) further shows the predetermined quantization index-reconstruction level mapping (265).

47. The data stream (45) according to claim 46, wherein the data stream (45) represents the predetermined quantization index-reconstruction level mapping (265) over the neural network range or the neural network layer (210, 30) range.

48. According to the aforementioned predetermined method, The mapping of each index value (32'') to a first reconstruction level according to the extended quantization index-reconstruction level mapping is replaced by, if there is a mapping of each index value (32'') to a second reconstruction level according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, and / or According to the extended quantization index-reconstruction level mapping, no reconstruction level is defined to which each of the index values ​​(32'') should be mapped, and according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, for any index value (32'') that is mapped to a corresponding reconstruction level, the mapping from each of the index values ​​(32'') to the corresponding reconstruction level is adopted, and / or According to any one of claims 44 to 47, a data stream (45) is adopted such that, according to the extended quantization index-reconstruction level mapping, no reconstruction level is defined to which each of the index values ​​(32'') should be mapped, and for any index value (32'') that is mapped to a corresponding reconstruction level according to the extended quantization index-reconstruction level mapping, the mapping from each of the index values ​​(32'') to the corresponding reconstruction level is adopted.

49. The data stream (45) is used to show the reconstruction rule (270) of a predetermined neural network portion. The quantization step size parameter (262) indicates the quantization step size (263), Includes a parameter set (264) that defines a quantization index-reconstruction level mapping (265), The reconstruction rule (270) for the predetermined neural network portion is: The quantization step size (263) for a quantization index (32'') within a predetermined index interval (268), and A data stream (45) according to any one of claims 31 to 48, defined by a quantization index-reconstruction level mapping (265) for quantization indices (32'') outside the predetermined index interval (268).

50. A data stream (45) in which neural network parameters (32) representing a neural network are encoded, The neural network parameters (32) are quantized (260) to a quantization index (32'') and encoded into the data stream (45). The data stream (45) is used to show the reconstruction rule (270) for inverse quantization (280) of the neural network parameters (32), It includes a quantization step size parameter (262) indicating the quantization step size (263), and a parameter set (264) defining a quantization index-reconstruction level mapping (265), The reconstruction rule (270) for the predetermined neural network portion is: The quantization step size (263) for a quantization index (32'') within a predetermined index interval (268), and A data stream (45) defined by a quantization index-reconstruction level mapping (265) for quantization indices (32'') outside the predetermined index interval (268).

51. The data stream (45) according to claim 49 or 50, wherein the predetermined index interval (268) includes zero.

52. The data stream (45) according to claim 51, wherein the predetermined index interval (268) is extended to a predetermined size threshold, and quantization indices (32'') exceeding the predetermined size threshold represent escape codes indicating that the quantization index-reconstruction level mapping (265) is used for inverse quantization (280).

53. The data stream (45) according to any one of claims 49 to 52, wherein the parameter set (264) defines the quantization index-reconstruction level mapping (265) by a list of reconstruction levels associated with quantization indices (32'') outside the predetermined index interval (268).

54. The data stream (45) according to any one of claims 31 to 53, wherein the neural network portion includes one or more sub-parts of the neural network layers (210, 30) of the neural network, and / or one or more neural network layers of the neural network.

55. The data stream (45) according to any one of claims 31 to 54, wherein the data stream (45) is structured into individually accessible portions (200), each individually accessible portion having the neural network parameters (32) for the corresponding neural network portion encoded therein.

56. The data stream (45) according to claim 55, wherein each individually accessible portion (200) is encoded at the starting point of each individually accessible portion using context-adaptive arithmetic coding (600) and context initialization.

57. The data stream (45) has, for each individually accessible portion, The start code (242) where each of the aforementioned individually accessible parts begins, and / or A pointer (220, 244) that points to the beginning of each of the individually accessible portions, and / or The data stream (45) according to claim 55 or 56, wherein when analyzing the data stream (45), a data stream length parameter is included, which indicates the data stream length (246) of each individually accessible portion in order to skip each individually accessible portion.

58. The data stream (45) includes the reconstruction rule (270) for inverse quantization (280) of the neural network parameters (32) associated with each of the neural network portions, The main header portion (47) of the data stream (45) related to the entire neural network, A neural network layer (210, 30) related to the header portion (110) of the data stream (45) related to the neural network layer (210) to which each of the aforementioned neural network portions belongs, or The data stream (45) according to any one of claims 55 to 57, shown in the header portion specific to the neural network portion of the data stream (45) associated with the neural network portion to which each of the aforementioned neural network portions belongs.

59. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) according to any one of claims 1 to 58, wherein the data stream (45) is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), an identification parameter (310) for identifying each predetermined individually accessible portion.

60. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), an identification parameter (310) for identifying each predetermined individually accessible portion.

61. The data stream (45) according to claim 59 or 60, wherein the identification parameter (310) is associated with each of the predetermined individually accessible portions via a hash function or an error detection code or an error correction code.

62. The data stream (45) according to any one of claims 59 to 61, further comprising a higher-level identification parameter (310) for identifying two or more predetermined sets of individually accessible portions.

63. The data stream (45) according to claim 62, wherein the higher-level identification parameter (310) is associated with the identification parameter (310) of two or more predetermined individually accessible portions via a hash function or an error detection code or an error correction code.

64. The data stream (45) according to any one of claims 59 to 63, wherein the individually accessible portions (200) are encoded at the starting point of each individually accessible portion using context-adaptive arithmetic coding (600) and context initialization.

65. The aforementioned data stream (45) provides access to each individually accessible portion, The start code (242) where each of the aforementioned individually accessible parts begins, and / or A pointer (220, 244) that points to the beginning of each of the individually accessible portions, and / or A data stream (45) according to any one of claims 59 to 64, which includes a data stream length parameter indicating the data stream length (246) of each individually accessible portion in order to skip each of the individually accessible portions when analyzing the data stream.

66. The data stream (45) according to any one of claims 59 to 65, wherein the neural network portion includes one or more sub-parts of the neural network layers (210, 30) of the neural network, and / or one or more neural network layers of the neural network.

67. The data stream (45) is encoded in a layered manner, with different versions (330) of the neural network (10) encoded in the data stream (45), and the data stream (45) is structured into one or more individually accessible parts (200), each part relating to a corresponding version (330) of the neural network. The data stream (45) is the first version (330) of the neural network. 2 )but, The second version of the neural network encoded in the second part (330 1 ) is delta-coded for and / or The first version of the neural network (330 2 To perform inference based on ), each of them encodes a second version of the neural network (330) into the second part. 1 A form of one or more compensating neural network portions (332) that are executed in addition to the execution of the corresponding neural network portion (334) of ), The data stream (45) according to any one of claims 1 to 66, which is encoded in the first part in the form of a compensated neural network portion where the outputs of each of the compensated neural network portions (332) and the corresponding neural network portion (334) are summed.

68. The representation of the neural network (10) is a layered encoded data stream (45), Different versions (330) of the neural network are encoded into a data stream (45), the data stream (45) is structured into one or more individually accessible parts (200), each part relating to a corresponding version of the neural network, The data stream (45) is the first version (330) of the neural network. 2 )but, The second version of the neural network encoded in the second part (330 1 ) is delta-coded for and / or The first version of the neural network (330 2 To perform inference based on ), each of them encodes a second version of the neural network (330) into the second part. 1 A form of one or more compensating neural network portions (332) that are executed in addition to the execution of the corresponding neural network portion (334) of ), A data stream (45) encoded in the first part in the form of a compensated neural network portion (332) to which the outputs of each of the compensated neural network portions (332) and the corresponding neural network portion (334) are summed.

69. The data stream (45) is the first version (330) of the neural network. 2 )but, The difference between weights and / or biases, and / or The second version of the neural network (330) encoded in the second part in terms of additional neurons (14, 18, 20) or neuronal interconnections (22, 24) 1 The data stream (45) according to claim 67 or 68, which is delta encoded with respect to the first part.

70. The data stream (45) according to any one of claims 67 to 69, wherein the individually accessible portions (200) are encoded at the starting point of each individually accessible portion using context-adaptive arithmetic coding (600) and context initialization.

71. The data stream (45) has, for each individually accessible portion, The start code (242) where each of the aforementioned individually accessible parts begins, and / or A pointer (220, 244) that points to the beginning of each of the individually accessible portions, and / or The data stream (45) according to any one of claims 67 to 70, wherein when analyzing the data stream (45), a data stream length parameter is included that indicates the data stream length (246) of each individually accessible portion in order to skip each individually accessible portion.

72. The data stream (45) according to any one of claims 67 to 71, wherein the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), an identification parameter (310) for identifying each predetermined individually accessible portion.

73. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) according to any one of claims 1 to 72, wherein the data stream (45) is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), supplemental data (350) for supplementing the representation of the neural network.

74. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the data stream (45) includes, for each of one or more predetermined individually accessible portions (200), supplemental data (350) to supplement the representation of the neural network.

75. The data stream (45) according to claim 73 or 74, wherein the data stream (45) indicates that the supplemental data (350) is not essential for the inference based on the neural network.

76. The data stream (45) according to any one of claims 73 to 75, wherein the data stream (45) is such that the supplemental data (350) for supplementing the representation of the neural network for one or more predetermined individually accessible portions (200) is encoded in another individually accessible portion (200), so that the data stream (45) includes, for each of the one or more predetermined individually accessible portions (200), a corresponding predetermined individually accessible portion relating to the neural network portion to which each predetermined individually accessible portion corresponds.

77. The data stream (45) according to any one of claims 73 to 76, wherein the neural network portion includes the neural network layers (210, 30) of the neural network and / or a subdivided layer portion of a predetermined neural network layer of the neural network.

78. The data stream (45) according to any one of claims 73 to 77, wherein the individually accessible portions (200) are encoded at the starting point of each individually accessible portion using context-adaptive arithmetic coding (600) and context initialization.

79. The data stream (45) has, for each individually accessible portion, The start code (242) where each of the aforementioned individually accessible parts begins, and / or A pointer (220, 244) indicating the beginning of each of the individually accessible portions, and / or The data stream (45) according to any one of claims 73 to 78, wherein when analyzing the data stream (45), a data stream length parameter is included, which indicates the data stream length (246) of each individually accessible portion in order to skip each of the individually accessible portions.

80. The aforementioned supplementary data (350) is, The relevance score of the neural network parameters (32), and / or A data stream (45) according to any one of claims 73 to 79, relating to the perturbation robustness of the neural network parameters (32).

81. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) according to any one of claims 1 to 80, wherein the data stream (45) includes hierarchical control data (400) structured in a sequence (410) of control data portions (420), and the control data portions (420) provide information about the neural network with increasing detail along the sequence of control data portions (420).

82. A data stream (45) in which a representation of the neural network (10) is encoded, The data stream (45) includes hierarchical control data (400) structured in a sequence (410) of control data portions (420), the data stream (45) providing information of the neural network with increasing detail along the sequence of control data portions (420).

83. The data stream (45) according to claim 81 or 82, wherein at least some of the control data portions (420) provide partially redundant information relating to the neural network.

84. The data stream (45) according to any one of claims 81 to 83, wherein the first control data portion provides the information relating to the neural network by indicating a default neural network type with default settings, and the second control data portion includes parameters indicating each of the default settings.

85. A device for encoding a representation of a neural network (10) into a data stream (45), The apparatus is configured to supply to the data stream (45) serialization parameters (102) indicating the coding order (104) in which neural network parameters (32) defining the neuronal interconnections (22, 24) of the neural network are encoded in the data stream (45).

86. The apparatus according to claim 85, wherein the apparatus is configured to encode the neural network parameters (32) into the data stream (45) using context-adaptive arithmetic coding.

87. The aforementioned device is The data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, and The apparatus according to claim 85 or 86, wherein the data stream (45) is configured to encode neural network parameters that define neuronal interconnections (22, 24) of the neural network in a predetermined neural network layer, according to the encoding order (104) indicated by the serialization parameter (102).

88. The apparatus according to any one of claims 85 to 87, wherein the serialization parameter (102) is an n-ary parameter that indicates the coding order (104) from a set (108) of n coding orders (104).

89. The set (108) of the n coding order (104) is, The predetermined coding order is a first predetermined coding order (106) which differs in the order in which it traverses the dimensions (34) of the tensor (30) that describes the predetermined neural network layers (210, 30) of the neural network (10). 1 ), and / or, For scalable coding of the neural network, a second predetermined coding order (106) is used, where the predetermined coding order differs in the number of times (107) that the predetermined neural network layer of the neural network is scanned. 2 ), and / or, The aforementioned predetermined coding order is a third predetermined coding order (106) in which the order in which the neural network layers of the neural network are scanned is different. 3 ), and / or, A fourth predetermined coding order (106) in which the neurons (14, 18, 20) of the neural network layers (210, 30) of the aforementioned neural network are scanned in a different order. 4 The apparatus according to claim 88, including ).

90. The apparatus according to any one of claims 85 to 89, wherein the serialization parameter (102) indicates a permutation used by the coding order (104) to permutate the neurons (14, 18, 20) of the neural network layer (210, 30) relative to the default order.

91. The apparatus according to claim 90, wherein the permutation orders the neurons (14, 18, 20) of the neural network layer (210, 30) such that the neural network parameter (32) increases monotonically along the coding order (104) or decreases monotonically along the coding order (104).

92. The apparatus according to claim 90, wherein the permutation is one of a predetermined coding order that can be signaled by the serialization parameter (102), and the neurons (14, 18, 20) of the neural network layer (210, 30) are ordered such that the bit rate for encoding the permutation indicated by the serialization parameter (102) into the data stream (45) of the neural network parameter (32) is minimized.

93. The apparatus according to any one of claims 85 to 92, wherein the neural network parameters (32) include weights and biases.

94. The apparatus according to any one of claims 85 to 93, wherein the data stream (45) is structured into individually accessible sub-parts (43, 44, 240), each sub-part (43, 44, 240) representing a corresponding neural network part of the neural network, and each sub-part (43, 44, 240) is configured to be fully scanned by the coding sequence (104) before subsequent sub-parts are scanned by the coding sequence (104).

95. The apparatus according to any one of claims 87 to 94, wherein the neural network parameters (32) are encoded into the data stream at the starting point of any individually accessible portion (200) or sub-portions (43, 44, 240) using context-adaptive arithmetic coding and context initialization.

96. The aforementioned device The apparatus according to any one of claims 87 to 95, wherein the data stream is configured to encode a start code (242) in which each individually accessible portion (200) or sub-portion (43, 44, 240) begins, and / or a pointer (220, 244) pointing to the beginning of each individually accessible portion or sub-portion, and / or a pointer (246) to the data stream length of each individually accessible portion or sub-portion in order to skip each individually accessible portion or sub-portion when parsing the data stream.

97. The apparatus according to any one of claims 85 to 96, wherein the apparatus is configured to encode in the data stream numerical representations of the neural network parameters (32) and numerical computation representation parameters (120) indicating the bit size when the neural network (10) is used for inference.

98. A device for encoding a representation of a neural network (10) into a data stream (45), The apparatus is configured to provide the data stream (45) with numerical representations of the neural network parameters (32) of the neural network encoded in the data stream, and numerical computation representation parameters (120) indicating the bit size, when the neural network (10) is used for inference.

99. The apparatus according to any one of claims 85 to 98, wherein the apparatus is configured to structure the data stream (45) into individually accessible sub-parts (43, 44, 240), each individually accessible sub-part representing a corresponding neural network part of the neural network, such that each individually accessible sub-part is fully scanned by the coding sequence (104) before a subsequent individually accessible sub-part is scanned by the coding sequence (104), and the apparatus is configured to encode in the data stream (45) the neural network parameters and type parameters indicating the parameter types of the neural network parameters encoded in the predetermined individually accessible sub-parts for a given individually accessible sub-part.

100. The apparatus according to claim 99, wherein the type parameter distinguishes at least the weights of the neural network from the biases of the neural network.

101. The device is configured to structure the data stream (45) into one or more individually accessible portions (200), each of which represents a corresponding neural network layer (210, 30) of the neural network. The apparatus according to any one of claims 85 to 100, wherein the data stream (45) is configured to encode a neural network layer type parameter (130) indicating the neural network layer type of a predetermined neural network layer of the neural network for a predetermined neural network layer.

102. A device for encoding a representation of a neural network (10) into a data stream (45), The device is configured such that the data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, and the device provides the data stream (45) with a neural network layer type parameter (130) indicating the neural network layer type of the neural network layer of a predetermined neural network layer of the neural network.

103. The apparatus according to claim 101 or 102, wherein the neural network layer type parameter (130) distinguishes at least between fully connected layer types and convolutional layer types.

104. The device is configured to structure the data stream (45) into individually accessible portions (200), each of which represents a corresponding neural network portion of the neural network. The apparatus according to any one of claims 85 to 103, wherein the data stream (45) is configured to encode pointers (220, 244) that point to the beginning of each of the one or more predetermined individually accessible portions.

105. A device for encoding a representation of a neural network (10) into a data stream (45), The device is configured to provide the data stream (45) with pointers (220, 244) that point to the beginning of each of the one or more predetermined individually accessible portions of the data stream (45).

106. Each individually accessible part is, The corresponding neural network layer (210) of the aforementioned neural network, or The apparatus according to claim 104 or 105, which represents the neural network portion (43, 44, 240) of the neural network layer (210) of the aforementioned neural network.

107. The device is configured to encode a representation of the neural network (10) into the data stream (45), the data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, the data stream (45) is further structured within a predetermined portion into individually accessible sub-parts (43, 44, 240), each sub-part (43, 44, 240) representing a corresponding neural network portion of the respective neural network layer of the neural network, and the device encodes the data stream (45) into one or more predetermined individually accessible sub-parts (43, 44, 240), A start code (242) in which each of the aforementioned predetermined individually accessible sub-parts begins, and / or, A pointer (244) that points to the beginning of each of the predetermined individually accessible sub-parts, and / or The apparatus according to any one of claims 85 to 106, configured to provide a data stream length parameter indicating the data stream length (246) of each predetermined individually accessible sub-part for skipping each predetermined individually accessible sub-part when analyzing the data stream.

108. The apparatus according to claim 107, wherein the apparatus is configured to encode the representation of the neural network in the data stream (45) at the starting point of each individually accessible portion and each individually accessible sub-portion using context-adaptive arithmetic coding and context initialization.

109. A device for encoding a representation of a neural network (10) into a data stream (45), The data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, and the data stream (45) is further structured within a predetermined portion into individually accessible sub-parts (43, 44, 240), each sub-part (43, 44, 240) representing a corresponding neural network portion of the respective neural network layer of the neural network, and the device provides the data stream (45) to each of the one or more predetermined individually accessible sub-parts (43, 44, 240), A start code (242) in which each of the aforementioned predetermined individually accessible sub-parts begins, and / or, A pointer (244) that points to the beginning of each of the predetermined individually accessible sub-parts, and / or A device configured to provide a data stream length parameter indicating the data stream length (246) of each predetermined individually accessible sub-part for skipping each predetermined individually accessible sub-part when analyzing the data stream (45).

110. The apparatus according to claim 109, wherein the apparatus is configured to encode the representation of the neural network in the data stream (45) at the starting point of each individually accessible portion and each individually accessible sub-portion using context-adaptive arithmetic coding and context initialization.

111. The apparatus according to any one of claims 85 to 110, wherein the apparatus is configured to encode a representation of a neural network (10) into a data stream, the data stream (45) is structured into individually accessible portions (200), each individually accessible portion representing a corresponding neural network portion of the neural network, and the apparatus is configured to provide the data stream (45) with processing option parameters (250) indicating one or more processing options (252) that must be used or can be optionally used when using the neural network (10) for inference.

112. The processing option parameter (250) indicates one or more available processing options (252) from a predetermined set of processing options (252). The predetermined processing option (252) is, The parallel processing capability of each of the aforementioned predetermined individually accessible portions, and / or, The sample-unit parallel processing capability of each of the aforementioned predetermined individually accessible portions (252 2 ), and / or, The channel-unit parallel processing capability of each of the predetermined individually accessible portions (252 1 ), and / or, The parallel processing capability per classification category for each of the aforementioned predetermined individually accessible parts, and / or, The apparatus according to claim 111, wherein the neural network portion represented by each of the predetermined individually accessible portions has a dependency on computation results obtained from another individually accessible portion of the data stream (45) that relates to the same neural network portion but belongs to another version of the version of the neural network (330) that is layer-encoded in the data stream (45).

113. A device for encoding a representation of a neural network (10) into a data stream (45), A device for data stream coding, wherein the data stream (45) is structured into individually accessible portions (200), each of which represents a corresponding neural network portion of the neural network, and the device is configured to provide the data stream (45) with processing option parameters (250) indicating one or more processing options (252) that must be used or can be optionally used when using the neural network (10) for inference.

114. The processing option parameter (250) indicates one or more available processing options (252) from a predetermined set of processing options (252). The predetermined processing option (252) is, The parallel processing capability of each of the aforementioned predetermined individually accessible portions, and / or, The sample-unit parallel processing capability of each of the aforementioned predetermined individually accessible portions (252 2 ), and / or, The channel-unit parallel processing capability of each of the predetermined individually accessible portions (252 1 ), and / or, The parallel processing capability per classification category for each of the aforementioned predetermined individually accessible parts, and / or, The apparatus according to claim 113, wherein the neural network portion represented by each of the predetermined individually accessible portions has a dependency on computation results obtained from another individually accessible portion of the data stream (45) that relates to the same neural network portion but belongs to another version of the version of the neural network (330) that is layer-encoded in the data stream (45).

115. The apparatus according to any one of claims 85 to 114, wherein the apparatus is configured to encode neural network parameters (32) representing a neural network into a data stream (45), the neural network parameters (32) are encoded into the data stream (45) by being quantized (260) to a quantization index (32''), and the neural network parameters (32) are encoded into the data stream (45) such that the neural network parameters (32) in different neural network portions of the neural network are quantized (260) differently, and the apparatus is configured to provide, for each of the neural network portions, the data stream (45) showing a reconstruction rule (270) for dequantizing (280) the neural network parameters (32) associated with each of the neural network portions.

116. A device for encoding neural network parameters (32) representing a neural network into a data stream (45), The neural network parameters (32) are quantized (260) to a quantization index (32'') and encoded into the data stream (45), such that the neural network parameters (32) in different neural network portions of the neural network are quantized (260) differently, and the device is configured to provide, for each of the neural network portions, the data stream (45) showing a reconstruction rule (270) for inverse quantization (280) of the neural network parameters (32) associated with each of the neural network portions.

117. The apparatus according to claim 115 or 116, wherein the neural network portion includes a neural network layer (210, 30) of the neural network and / or a subdivided portion of a predetermined neural network layer of the neural network.

118. The apparatus applies a first reconstruction rule (270) to the data stream (45) for inverse quantization (280) of the neural network parameters (32) related to the first neural network portion. 1 ,270a 1 ) is used as the second reconstruction rule (270) for inverse quantization (280) of the neural network parameters (32) related to the second neural network portion. 2 ,270a 2 The apparatus according to any one of claims 115 to 117, which is configured to encode by delta encoding for ).

119. The device applies the first reconstruction rule (270) to the data stream (45). 1 ,270a 1 A first index value to indicate ) and the second reconstruction rule (270 2 ,270a 2 It is configured to encode a second exponential value to indicate ), The first reconstruction rule (270 1 ,270a 1 ) is defined by a first quantization step size (263) defined by a predetermined basis exponent and a first exponent defined by the first exponent value, and The second reconstruction rule (270 2 ,270a 2 The apparatus according to claim 118, wherein the second quantization step size (263) is defined by the exponent of the predetermined basis, and the second exponent is defined by the sum of the first and second exponent values.

120. The apparatus according to claim 119, wherein the data stream further represents a predetermined basis.

121. The apparatus applies a first reconstruction rule (270) to the data stream for dequantizing (280) the neural network parameters (32) related to the first neural network portion. 1 ,270a 1 A first exponential value to show ) and a second reconstruction rule (270) to inverse quantize (280) the neural network parameters (32) related to the second neural network portion. 2 ,270a 2 It is configured to encode a second exponential value to indicate ), The first reconstruction rule (270 1 ,270a 1 ) is defined by a first quantization step size (263) defined by a predetermined basis exponent, and a first exponent defined by the sum of the first exponent value and the predetermined exponent value, The second reconstruction rule (270 2 ,270a 2 The apparatus according to any one of claims 115 to 118, wherein the second quantization step size (263) is defined by the exponent of the predetermined basis, and the second exponent is defined by the sum of the second exponent value and the predetermined exponent value.

122. The apparatus according to claim 121, wherein the data stream further represents a predetermined basis.

123. The apparatus according to claim 122, wherein the data stream represents the predetermined basis within the neural network range.

124. The apparatus according to any one of claims 121 to 123, wherein the data stream further indicates the predetermined exponential value.

125. The apparatus according to claim 125, wherein the data stream exhibits the predetermined exponential value in the neural network layer (210, 30) range.

126. The apparatus according to claim 124 or 125, wherein the data stream further indicates the predetermined basis, and the data stream indicates the predetermined exponential value in a range finer than the range indicated by the data stream.

127. The apparatus according to any one of claims 119 to 126, wherein the apparatus is configured to encode the predetermined basis in a non-integer format and the first and second exponential values ​​in an integer format in the data stream.

128. The device applies the first reconstruction rule (270) to the data stream. 1 ,270a 1 To demonstrate this, a first parameter set (264) defines a first quantization index-reconstruction level mapping (265), and the second reconstruction rule (270 2 ,270a 2 To demonstrate this, it is configured to encode a second set of parameters (264) that defines a second quantization index-reconstruction level mapping (265), The first reconstruction rule (270 1 ,270a 1 ) is defined by the first quantization index-reconstruction level mapping (265), and The second reconstruction rule (270 2 , 270a 2 ) is defined by extending the first quantization index - reconstruction level mapping (265) in a predetermined manner by the second quantization index - reconstruction level mapping (265), the apparatus according to any one of claims 118 to 127.

129. The device applies the first reconstruction rule (270) to the data stream. 1 ,270a 1 To demonstrate this, a first parameter set (264) defines a first quantization index-reconstruction level mapping (265), and the second reconstruction rule (270 2 ,270a 2 To demonstrate this, it is configured to encode a second set of parameters (264) that defines a second quantization index-reconstruction level mapping (265), The first reconstruction rule (270 1 ,270a 1 ) is defined by extending a predetermined quantization index-reconstruction level mapping (265) in a predetermined manner by a first quantization index-reconstruction level mapping (265), The second reconstruction rule (270 2 , 270a 2 ) is defined by extending the predetermined quantization index-reconstruction level mapping (265) in a predetermined manner by the second quantization index-reconstruction level mapping (265), the apparatus according to any one of claims 118 to 128.

130. The apparatus according to claim 129, wherein the data stream further represents the predetermined quantization index-reconstruction level mapping (265).

131. The apparatus according to claim 130, wherein the data stream exhibits the predetermined quantization index-reconstruction level mapping (265) over the neural network range or the neural network layer (210, 30) range.

132. According to the aforementioned predetermined method, The mapping of each index value (32'') to a first reconstruction level according to the extended quantization index-reconstruction level mapping is replaced by, if there is a mapping of each index value (32'') to a second reconstruction level according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, and / or According to the extended quantization index-reconstruction level mapping, no reconstruction level is defined to which each index value (32'') should be mapped, and according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, for any index value (32'') that is mapped to a corresponding reconstruction level, the mapping from each index value (32'') to the corresponding reconstruction level is adopted, and / or The apparatus according to any one of claims 128 to 131, wherein, according to the extended quantization index-reconstruction level mapping, no reconstruction level is defined to which each of the index values ​​(32'') should be mapped, and, according to the extended quantization index-reconstruction level mapping, for any index value (32'') that is mapped to a corresponding reconstruction level, the mapping from each of the index values ​​(32'') to the corresponding reconstruction level is adopted.

133. The device provides the data stream with the reconstruction rule (270) for a predetermined neural network portion, The quantization step size parameter (262) indicates the quantization step size (263), It is configured to encode a parameter set (264) that defines a quantization index-reconstruction level mapping (265), The reconstruction rule (270) for the predetermined neural network portion is: The quantization step size (263) for a quantization index (32'') within a predetermined index interval (268), and The apparatus according to any one of claims 115 to 132, defined by a quantization index-reconstruction level mapping (265) for quantization indices (32'') outside the predetermined index interval (268).

134. A device for encoding neural network parameters (32) representing a neural network into a data stream (45), The neural network parameters (32) are quantized (260) to a quantization index (32'') and encoded into the data stream (45). The apparatus provides the data stream (45) with a reconstruction rule (270) for inverse quantization (280) of the neural network parameters (32), The quantization step size parameter (262) indicates the quantization step size (263), It is configured to provide a parameter set (264) that defines a quantization index-reconstruction level mapping (265), The reconstruction rule (270) for the predetermined neural network portion is: The quantization step size (263) for a quantization index (32'') within a predetermined index interval (268), and A device defined by a quantization index-reconstruction level mapping (265) for quantization indices (32'') outside the predetermined index interval (268).

135. The apparatus according to claim 133 or 134, wherein the predetermined index interval (268) includes zero.

136. The apparatus according to claim 135, wherein the predetermined index interval (268) is extended to a predetermined size threshold, and quantization indices (32'') exceeding the predetermined size threshold represent escape codes indicating that the quantization index-reconstruction level mapping (265) is used for inverse quantization (280).

137. The apparatus according to any one of claims 133 to 136, wherein the parameter set (264) defines a quantization index-reconstruction level mapping (265) by a list of reconstruction levels associated with quantization indices (32'') outside the predetermined index interval (268).

138. The apparatus according to any one of claims 115 to 137, wherein the neural network portion includes one or more sub-parts of the neural network layers (210, 30) of the neural network, and / or one or more neural network layers of the neural network.

139. The apparatus according to any one of claims 115 to 138, wherein the apparatus is configured to structure the data stream (45) into individually accessible portions (200) and encode the neural network parameters (32) for the corresponding neural network portions into each individually accessible portion.

140. The apparatus according to claim 139, wherein the apparatus is configured to encode the individually accessible portion (200) into a data stream using context-adaptive arithmetic coding and context initialization at the starting point of each individually accessible portion.

141. The device provides access to each portion of the data stream individually. The start code (242) where each of the aforementioned individually accessible parts begins, and / or A pointer (220, 244) indicating the beginning of each of the individually accessible parts, and / or The apparatus according to claim 139 or 140, configured to encode a data stream length parameter indicating the data stream length (246) of each individually accessible portion for skipping each individually accessible portion when analyzing the data stream.

142. The device provides instructions for each of the neural network portions of a reconstruction rule (270) for inverse quantization (280) of the neural network parameters (32) associated with each of the neural network portions of the neural network, The main header portion (47) of the data stream related to the entire neural network, A neural network layer (210, 30) related to the header portion (110) of the data stream related to the neural network layer (210) to which each of the aforementioned neural network portions belongs, or In the neural network portion-specific header portion of the data stream to which each of the aforementioned neural network portions belongs and which is related to each of the aforementioned neural network portions, The apparatus according to any one of claims 139 to 141, configured to encode the data stream.

143. The apparatus according to any one of claims 85 to 142, wherein the apparatus is configured to encode a representation of a neural network (10) into a data stream (45), the data stream (45) is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the apparatus is configured to provide the data stream (45) with an identification parameter (310) for identifying each of the one or more predetermined individually accessible portions.

144. A device for encoding a representation of a neural network (10) into a data stream (45), The device is configured such that the data stream (45) is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the device provides, for each of one or more predetermined individually accessible portions, the data stream (45) with an identification parameter (310) for identifying each predetermined individually accessible portion.

145. The apparatus according to claim 143 or 144, wherein the identification parameter (310) relates to each of the predetermined individually accessible portions via a hash function or an error detection code or an error correction code.

146. The apparatus according to any one of claims 143 to 145, wherein the apparatus is configured to encode a higher-level identification parameter (310) for identifying two or more predetermined sets of individually accessible portions in the data stream (45).

147. The apparatus according to claim 146, wherein the higher-level identification parameter (310) is associated with the identification parameter (310) of one or more predetermined individually accessible portions via a hash function or an error detection code or an error correction code.

148. The apparatus according to any one of claims 143 to 147, wherein the apparatus is configured to encode the individually accessible portion (200) into the data stream using context-adaptive arithmetic coding and context initialization at the starting point of each individually accessible portion.

149. The device, for each individually accessible portion, applies the following to the data stream: The start code (242) where each of the aforementioned individually accessible parts begins, and / or A pointer (220, 244) that points to the beginning of each of the individually accessible portions, and / or The apparatus according to any one of claims 143 to 148, wherein when analyzing the data stream, it is configured to encode a data stream length parameter indicating the data stream length (246) of each individually accessible portion for skipping each of the individually accessible portions.

150. The apparatus according to any one of claims 143 to 149, wherein the neural network portion includes neural network layers (210, 30) of the neural network, and / or one or more sub-portions of one or more neural network layers (210, 30) of the neural network.

151. The device is configured to encode a representation of the neural network (10) into a data stream (45) in a layered manner, wherein different versions (330) of the neural network are encoded into the data stream (45), and the data stream (45) is structured into one or more individually accessible portions (200), each portion relating to a corresponding version of the neural network. The aforementioned device is The first version of the aforementioned neural network (330 2 )of, The second version of the neural network encoded in the second part (330 1 ) is delta-coded for and / or The first version of the neural network (330 2 To perform inference based on ), a second version of the neural network (330) is used, each encoded in the second part. 1 A form of one or more compensating neural network portions (332) that are executed in addition to the execution of the corresponding neural network portion (334) of ), and In the form of a compensated neural network portion, the outputs of each of the aforementioned compensated neural network portions (332) and the corresponding neural network portion (334) are summed, The apparatus according to any one of claims 85 to 150, configured to encode the first part.

152. A device for encoding the representation of a neural network (10) into a data stream (45) in a layered manner, Different versions (330) of the neural network are encoded in the data stream (45), and the data stream (45) is structured into one or more individually accessible parts (200), each part relating to a corresponding version of the neural network. The aforementioned device is The first version of the aforementioned neural network (330 2 )of, The second version of the neural network encoded in the second part (330 1 ) is delta coded, and / or, The first version of the neural network (330 2 To perform inference based on ), a second version of the neural network (330) is used, each encoded in the second part. 1 A form of one or more compensating neural network portions (332) that are executed in addition to the execution of the corresponding neural network portion (334) of ), and In a form of a compensated neural network portion (332) in which the outputs of each of the aforementioned compensated neural network portions (332) and the corresponding neural network portion (334) are summed, A device configured to encode into a first part.

153. The apparatus provides the second version of the neural network (330) to the second portion of the data stream. 1 ) is configured to encode; and, The aforementioned device is The first version of the neural network (330 2 )of, The difference between weights and / or biases, and / or Additional neurons (14, 18, 20) or neuronal interconnections (22, 24) The second version of the neural network encoded in the second part (330 1 The apparatus according to claim 151 or 152, configured to delta encode the first portion of the data stream.

154. The apparatus according to any one of claims 151 to 153, wherein the apparatus is configured to encode the individually accessible portion (200) into the data stream using context-adaptive arithmetic coding (600) and context initialization at the starting point of each individually accessible portion.

155. The device, for each individually accessible portion, includes the data stream, The start code (242) where each of the aforementioned individually accessible parts begins, and / or A pointer (220, 244) that points to the beginning of each of the individually accessible portions, and / or The apparatus according to any one of claims 151 to 154, wherein when analyzing the data stream, it is configured to encode a data stream length parameter indicating the data stream length (246) of each individually accessible portion for skipping each of the individually accessible portions.

156. The apparatus according to any one of claims 151 to 155, wherein the apparatus is configured to encode an identification parameter (310) in a data stream for each of one or more predetermined individually accessible portions (200) to identify each of the predetermined individually accessible portions.

157. The apparatus according to any one of claims 85 to 156, wherein the apparatus is configured to encode a representation of a neural network (10) into a data stream (45), the data stream (45) being structured into individually accessible portions (200), each portion representing a corresponding portion of the neural network, and the apparatus is configured to provide the data stream (45) with supplemental data (350) to supplement the representation of the neural network for each of one or more predetermined individually accessible portions (200).

158. A device for encoding a representation of a neural network (10) into a data stream (45), The device is configured such that the data stream (45) is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the device provides the data stream (45) with supplemental data (350) to supplement the representation of the neural network for each of one or more predetermined individually accessible portions (200).

159. The apparatus according to claim 157 or 158, wherein the data stream (45) indicates that the supplemental data (350) is not essential for inference based on the neural network.

160. The apparatus according to any one of claims 157 to 159, wherein the apparatus is configured to encode the supplementary data (350) for supplementing the representation of the neural network for one or more predetermined individually accessible portions (200) into another individually accessible portion (200), and the data stream includes, for one or more predetermined individually accessible portions (200), a corresponding other predetermined individually accessible portion relating to the neural network portion to which each predetermined individually accessible portion corresponds.

161. The apparatus according to any one of claims 157 to 160, wherein the neural network portion includes the neural network layers (210, 30) of the neural network and / or a subdivided layer portion of a predetermined neural network layer (210, 30) of the neural network.

162. The apparatus according to any one of claims 157 to 161, wherein the apparatus is configured to encode the individually accessible portion (200) at the starting point of each individually accessible portion using context-adaptive arithmetic coding and context initialization.

163. The aforementioned device, with respect to each individually accessible part, The start code (242) where each of the aforementioned individually accessible parts begins, and / or A pointer (220, 244) indicating the beginning of each of the individually accessible portions, and / or The apparatus according to any one of claims 157 to 162, wherein when analyzing the data stream, a data stream length parameter indicating the data stream length (246) of each individually accessible portion for skipping each individually accessible portion is encoded in the data stream.

164. The aforementioned supplementary data (350) is, The relevance score of the neural network parameters (32), and / or The apparatus according to any one of claims 157 to 163, relating to the perturbation robustness of the neural network parameters (32).

165. A device for encoding a representation of a neural network (10) into a data stream (45), The apparatus according to any one of claims 85 to 164, wherein the apparatus is configured to provide the data stream (45) with hierarchical control data (400) structured in a sequence (410) of control data portions (420), and the control data portions provide information about the neural network with increasing detail in the sequence of control data portions.

166. A device for encoding a representation of a neural network (10) into a data stream (45), The apparatus is configured to provide the data stream (45) with hierarchical control data (400) structured in a sequence (410) of control data portions (420), wherein the control data portions provide information of the neural network with increasing detail along the sequence of control data portions.

167. The apparatus according to claim 165 or 166, wherein at least some of the control data portions (420) provide partially redundant information relating to the neural network.

168. The apparatus according to any one of claims 165 to 167, wherein the first control data portion provides the information relating to the neural network by indicating a default neural network type with default settings, and the second control data portion includes parameters indicating each of the default settings.

169. A device for decoding a representation of a neural network (10) from a data stream (45), The apparatus is configured to decode serialization parameters (102) from the data stream (45) that indicate the encoding order (104) in which neural network parameters (32) defining the neuronal interconnections (22, 24) of the neural network are encoded in the data stream (45).

170. The apparatus according to claim 169, wherein the apparatus is configured to decode the neural network parameters (32) from the data stream (45) using context-adaptive arithmetic decoding.

171. The data stream is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network. The device is configured to continuously decode neural network parameters that define the neuronal interconnections (22, 24) of the neural network within a predetermined neural network layer from the data stream (45). The apparatus according to claim 169 or 170, wherein the neural network parameters sequentially decoded from the data stream (45) are assigned to the neuron interconnects (22, 24) using the coding sequence (104).

172. The apparatus according to any one of claims 169 to 171, wherein the serialization parameter (102) is an n-ary parameter that indicates the coding order (104) from a set (108) of n coding orders (104).

173. The set (108) of the n coding order (104) is, The predetermined coding order is a first predetermined coding order (106) which differs in the order in which it traverses the dimensions (34) of the tensor (30) that describes the predetermined neural network layers (210, 30) of the neural network (10). 1 ), and / or, For scalable coding of the neural network, a second predetermined coding sequence (106) is used, where the predetermined coding sequence differs in the number of times (107) that it scans the predetermined neural network layers (210, 30) of the neural network. 2 ), and / or, The aforementioned predetermined coding order is a third predetermined coding order (106) in which the order in which the neural network layers of the neural network are scanned is different. 3 ), and / or, A fourth predetermined coding order in which the neurons (14, 18, 20) of the neural network layer of the aforementioned neural network are scanned in a different order (106 4 The apparatus according to claim 172, including ).

174. The apparatus according to any one of claims 169 to 173, wherein the serialization parameter (102) indicates a permutation used by the coding order (104) to permutate the neurons (14, 18, 20) of the neural network layer (210, 30) relative to the default order.

175. The apparatus according to claim 174, wherein the permutation orders the neurons (14, 18, 20) of the neural network layer (210, 30) such that the neural network parameter (32) increases monotonically along the coding order (104) or decreases monotonically along the coding order (104).

176. The apparatus according to claim 174, wherein the permutation is one of a predetermined coding order that can be signaled by the serialization parameter (102), and the neurons (14, 18, 20) of the neural network layer (210, 30) are ordered such that the bit rate for encoding the neural network parameter (32) into the data stream (45) is minimized for the permutation indicated by the serialization parameter (102).

177. The apparatus according to any one of claims 169 to 176, wherein the neural network parameters (32) include weights and biases.

178. The apparatus according to any one of claims 169 to 177, wherein the apparatus is configured to decode from the data stream into individually accessible sub-parts (43, 44, 240) in which the data stream is structured into individually accessible parts (200), and each sub-part (43, 44, 240) represents a corresponding neural network part of the neural network, such that each sub-part (43, 44, 240) is fully scanned by the coding sequence (104) before subsequent sub-parts are scanned by the coding sequence (104).

179. The apparatus according to any one of claims 171 to 178, wherein the neural network parameters (32) are decoded from the data stream using context-adaptive arithmetic decoding and context initialization at the starting point of any individually accessible portion (200) or sub-portion (43, 44, 240).

180. The apparatus according to any one of claims 171 to 179, wherein the apparatus is configured to decode from the data stream a start code (242) in which each individually accessible portion (200) or sub-portion (43, 44, 240) begins, and / or a pointer (220, 244) pointing to the beginning of each individually accessible portion or sub-portion, and / or a pointer (246) to the data stream length of each individually accessible portion or sub-portion in order to skip each individually accessible portion or sub-portion when parsing the data stream.

181. The apparatus according to any one of claims 169 to 180, wherein the apparatus is configured to decode from the data stream numerical representations of the neural network parameters (32) and numerical computation representation parameters (120) indicating the bit size when the neural network (10) is used for inference.

182. A device for decoding a representation of a neural network (10) from a data stream (45), The device is configured to decode numerical computation representation parameters (120) that indicate numerical representations and bit sizes of the neural network parameters (32) of the neural network encoded in the data stream (45) when the neural network (10) is used for inference, and is configured to use the numerical representations and bit sizes for representing the neural network parameters (32) decoded from the data stream (45).

183. The apparatus according to any one of claims 169 to 182, wherein the data stream (45) is structured into individually accessible sub-parts (43, 44, 240), and each individually accessible sub-part represents a corresponding neural network part of the neural network, such that each individually accessible sub-part is fully scanned by the coding sequence (104) before a subsequent individually accessible sub-part is scanned by the coding sequence (104), and the apparatus is configured to decode from the data stream (45) for a predetermined individually accessible sub-part the neural network parameters and type parameters indicating the parameter type of the neural network parameters decoded from the predetermined individually accessible sub-part.

184. The apparatus according to claim 183, wherein the type parameter distinguishes at least the weights of the neural network from the biases of the neural network.

185. The data stream (45) is structured into one or more individually accessible parts (200), and each of the one or more individually accessible parts represents the corresponding neural network layers (210, 30) of the neural network. The apparatus according to any one of claims 169 to 184, wherein the apparatus is configured to decode from the data stream (45) a neural network layer type parameter (130) indicating the neural network layer type of a predetermined neural network layer of the neural network.

186. A device for decoding a representation of a neural network (10) from a data stream (45), The device is configured to decode from the data stream (45) into one or more individually accessible portions (200), each portion representing a corresponding neural network layer (210, 30) of the neural network, and to decode from the data stream (45) a neural network layer type parameter (130) indicating the neural network layer type of a predetermined neural network layer (210, 30) of the neural network.

187. The apparatus according to claim 185 or 186, wherein the neural network layer type parameter (130) distinguishes at least between fully connected layer types and convolutional layer types.

188. The data stream (45) is structured into individually accessible portions (200), and each individually accessible portion represents the corresponding neural network portion of the neural network. The apparatus according to any one of claims 169 to 187, wherein the apparatus is configured to decode, for each of one or more predetermined individually accessible portions (200) from the data stream (45), pointers (220, 244) pointing to the beginning of each individually accessible portion.

189. A device for decoding a representation of a neural network (10) from a data stream (45), The data stream (45) is structured into one or more individually accessible portions (200), each portion representing a corresponding neural network layer (210, 30) of the neural network, and the device is configured to decode from the data stream (45) one or more predetermined individually accessible portions, for each of the predetermined individually accessible portions, pointers (220, 244) pointing to the beginning of each predetermined individually accessible portion.

190. Each individually accessible part is, The corresponding neural network layer (210) of the aforementioned neural network, or The apparatus according to claim 188 or 189, which represents the neural network portion (43, 44, 240) of the neural network layer (210) of the aforementioned neural network.

191. The device is configured to decode a representation of the neural network (10) from the data stream (45), wherein the data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, and the data stream (45) is further structured within a predetermined portion into individually accessible sub-parts (43, 44, 240), each sub-part (43, 44, 240) representing a corresponding neural network portion of the respective neural network layer (210, 30) of the neural network, and the device decodes from the data stream (45) for each of the one or more predetermined individually accessible sub-parts (43, 44, 240): A start code (242) in which each of the aforementioned predetermined individually accessible sub-parts begins, and / or, A pointer (244) that points to the beginning of each of the predetermined individually accessible sub-parts, and / or The apparatus according to any one of claims 169 to 190, configured to decode a data stream length parameter indicating the data stream length (246) of each predetermined individually accessible sub-part for skipping each predetermined individually accessible sub-part when analyzing the data stream (45).

192. The apparatus according to claim 191, wherein the apparatus is configured to decode the representation of the neural network from the data stream (45) at the starting points of each individually accessible portion and each individually accessible sub-portion, using context-adaptive arithmetic decoding and context initialization.

193. A device for decoding a representation of a neural network (10) from a data stream (45), The data stream (45) is structured into one or more individually accessible portions (200), each individually accessible portion representing a corresponding neural network layer (210, 30) of the neural network, and the data stream (45) is further structured within a predetermined portion into individually accessible sub-parts (43, 44, 240), each sub-part (43, 44, 240) representing a corresponding neural network portion of the respective neural network layer (210, 30) of the neural network, and the device accesses each of the one or more predetermined individually accessible sub-parts (43, 44, 240) from the data stream (45), A start code (242) in which each of the aforementioned predetermined individually accessible sub-parts begins, and / or, A pointer (244) to the start of each of the predetermined individually accessible sub-parts, and / or A device configured to decode a data stream length parameter indicating the data stream length (246) of each predetermined individually accessible sub-part for skipping each predetermined individually accessible sub-part when analyzing the data stream (45).

194. The apparatus according to claim 193, wherein the apparatus is configured to decode the representation of the neural network from the data stream (45) at the starting points of each individually accessible portion and each individually accessible sub-portion, using context-adaptive arithmetic decoding and context initialization.

195. The apparatus according to any one of claims 169 to 194, wherein the apparatus is configured to decode a representation of a neural network (10) from a data stream (45), the data stream (45) is structured into individually accessible portions (200), each individually accessible portion representing a corresponding neural network portion of the neural network, and the apparatus is configured to decode from the data stream (45) for each of one or more predetermined individually accessible portions (200) one or more processing option parameters (250) indicating one or more processing options (252) that must be used or can be optionally used when using the neural network (10) for inference.

196. The processing option parameter (250) indicates one or more available processing options (252) from a predetermined set of processing options (252). The predetermined processing option (252) is, The parallel processing capability of each of the aforementioned predetermined individually accessible portions, and / or, The sample-unit parallel processing capability of each of the aforementioned predetermined individually accessible portions (252 2 ), and / or, The channel-unit parallel processing capability of each of the predetermined individually accessible portions (252 1 ), and / or, The parallel processing capability per classification category for each of the aforementioned predetermined individually accessible parts, and / or, The apparatus according to claim 195, wherein the neural network portion represented by each of the predetermined individually accessible portions has a dependency on computation results obtained from another individually accessible portion of the data stream (45) that relates to the same neural network portion but belongs to another version of the version of the neural network (330) that is layer-encoded in the data stream (45).

197. A device for decoding a representation of a neural network (10) from a data stream (45), The device is configured to decode, for each of the one or more predetermined one or more individually accessible portions, from the data stream (45) the processing option parameters (250) indicating one or more processing options (252) that must be used or can be used optionally when using the neural network (10) for inference.

198. The processing option parameter (250) indicates one or more available processing options (252) from a predetermined set of processing options (252). The predetermined processing option (252) is, The parallel processing capability of each of the aforementioned predetermined individually accessible portions, and / or, The sample-unit parallel processing capability of each of the aforementioned predetermined individually accessible portions (252 2 ), and / or, The channel-unit parallel processing capability of each of the predetermined individually accessible portions (252 1 ), and / or, The parallel processing capability per classification category for each of the aforementioned predetermined individually accessible parts, and / or, The apparatus according to claim 197, wherein the neural network portion represented by each of the predetermined individually accessible portions has a dependency on computation results obtained from another individually accessible portion of the data stream (45) that relates to the same neural network portion but belongs to another version of the version of the neural network (330) that is layer-encoded in the data stream (45).

199. The apparatus according to any one of claims 169 to 198, wherein the apparatus is configured to decode neural network parameters (32) representing a neural network from a data stream (45), the neural network parameters (32) are encoded in the data stream (45) by being quantized (260) to a quantization index (32''), the neural network parameters (32) are encoded in the data stream (45) such that the neural network parameters (32) in different neural network portions of the neural network are quantized (260) differently, and the apparatus is configured to decode, for each of the neural network portions, a reconstruction rule (270) for dequantizing (280) the neural network parameters (32) associated with each neural network portion from the data stream (45).

200. A device for decoding neural network parameters (32) representing a neural network from a data stream (45), The neural network parameters (32) are quantized (260) to a quantization index (32'') and encoded into the data stream (45), and the neural network parameters (32) are encoded into the data stream (45) such that the neural network parameters (32) in different neural network portions of the neural network are quantized (260) differently, and the device is configured to decode a reconstruction rule (270) from the data stream (45) for each of the neural network portions to dequantize (280) the neural network parameters (32) associated with each of the neural network portions.

201. The apparatus according to claim 199 or 200, wherein the neural network portion includes a neural network layer (210, 30) of the neural network and / or a subdivided layer portion of a predetermined neural network layer of the neural network.

202. The apparatus uses a first reconstruction rule (270) to dequantize the neural network parameters (280) related to the first neural network portion from the data stream (45). 1 ,270a 1 ) is used as the second reconstruction rule (270) for inverse quantizing the neural network parameters (280) related to the second neural network portion. 2 ,270a 2 The apparatus according to any one of claims 199 to 201, configured to decode by delta decoding against ).

203. The device uses the data stream (45) to determine the first reconstruction rule (270 1 ,270a 1 A first index value to indicate ) and the second reconstruction rule (270 2 ,270a 2 It is configured to decode a second exponential value to indicate ), The first reconstruction rule (270 1 ,270a 1 ) is defined by a first quantization step size (263) defined by a predetermined basis exponent and a first exponent defined by the first exponent value, and The second reconstruction rule (270 2 ,270a 2 The apparatus according to claim 202, wherein the second quantization step size (263) is defined by the exponent of the predetermined basis, and the second exponent is defined by the sum of the first and second exponent values.

204. The apparatus according to claim 203, wherein the data stream (45) further represents a predetermined basis.

205. The apparatus uses a first reconstruction rule (270) to dequantize (280) the neural network parameters (32) related to the first neural network portion from the data stream (45). 1 ,270a 1 A first exponential value to show ) and a second reconstruction rule (270) to inverse quantize (280) the neural network parameters (32) related to the second neural network portion. 2 ,270a 2 It is configured to decode a second exponential value to show ), The first reconstruction rule (270 1 ,270a 1 ) is defined by a first quantization step size (263) defined by a predetermined basis exponent, and a first exponent defined by the sum of the first exponent value and the predetermined exponent value, The second reconstruction rule (270 2 ,270a 2 The apparatus according to any one of claims 199 to 202, wherein the second quantization step size (263) is defined by the exponent of the predetermined basis, and the second exponent is defined by the sum of the second exponent value and the predetermined exponent value.

206. The apparatus according to claim 205, wherein the data stream further represents a predetermined basis.

207. The apparatus according to claim 206, wherein the data stream represents the predetermined basis within the neural network range.

208. The apparatus according to any one of claims 205 to 207, wherein the data stream further indicates the predetermined exponential value.

209. The apparatus according to claim 208, wherein the data stream (45) exhibits the predetermined exponential value in the neural network layer (210, 30) range.

210. The apparatus according to claim 208 or 209, wherein the data stream further indicates the predetermined basis, and the data stream indicates the predetermined exponential value in a range finer than the range indicated by the data stream.

211. The apparatus according to any one of claims 203 to 210, wherein the apparatus is configured to decode from the data stream the predetermined basis in non-integer form and the first and second exponential values ​​in integer form.

212. The device uses the first reconstruction rule (270) from the data stream. 1 ,270a 1 To demonstrate this, a first parameter set (264) defines a first quantization index-reconstruction level mapping (265), and the second reconstruction rule (270 2 ,270a 2 To demonstrate this, it is configured to decode a second set of parameters (264) that defines a second quantization index-reconstruction level mapping (265), The first reconstruction rule (270 1 ,270a 1 ) is defined by the first quantization index-reconstruction level mapping (265), The second reconstruction rule (270 2 ,270a 2 The apparatus according to any one of claims 202 to 211, wherein the first quantization index-reconstruction level mapping (265) is extended in a predetermined manner with the second quantization index-reconstruction level mapping (265).

213. The device uses the first reconstruction rule (270) from the data stream. 1 ,270a 1 To demonstrate this, a first parameter set (264) defines a first quantization index-reconstruction level mapping (265), and the second reconstruction rule (270 2 ,270a 2 To demonstrate this, it is configured to decode a second set of parameters (264) that defines a second quantization index-reconstruction level mapping (265), The first reconstruction rule (270 1 ,270a 1 ) is defined by extending the predetermined quantization index-reconstruction level mapping (265) in a predetermined manner with the first quantization index-reconstruction level mapping (265), The second reconstruction rule (270 2 ,270a 2 The apparatus according to any one of claims 202 to 212, wherein the predetermined quantization index-reconstruction level mapping (265) is extended in a predetermined manner with the second quantization index-reconstruction level mapping (265).

214. The apparatus according to claim 213, wherein the data stream further represents the predetermined quantization index-reconstruction level mapping (265).

215. The apparatus according to claim 214, wherein the data stream exhibits the predetermined quantization index-reconstruction level mapping (265) over the neural network range or the neural network layer (210, 30) range.

216. According to the aforementioned predetermined method, The mapping of each index value (32'') to a first reconstruction level according to the extended quantization index-reconstruction level mapping is replaced by, if there is a mapping of each index value (32'') to a second reconstruction level according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, and / or According to the extended quantization index-reconstruction level mapping, no reconstruction level is defined to which each of the index values ​​(32'') should be mapped, and according to the quantization index-reconstruction level mapping that extends the extended quantization index-reconstruction level mapping, for any index value (32'') that is mapped to a corresponding reconstruction level, the mapping from each of the index values ​​(32'') to the corresponding reconstruction level is adopted, and / or The apparatus according to any one of claims 212 to 215, wherein, according to the extended quantization index-reconstruction level mapping, no reconstruction level is defined to which each of the index values ​​(32'') should be mapped, and, according to the extended quantization index-reconstruction level mapping, for any index value (32'') that is mapped to a corresponding reconstruction level, the mapping from each of the index values ​​(32'') to the corresponding reconstruction level is adopted.

217. The device is configured to perform decoding from the data stream to indicate a predetermined neural network reconstruction rule (270), The quantization step size parameter (262) indicates the quantization step size (263), Includes a parameter set (264) that defines a quantization index-reconstruction level mapping (265), The reconstruction rule (270) for the predetermined neural network portion is: The quantization step size (263) for a quantization index (32'') within a predetermined index interval (268), and The apparatus according to any one of claims 199 to 216, defined by a quantization index-reconstruction level mapping (265) for quantization indices (32'') outside the predetermined index interval (268).

218. A device for decoding neural network parameters (32) representing a neural network from a data stream (45), The neural network parameters (32) are quantized (260) to a quantization index (32'') and encoded into the data stream (45). The aforementioned device The quantization step size parameter (262) indicates the quantization step size (263), The system is configured such that a set of parameters (264) defining a quantization index-reconstruction level mapping (265) can be decoded from the data stream (45) to derive a reconstruction rule (270) for dequantizing (280) the neural network parameters (32). The reconstruction rule (270) for the predetermined neural network portion is: The quantization step size (263) for a quantization index (32'') within a predetermined index interval (268), and A device defined by a quantization index-reconstruction level mapping (265) for quantization indices (32'') outside the predetermined index interval (268).

219. The apparatus according to claim 217 or 218, wherein the predetermined index interval (268) includes zero.

220. The apparatus according to claim 219, wherein the predetermined index interval (268) is extended to a predetermined threshold, and quantization indices (32'') exceeding the predetermined threshold represent escape codes indicating that the quantization index-reconstruction level mapping (265) is used for inverse quantization (280).

221. The apparatus according to any one of claims 217 to 220, wherein the parameter set (264) defines a quantization index-reconstruction level mapping (265) by a list of reconstruction levels associated with quantization indices (32'') outside the predetermined index interval (268).

222. The apparatus according to any one of claims 199 to 221, wherein the neural network portion includes one or more sub-parts of the neural network layers (210, 30) of the neural network, and / or one or more neural network layers of the neural network.

223. The apparatus according to any one of claims 199 to 222, wherein the data stream (45) is structured into individually accessible portions (200), and the apparatus is configured to decode the neural network parameters (32) for the corresponding neural network portions from each individually accessible portion.

224. The apparatus according to claim 223, wherein the apparatus is configured to decode the individually accessible portions (200) from the data stream (45) at the starting point of each individually accessible portion using context-adaptive arithmetic decoding and context initialization.

225. The device accesses each individually accessible portion of the data stream (45): The start code (242) where each of the aforementioned individually accessible parts begins, and / or A pointer (220, 244) that points to the beginning of each of the individually accessible portions, and / or The apparatus according to claim 223 or 224, configured to read a data stream length parameter indicating the data stream length (246) of each individually accessible portion for skipping each individually accessible portion when analyzing the data stream (45).

226. The device provides instructions for each of the neural network portions of a reconstruction rule (270) for inverse quantization (280) of the neural network parameters (32) associated with each of the neural network portions of the neural network, The main header portion (47) of the data stream (45) relating to the entire neural network, A neural network layer (210, 30) related to the header portion (110) of the data stream (45) related to the neural network layer to which each of the aforementioned neural network portions belongs, or The apparatus according to any one of claims 223 to 225, wherein the neural network portion-specific header portion of the data stream associated with each of the aforementioned neural network portions is configured to read from the data stream (45).

227. The apparatus according to any one of claims 169 to 226, wherein the apparatus is configured to decode a representation of a neural network (10) from a data stream (45), the data stream (45) being structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the apparatus is configured to decode, for each of one or more predetermined individually accessible portions, an identification parameter (310) for identifying each predetermined individually accessible portion from the data stream (45).

228. A device for decoding a representation of a neural network (10) from a data stream (45), The data stream (45) is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the device is configured to decode from the data stream (45) one or more predetermined individually accessible portions, for each of the predetermined individually accessible portions, an identification parameter (310) for identifying each predetermined individually accessible portion.

229. The apparatus according to claim 227 or 228, wherein the identification parameter (310) relates to each of the predetermined individually accessible portions via a hash function or an error detection code or an error correction code.

230. The apparatus according to any one of claims 227 to 229, wherein the apparatus is configured to decode a higher-level identification parameter (310) for identifying two or more predetermined sets of individually accessible portions from the data stream (45).

231. The apparatus according to claim 230, wherein the higher-level identification parameter (310) is associated with the identification parameter (310) of one or more predetermined individually accessible portions via a hash function or an error detection code or an error correction code.

232. The apparatus according to any one of claims 227 to 231, wherein the apparatus is configured to decode the individually accessible portion (200) from the data stream (45) at the starting point of each individually accessible portion using context-adaptive arithmetic decoding and context initialization.

233. The device, with respect to each individually accessible portion of the data stream, The start code (242) where each of the aforementioned individually accessible parts begins, and / or A pointer (220, 244) that points to the beginning of each of the individually accessible portions, and / or The apparatus according to any one of claims 227 to 232, configured to read a data stream length parameter indicating the data stream length (246) of each individually accessible portion for skipping each of the individually accessible portions when analyzing the data stream.

234. The apparatus according to any one of claims 227 to 233, wherein the neural network portion includes neural network layers (210, 30) of the neural network, and / or one or more sub-portions of one or more neural network layers of the neural network.

235. The device is configured to decode a representation of a neural network (10) from a layered encoded data stream (45), wherein different versions (330) of the neural network are encoded in the data stream (45), and the data stream (45) is structured into one or more individually accessible portions (200), each portion relating to a corresponding version of the neural network. The aforementioned device is By using delta decoding on the second version (3301) of the neural network encoded in the second part, and / or, From the aforementioned data stream (45), each of the first versions (330) of the neural network is obtained. 2 In order to perform inference based on ), one or more compensating neural network portions (332) are decoded, The first version of the neural network encoded from the first part (330 2 It is configured to decrypt ) This is performed in addition to the execution of the corresponding neural network portion (334) of the second version (3301) of the neural network encoded in the second portion, The apparatus according to any one of claims 169 to 234, wherein the outputs of each of the aforementioned compensating neural network portions (332) and the corresponding neural network portion (334) are summed.

236. The device is configured to decode a representation of a neural network (10) from a layered encoded data stream (45), wherein different versions (330) of the neural network are encoded in the data stream (45), and the data stream (45) is structured into one or more individually accessible portions (200), each portion relating to a corresponding version of the neural network. The aforementioned device is By using delta decoding on the second version (3301) of the neural network encoded in the second part, and / or, From the data stream (45), the first version (330) of the neural network 2 By decoding one or more compensating neural network portions (332), each of which is such, in order to perform inference based on ), From the first part, the first version of the neural network (330 2 It is configured to decrypt ) The second part is executed in addition to the execution of the corresponding neural network portion (334) of the second version (3301) of the encoded neural network, A device that sums the outputs of each of the aforementioned compensating neural network portions (332) and the corresponding neural network portions (334).

237. The device extracts the second version (330) of the neural network from the second portion of the data stream (45). 1 ) is configured to decrypt; and, The aforementioned device is The first version of the neural network (330 2 )of, The difference between weights and / or biases, and / or Additional neurons (14, 18, 20) or neuronal interconnections (22, 24) The second version of the neural network encoded in the second part (330 1 The apparatus according to claim 235 or 236, configured to perform delta decoding on the first portion of the data stream (45).

238. The apparatus according to any one of the preceding claims 235 to 237, wherein the apparatus is configured to decode the individually accessible portion (200) from the data stream (45) at the starting point of each individually accessible portion using context-adaptive arithmetic decoding (600) and context initialization.

239. The device accesses each individually accessible portion of the data stream (45): The start code (242) where each of the aforementioned individually accessible parts begins, and / or A pointer (220, 244) that points to the beginning of each of the individually accessible portions, and / or The apparatus according to any one of claims 235 to 238, wherein when analyzing the data stream, it is configured to decode a data stream length parameter indicating the data stream length (246) of each individually accessible portion for skipping each of the individually accessible portions.

240. The apparatus according to any one of claims 235 to 239, wherein the apparatus is configured to decode, for each of one or more predetermined individually accessible portions (200) from the data stream, an identification parameter (310) for identifying each predetermined individually accessible portion.

241. The apparatus according to any one of claims 169 to 240, wherein the apparatus is configured to decode a representation of a neural network (10) from a data stream (45), the data stream (45) is composed of individually accessible portions (200), each portion representing a corresponding portion of the neural network, and the apparatus is configured to decode supplemental data (350) from the data stream (45) for each of one or more predetermined individually accessible portions to supplement the representation of the neural network.

242. A device for decoding a representation of a neural network (10) from a data stream (45), The data stream (45) is structured into individually accessible portions (200), each portion representing a corresponding neural network portion of the neural network, and the device is configured to decode supplemental data (350) from the data stream (45) for each of one or more predetermined individually accessible portions (200) to supplement the representation of the neural network.

243. The apparatus according to claim 241 or 242, wherein the data stream (45) indicates that the supplemental data (350) is not essential for inference based on the neural network.

244. The apparatus according to any one of claims 241 to 243, wherein the apparatus is configured to decode supplemental data (350) for supplementing the representation of the neural network for one or more predetermined individually accessible portions (200) from another individually accessible portion, and the data stream (45) includes, for each of the one or more predetermined individually accessible portions, a corresponding other predetermined individually accessible portion relating to the neural network portion to which each predetermined individually accessible portion corresponds.

245. The apparatus according to any one of claims 241 to 244, wherein the neural network portion includes the neural network layers (210, 30) of the neural network and / or a subdivided layer portion of a predetermined neural network layer of the neural network.

246. The apparatus according to any one of claims 241 to 245, wherein the apparatus is configured to decode the individually accessible portion (200) at the starting point of each individually accessible portion using context-adaptive arithmetic decoding and context initialization.

247. The device, with respect to each individually accessible portion of the data stream, The start code (242) where each of the aforementioned individually accessible parts begins, and / or A pointer (220, 244) that points to the beginning of each of the individually accessible portions, and / or The apparatus according to any one of claims 241 to 246, wherein when analyzing the data stream (45), it is configured to read a data stream length parameter indicating the data stream length (246) of each individually accessible portion for skipping each of the individually accessible portions.

248. The aforementioned supplementary data (350) is, The relevance score of the neural network parameters (32), and / or The apparatus according to any one of claims 241 to 247, relating to the perturbation robustness of the neural network parameters (32).

249. A device for decoding a representation of a neural network (10) from a data stream (45), The apparatus according to any one of claims 169 to 248, wherein the apparatus is configured to decode hierarchical control data (400) structured into a sequence (410) of control data portions (420) from the data stream (45), and the control data portions provide information about the neural network with increasing detail along the sequence of control data portions.

250. A device for decoding a representation of a neural network (10) from a data stream (45), The apparatus is configured to decode hierarchical control data (400) structured into a sequence (410) of control data portions (420) from the data stream (45), wherein the control data portions provide information about the neural network with increasing detail along the sequence of control data portions.

251. The apparatus according to claim 249 or 250, wherein at least some of the control data portions (420) provide partially redundant information relating to the neural network.

252. The apparatus according to any one of claims 249 to 251, wherein the first control data portion provides the information relating to the neural network by indicating a default neural network type with default settings, and the second control data portion includes parameters indicating each of the default settings.

253. A device for performing inference using a neural network, An apparatus for decoding a data stream (45) according to any one of claims 169 to 252 and deriving the neural network from the data stream (45), An apparatus including a processor configured to perform the inference based on the neural network.

254. A method for encoding a representation of a neural network into a data stream (45), A method comprising the step of providing the data stream with serialization parameters indicating the encoding order in which neural network parameters defining the neuronal interconnections of the neural network are encoded in the data stream.

255. A method for encoding a representation of a neural network into a data stream, A method for using the neural network for inference, comprising the step of providing the data stream with numerical computational representation parameters indicating the numerical representation and bit size of the neural network parameters of the neural network that are encoded in the data stream.

256. A method for encoding a representation of a neural network into a data stream, A method wherein the data stream is structured into one or more individually accessible portions, each individually accessible portion representing a corresponding neural network layer of the neural network, and the method includes providing the data stream with a neural network layer type parameter indicating the neural network layer type of the predetermined neural network layer of the neural network for a given neural network layer.

257. A method for encoding a representation of a neural network into a data stream, A method wherein the data stream is structured into one or more individually accessible portions, each portion representing a corresponding neural network layer of the neural network, and the method includes the step of providing the data stream with a pointer to the beginning of each of the one or more predetermined individually accessible portions.

258. A method for encoding a representation of a neural network into a data stream, The data stream is structured into one or more individually accessible parts, each individually accessible part representing a corresponding neural network layer of the neural network, and within a predetermined part, the data stream is further structured into individually accessible sub-parts, each sub-part representing a corresponding neural network part of the respective neural network layer of the neural network, and the method provides the data stream to each of the one or more predetermined individually accessible sub-parts: The start code in which each of the aforementioned predetermined individually accessible subparts begins, and / or A pointer to the beginning of each of the aforementioned predetermined individually accessible sub-parts, and / or A method comprising the step of providing a data stream length parameter indicating the data stream length of each predetermined individually accessible sub-part for skipping each predetermined individually accessible sub-part when analyzing the data stream.

259. A method for encoding a representation of a neural network into a data stream, A method comprising the step of providing the data stream, for each of one or more predetermined individually accessible portions, a processing option parameter indicating one or more processing options that must be used or may be used when the neural network is used for inference, to the data stream.

260. A method for encoding neural network parameters representing a neural network into a data stream, The method comprises the step of providing the data stream, which contains the neural network parameters quantized to a quantization index and encoded in the data stream such that the neural network parameters of different neural network portions of the neural network are quantized differently, and for each of the neural network portions, the data stream which contains the reconstruction rules for dequantizing the neural network parameters associated with each of the neural network portions.

261. A method for encoding neural network parameters representing a neural network into a data stream, The neural network parameters are quantized into quantization indices and encoded into the data stream, and the method is used to demonstrate reconstruction rules for dequantizing the neural network parameters. The quantization step size parameter indicates the quantization step size, The step includes providing a set of parameters that define a quantization index-reconstruction level mapping to the data stream, The reconstruction rule for the predetermined neural network portion is, The quantization step size for quantization indices within a predetermined index interval, A method defined by the quantization index-reconstruction level mapping for quantization indices outside the predetermined index interval.

262. A method for encoding a representation of a neural network into a data stream, A method wherein the data stream is structured into individually accessible portions, each portion representing a corresponding neural network portion of the neural network, and the method includes the step of providing the data stream with an identification parameter for each of one or more predetermined individually accessible portions to identify each of the predetermined individually accessible portions.

263. A method for encoding a representation of a neural network into a data stream using a layered approach, Different versions of the neural network are encoded in the data stream, and the data stream is structured into one or more individually accessible parts, each part relating to a corresponding version of the neural network, and the method provides a first version of the neural network, Delta coding is performed on the second version of the neural network encoded in the second part, and / or To perform inference based on the first version of the neural network, one or more compensating neural network portions are performed in addition to the execution of the corresponding neural network portion of the second version of the neural network, each encoded in the second portion, In the form of a compensated neural network portion, the outputs of each of the aforementioned compensated neural network portions and the corresponding neural network portions are summed, A method comprising the step of encoding the first part.

264. A method for encoding a representation of a neural network into a data stream, The method comprises the step of providing, for each of one or more predetermined individually accessible portions, supplemental data to supplement the representation of the neural network in the data stream.

265. A method for encoding a representation of a neural network into a data stream, The method comprises the step of providing the data stream with hierarchical control data structured into a sequence of control data portions, wherein the control data portions provide information about the neural network with increasing detail along the sequence of control data portions.

266. A method for decoding a neural network representation from a data stream, A method comprising the step of decoding serialization parameters from the data stream that indicate the encoding order in which neural network parameters defining the neuronal interconnections of the neural network are encoded in the data stream.

267. A method for decoding a neural network representation from a data stream, The method comprises the steps of: decoding from the data stream numerical computation representation parameters that indicate the numerical representation and bit size to which the neural network parameters of the neural network, encoded in the data stream, should be represented when the neural network is used for inference; and using the numerical representation and bit size decoded from the data stream for representing the neural network parameters.

268. A method for decoding a neural network representation from a data stream, A method wherein the data stream is structured into one or more individually accessible parts, each part representing a corresponding neural network layer of the neural network, and the method includes the step of decoding from the data stream, for a given neural network layer, a neural network layer type parameter indicating the neural network layer type of the given neural network layer of the neural network.

269. A method for decoding a neural network representation from a data stream, A method wherein the data stream is structured into one or more individually accessible portions, each portion representing a corresponding neural network layer of the neural network, and the method includes the step of decoding from the data stream, for each of the one or more predetermined individually accessible portions, a pointer pointing to the beginning of each predetermined individually accessible portion.

270. A method for decoding a neural network representation from a data stream, The data stream is structured into one or more individually accessible parts, each individually accessible part representing a corresponding neural network layer of the neural network, and the data stream is further structured into individually accessible sub-parts within a predetermined part, each sub-part representing a corresponding neural network part of the respective neural network layer of the neural network, and the method extracts from the data stream, for each of the one or more predetermined individually accessible sub-parts, The start code in which each of the aforementioned predetermined individually accessible subparts begins, and / or A pointer indicating the beginning of each of the aforementioned predetermined individually accessible sub-parts, and / or A method for analyzing the data stream, comprising the step of decoding a data stream length parameter indicating the data stream length of each of the predetermined individually accessible sub-parts for skipping the predetermined individually accessible sub-parts.

271. A method for decoding a neural network representation from a data stream, A method wherein the data stream is structured into individually accessible portions, each of which represents a corresponding neural network portion of the neural network, and the method comprises the step of decoding from the data stream, for each of one or more predetermined individually accessible portions, one or more processing option parameters indicating that the neural network must be used for inference, or can be optionally used.

272. A method for decoding neural network parameters representing a neural network from a data stream, The neural network parameters are quantized into a quantization index and encoded into the data stream, and the neural network parameters of different neural network portions of the neural network are quantized differently and encoded into the data stream, the method comprising the step of decoding from the data stream a reconstruction rule for each of the neural network portions to dequantize the neural network parameters associated with each of the neural network portions.

273. A method for decoding neural network parameters representing a neural network from a data stream, The neural network parameters are quantized into a quantization index and encoded into the data stream, and the method is, The quantization step size parameter indicates the quantization step size, The process includes the step of deriving a reconstruction rule from the data stream for dequantizing the neural network parameters by decoding a parameter set that defines a quantization index-reconstruction level mapping from the data stream, The reconstruction rule for the predetermined neural network portion is, The quantization step size for quantization indices within a predetermined index interval, A method defined by a quantization index-reconstruction level mapping for quantization indices outside the predetermined index interval.

274. A method for decoding a neural network representation from a data stream, A method wherein the data stream is structured into individually accessible portions, each portion representing a corresponding neural network portion of the neural network, and the method comprises the step of decoding from the data stream, for each of one or more predetermined individually accessible portions, an identification parameter for identifying each predetermined individually accessible portion.

275. A method for decoding a representation of a neural network encoded in a data stream using a layered approach from the data stream, Different versions of the neural network are encoded in the data stream, and the data stream is structured into one or more individually accessible parts, each part relating to a corresponding version of the neural network, and the method provides a first version of the neural network, By applying delta decoding to the second version of the neural network encoded in the second part, and / or, In order to perform inference based on the first version of the neural network, each of the following is performed by decoding one or more compensating neural network portions from the data stream, in addition to the execution of the corresponding neural network portion of the second version of the neural network encoded in the second portion: The first part includes a step of decryption, A method in which the outputs of each of the aforementioned compensating neural network portions and the corresponding neural network portions are summed.

276. A method for decoding a neural network representation from a data stream, The method comprises the step of decoding from the data stream, for each of one or more predetermined individually accessible portions, supplemental data to supplement the representation of the neural network.

277. A method for decoding a neural network representation from a data stream, The method comprises the step of decoding hierarchical control data structured into a sequence of control data portions from the data stream, wherein the control data portions provide information about the neural network with increasing detail along the sequence of control data portions.

278. A computer program, when executed by a computer, that causes a computer to perform the method according to any one of claims 254 to 277.

Images