Method and electronic apparatus for decoding a data stream, and related computer programs and data streams.

Abstract

The data stream comprises a plurality of identifiers and a bit sequence (Fnn). A method for decoding said data stream into a sequence of data (V) of a respective predetermined type comprises the following steps for obtaining each item of data (V) of the sequence: - determining a context (C) based on an identifier from the plurality of identifiers having the type of the associated data item, - decoding a portion of the bit sequence (Fnn) by an entropy decoder (23) receiving the bit sequence (Fnn) as input and parameterized with the determined context (C). The invention also relates to an electronic decoding device (20) and an associated computer program.

Description

Technical Field

[0001] The present invention relates to the technical field of data decoding.

[0002] In particular, the present invention relates to a method for decoding a data stream, an electronic device, and related computer programs and data streams.

Background Art

[0003] Entropy coding is used particularly in the field of encoding of audio or video content in order to optimally compress such data taking into account the occurrence statistics of different symbols within the data.

[0004] Within this framework, for example, CABAC ("Context-based Adaptive Binary Arithmetic Coding") may be mentioned as described in the paper "Context-based adaptive binary arithmetic coding in the H.264 / AVC video compression standard" by D. Marpe, H. Schwarz, and T. Wiegand in IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, pp. 620 - 636, July 2003.

[0005] When used for entropy coding or entropy decoding of data within the framework of a standard (e.g., a standard defining a method for compressing audio or video content), an entropy decoder is always parameterized in a context that depends on previously encoded or decoded syntax elements in a method predefined by the relevant standard.

Summary of the Invention

Means for Solving the Problems

[0006] The present invention provides a method for decoding a data stream containing a sequence of multiple identifiers and binary elements into sequences of data of a predetermined type, comprising the following steps for obtaining each piece of data in the sequence: - A step of determining the context based on the identifier associated with the relevant data type from among multiple identifiers. - A step of receiving a sequence of binary elements as input and decoding a portion of the sequence of binary elements using an entropy decoder parameterized in a determined context, We propose a method that includes this.

[0007] The presence of identifiers within the data stream that indicate the context used to decode different types of data increases flexibility when using entropy coding, which is particularly interesting when the data format is not fully predefined.

[0008] The acquired data are, for example, values ​​representing audio or video content. These representative values ​​may be generated during the encoding process by an encoding artificial neural network (or, more generally, a machine learning method such as deep learning or random forest learning), as described below.

[0009] The data stream may also contain information indicating the set of contexts available within the entropy decoder. Such information might, for example, indicate the number of contexts available within the entropy decoder.

[0010] The data stream may further include data for parameterizing the relevant context for each context available within the entropy decoder.

[0011] The decoding method may therefore include the step of initializing each context available within the entropy decoder using data to parameterize the relevant context.

[0012] Therefore, the above-mentioned instruction information and parameterized data make it possible to construct an entropy decoder before the effective implementation of entropy decoding.

[0013] The decoding method may further include the step of applying the acquired data as input to an artificial neural network.

[0014] Such artificial neural networks are implemented, for example, by processing units (and possibly parallel processing units). Therefore, this method may include the step of configuring processing units according to the data contained in the data stream.

[0015] The present invention is particularly interesting in this situation where the technical properties of each of the different data (or representative values) being handled depend on the artificial neural network used (which itself is defined by the data as described above) and are therefore not predetermined, and the context used for entropy coding of a particular data cannot be defined in advance (as already shown, such a pre-definition may be given, for example, by a standard).

[0016] However, as already shown, the present invention is not limited to this situation and is of interest in cases where the association between the data to be encoded and the entropy encoding context cannot be defined in advance. Therefore, the present invention can be advantageously used for entropy encoding of text written in several languages ​​using different alphabets. The number of alphabetic symbols and their probabilities of occurrence differ from language to language, and in such situations, the present invention makes it possible to specify the context used for entropy encoding of symbols or words specific to a particular language.

[0017] Generally, the decoding method may further include machine learning methods such as deep learning or random forest learning. Therefore, the data stream may include data for constructing this machine learning method, and / or steps may be provided for constructing a processing unit using this construct data to implement this machine learning method.

[0018] Generally, the present invention can be used to encode data originating from a neural network or to decode data provided to a neural network. A neural network should be understood as a processing process comprising a number of similar steps, where only the parameters of these steps differ from one another and are set by a learning process, and these steps are adapted to be implemented in large parallel. Thus, "neural network" may refer to a series of layers that perform linear filtering of data originating from an input layer or a preceding layer, followed by the application of a nonlinear function (a neural network in the conventional sense) after each filtering. Alternatively, a neural network may refer to a series of tests, where the result of each test determines which subsequent test to apply (random forest), or it may refer to other processing processes having the characteristics described above.

[0019] According to the embodiments described below, the above sequence of data forms a set of feature maps.

[0020] In this case, according to the first possibility, each identifier among the multiple identifiers is associated with a feature map in the set of feature maps. In this case, it can be stipulated that the context determined for retrieving elements of a given feature map is determined based on the identifiers associated with the given feature map.

[0021] According to the second possibility, the feature maps of a set of feature maps have a common structure in which each element of the feature map is defined by its location, and identifiers among multiple identifiers can each be associated with different locations in this common structure. Therefore, it can be stipulated that the context determined to retrieve a given element of the feature map is determined based on the identifier associated with the location that defines this given element.

[0022] The present invention also proposes an electronic device for decoding a data stream containing a plurality of identifiers and a sequence of binary elements into sequences of data of a predetermined type, the device comprising: an entropy decoder that receives a sequence of binary elements as input; and a configuration module designed to determine a context based on an identifier among the plurality of identifiers associated with the type of data to be retrieved, and to parameterize the entropy decoder in the determined context in order to retrieve the data at the output of the entropy decoder.

[0023] The present invention further proposes a computer program that includes instructions executable by a processor, and is designed to implement the decoding method proposed above when these instructions are executed by the processor.

[0024] Finally, the present invention proposes a data stream that represents a sequence of data of a predetermined type and includes a plurality of identifiers and a sequence of binary elements, each identifier representing a context that is parameterized so that when the entropy decoder receives a portion of the sequence of binary elements as input, the entropy decoder obtains at least one piece of data having a type associated with the identifier.

[0025] Naturally, different features, alternative forms, and embodiments of the present invention can be associated with each other in various combinations, provided they are not incompatible or exclusive with one another.

[0026] Furthermore, various other features of the present invention will become apparent from the accompanying description taken in conjunction with the drawings that illustrate non-limiting embodiments of the present invention.

Brief Description of the Drawings

[0027] [Figure 1] Shows an electronic encoding device used within the framework of the present invention. [Figure 2] Schematically shows the feature map used by the electronic encoding device of FIG. 1. [Figure 3] It is a flowchart showing the steps of an encoding method implemented within the electronic encoding device of FIG. 1. [Figure 4] Shows the data stream generated by the electronic encoding device of FIG. 1. [Figure 5] Shows an example of an electronic decoding device according to the present invention. [Figure 6] It is a flowchart showing the steps of a decoding method implemented within the electronic decoding device of FIG. 5.

Modes for Carrying Out the Invention

[0028] FIG. 1 shows an electronic encoding device 2 including an entropy encoder 10.

[0029] This electronic encoding device 2 includes a processor 4 (e.g., a microprocessor) and a parallel processing unit 6, such as a graphics processing unit, i.e., a GPU or a tensor processing unit, i.e., a TPU.

[0030]

[0031] ​As schematically shown in Figure 1, the control module 5 receives data P and B, in this case format data P and content data B, representing the audio or video content to be compressed.

[0032] Format data P indicates the format characteristics of the representation of audio or video content, such as image size (in pixels), frame rate, binary depth of luminance information, and binary depth of chrominance information in the case of video content.

[0033] Content data B forms a representation (in this case, uncompressed) of audio or video content. For example, in the case of video content, the content data includes, for each pixel of an image in an image sequence, data representing the brightness value of the pixel and data representing the chrominance value of the pixel.

[0034] The parallel processing unit 6 is configured by the processor 4 (for example, the control module 5) and then designed to implement the artificial neural network 8. To this end, the parallel processing unit 6 is designed to execute multiple operations of the same type in parallel at a given time.

[0035] As described below, the artificial neural network 8 is used within a framework for processing content data B, with the aim of obtaining a value V representing audio or video content.

[0036] Alternatively, a different type of machine learning method, such as another type of deep learning or random forest learning, can be used instead of artificial neural networks.

[0037] In the embodiment described herein, when content data B is applied to the input of the artificial neural network 8, the artificial neural network 8 generates a representative value V as its output.

[0038] The content data B applied to the input of the artificial neural network 8 (i.e., applied to the input layer of the artificial neural network 8) may also represent a block of an image, or a block of an image component (e.g., a block of the luminance or chrominance component of the image, or a block of the color component of the image), or an image in a video sequence, or a component of an image in a video sequence (e.g., a luminance or chrominance component, or a color component), or a series of images in a video sequence.

[0039] For example, in this case, at least some of the neurons (or nodes) in the input layer may be defined as each receiving a pixel value of an image component, and said value may be represented by one of the content data B.

[0040] Alternatively, processing content data B may involve the use of several artificial neural networks, as described, for example, in the aforementioned paper "DVC: An End-to-end Deep Video Compression Framework" by Guo Lu et al. (2019 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019).

[0041] The representative values ​​V generated by the output of the artificial neural network 8 are then organized into a sequence of feature maps F, as schematically shown in Figure 2. Here, the artificial neural network 8 generates, for example, N feature maps F.

[0042] Each feature map F has, for example, a two-dimensional structure (or matrix structure). Therefore, each feature map F here forms a matrix with H rows and W columns.

[0043] An element at a given location within a given feature map F corresponds to a representative value V generated by an output node (or node of the output layer) of an artificial neural network, which is associated with this given location and this given feature map F in a predefined manner. According to one possible embodiment, the artificial neural network 8 generates all N feature maps as outputs (i.e., on its output layer) at a given time point. According to another possible embodiment, different sets of content data B (corresponding to different locations in an image, for example) are applied to the input (i.e., the input layer) of the artificial neural network 8 at different time points, and the artificial neural network 8 generates the corresponding feature map F as an output (i.e., on its output layer) at each of these different time points (in this case, the output nodes of the artificial neural network 8 are each associated with different locations on a single feature map F).

[0044] Alternatively, the representative values ​​V generated by the output of the artificial neural network 8 can be organized into an ordered sequence of representative values ​​V. If the content data B applied to the input of the artificial neural network 8 represents a block of an image (or a block of an image component), then the ordered sequence of representative values ​​V generated by the output of the artificial neural network 8 is associated with this block. Thus, different sequences of representative values ​​generated consecutively by the artificial neural network 8 are each associated with different blocks of an image (or different blocks of relevant components of an image).

[0045] In yet another alternative form, the representative values ​​V generated by the output of the artificial neural network 8 are placed within a multidimensional (e.g., M-dimensional) structure of the data. Each element of this structure is then identified by its position within the structure, i.e., a set of M coordinates in the example above. The representative values ​​V generated by a given output node of the artificial neural network 8 (i.e., a given node in the output layer) then form elements of the structure, identified by their positions within the structure associated with this output node in a predefined manner (i.e., coordinates associated with this output node in a predefined manner).

[0046] The representative value V generated by the output of the artificial neural network 8 is applied to the input of the entropy encoder 10.

[0047] These representative values ​​V are applied, for example, in a predefined order. Therefore, if the representative values ​​V are organized into an (ordered) sequence in a feature map F, then different feature maps F are applied one after another (in the aforementioned sequence order), and different elements of the same feature map F (each corresponding to one representative value) are applied according to a predefined scheme (their positions) within the feature map F.

[0048] The entropy encoder 10 is designed to encode several statistical sources, each corresponding to a specific probability of occurrence of the symbol to be encoded. To this end, the entropy encoder 10 can be parameterized in a specific context associated with a given statistical source, in which the entropy encoder 10 produces the optimal entropy encoding if the effectively encoded symbol (here, the representative value V) satisfies the expected probability of this statistical source.

[0049] The entropy encoder 10 is, here, of the CABAC ("Context-Adaptive Binary Arithmetic Coding") type. Alternatively, it could be another type of entropy encoder, such as a Huffman type encoder, an arithmetic encoder, or an LZW ("Lempel-Ziv-Welch") encoder.

[0050] As described below, the entropy encoder 10 is always parameterized by the control module 5 in a context C that depends on the type of representative value V (representative value V derived from the artificial neural network 8) being encoded at that time.

[0051] In the example described herein, where representative values ​​V are organized into a sequence of feature maps F, the context C selected by the control module 5 to parameterize the entropy encoder 10 for entropy coding of elements in feature map F may, for example, depend (in a predefined manner) on the relevant feature map F and / or the location of elements within feature map F. (The association between a particular context and a certain feature map and / or location within that feature map is made, for example, using previous statistical measurements of the probability of occurrence of symbols in this feature map or at this location for a number of images processed by the coding artificial neural network 8. Such associations are intended to separate these statistical sources in representative values ​​V in order to encode each of the different statistical sources in a particular context and thus maximize compression.)

[0052] The entropy encoder 10 generates a sequence of binary elements (or a sequence of bits) Fnn as its output.

[0053] Here, an example of an encoding method implemented by the electronic encoding device 2 will be explained with reference to Figure 3.

[0054] The method in Figure 3 begins with step E2, which involves selecting an encoding process-decoding process pair. As already shown for the encoding process, both the encoding and decoding processes utilize at least one artificial neural network.

[0055] In the example described here, the encoding process is implemented by an encoding artificial neural network, and the decoding process is implemented by a decoding artificial neural network.

[0056] A unit formed by an encoding artificial neural network and a decoding artificial neural network (where the output of the encoding artificial neural network is applied to the input of the decoding artificial neural network) forms, for example, an autoencoder.

[0057] The encoding-decoding process pair is selected from, for example, several predefined encoding-decoding process pairs, i.e., several encoding artificial neural network-decoding artificial neural network pairs.

[0058] The encoding-decoding process pair may be selected from among the encoding-decoding process pairs in which the decoding process uses an artificial neural network available to the electronic decoding device (such as the electronic decoding device 20 shown in Figure 5 and described below). To this end, the electronic encoding device may, in some cases, receive in advance a list of artificial neural networks accessible by the electronic decoding device (from the electronic decoding device or a dedicated server).

[0059] The encoding-decoding process pair can also be selected according to the intended use (e.g., indicated by the user via a user interface not shown on the electronic encoding device 2). For example, if the intended use is video conferencing, the selected encoding-decoding process pair includes a low-latency decoding process. For other uses, the selected encoding-decoding process pair includes a random-access decoding process.

[0060] In a low-latency video sequence decoding process, the images in a video sequence are represented, for example, by encoded data that can be transmitted and decoded immediately. This data can then be transmitted in the order in which the video images are displayed, thereby ensuring a latency of one frame between encoding and decoding.

[0061] In the random access video sequence decoding process, the encoded data associated with multiple images is transmitted in an order different from the display order of these images, thereby improving compression. Next, encoded images that do not reference other images (so-called intraframes) can be encoded periodically, making it possible to start decoding the video sequence from several points in the encoded stream.

[0062] For this purpose, you can refer to the paper "Overview of the High Efficiency Video Coding (HEVC) Standard" by G.S. Ullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand in IEEE Transactions on Circuits and Systems for Video Technology, vol.22, no.12, pp.1649-1668, Dec.2012.

[0063] Various criteria for selecting encoding-decoding process pairs can be combined in different ways depending on the situation.

[0064] Once an encoding process-decoding process pair is selected, the control module 5 proceeds to configure the parallel processing unit 6 in step E4 so that the parallel processing unit 6 can implement the selected encoding process.

[0065] Step E4 specifically includes creating an instance of the coding artificial neural network 8 used by the selected coding process within the parallel processing unit 6.

[0066] This instance creation involves the following steps: - Within the parallel processing unit 6, the steps of securing the memory space necessary to implement the encoded artificial neural network, and / or - A step of programming a parallel processing unit 6 using the weights Γ and activation function that define the encoded artificial neural network 8, and / or, - A step of loading at least a portion of content data B into the local memory of the parallel processing unit 6. It may include.

[0067] Next, the method in Figure 3 includes step E6, which implements the encoding process, namely, the step of applying content data B as input to the encoding artificial neural network 8 (or, in other words, the step of activating the encoding artificial neural network 8 using content data B as input).

[0068] Therefore, step E6 makes it possible to generate a representative value V (here in the output of the encoded artificial neural network 8).

[0069] The following steps relate to encoding (i.e., preparing) the data stream for the electronic decoding device (for example, the electronic decoding device 20 described below with reference to Figure 5).

[0070] Therefore, this method includes, in particular, step E8 of encoding a first header portion Fc which contains data specific to the audio or video content representation format (for example, data linked to the format of the video sequence being encoded).

[0071] These data forming the first header portion Fc indicate, for example, the image size (in pixels), frame rate, bit depth of luminance information, and bit depth of chrominance information. These data are constructed, for example, based on the format data P described above (after potential reformatting).

[0072] In step E10, the control module 5 performs encoding of a second header portion containing data R that represents the decoded artificial neural network (related to the encoding-decoding process pair selected in step E2).

[0073] According to the first possible embodiment, these instruction data R may include identifiers for the decoded artificial neural network.

[0074] Such identifiers indicate the decoding artificial neural network corresponding to the above-mentioned encoding artificial neural network 8 (among multiple decoding artificial neural networks, e.g., all decoding artificial neural networks available in an electronic decoding device), and therefore such a decoding artificial neural network should be used to decode the representative value V.

[0075] In other words, such identifiers, by convention (particularly shared by the electronic coding device and the electronic decoding device), define this decoding artificial neural network from among all decoding artificial neural networks available to (or accessible by) the electronic decoding device. As already shown, the electronic coding device 2 may, if applicable, receive in advance a list of artificial neural networks accessible by the electronic decoding device (from the electronic decoding device or a dedicated server).

[0076] According to a second possible embodiment, these instruction data R may include descriptive data of the decoded artificial neural network.

[0077] The decoding artificial neural network (corresponding to the encoding artificial neural network 8 described above) is encoded (i.e., represented) by these descriptive data (or data encoding the decoding artificial neural network) in accordance with standards such as MPEG-7 Part 17 or using a format such as JSON.

[0078] For this purpose, you can refer to the paper "DeepCABAC: Context-adaptive binary arithmetic coding for deep neural network compression" by S. Wiedemann et al. in the Proceedings of the 36th International Conference on Machine Learning, Long Beach, California, PMLR 97, 2019, or the paper "Compact and Computationally Efficient Representation of Deep Neural Networks" by S. Wiedemann et al. in IEEE Transactions on Neural Networks and Learning Systems (Vol.31, Iss.3), March 2020.

[0079] The instruction data R may also be defined to include an indicator indicating whether the decoded artificial neural network belongs to a predetermined set of artificial neural networks (in this case, the first possible embodiment described above is used), or whether the decoded artificial neural network is encoded in a data stream, i.e., represented by the description data described above (in this case, the second possible embodiment described above is used).

[0080] The method in Figure 3 follows step E12, which determines whether the electronic decoding device can implement the decoding process using a decoding artificial neural network.

[0081] The control module 5 determines this possibility, for example, by determining (potentially using previous interactions between the electronic encoding device 2 and the electronic decoding device) whether the electronic decoding device includes a module suitable for implementing this decoding process or software suitable for the implementation of this decoding process by the electronic decoding device (if this software is executed by the processor of the electronic decoding device).

[0082] If the control module 5 determines that the electronic decoding device is capable of implementing the decoding process, the method proceeds to step E16, which is described below.

[0083] If control module 5 determines that the electronic decryption device is unable to implement the decryption process, the method performs step E14, as described below (before proceeding to step E16).

[0084] Alternatively, the choice of whether or not to perform step E14 (before performing step E16) may be made based on another criterion, for example, a dedicated indicator stored in the electronic coding device 2 (and optionally adjustable by the user via the user interface of the electronic coding device 2) or a choice made by the user (for example, obtained via the user interface of the electronic coding device 2).

[0085] In step E14, the control module 5 encodes a third header portion in the data stream, which includes a computer program (exe) (or code) executable by the processor of the electronic decryption device. (The use of the computer program (exe) within the electronic decryption device is described below with reference to Figure 5.)

[0086] To adapt for execution within the electronic decoding device, the computer program is selected, for example, within a library, based on information related to the hardware configuration of the electronic decoding device (e.g., information received during previous communication between the electronic encoding device 2 and the electronic decoding device).

[0087] Next, in step E16, the control module 5 performs the entropy coding configuration.

[0088] During this configuration step 16, the control module 5 determines the set of statistical sources by which to perform entropy coding. For example, the control module 5 determines the number of statistical sources (and therefore contexts) to be used for entropy coding.

[0089] To this end, the control module 5 can, for example, analyze the generated representative value V as shown above and identify how many different statistical sources are included within it. Alternatively, the control module 5 may select the number of statistical sources according to the complexity that is desired to be tolerated during encoding and / or decoding.

[0090] Therefore, the control module 5 can subdivide the signal formed by the set of representative values ​​V into a number of sub-signals (for example, up to a predetermined number or, alternatively, without limit on the number) according to a predefined criterion.

[0091] In the following, the number of different statistical sources identified in the signal formed by the representative value V (i.e., the number of contexts in which the entropy encoder is parameterized during the entropy coding of the representative value V) is denoted by K. For example, K = 160.

[0092] In the example described here, control module 5 can generate the same number of subsignals as feature map F, or the same number of subsignals as locations (i.e., distinct locations) within feature map F. It may be further specified that, depending on the circumstances, control module 5 can merge subsignals having the same (or very similar) statistical distribution.

[0093] Therefore, during configuration step E16, the control module 5 also determines which statistical source (and thus which context) is associated with each type of representative value V (i.e., in this case, each element of the feature map).

[0094] For example, if control module 5 generates the same number of sub-signals as feature map F, the context associated with one element of feature map F (i.e., used to parameterize the entropy coding of that element) depends on the relevant feature map F.

[0095] On the other hand, if the control module 5 generates as many sub-signals as there are locations within each feature map F, then the context associated with one element of feature map F (i.e., used to parameterize the entropy coding of this element) depends on the location of that element in feature map F.

[0096] Next, the method in Figure 3 proceeds to the step of encoding data representing the entropy encoder configuration determined in step E16.

[0097] Therefore, the method in Figure 3 first includes step E18 of encoding a fourth header portion containing information I1 indicating the set of contexts to be used for entropy coding. In the example described herein, information I1 indicates the number K of contexts to be used for entropy coding.

[0098] Next, the method in Figure 3 includes step E20, which encodes a fifth header portion containing data Iinit for parameterizing the relevant context for each context used in entropy coding.

[0099] In the example described here, where the entropy coding used is of type CABAC, the parameterized data Iinit associated with a given context is data for initializing this context, as described, for example, in Recommendation ITU-T H.265, part "9.3.2.2 Initialization process for context variables".

[0100] In other embodiments, the parameterized data associated with a given context may be data representing a probabilistic model used for the context in relation to the entropy coding.

[0101] Next, the method in Figure 3 includes a step E22 in which, for each type of representative value V, a sixth header portion is encoded, which includes data I2 indicating the context in which the entropy coding of the representative value V having this type is performed.

[0102] In the example described, as already shown, the control module 5 selects, as already shown, the assignment of different contexts for each of the different feature maps F (each feature map F forming a sub-signal) or for each of the different locations (or places) defined in the feature map F (the elements have a given location that forms a sub-signal).

[0103] Therefore, here, data I2 is, - An indicator I2_mode, where a value of 0 indicates that the context is associated with each feature map F, and a value of 1 indicates that the context is used for each location in feature map F. - Multiple identifiers I2_map[i] that indicate the context used for different types of representative values ​​V, It is proposed that this be included.

[0104] If marker I2_mode is equal to 0, different types of representative values ​​correspond to different feature maps F. If marker I2_mode is equal to 1, different types of representative values ​​correspond to different locations within feature map F.

[0105] The following describes cases where (unless otherwise specified) the control module 5 selects different context assignments for different feature maps, i.e., cases where the indicator I2_mode is equal to 0.

[0106] The method shown in Figure 3 follows step E24, in which the entropy encoder 10 is initialized (by the control module 5) using the parameterized data associated with different contexts. This type of initialization is described in the aforementioned document (Recommendation ITU-T H.265, part “9.3.2.2 Initialization process for context variables”).

[0107] Next, the method in Figure 3 includes step E26 of entropy coding of a representative value V by an entropy encoder 10, the entropy encoder 10 is always parameterized (by the control module 5) in the context C used with respect to the type of representative value V being processed (based on the selection made as described above).

[0108] For example, in the example described herein, in order to entropy encode representative values ​​V contained in a given feature map F, the control module 5 parameterizes the entropy encoder 10 in the context C associated with the given feature map F and applies the elements of the feature map F sequentially (in a predefined order) to the input of the entropy encoder 10.

[0109] Therefore, the entropy encoder 10 generates a sequence of binary elements Fnn as output, which represents the audio or video content in a compressed format.

[0110] Step E26 may, depending on the circumstances, include a substep of binarizing the representative value V (and therefore before entropy coding), as described in the aforementioned paper, "Context-based adaptive binary arithmetic coding in the H.264 / AVC video compression standard." The purpose of this binarization step is to convert the representative value V, which can take many values, into a sequence of binary elements, each of which is encoded by entropy coding (and in this case, the context is associated with the encoding of each binary element).

[0111] In particular, if step E6 allows processing only a portion of the audio or video content to be compressed (for example, if step E6 processes blocks, components, or images of a video sequence to be compressed), it is possible to repeat the execution of step E6 (to obtain representative values ​​of consecutive portions of the content) and step E26 (to perform entropy coding of these representative values).

[0112] Therefore, in step E28, the processor 4 can construct a complete data stream including the header Fet and the sequence of binary elements Fnn.

[0113] The complete data stream is constructed such that the header Fet and the sequence of binary elements Fnn are individually identifiable.

[0114] According to one possible embodiment, the header Fet includes an indicator of a binary element sequence Fnn that begins within the complete data stream. This indicator is, for example, the starting location of the binary element sequence Fnn from the beginning of the complete data stream, bitwise. (i.e., in this case, the header has a predetermined fixed length.)

[0115] Other means for identifying the header Fet and the sequence Fnn of binary elements may be considered as alternatives, for example, as markers (i.e., a combination of bits used to indicate the beginning of the sequence Fnn of binary elements, the use of which is prohibited for the rest of the data stream or at least within the header Fet).

[0116] The data stream constructed in step E28 can be encapsulated in a transmission format that is known in itself, such as a “packet transport system” or “byte stream” format.

[0117] In the case of a "packet transport system" format (for example, as proposed by the RTP protocol), data is encoded in identifiable packets and transmitted over a communication network. The network can easily identify the boundaries of the data (images, sets of images, and in this case, the header Fet and the sequence of binary elements Fnn) using packet identification information provided by the network layer.

[0118] In the "byte stream" format, specifically, packets do not exist, and the construction of step E28 requires the identification of boundaries between related data (boundaries between parts of the stream corresponding to each image, and in this case, the boundary between the header Fet and the sequence of binary elements Fnn) using further means such as the use of a Network Abstraction Layer (NAL) unit that allows a unique binary combination (such as 0x00000001) to identify boundaries between data.

[0119] Next, the complete data stream constructed in step E28 may be sent in step E30 to the electronic decoding device 20 described below (by means of communication not shown and / or via at least one communication network), or stored in the electronic encoding device 2 (for later transmission or, as an alternative, for later decoding within the electronic encoding device itself, in which case the electronic encoding device is designed to further implement the decoding method 20 described below with reference to Figure 5).

[0120] Therefore, this data stream includes a header Fet and a sequence of binary elements Fnn, as shown in Figure 4.

[0121] As is clear from the above, the header Fet is, - The first part Fc, which includes the data characteristics of the audio or video content representation format. - The second part includes data R showing a decoding artificial neural network (or data showing a machine learning method commonly used for decoding), - A third part, which may include a computer program (exe) that can be executed by the processor of the electronic decoding device. - A fourth part containing information I1 that shows the set of contexts used for entropy coding, - For each context used in entropy coding, a fifth part containing data Iinit for parameterizing the relevant context, - As described below, when an entropy encoder is parameterized for entropy coding of a particular type of data, and therefore an entropy decoder receives a portion of a sequence of binary elements as input, a sixth part I2 contains multiple identifiers I2_map[i] that represent the context in which the entropy decoder is parameterized to obtain this type of data, Includes.

[0122] According to possible alternatives, it may be specified that information I1 indicating the set of contexts used for entropy coding is not transmitted, in which case the entropy encoder and entropy decoder can use several contexts that are defined in advance (by convention).

[0123] Figure 5 shows an electronic decoding device 20 that uses an entropy decoder 23.

[0124] This electronic decoding device 20 includes a receiving unit 21, a processor 24 (e.g., a microprocessor), and a parallel processing unit 26, such as a graphics processing unit, i.e., a GPU, or a tensor processing unit, i.e., a TPU.

[0125] The receiving unit 21 is, for example, a communication circuit (such as a radio frequency communication circuit) that can receive data (in particular the data stream described above) from an external electronic device such as an electronic coding device 2 and communicate this data to the processor 24 (for example, the receiving unit 21 is connected to the processor 24 by a bus).

[0126] The electronic decoding device 20 also includes a storage unit 22, such as memory (optionally rewritable non-volatile memory) or a hard drive. Although the storage unit 22 is shown as a separate element from the processor 24 in Figure 5, alternatively, the storage unit 22 may be integrated (i.e., included) with the processor 24.

[0127] In this case, the processor 24 is adapted to execute, for example, multiple instructions of a computer program stored in the storage unit 22 in sequence.

[0128] Some of these instructions, when executed by the processor 24, enable the implementation of a control module 25 having functions described later. Alternatively, some of the functions of the control module 25 can be implemented by having the processor 24 execute instructions identified in the header Fet in step E52, as described later.

[0129] Another portion of the instructions stored in the storage unit 22, when executed by the processor 24, enables the implementation of the aforementioned entropy decoder 23. Alternatively, the entropy decoder 23 may be implemented in step E52 by having the processor 24 execute the instructions identified in the header Fet, as will be described later.

[0130] The entropy decoder 23 is designed to perform entropy decoding, which is the inverse of entropy coding performed by the entropy encoder 10 of the electronic coding device 2 described above with reference to Figure 1. Thus, the entropy decoder 23 is here a CABAC ("Context Adaptive Binary Arithmetic Coding") type entropy decoder. Alternatively, it could be another type of entropy encoder, such as a Huffman type decoder, an arithmetic decoder, or an LZW ("Lempel-Ziv-Welch") decoder.

[0131] The parallel processing unit 26 is configured by the processor 24 (specifically the control module 25 in this case) and then designed to implement the artificial neural network 28. To this end, the parallel processing unit 26 is designed to execute multiple operations of the same type in parallel at a given time.

[0132] Generally, the parallel processing unit 26 is designed to implement a machine learning method (e.g., a deep learning method or a random forest learning method) after being configured by the processor 24 using data received in a data stream.

[0133] As schematically shown in Figure 5, the processor 24 receives a data stream containing the header Fet and the sequence of binary elements Fnn (in this case, via the receiving unit 21).

[0134] As described below, the artificial neural network 28 is used within a framework for processing data obtained by entropy decoding of a sequence of binary elements Fnn, and this data processing aims to obtain the audio or video content corresponding to the initial audio or video content B.

[0135] The storage unit 22 can store multiple parameter sets, each parameter set defining a decoding artificial neural network. As described below, the processor 24 can configure the parallel processing unit 26 to use a specific parameter set from these parameter sets, so that the parallel processing unit 26 can then implement the artificial neural network defined by this specific parameter set.

[0136] The storage unit 22 may store, in particular, a first parameter set defining a first artificial neural network that forms a random access decoder and / or a second parameter set defining a second artificial neural network that forms a low latency decoder.

[0137] In this case, the electronic decoding device 20 has pre-decoding options for both situations where random access to the content is desired and situations where the content should be displayed without delay.

[0138] Here, with reference to Figure 6, we will describe a decoding method that uses an artificial neural network 28 implemented by a parallel processing unit 26, which is implemented in an electronic decoding device 20 and uses an entropy decoder 23 on the one hand.

[0139] The method in Figure 6 begins with step E50, in which a data stream containing the header Fet and the sequence of binary elements Fnn is received (by the electronic decoder 20, specifically the receiving unit 21 in this case). The receiving unit 21 transmits the received data stream to the processor 24 for processing by the control module 25.

[0140] Next, the control module 25 proceeds to step E52, which identifies the header Fet and the sequence of binary elements Fnn in the received data stream, for example, by an indicator at the beginning of the binary element sequence (as already described in step E28).

[0141] In step E52, the control module 25 can also identify different parts of the header Fet (as described above with reference to Figure 4).

[0142] If, in step E52, executable instructions (such as instructions for a computer program Exe) are identified (i.e., detected) within the first data, the control module 25 may, in step E54, initiate the execution of these executable instructions to implement at least some of the (subject-descriptive) steps of processing the header data (and optionally entropy decoding). These instructions may be executed by the processor 24 or, alternatively, by a virtual machine instantiated within the electronic decoding device 20.

[0143] The method in Figure 6 follows step E56, which involves decoding data Fc, which is a feature of the audio or video content representation format, in order to obtain the features of the format. For example, in the case of video content, decoding of data portion Fc allows obtaining the image size (in pixels), and / or frame rate, and / or the binary depth of the luminance information, and / or the binary depth of the chrominance information.

[0144] Next, the control module 25 proceeds to step E58, which decodes the data R that represents the decoded artificial neural network to be used.

[0145] According to the first possibility, as already shown, these data R are identifiers that indicate the decoded artificial neural network 28 within a given set of artificial neural networks, for example.

[0146] This predetermined set is, for example, a set of decoded artificial neural networks accessible by the electronic decoding device 20, that is, a set of artificial neural networks in which the electronic decoding device 20 stores a set of parameters that define the associated artificial neural network (as shown above), or a set of artificial neural networks in which the electronic decoding device 20 can access this set of parameters by connecting to a remote electronic device such as a server (as described below).

[0147] In this case, the control module 25 may, for example, proceed to read the set of parameters associated with the decoded identifier (this set of parameters defines the artificial neural network identified by the decoded identifier) ​​from the storage unit 22.

[0148] Alternatively (or if the storage unit 22 does not store a set of parameters for the artificial neural network identified by the decoded identifier), the control module 25 may transmit a request for a set of parameters to a remote server (this request may include, for example, the decoded identifier) ​​and receive a set of parameters defining the artificial neural network identified by the decoded identifier as a response.

[0149] According to a second possible embodiment, data R is data Rc describing a decoded artificial neural network 28.

[0150] As already shown, this descriptive data (or encoded data) is encoded according to standards such as MPEG-7 Part 17 or using a format such as JSON.

[0151] Decoding these descriptive data allows us to obtain parameters that define the artificial neural network used to decode the data obtained (by entropy decoding) from the binary element sequence Fnn.

[0152] In some embodiments, the use of the first or second possibility described above depends on the labels, which are also included in the data R, as already shown.

[0153] Regardless of which option is used, decoding the data R representing the decoding artificial neural network to be used allows (in this case, the control module 25) to determine the features of the decoding artificial neural network 28. Thus, for example, in the example described herein, the control module 25 determines the number N of feature maps expected at the input of the decoding artificial neural network 28 and the sizes H, W of these feature maps. In fact, as described above (see, for example, the description of step E2), each element of the feature map F in the input layer of the decoding artificial neural network 28, corresponding to the output layer of the encoding artificial neural network 8, is associated with the input node (or node of the input layer) of the decoding artificial neural network in a predetermined manner. Thus, the number and size of the feature maps F are linked to the features of the decoding artificial neural network 28 and specific header data such as the data Fc described above (in particular, including the image size).

[0154] Next, in step E60, the control module 25 proceeds to configure the parallel processing unit 26 using parameters that define the decoding artificial neural network 28 (parameters obtained in step E58) or parameters that define a commonly used machine learning method, so that the parallel processing unit 26 can implement the decoding artificial neural network 28 (or a machine learning method in general, such as another deep learning method or random forest learning method as an alternative).

[0155] This configuration step E60 specifically involves creating an instance of the decoding artificial neural network 28 within the parallel processing unit 26, using the parameters obtained in step E58.

[0156] This instance creation involves the following steps: - The step of securing memory space necessary to implement the decoding artificial neural network 28 within the parallel processing unit 26, and / or, - A step of programming a parallel processing unit 26 using parameters that define the decoding artificial neural network 28 (e.g., including weights Γ' and activation function) (parameters obtained in step E58), It may include.

[0157] Next, in step E62, the control module 25 proceeds to decode information I1 which indicates the set of contexts available in the entropy decoder 23 (in this case, the number of contexts K available in the entropy decoder 23). As already shown, for example, K = 160.

[0158] Next, in step E64, the control module 25 proceeds to decode the parameterized data Iinit associated with each of the different contexts of the set of contexts used (this set is determined for the information I1 decoded in step E62).

[0159] Next, the control module 25 can implement step E66, which initializes each context available in the entropy decoder 23 using the parameterized data Iinit of the relevant context (decoded in step E64).

[0160] More precisely, the entropy encoder 23 is adaptive, and each context is initialized with a probabilistic model defined by parameterized data Iinit associated with that context.

[0161] Alternatively, if the entropy decoder 23 uses a fixed probability model for each context, the control module 25, in step E66, configures each context available to the entropy decoder using a probability model defined by the parameterized data Iinit associated with that context.

[0162] Next, the control module 25 performs step E68, for each type of data to be obtained by entropy decoding, decodes data I2 which indicates the context that needs to be parameterized for the entropy decoding of the data having that type, and therefore the entropy encoder 23 needs to decode this data.

[0163] In the examples described herein, the control module 25 first decodes the indicator I2_mode (i.e., referred to herein) which, if the value is 0, indicates that the context is associated with each feature map F, and if the value is 1, indicates that the context is used for each location in the feature map F.

[0164] Therefore, depending on the value of the indicator I2_mode, number K, and the size H and W of the feature map, the control module 25 decodes an identifier I2_map[i] that represents the context in which the entropy decoder 23 needs to be parameterized to obtain this type of data when the entropy decoder 23 receives a portion of the sequence of binary elements Fnn as input (i.e., reads the data stream in this case).

[0165] For example, if the indicator I2_mode is equal to 0, the control module 25 decodes (or reads) an identifier I2_map[i] for each of the N feature maps F, which represents the context to which the entropy decoder 23 needs to be parameterized in order to obtain data associated with that feature map F (in this case, the representative value V). That is, in this case, the type of data is defined by the feature map F to which this data belongs (in this case, the representative value V).

[0166] If the indicator I2_mode is equal to 1, the control module 25 decodes (or reads) for each of the W × H locations (or places) in each feature map F an identifier I2_map[i] that represents the context in which the entropy decoder 23 needs to be parameterized to obtain the data (in this case, representative value V) at that location. In other words, in this case, the type of data is defined by the location of this data (in this case, representative value V) in the associated feature map F.

[0167] For each data point in the expected data sequence (i.e., for each of the representative values ​​V applied to the input of the decoded artificial neural network 28), in step E70, the control module 25, - Based on the identifier I2_map[i] associated with this data type, determine the context used to decrypt this data. - Parameterize the entropy decoder 23 in the context C determined in this way, and - A portion of the sequence of binary elements Fnn (in the order in which these binary elements were received) is applied to the input of the entropy decoder 23 so that the entropy decoder 23 produces the same expected data (here, the representative value V) as the output that was encoded by entropy coding in step E26.

[0168] As already shown (see the explanation of step E58 above), the control module 25 has previously determined the number N and sizes H and W of the feature maps, and therefore knows the number of representative values ​​V (or expected data) to be obtained during the entropy decoding step E70. Furthermore, as stated in the explanation of step E26, the different representative values ​​V are encoded by entropy coding in a predefined order, and therefore the representative values ​​V (expected data) are decoded in this same predefined order.

[0169] As already explained, when the indicator I2_mode is equal to 0, the context C determined to decode the elements (i.e., representative values ​​V) of a given feature map F is therefore determined based on the identifier I2_map[i] associated with this given feature map F (in this case, the type of data corresponding to the feature map containing this data).

[0170] Similarly, if the indicator I2_mode is equal to 1, the context C determined to decode a given element (i.e., representative value V) of the feature map F is determined based on the identifier I2_map[i] associated with the location of this given element in the feature map (in this case, the type of data corresponding to the location of the data).

[0171] Therefore, in step E70, the entropy decoder 23 generates the expected data sequence, which in this case is a set of values ​​V.

[0172] Next, the processor 24 (here directly or alternatively via the control module 25 at the output of the entropy encoder 23) may, in step E72, apply (i.e., present) these data to the artificial neural network 28 implemented by the parallel processing unit 26, such that the representative value V is processed at least partially by a decoding process using the artificial neural network 28.

[0173] In the example described here, the artificial neural network 28 receives a representative value V as input and generates an output representation I of the encoded content that is suitable for playback on an audio or video playback device. That is, the representative value V (in this case in the form of a feature map F) is applied to the input layer of the artificial neural network 28, and the output layer of the artificial neural network 28 generates the aforementioned representation I of the encoded content. Therefore, in the case of video content (including an image or a sequence of images), the artificial neural network 28 generates at least one matrix representation I of the image as output (i.e., on its output layer).

[0174] As is already understood regarding the encoding in step E6 described above, the association between the elements of feature map F (i.e., representative values ​​V) and the input nodes (or nodes of the input layer) is predefined.

[0175] In certain embodiments, in order to process a specific representative value V (for example, corresponding to a block or image), the artificial neural network 28 may receive as input at least some of the data generated at the output of the artificial neural network 28 while processing previous data (here, the previous representative value V) corresponding to a preceding block or preceding image. In this case, the process proceeds to step E74, in which the data generated at the output of the artificial neural network 28 is reinjected into the input of the artificial neural network 28.

[0176] Furthermore, according to other possible embodiments, the decoding process may use multiple artificial neural networks, as already described above for processing content data B.

[0177] Next, in step E76, the control module 25 determines whether the processing of the binary element sequence Fnn by the artificial neural network 28 has been completed.

[0178] In the case of a negative decision (N), this method loops back to step E70 to perform entropy decoding of the subsequent portion of the binary element sequence Fnn and applies the other representative value V (generated by this entropy decoding) to the artificial neural network 28.

[0179] If the decision is positive (P), this method terminates at step E78.

Claims

1. A method for decoding a data stream containing multiple identifiers (I2_map[i]) and sequences of binary elements (Fnn) into sequences of data (V) of a predetermined type, comprising the following steps for obtaining at least one data (V) from the sequence of data: - A step of determining the context (C) based on the identifier (I2_map[i]) associated with the current data type from among the multiple identifiers, - A step of receiving the sequence of binary elements (Fnn) as input and decoding a portion of the sequence of binary elements (Fnn) using an entropy decoder (23) parameterized in the determined context (C), Decryption methods including [specific methods].

2. The decoding method according to claim 1, wherein the data stream includes information (I1) indicating a set of contexts available for use within the entropy decoder (23).

3. The decoding method according to claim 2, wherein the information (I1) indicates the number of contexts available within the entropy decoder (23).

4. The decoding method according to any one of claims 1 to 3, wherein the data stream includes data (Iinit) for parameterizing each of at least one context available in the entropy decoder.

5. The decoding method according to claim 4, further comprising the step (E66) of initializing each context available in the entropy decoder (23) using the data (Iinit) for parameterizing each of the contexts.

6. A decoding method according to any one of claims 1 to 5, comprising the step (E72) of applying the acquired data (V) to the input of an artificial neural network (28).

7. The decoding method according to claim 6, wherein the artificial neural network (28) is implemented by a processing unit (26), and the method includes the step (E60) of configuring the processing unit (26) according to the data (R) contained in the data stream.

8. The decoding method according to any one of claims 1 to 7, wherein the sequence of data forms a set of feature maps (F).

9. The decoding method according to claim 8, wherein the identifier among the plurality of identifiers is associated with each feature map (F) of the set of feature maps, and the context determined to obtain elements of a given feature map (F) is determined based on the identifier associated with the given feature map (F).

10. The decoding method according to claim 8, wherein the feature map (F) of the set of feature maps has a common structure in which each element of the feature map is defined by location, the identifiers among the plurality of identifiers are each associated with different locations in the common structure, and the context determined to obtain a given element of the feature map (F) is determined based on the identifiers associated with the locations that define the given element.

11. An electronic device (20) for decoding a data stream containing multiple identifiers (I2_map[i]) and a sequence of binary elements (Fnn) into a sequence of data (V) of a predetermined type, - An entropy decoder (23) that receives the sequence of binary elements (Fnn) as input, - A configuration module (25) designed to determine a context (C) based on an identifier (I2_map[i]) associated with the type of data currently to be acquired from among the plurality of identifiers, and to parameterize the entropy decoder (23) in the determined context (C) in order to acquire the data at the output of the entropy decoder (23), Electronic device (20) including.

12. A computer program comprising instructions executable by a processor (24), wherein, when the instructions are executed by the processor (24), the computer program is adapted to implement the decoding method described in claim 1.

13. A method for encoding data into a data stream, A method comprising the step of generating a data stream that represents a sequence of data (V) of each predetermined type and includes a plurality of identifiers (I2_map[i]) and a sequence of binary elements (Fnn), wherein each identifier (I2_map[i]) represents a context (C), and the entropy decoder (23) is configured to be parameterized to obtain at least one data having a type associated with the identifier (I2_map[i]) when the entropy decoder (23) receives a portion of the sequence of binary elements (Fnn) as input in the context (C).

Images