Method and electronic device for decoding a data stream, and associated data stream

Abstract

In a method for decoding a data stream using an electronic device (10) comprising a processor (14) and a parallel processing unit (16) designed to perform a plurality of operations of the same type in parallel at any given time, the data stream comprises a first set of data (Fet) and a second set of data (Fnn) representing audio or video content. The decoding method comprises the following steps: - processing of the data in the first set of data (Fet) by the processor (14); - obtaining the audio or video content by processing (E70) the data in the second set of data (Fnn) according to a process dependent on at least a part of the data in the first set (Fet) and using an artificial neural network (18) implemented by the parallel processing unit (16). Figure for the abstract: Figure 7

Description

Title of the invention: Method and electronic device for decoding a data stream, and associated data stream Technical field of the invention

[0001] The present invention relates to the technical field of decoding content, in particular audio or video content.

[0002] It relates in particular to a method and an electronic device for decoding a data stream, as well as an associated data stream. State of the art

[0003] It is known, particularly in the field of video decoding, to use an electronic device comprising both a processor (generally a central processing unit of the electronic device, or CPU for "Central Processing Unit") and a parallel processing unit designed to perform in parallel at a given time a plurality of operations of the same type. Such a parallel processing unit is for example a graphics processing unit or GPU (for "Graphical Processing Unit Tensor Processing Unit"), as described for example in the article "Google's Tensor Processing Unit explained: this is what the future of computing looks like", by Joe Osborne, Techradar, August 22, 2016.

[0004] It has also been proposed to compress data representative of video content using an artificial neural network. The decoding of the compressed data can then be carried out using another artificial neural network, as described for example in the article "DVC: An End-to-end Deep Video Compression Framework", by Guo Lu et al., 2019 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 10998-11007. Presentation of the invention

[0005] In this context, the present invention proposes a method for decoding a data stream by means of an electronic device comprising a processor, and a parallel processing unit designed to perform in parallel at a given instant a plurality of operations of the same type, characterized in that the data stream comprises a first set of data and a second set of data representative of an audio or video content, and in that the method comprises the following steps:

[0006] - processing of the data of the first data set by the processor;

[0007] - obtaining audio or video content by processing the data of the second set of data according to a process depending on at least part of the data of the first set and using an artificial neural network implemented by the parallel processing unit.

[0008] The processing of the data of the second set by means of the artificial neural network implemented by the parallel processing unit can thus be adapted according to the data of the first set contained in the data stream. This provides flexible and efficient processing of the second data with a view to their decoding by an artificial neural network.

[0009] The method may thus, for example, comprise a step of configuring the parallel processing unit as a function of at least part of the data of the first data set.

[0010] The configuration of the parallel processing unit may comprise an allocation of memory of the parallel processing unit and / or an instantiation of the memory of the parallel processing unit and / or an assignment of values ​​(as a function of said at least part of the data of the first data set) to the processing implemented on the parallel processing unit (here in practice an assignment of weights and / or activation functions defining the artificial neural network, these weights and / or activation functions being determined as a function of said at least part of the data of the first data set).

[0011] According to a first possibility, the first data set may comprise descriptive data of the artificial neural network (for example coding data of the artificial neural network). In this case, in the configuration step, the processor may configure the parallelized processing unit on the basis of this descriptive data.

[0012] According to a second possibility, the electronic device may comprise a unit for storing a plurality of parameter sets respectively defining a plurality of artificial neural networks. The first data set may in this case comprise an identifier. In the configuration step, the processor may then configure the parallelized processing unit on the basis of a parameter set associated with this identifier among the plurality of parameter sets.

[0013] The processor is for example a microprocessor; the processor can thus successively execute a plurality of instructions of a computer program.

[0014] It may further be provided that the data stream further comprises instructions executable (for example by the processor, or alternatively by a virtual machine) within the electronic device. The processing of the data of the first data set may in this case be at least partly carried out due to the execution of at least part of these instructions.

[0015] In particular, the step of configuring the parallel processing unit can then be carried out due to the execution of at least part of these instructions.

[0016] The decoding method may further comprise a step of identifying the first data set and the second data set within the data stream (for example by using a predetermined bit length for the first data set and / or the second data set, or by using a boundary indicator between the first data set and the second data set).

[0017] The first data set may further comprise data representative of characteristics of the format of the content encoded by the data stream.

[0018] When the content is video content (i.e. this content comprises at least one image, possibly a sequence of images), the processing of the data of the second set can produce for example at least one matrix representation of at least part of an image (for example a block of the image or of a component of the image, or even the entire image).

[0019] The artificial neural network can receive as input (i.e. at an input layer of the artificial neural network) data from the second data set. The artificial neural network can further produce as output (i.e. at an output layer of the artificial neural network) the aforementioned matrix representation.

[0020] According to one possible embodiment, the artificial neural network can further receive as input (i.e. at the level of the aforementioned input layer) data produced beforehand as output (i.e. at the level of the output layer) of the artificial neural network.

[0021] The invention also proposes an electronic device for decoding a data stream comprising a first set of data and a second set of data representative of audio or video content, the electronic device comprising:

[0022] - a processor adapted to process the data of the first data set;

[0023] - a parallel processing unit designed to perform in parallel at a time given a plurality of operations of the same type and adapted to obtain the audio or video content by processing the data of the second set of data according to a process dependent on at least part of the data of the first set and using an artificial neural network implemented by the parallel processing unit.

[0024] As already indicated, the processor may further be adapted to configure the parallel processing unit as a function of at least part of the data of the first data set.

[0025] The parallel processing unit can be adapted to produce at least one matrix representation of at least part of an image.

[0026] The invention finally proposes a data stream comprising a first set of data and a second set of data representative of audio or video content, in which the first set of data comprises data defining at least in part a process for processing the data of the second set of data using an artificial neural network.

[0027] As already indicated and explained below, these data defining at least in part the processing process may be an identifier of the artificial neural network (among a predetermined set of artificial neural networks) or descriptive data (for example coding data) of the artificial neural network.

[0028] Of course, the various features, variants and embodiments of the invention may be combined with each other in various combinations to the extent that they are not incompatible or mutually exclusive. Detailed description of the invention

[0029] Furthermore, various other characteristics of the invention emerge from the appended description given with reference to the drawings which illustrate non-limiting forms of embodiment of the invention and where:

[0030] [fig. 1] represents an electronic coding device used in the context of the invention;

[0031] [fig.2] is a flowchart representing steps of a coding process implemented work within the electronic coding device of figure 1;

[0032] [fig.3] is a first example of data flow obtained by the method of figure 2 ;

[0033] [fig.4] is a second example of data flow obtained by the method of figure 2 ;

[0034] [fig.5] is a third example of data flow obtained by the method of figure 2 ;

[0035] [fig.6] is a fourth example of data flow obtained by the method of the figure 2;

[0036] [fig.7] represents an electronic coding device according to one embodiment of the invention; and

[0037] [fig.8] is a flowchart representing steps of a decoding method implemented work within the electronic decoding device of figure 7.

[0038] Figure 1 represents an electronic coding device 2 using at least one artificial neural network 8.

[0039] This electronic coding device 2 comprises a processor 4 (for example a microprocessor) and a parallel processing unit 6, for example a unit of graphics processing unit (or GPU) or a tensor processing unit (or TPU).

[0040] As schematically represented in FIG. 1, the processor 4 receives data P, B representing audio or video content to be compressed, here data of format P and data of content B.

[0041] The P format data indicates characteristics of the format for representing the audio or video content, for example for video content the dimensions (in pixels) of the images, the frame rate, the bit depth of the luminance information and the bit depth of the luminance information.

[0042] The content data B forms a representation (here uncompressed) of the audio or video content. For example, in the case of video content, the content data comprises, for each pixel of each image of an image sequence, data representing a luminance value of the pixel and data representing chrominance values ​​of the pixel.

[0043] The parallel processing unit 6 is designed to implement an artificial neural network 8 after having been configured by the processor 4. To do this, the parallel processing unit is designed to perform in parallel at a given time a plurality of operations of the same type.

[0044] As explained below, the artificial neural network 8 is used in the context of processing content data B aimed at obtaining compressed data C.

[0045] In the embodiment described herein, when the content data B is input to the artificial neural network 8, the artificial neural network 8 outputs the compressed data C.

[0046] The content data B applied as input to the artificial neural network 8 (i.e. applied to an input layer of the artificial neural network 8) may represent a block of an image, or a block of a component of an image (for example a block of a luminance or chrominance component of this image, or a block of a color component of this image), or an image of a video sequence, or a component of an image of a video sequence (for example a luminance or chrominance component, or a color component), or even a series of images of the video sequence.

[0047] It can be provided for example in this case that at least some of the neurons of the input layer each receive a pixel value of a component of an image, value represented by one of the content data B.

[0048] Alternatively, the processing of the content data B may comprise the use of several artificial neural networks, as described for example in the aforementioned article "DVC: An End-to-end Deep Video Compression Framework", by Guo Lu et al. ., 2019 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.

[0049] An example of a coding method implemented by the electronic coding device 2 is now described with reference to FIG. 2.

[0050] A memory linked to the processor 4 stores, for example, computer program instructions designed for the implementation of at least part of the steps of the method of FIG. 2 when these instructions are executed by the processor 4. In other words, the processor 4 is programmed to implement at least part of the steps of FIG. 2.

[0051] The method of figure 2 begins here with an optional step E2 of receiving a list of artificial neural networks accessible by an electronic decoding device.

[0052] The list is for example received by the processor 4 of the electronic coding device 2 directly from an electronic decoding device (conforming for example to the electronic decoding device 10 of FIG. 7 described later).

[0053] As explained below, the artificial neural networks accessible by the electronic decoding device are artificial neural networks for which the electronic decoding device stores parameters defining the artificial neural network concerned or can have access to these parameters by connection to remote electronic equipment such as a server.

[0054] Alternatively, the list could be received by the processor 4 of the electronic device 2 from a remote server, such as the aforementioned server.

[0055] The method of Figure 2 continues with a step E4 of selecting a coding process - decoding process pair. As already indicated for the coding process, the coding process and the decoding process each use at least one artificial neural network.

[0056] In the example described herein, the encoding process is implemented by an encoding artificial neural network and the decoding process is implemented by a decoding artificial neural network.

[0057] The assembly formed by the coding artificial neural network and by the decoding artificial neural network (the output of the coding artificial neural network being applied to the input of the decoding artificial neural network) constitutes for example an auto-encoder.

[0058] The coding process - decoding process pair is for example selected from a plurality of predefined coding process - decoding process pairs, i.e. here from a plurality of coding artificial neural network - decoding artificial neural network pairs.

[0059] When a list of artificial neural networks accessible by an electronic decoding device is previously received (as explained above in step E2), the coding process - decoding process pair is for example selected from coding process - decoding process pairs for which the decoding process uses an artificial neural network present in the received list.

[0060] The coding process - decoding process pair can also be selected according to the intended application (indicated for example by a user by means of a user interface not shown of the electronic coding device 2). For example, if the intended application is a videoconference, the selected coding process - decoding process pair comprises a low latency decoding process. In other applications, the selected coding process - decoding process pair will comprise a random access decoding process.

[0061] In a low-latency decoding process of a video sequence, an image of the video sequence is for example represented by coded data which can be sent and decoded immediately; the data can then be sent in the order in which the images of the video are displayed, which in this case guarantees a latency of one image between encoding and decoding.

[0062] In a random access decoding process of a video sequence, the coded data relating respectively to a plurality of images are sent in an order different from the display order of these images, which makes it possible to increase the compression. Images coded without reference to the other images (so-called intra images) can then be coded regularly, which makes it possible to start the decoding of the video sequence from several places in the coded stream.

[0063] On this subject, reference may be made to the article "OverView of the High Efficiency Video Coding (HEVC) Standard", by G.J. Sullivan, J.-R. Ohm, W.-J. Han and T. Wiegand, in IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649-1668, Dec. 2012.

[0064] The coding process - decoding process pair can also be selected in order to obtain the best possible compression-distortion compromise.

[0065] To do this, a plurality of coding process - decoding process sets can be applied to the content data B and the set for which the best compression-distortion compromise is achieved can be selected.

[0066] Alternatively, the content type can be determined (for example by analyzing the content data B) and the coding process - decoding process pair can be selected based on the determined type.

[0067] The coding process - decoding process pair can also be selected according to the computing power available at the electronic device level. electronic decoding device. Information representative of this computing power may have previously been transmitted from the electronic decoding device to the electronic coding device (and received for example by the electronic coding device in step E2 described above).

[0068] The different selection criteria for the coding process - decoding process pair can possibly be combined.

[0069] Once the coding process - decoding process pair has been selected, the processor 4 proceeds in step E6 to configure the parallel processing unit 6 so that the parallel processing unit 6 can implement the selected coding process.

[0070] This step E6 comprises in particular the instantiation, within the parallel processing unit 6, of the artificial coding neural network 8 used by the selected coding process.

[0071] This instantiation may notably include the following steps:

[0072] - reservation, within the parallel processing unit 6, of the memory space born necessary for the implementation of the coding artificial neural network; and / or

[0073] - programming of the parallel processing unit 6 with the weights W and the activation functions defining the coding artificial neural network 8; and / or

[0074] - loading at least part of the content data B into a memory local of the parallel processing unit 6.

[0075] The method of figure 2 then comprises a step E8 of implementing the coding process, that is to say here of applying the content data B as input to the coding artificial neural network 8 (or in other words of activating the coding artificial neural network 8 by taking the content data B as input).

[0076] Step E8 thus makes it possible to produce (here at the output of the coding artificial neural network 8) the compressed data C.

[0077] The following steps aim at the coding (i.e. the preparation) of the data stream containing in particular the compressed data C and intended for the electronic decoding device (for example the electronic decoding device 10 described below with reference to FIG. 7).

[0078] The method thus notably comprises a step E10 of coding a first header part Fc which comprises data characteristic of the representation format of the audio or video content (here for example data linked to the format of the video sequence being coded).

[0079] These data forming the first header part Fc indicate for example the dimensions (in pixels) of the images, the image frequency, the bit depth of the luminance information and the bit depth of the luminance information. These data are for example constructed on the basis of the above format data F data (after possible reformatting).

[0080] The method of figure 2 then continues with a step E12 of determining the availability of the artificial decoding neural network (used by the decoding process selected in step E4) for the electronic decoding device capable of decoding the data stream (for example the electronic decoding device 10 described below with reference to figure 7).

[0081] This determination can be made on the basis of the list received in step E2: the processor 4 determines in this case whether the decoding artificial neural network used by the decoding process selected in step E4 is part of the list received in step E2. (Naturally, in the embodiments where the coding process - decoding process pair is systematically chosen to correspond to a decoding artificial neural network available for the electronic decoding device, step E12 can be omitted and the method then continues in step E14.)

[0082] According to one possible embodiment, in the absence of information on the availability of the decoding artificial neural network for the electronic decoding device, the method continues at step E16 (so that descriptive data of the decoding artificial neural network are transmitted to the electronic decoding device as explained below).

[0083] If the processor 4 determines in step E12 that the decoding artificial neural network is available for the electronic decoding device (arrow P), the method continues in step E14 described below.

[0084] If the processor 4 determines in step E12 that the decoding artificial neural network is not available for the electronic decoding device (arrow N), the method continues in step E16 described below.

[0085] Alternatively, the choice of step E14 or step E16 as a step subsequent to step E12 could be made on another criterion, for example as a function of a dedicated indicator stored within the electronic coding device 2 (and possibly adjustable by the user via a user interface of the electronic coding device 2) or as a function of a choice of the user (obtained for example via a user interface of the electronic coding device 2).

[0086] The processor 4 proceeds in step E14 to the coding of a second header part comprising an indicator IND and a third header part comprising here an identifier Inn of the decoding artificial neural network.

[0087] The indicator IND coded in the data stream in step E14 indicates that the decoding artificial neural network is part of a predetermined set of artificial neural networks, here the set of artificial neural networks available (or accessible) for the electronic decoding device (i.e. for example the set of artificial neural networks in the list received in step E2).

[0088] The identifier Inn of the decoding artificial neural network is an identifier defining by convention (shared in particular by the electronic coding device and the electronic decoding device) this decoding artificial neural network, for example within the aforementioned predetermined set.

[0089] The processor 4 proceeds in step E16 to the coding of a second header part comprising an indicator IND' and a third header part here comprising descriptive data Rc of the decoding artificial neural network.

[0090] The indicator IND' coded in the data stream in step El6 indicates that the decoding artificial neural network is coded in the data stream, i.e. represented by means of the aforementioned descriptive data Rc.

[0091] The decoding artificial neural network is for example encoded (i.e. represented) by the descriptive data (or coding data of the decoding artificial neural network) Rc in accordance with a standard such as the MPEG-7 part 17 standard or a format such as the JSON format.

[0092] On this subject, reference may be made to the article "DeepCABAC: Context-adaptive binary arithmetic coding for deep neural network compression", by S. Wiedemann et al., in Proceedings of the 36th International Conference on Machine Learning, Long Beach, California, PMLR 97, 2019, or to the article "Compact and Computationally Efficient Representation of Deep Neural Networks", by S. Wiedemann et al., in IEEE Transactions on Neural Networks and Learning Systems (Vol. 31, Iss. 3), March 2020.

[0093] After step E14 as after step E16, the method of FIG. 2 continues with a step El8 of determining the possibility for the electronic decoding device to implement the decoding process using the artificial decoding neural network.

[0094] The processor 4 determines for example this possibility by determining (possibly by means of prior exchanges between the electronic coding device and the electronic decoding device) whether the electronic decoding device comprises a module adapted to implement this decoding process or software adapted to the implementation of this decoding process by the electronic decoding device when this software is executed by a processor of the electronic decoding device.

[0095] If the processor 4 determines that it is possible for the electronic decoding device to implement the decoding process, the method continues at step E22 described below.

[0096] If the processor 4 determines that it is not possible for the electronic decoding device to implement the decoding process, the method performs step E20 described below (before moving on to step E22).

[0097] Alternatively, the choice of whether or not to carry out step E20 (before carrying out step E22) could be carried out on another criterion, for example based on a dedicated indicator stored within the electronic coding device 2 (and possibly adjustable by the user via a user interface of the electronic coding device 2) or based on a choice of the user (obtained for example via a user interface of the electronic coding device 2).

[0098] The processor 4 encodes in the data stream at step E20 a fourth header part containing a computer program Exe (or code) executable by a processor of the electronic decoding device. (The use of the computer program Exe within the electronic decoding device is described below with reference to FIG. 8.)

[0099] In order to be adapted to execution within the electronic decoding device, the computer program is for example chosen from within a library as a function of information relating to the hardware configuration of the electronic decoding device (information received for example during prior exchange between the electronic coding device 2 and the electronic decoding device).

[0100] The processor 4 then proceeds to a step E22 of coding a compressed stream Fnn on the basis of the compressed data C obtained in step E8.

[0101] It should be noted in this regard that, in the preceding description, step E8 was described before the steps of coding the Fet header (steps E10 to E20). Step E8 could, however, in practice be carried out just before step E22.

[0102] In particular, when step E8 allows the processing of only a part of the audio or video content to be compressed (for example when step E8 performs the processing of a block, or a component, or an image of a video sequence to be compressed), it is possible to repeat the implementation of steps E8 (to obtain compressed data relating to the successive parts of the content) and E22 (to code the compressed data obtained in the data stream).

[0103] The processor 4 can thus construct in step E24 the complete data stream comprising the header Fet and the compressed stream Fnn.

[0104] The complete data stream is constructed so that the header Fet and the compressed stream Fnn are individually identifiable.

[0105] According to one possible embodiment, the header Fet contains an indicator of the start of the compressed stream Fnn in the complete data stream. This indicator is, for example, the location, in bits, of the start of the compressed stream Fnn from the start of the complete data stream. (In other words, the header in this case has a predetermined fixed length.)

[0106] Other means of identifying the header Fet and the compressed stream Fnn are conceivable as a variant, such as for example a marker (i.e. a combination of bits used to indicate the start of the compressed stream Fnn and the use of which is prohibited in the rest of the data stream, or at least in the Fet header).

[0107] Figures 3 to 6 show examples of complete data flows that can be obtained using the method of Figure 2.

[0108] As explained above, these data streams comprise a Fet header and a compressed Fnn stream.

[0109] In the case of Figure 3 (which corresponds to the situation where step E14 has been implemented and where step E20 has not been implemented), the header comprises:

[0110] - a first part Fc comprising the characteristic data of the representation format presentation of audio or video content;

[0111] - a second part comprising the indicator IND indicating that the network of artificial decoding neurons is part of a predetermined set of artificial neural networks; and

[0112] - a third part comprising the Inn identifier of the artificial neural network of decoding.

[0113] In the case of Figure 4 (which corresponds to the situation where step E16 has been implemented and where step E20 has not been implemented), the header comprises:

[0114] - a first part Fc comprising the characteristic data of the representation format presentation of audio or video content;

[0115] - a second part comprising the indicator IND' indicating that the network of artificial decoding neurons is encoded in the data stream; and

[0116] - a third part comprising the descriptive Rc data (here the data of coding) of the decoding artificial neural network.

[0117] In the case of Figure 5 (which corresponds to the situation where steps E14 and E20 have been implemented), the header comprises:

[0118] - a first part Fc comprising the characteristic data of the representation format presentation of audio or video content;

[0119] - a second part comprising the indicator IND indicating that the network of artificial decoding neurons is part of a predetermined set of artificial neural networks;

[0120] - a third part comprising the Inn identifier of the artificial neural network decoding; and

[0121] - a fourth part comprising the computer program Exe.

[0122] In the case of Figure 6 (which corresponds to the situation where steps E16 and E20 have been implemented), the header comprises:

[0123] - a first part Fc comprising the characteristic data of the representation format presentation of audio or video content;

[0124] - a second part comprising the indicator IND' indicating that the network of artificial decoding neurons is encoded in the data stream; and

[0125] - a third part comprising the descriptive Rc data (here the data of coding) of the decoding artificial neural network; and

[0126] - a fourth part comprising the computer program Exe.

[0127] The data stream constructed in step E24 can be encapsulated in transmission formats known per se, such as the “Packet-Transport System” format or the “Byte-Stream” format.

[0128] In the case of the "Packet-Transport System" format (as proposed for example by the RTP protocol), the data is encoded in identifiable packets and transmitted over a communication network. The network can easily identify the boundaries of the data (images, groups of images and here Fet header and Fnn compressed stream), using the packet identification information provided by the network layer.

[0129] In the "Byte-Stream" format, there are no packets specifically and the construction of step E24 must make it possible to identify the boundaries of the relevant data (such as boundaries between parts of the stream corresponding to each image, and here between header Fet and compressed stream Fnn) using additional means, such as the use of network abstraction layer units (or NAL units for "Network Abstraction Layer"). where unique combinations of bits (such as 0x00000001) make it possible to identify the boundaries between data).

[0130] The complete data stream constructed in step E24 can then be transmitted in step E26 to the electronic decoding device 26 (by communication means not shown and / or through at least one communication network), or stored within the electronic coding device 2 (for subsequent transmission or, as a variant, subsequent decoding, for example within the electronic coding device itself, which is in this case designed to further implement the decoding method described below with reference to FIG. 8).

[0131] When the audio or video content comprises a plurality of parts (for example a plurality of groups of images when the content is a video sequence), the method of steps E4 to E24 may optionally be implemented for each of the parts of the content (for example for each group of images) so as to obtain a data stream as represented in one of FIGS. 3 to 6 for each part of the content (for example for each group of images). Thus, the compressed stream Fnn relating to each group of images may be decoded using an artificial neural network specific to the group of images concerned and possibly different from the artificial neural networks used for the other groups of images, as described below. The artificial neural networks may optionally have identical structures (and differ only by the weights and / or the activation functions which define a particular artificial neural network).

[0132] Figure 7 represents an electronic decoding device 10 using at least an artificial neural network 18.

[0133] This electronic decoding device 10 comprises a reception unit 11, a processor 14 (for example a microprocessor) and a parallel processing unit 16, for example a graphics processing unit (or GPU for "Graphical Processing Unit") or a tensor processing unit (or TPU for "Tensor Processing Unit").

[0134] The reception unit 11 is for example a communication circuit (such as a radiofrequency communication circuit) and makes it possible to receive data (and in particular here the coded data stream) from an external electronic arrangement, such as the electronic coding device 2, and to communicate this data to the processor 14 (to which the reception unit 11 is for example connected by a bus).

[0135] The electronic coding device 10 also comprises a storage unit 12, for example a memory (possibly a rewritable non-volatile memory) or a hard disk. Although the storage unit 12 is shown in FIG. 7 as a separate element from the processor 14, the storage unit 12 could alternatively be integrated into (i.e., included in) the processor 14.

[0136] The processor 14 is in this case designed to successively execute a plurality of instructions of a computer program stored for example in the storage unit 12.

[0137] The parallel processing unit 16 is designed to implement the artificial neural network 18 after having been configured by the processor 14. To do this, the parallel processing unit 16 is designed to perform in parallel at a given instant a plurality of operations of the same type.

[0138] As schematically represented in figure 7, the processor 14 receives a data stream (for example via communication means not shown of the electronic decoding device 10) comprising a first set of data, here the header Fet, and a second set of data representative of the audio or video content, here the compressed stream Fnn.

[0139] As explained below, the artificial neural network 18 is used in the context of processing the second set of data (i.e. here compressed data Fnn) to obtain audio or video content corresponding to the initial audio or video content B.

[0140] The storage unit 12 can store a plurality of parameter sets, each parameter set defining a decoding artificial neural network. As explained below, the processor 14 can in this case configure the parallelized processing unit 16 by means of a particular parameter set among these parameter sets so that the parallelized processing unit 16 can then implement the artificial neural network defined by this particular parameter set.

[0141] The storage unit 12 can in particular store a first set of parameters defining a first artificial neural network forming a random access decoder and / or a second set of parameters defining a second artificial neural network forming a low latency decoder.

[0142] The electronic decoding device 10 in this case has in advance decoding possibilities both for situations where random access to the content is desired and for situations where the content is desired to be displayed without delay.

[0143] A decoding method implemented within the electronic decoding device 10 and using the artificial neural network 18 implemented by the parallel processing unit 16 is now described with reference to FIG. 8.

[0144] This method may begin with an optional step of transmission, by the electronic decoding device 10 and to a device for controlling the transmission of the data stream to be decoded, of a list L of artificial neural networks accessible by the electronic decoding device 10. The device for controlling the transmission of the data stream may be, for example, the electronic coding device 2. (The electronic coding device 2 in this case receives this list L in step E2 described above with reference to FIG. 2.) Alternatively, the device for controlling the transmission of the data stream could be a dedicated server, operating in cooperation with the electronic coding device 2.

[0145] The artificial neural networks accessible by the electronic decoding device 10 are the artificial neural networks for which the electronic decoding device 10 stores a set of parameters defining the artificial neural network concerned (as indicated above) or can have access to this set of parameters by connection to remote electronic equipment such as a server (as explained below).

[0146] The method of FIG. 8 comprises a step E52 of reception (by the electronic decoding device 10, and specifically here by the reception unit 11) of the data stream comprising the first set of data, namely the header Fet; and the second set of data, namely the compressed stream Fnn. The reception unit 11 transmits the received data stream to the processor 14.

[0147] The processor 14 then proceeds to a step E54 of identifying the first set of data (header Fet) and the second set of data (compressed stream Fnn) within the received data stream, for example by means of the compressed stream start indicator (already mentioned during the description of step E24).

[0148] The processor 14 can also identify in step E54 the different parts of the first set of data (header), namely here within the header Fet: the first part Fc (comprising data representative of characteristics of the format of the content coded by the data stream), the second part (IND indicator or IND'), the third part (Inn identifier or Rc coding data) and possibly the fourth part (Exe computer program), as illustrated in Figures 3 to 6 described above.

[0149] In cases where executable instructions (such as the instructions of the computer program Exe) are identified (i.e. detected) within the first data in step E54, the processor 14 can launch in step E56 the execution of these executable instructions in order to implement at least some of the steps (described below) of processing the data of the first data set. These instructions can be executed by the processor 14 or, alternatively, by a virtual machine instantiated within the electronic decoding device 10.

[0150] The method of Figure 7 continues with a step E58 of decoding the Fc data characteristic of the representation format of the audio or video content so as to obtain characteristics of this format. In the case of video content for example, the decoding of the Fc data makes it possible to obtain the dimensions (in pixels) of the images and / or the image frequency and / or the bit depth of the luminance information and / or the bit depth of the luminance information.

[0151] The processor 14 then proceeds (in certain embodiments, due to the execution of instructions identified within the first set of data in step E54, as already indicated) to a step E60 of decoding the indicator IND, IND' contained here in the second part of the header Fet.

[0152] If the decoding of the indicator IND, IND' present in the received data stream indicates that the artificial neural network 18 to be used for the decoding is part of a predetermined set of artificial neural networks (i.e. if the indicator present in the first data set is the indicator IND indicating that the decoding artificial neural network 18 is part of a predetermined set of artificial neural networks), the method continues at step E62 described below.

[0153] If the decoding of the indicator IND, IND' present in the received data stream indicates that the artificial neural network 18 to be used for the decoding is coded in the data stream (i.e. if the indicator present in the first data set is the indicator IND' indicating that the decoding artificial neural network 18 is coded in the data stream), the method continues at step E66 described below.

[0154] In step E62, the processor 14 proceeds (in certain embodiments, due to the execution of instructions identified within the first data set in step E54, as already indicated) to decode the identifier Inn (contained here in the third part of the header Fet). As already indicated, this identifier Inn is an identifier designating the decoding artificial neural network 18, for example at within the aforementioned predetermined set of artificial neural networks.

[0155] The processor 14 can then proceed (in certain embodiments, due to the execution of instructions identified within the first set of data in step E54, as already indicated) in step E64 to read, for example in the storage unit 12, a set of parameters associated with the decoded identifier Inn (this set of parameters defining the artificial neural network identified by the decoded identifier Inn).

[0156] According to one possible embodiment, it can be provided that the processor 14 generates an error message in the event of absence (here within the storage unit 12) of data (in particular parameters) relating to this artificial neural network identified by the decoded identifier Inn.

[0157] Alternatively (or in the case where no set of parameters is stored in the storage unit 12 for the artificial neural network identified by the decoded identifier Inn), the electronic decoding device 10 can transmit (in certain embodiments, due to the execution of instructions identified within the first data set in step E54, as already indicated) a request for a set of parameters to a remote server (this request including for example the decoded identifier Inn) and receive in response to step E64 the set of parameters defining the artificial neural network identified by the decoded identifier Inn.

[0158] The method then continues at step E68 described below.

[0159] In step E66, the processor 14 proceeds (in certain embodiments, due to the execution of instructions identified within the first set of data in step E54, as already indicated) to decode the descriptive data Rc of the artificial neural network 18 (contained here in the third part of the header Fet).

[0160] As already indicated, these descriptive data (or coding data) Rc are for example coded in accordance with a standard such as the MPEG-7 part 17 standard or a format such as the JSON format.

[0161] The decoding of the descriptive data Rc makes it possible to obtain the parameters defining the artificial neural network to be used for the decoding of the data of the second data set (i.e. here data of the compressed flow Fnn).

[0162] The method continues in this case also at step E68 described now.

[0163] The processor 14 then proceeds (in certain embodiments, due to the execution of instructions identified within the first data set in step E54, as already indicated) in step E68 to configure the parallelized processing unit 16 by means of the parameters defining the decoding artificial neural network 18 (parameters obtained in step E64 or in step E66) so that the parallelized processing unit 16 can implement the decoding artificial neural network 18.

[0164] This configuration step E68 notably comprises the instantiation of the decoding artificial neural network 18 within the parallel processing unit 16, here using the parameters obtained in step E64 or in step E66.

[0165] This instantiation may notably include the following steps:

[0166] - reservation, within the parallel processing unit 16, of the memory space born necessary for the implementation of the decoding artificial neural network 18; and / or

[0167] - programming the parallel processing unit 16 with the parameters (including for example weights W' and activation functions) defining the decoding artificial neural network 18 (parameters obtained in step E64 or in step E66); and / or

[0168] - loading at least part of the data from the second data set (i.e. at least part of the data of the compressed stream Fnn) on a local memory of the parallel processing unit 16.

[0169] As is clear from the description of steps E58 to E68 above, the data of the first data set Fet are thus processed by the processor 14.

[0170] The processor 14 can then apply (i.e. present) at step E70 the data of the second set of data (here data of the compressed flow Fnn) to the artificial neural network 18 implemented by the parallel processing unit 16 so that this data is processed by a decoding process using at least in part the artificial neural network 18.

[0171] In the example described here, the artificial neural network 18 receives as input the data of the second data set Fnn and produces as output a representation I of the coded content suitable for reproduction on an audio or video reproduction device. In other words, at least some of the data of the second data set Fnn are applied to the input layer of the artificial neural network 18 and the output layer of the artificial neural network 18 produces the aforementioned representation I of the coded content. In the case of video content (comprising an image or a sequence of images), the artificial neural network 18 thus produces as output (i.e. at its output layer) at least one matrix representation I of an image.

[0172] In certain embodiments, for the processing of certain data of the compressed stream Fc (corresponding for example to a block or an image), the artificial neural network 18 can receive as input at least some of the data produced at the output of the artificial neural network 18 during the processing of previous data (corresponding for example to the previous block or to the previous image) in the compressed stream Fc. In this case, a step E72 is carried out for reinjecting data produced at the output of the artificial neural network 18 towards the input of the artificial neural network 18.

[0173] Furthermore, according to other possible embodiments, the decoding process could use a plurality of artificial neural networks, as already mentioned above with regard to the processing of content data B.

[0174] The data of the second set (here at least some data of the compressed flow Fnn) were thus processed by a process dependent on part of the data of the first set (process dependent here on the identifier Inn or the coding data Rc) and using the artificial neural network 18 implemented by the parallel processing unit 16.

[0175] The processor 14 then determines in step E74 whether the processing of the compressed stream Fnn by means of the artificial neural network 18 is complete.

[0176] In the event of a negative determination (N), the method loops to step E70 to apply other data from the compressed stream Fnn to the artificial neural network 18.

[0177] In the event of a positive determination (P), the method continues at step E76 where the processor 14 determines whether there is data remaining to be processed in the received data stream.

[0178] In the event of a negative determination (N) in step E76, the method is terminated in step E78.

[0179] In the event of a positive determination (P) in step E76, the method loops to step E52 for processing a new part of the data flow as represented in one of figures 3 to 6.

[0180] As indicated above with regard to the reiteration of the coding steps E4 to E24, this other part of the data stream then also comprises a first set of data and a second set of data representative of another audio or video content (for example, in the case of video content, another group of images for the content representation format used). Another artificial neural network can in this case be determined on the basis of some of these first data (identifier Inn or coding data Rc), as described above in steps E54 to E66, then the parallelized processing unit 16 can be configured to implement this other artificial neural network (in accordance with step E68 described above).The data of the second data set of this other part of the data stream (relating for example to the other group of images mentioned above) can thus be decoded by means of this other artificial neural network (as described above in step E70).

[0181] The other artificial neural network which has just been mentioned may have a structure identical to the artificial neural network 18 mentioned above, which simplifies the configuration step of the parallel processing unit 16 (only the weights and / or the activation functions defining the current artificial neural network being for example updated).

Claims

Claims

1. Method for decoding a data stream by means of an electronic device (10) comprising a processor (14), and a parallel processing unit (16) designed to perform in parallel at a given instant a plurality of operations of the same type, characterized in that the data stream comprises a first set of data (Fet) and a second set of data (Fnn) representative of an audio or video content, and in that the method comprises the following steps: - processing (E56; E58; E60; E62; E66) of the data of the first set of data (Fet) by the processor (14); - obtaining the audio or video content by processing (E70) of the data of the second set of data (Fnn) according to a process dependent on at least part of the data of the first set (Fet) and using an artificial neural network (18) implemented by the parallel processing unit (16).

2. Decoding method according to claim 1, comprising a step of configuring (E68) the parallel processing unit (16) as a function of at least part of the data of the first data set (Fet).

3. Decoding method according to claim 2, wherein the first data set (Fet) comprises descriptive data (Rc) of the artificial neural network (18) and wherein, in the configuration step (E68), the processor (14) configures the parallel processing unit (16) on the basis of said descriptive data (Rc).

4. Decoding method according to claim 2, wherein the electronic device (10) comprises a storage unit (12) of a plurality of parameter sets respectively defining a plurality of artificial neural networks, wherein the first data set (Fet) comprises an identifier (Inn) and wherein, in the configuration step (E68), the processor (14) configures the parallelized processing unit (16) on the basis of a parameter set associated with this identifier (Inn) among the plurality of parameter sets.

5. Decoding method according to one of claims 1 to 4, wherein the data stream further comprises instructions (Exe) executable within the electronic device (10) and wherein the processing of the data of the first data set (Fet) is at least partly carried out due to the execution of at least part of said ins- pull-ups (Exe).

6. Decoding method according to claim 5 taken in direct or indirect dependence on claim 2, in which the configuration step (E68) of the parallel processing unit (16) is carried out due to the execution of at least part of said instructions (Exe).

7. Decoding method according to one of claims 1 to 6, comprising a step (E54) of identifying the first data set (Fet) and the second data set (Fnn) within the data stream.

8. Decoding method according to one of claims 1 to 7, in which the first data set (Fet) comprises data (Fc) representative of characteristics of the format of the content coded by the data stream.

9. Decoding method according to one of claims 1 to 8, in which the processing of the data of the second set (Fnn) produces at least one matrix representation (I) of at least part of an image.

10. Decoding method according to one of claims 1 to 9, in which the artificial neural network (18) receives as input data from the second data set (Fnn).

11. Decoding method according to one of claims 1 to 10, in which the artificial neural network (18) receives as input data previously produced as output from the artificial neural network (18).

12. Electronic device (10) for decoding a data stream comprising a first set of data (Fet) and a second set of data (Fnn) representative of audio or video content, the electronic device (10) comprising: - a processor (14) adapted to process the data of the first set of data (Fet); - a parallel processing unit (16) designed to perform in parallel at a given instant a plurality of operations of the same type and adapted to obtain the audio or video content by processing the data of the second set of data (Fnn) according to a process dependent on at least part of the data of the first set (Fet) and using an artificial neural network (18) implemented by the parallel processing unit (16).

13. Electronic device according to claim 12, wherein the processor (14) is adapted to configure the parallel processing unit (16) according to at least part of the data of the first data set (Fet).

14. Electronic device according to claim 13, comprising a storage unit (12) of a plurality of parameter sets respectively defining a plurality of artificial neural networks, in which the first data set (Fet) comprises an identifier (Inn) and in which the processor (14) is adapted to configure the parallel processing unit (16) on the basis of a parameter set associated with this identifier (Inn) among the plurality of parameter sets.

15. Electronic device according to one of claims 12 to 14, in which the parallel processing unit (16) is adapted to produce at least one matrix representation (I) of at least part of an image.

16. Data stream comprising a first data set (Fet) and a second data set (Fnn) representative of audio or video content, wherein the first data set (Fet) comprises data (Inn; Rc) defining at least in part a process for processing the data of the second data set (Fnn) using an artificial neural network.

Images