Decoding method and device, associated computer program and data stream

Abstract

A data stream contains data packets each comprising at least first data and second data. A method for decoding this data stream comprises the following steps: - identifying, among said data packets, a first data packet (12) the first data of which include information (T1) indicating a predetermined type of data packet; - processing the second data (NNC) of the first data packet to obtain an artificial neural network; - decoding the second data (C) contained in a second data packet (14) among said data packets, using at least the obtained artificial neural network, so as to produce data representative of audio or video content. An associated decoding device and computer program are also proposed.

Description

[0001] Decoding method and device, computer program and associated data streams

[0002] Technical field of the invention

[0003] The present invention relates to the technical field of coding audio or video content.

[0004] It relates in particular to a decoding method and device, as well as an associated computer program and data stream.

[0005] State of the art

[0006] It has been proposed to use artificial neural networks to perform all or part of the decoding of data representative of audio or video content.

[0007] Document WO 2022 / 013 249 discloses a decoding method in which an indicator is decoded to determine whether an artificial neural network is encoded in the received data stream or is part of a predetermined set of artificial neural networks, and in which the artificial neural network is then used to decode data representative of audio or video content.

[0008] Presentation of the invention

[0009] In this context, the present invention proposes a method for decoding a data stream comprising data packets each comprising at least first data and second data, characterized in that it comprises the following steps:

[0010] - identification, among said data packets, of a first data packet whose first data includes information indicative of a predetermined type of data packet;

[0011] - processing the second data of the first data packet to obtain an artificial neural network;

[0012] - decoding the second data included in a second data packet among said data packets, using at least the artificial neural network obtained and so as to produce data representative of audio or video content.

[0013] The data for obtaining the artificial neural network and the data decodable using this artificial neural network to reproduce the audio or video content are thus transported in respective data packets, which facilitates their identification and use at the time of decoding. The first data packet, which contains the data for obtaining the artificial neural network, is in this regard specifically identified by means of the information indicative of the predetermined type of data packet.

[0014] The second data of the first data packet comprises, for example, descriptive data of the artificial neural network; the processing step may then be a step of decoding the descriptive data to obtain parameters of the artificial neural network.

[0015] The first data packet may further include an identifier of the artificial neural network. This identifier may be an element of a list of distinct identifiers associated respectively with distinct artificial neural networks.

[0016] The method may further comprise a step of receiving a third data packet, the first data of which includes information indicative of said predetermined type of packet, and comprising said identifier, and / or a step of reusing said artificial neural network obtained to decode second data included in a fourth data packet among said data packets. Thus, the presence of the identifier in the third data packet indicates that this third data packet also contains data that can be used to obtain the artificial neural network defined in the first data packet, and this artificial neural network can therefore be reused without having to process the data of the third data packet again.

[0017] The second data packet may also include the identifier. In other words, the identifier included in the first data packet and the identifier included in the second data packet are identical. The identifier may in this case indicate that the artificial neural network defined in the first data packet (which contains the identifier) ​​is to be used to decode the data contained in the second data packet (which also contains the identifier in this case).

[0018] However, other possibilities are possible to indicate the neural network to use for decoding.

[0019] The method may comprise a step of receiving another data packet containing parameters relating to at least one image of said content; these parameters may in this case comprise said identifier. The artificial neural network defined in the first data packet will then be used to decode the data making it possible to obtain said at least one image of the content.

[0020] According to a possible embodiment, the first data packet is, among the data packets whose first data includes information indicative of said predetermined type of packet, the last packet preceding the second data packet in the data stream. In this embodiment, the artificial neural network to be used for decoding the data of the second packet is thus that defined in the last received packet having the predetermined type.

[0021] According to one embodiment, the second data packet comprises a pointer to the first data packet.

[0022] Along the same lines, the process may include the following steps:

[0023] - reading a flag in the second data packet;

[0024] - if the flag has a predefined value, reading, in the second data packet, a pointer to the first data packet.

[0025] The process may further include the following steps:

[0026] - receiving another data packet containing parameters relating to at least one image of said content

[0027] - reading a flag among said parameters;

[0028] - if the flag has a predefined value, reading, among said parameters, a pointer to the first data packet.

[0029] The pointer may designate, for example, a location in a portion of the data stream relating to a sequence of images distinct from the sequence of images encoded at least in part by the second data of the second data packet.

[0030] Furthermore, the first data packet may include information indicative of the encoding format of the second data of the first data packet.

[0031] The first data packet may start with a predefined marker and the second data packet may in this case also start with said predefined marker. Such a marker in this case identifies the start of the data packets.

[0032] The first data of a data packet are for example included in a header of this data packet, while the second data can then be included in the useful data (in English "payload") of this data packet. The invention also proposes a device for decoding a data stream comprising data packets each comprising at least first data and second data, characterized in that it comprises a processor configured or programmed to:

[0033] - identifying, among said data packets, a first data packet whose first data includes information indicative of a predetermined type of data packet;

[0034] - process the second data of the first data packet to obtain an artificial neural network;

[0035] - decoding the second data included in a second data packet among said data packets, using at least the obtained artificial neural network and so as to produce data representative of audio or video content.

[0036] The invention also provides a computer program comprising instructions executable by a processor and designed to implement a method as proposed above, when these instructions are executed by the processor.

[0037] The invention finally proposes a data stream comprising data packets each comprising at least first data and second data, characterized in that the data packets comprise:

[0038] - a first data packet, the first data of which includes information indicative of a predetermined type of data packet and the second data of which defines an artificial neural network;

[0039] - a second data packet whose second data is decodable using at least said artificial neural network so as to produce data representative of audio or video content.

[0040] As explained above, the first data packet may comprise an identifier; the data stream may comprise another data packet which comprises said identifier, the first data of which comprises said information indicative of the predetermined type of data packet, and the second data of which is identical to the second data of the first data packet.

[0041] Of course, the various features, variants and embodiments of the invention may be combined with each other in various combinations to the extent that they are not incompatible or mutually exclusive.

[0042] Detailed description of the invention

[0043] In addition, various other characteristics of the invention emerge from the appended description given with reference to the drawings which illustrate non-limiting forms of embodiment of the invention and where:

[0044] - Figure 1 represents the main elements of a coding device;

[0045] - figure 2 schematically represents a first possible embodiment for a coding module of the coding device of figure 1;

[0046] - figure 3 schematically represents a second possible embodiment for the coding module of the coding device of figure 1;

[0047] - Figure 4 represents a first example of data flow;

[0048] - Figure 5 represents a second example of data flow;

[0049] - Figure 6 represents a third example of data flow;

[0050] - Figure 7 represents a fourth example of data flow;

[0051] - Figure 8 represents a fifth example of data flow;

[0052] - Figure 9 represents a sixth example of data flow;

[0053] - Figure 10 represents the main elements of a decoding device;

[0054] - figure 11 schematically represents a first possible embodiment for a decoding module of the decoding device of figure 10;

[0055] - figure 12 schematically represents a second possible embodiment for the decoding module of the decoding device of figure 10;

[0056] - Figure 13 is a flowchart showing steps of a decoding method;

[0057] - Figure 14 is a flowchart showing steps of a possible method for decoding the data stream of Figure 5;

[0058] - Figure 15 is a flowchart showing steps of a possible method for decoding the data stream of Figure 6;

[0059] - Figure 16 is a flowchart showing steps of a possible method for decoding the data stream of Figure 7; and

[0060] - Figure 17 is a flowchart showing steps of a possible method for decoding the data stream of Figure 8. It should be noted that, in these figures, the structural and / or functional elements common to the different variants may have the same references.

[0061] Figure 1 represents a coding device used in the context of the invention.

[0062] This coding device comprises a management module 2, a coding module 4, a stream formation module 6 and a stream transmission module 8.

[0063] Each of these modules can be implemented in practice by a processor programmed (for example by means of instructions stored in a memory associated with the processor) to implement the functionalities described below for the module concerned (in this example, due to the execution by the processor of some of the aforementioned instructions). Furthermore, several modules can in practice be implemented by means of the same processor, for example by means of the execution (by this processor) of several sets of instructions corresponding respectively to the different modules. Alternatively, one or other of the modules can be implemented by means of an application-specific integrated circuit.

[0064] The management module 2 is configured to control the operation of the coding module 4, in particular to determine which coding process must be used to code data B representative of audio or video content, as will be explained below.

[0065] The coding module 4 is configured to receive as input these data B representative of an audio or video content and to generate as output, on the basis of at least a part of the data B, a coded representation C of this content. The size of the coded representation C (in number of bits) is normally less than the size of the corresponding data B (in number of bits).

[0066] In the case of video content, the B data comprise, for example, values ​​respectively associated with pixels of an image (or a component of an image) of the video sequence. The B data can thus be luminance values ​​or chrominance values ​​respectively associated with pixels of a component of an image of the video sequence concerned.

[0067] In the case of audio content, B data is data representative of a sound signal, for example in the WAV format (used for storing audio compact discs).

[0068] In order to produce the coded representation C on the basis of the data B, the coding module 4 uses at least one artificial neural network N, N'. According to a first possible embodiment illustrated in Figure 2, the data B representative of the audio or video content are applied as input to the artificial neural network N, which then generates as output a corresponding part of the coded representation C.

[0069] The data B applied as input to the artificial neural network N (i.e. applied to an input layer of the artificial neural network N) may represent a block of an image, or a block of a component of an image (for example a block of a luminance or chrominance component of this image, or a block of a color component of this image), or an image of a video sequence, or a component of an image of a video sequence (for example a luminance or chrominance component, or a color component), or a series of images of the video sequence.

[0070] For example, in this case, we can predict that at least some of the neurons in the input layer each receive a pixel value from a component of an image, a value represented by one of the data B.

[0071] According to a second possible embodiment, the coding module 4 applies to the data B representative of the audio or video content a processing comprising several steps, at least one step of which is carried out by means of an artificial neural network N'.

[0072] Thus, as illustrated for example in figure 3, a part Cj-i previously obtained from the coded representation is applied as input to the artificial neural network N', which makes it possible to generate at the output of the artificial neural network N' predicted data Pj which are subtracted from the current data Bj so as to obtain (at the output of the coding module 4) a part Cj of the coded representation corresponding to the current data Bj.

[0073] In Figure 3, reference 10 represents a delay module to illustrate the fact that at the instant of processing the current data Bj to obtain the part Cj of the corresponding coded representation, it is a previously obtained part Cj-i of the coded representation which is applied as input to the artificial neural network N'.

[0074] In practice, the part Cj-i is for example previously obtained by processing (by the coding module 4) data Bj-i relating to an image preceding the image represented by the current data Bj. As a variant, the previously obtained part used as input to the artificial neural network N' can be a coded representation part corresponding to at least one block of the image neighboring the block whose pixel values ​​are represented by the current data Bj.

[0075] During encoding, the management module 2 determines which encoding process (i.e., which processing performed by the encoding module 4) is to be used for encoding a set of data B representative of the audio or video content.

[0076] The management module 2 thus determines in particular which artificial neural network N, N' must be used within the coding module 4.

[0077] The data set B for which the management module 2 determines the process (and in particular the artificial neural network N, N') to be used depends on the application concerned. This data set B is, for example, the set of data B relating to a given image or the set of data B relating to a given sequence of images.

[0078] The management module 2 selects for example the artificial neural network N, N' to be used when encoding the data set B from among a plurality of predefined artificial neural networks, for example in order to minimize a rate-distortion criterion (which takes into account the size of the coded representation C and the distortion between the content represented by the data B and the content reconstructed on the basis of the coded representation C).

[0079] Alternatively, the management module 2 carries out a step of training the artificial neural network N, N' so as to optimize a given criterion (for example the aforementioned rate distortion criterion) during the processing of the data set B concerned, and commands the coding module 4 to use the artificial neural network thus trained for the generation of the coded representation C on the basis of the data set B concerned.

[0080] The management module 2 can thus produce (in particular for the flow formation module 6) information i indicative of the artificial neural network to be used when decoding the coded representation C. Specifically, the management module 2 can provide such information i for each part C of the coded representation associated with a data set B as defined above.

[0081] In some cases, the artificial neural network to be used for decoding the coded representation C is distinct from the artificial neural network N. For example, in the case of Figure 2 (where the artificial neural network N receives the data B as input and produces the coded representation C as output), the artificial neural network to be used for decoding the coded representation C is designed (i.e., in practice trained) so as to minimize the distortion of the data B during their successive passages through the artificial neural network N (to produce the coded representation C) and through the artificial neural network to be used for decoding, and / or to minimize the size of the coded representation C (in the sense of a rate-distortion criterion).

[0082] In the case where the management module 2 selects the artificial neural network N from a plurality of predefined artificial neural networks, the artificial neural network to be used for decoding is the one associated in a predefined manner with the selected artificial neural network N. The information i can then designate (for example within a list of artificial neural networks) this network associated with the selected artificial neural network N.

[0083] In the case where the management module 2 obtains the artificial neural network N by means of a training step, this training step can allow the simultaneous training of the artificial neural network to be used for decoding. The information i can then comprise descriptive data of the artificial neural network to be used for decoding (this descriptive data can comprise, for example, weights respectively associated with the neurons of this artificial neural network and determined during the training step).

[0084] The stream formation module 6 receives the coded representation C produced by the coding module 4 and the information i provided by the management module 2, and constructs on the basis of these elements a data stream F. The stream formation module 6 can of course receive in practice other data from the coding module 4 and / or from the management module 6.

[0085] The stream formation module 6 constructs the data stream F in the form of different data packets intended to be successively communicated (for example transmitted) to the decoding device. These data packets are for example respectively units of the network abstraction layer (in English "NAL units" for "Network Abstraction Layer units").

[0086] The flow formation module 6 constructs the various data packets in accordance with what is described below. In the example described here, each data packet begins with a predefined marker M (i.e. formed from a predefined sequence, or predefined pattern, of bits). Every data packet therefore begins here with the same marker M, which makes it possible to identify the start of a data packet upon reception of the flow. It is proposed, for example, that the value corresponding to the marker M (i.e. the sequence of bits forming the marker M) be prohibited within the data flow F outside the start of the data packets.

[0087] Alternatively, one could consider other means of identifying data packets in the data stream F, for example a list listing the addresses of the different data packets in the data stream F.

[0088] Each data packet herein further comprises a type identifier which designates the type of the data packet concerned from a predetermined set of possible types.

[0089] In the examples described below, at least some of the following data packet types are used:

[0090] - a data packet carrying descriptive data of an artificial neural network (to be used for decoding), designated by a first type identifier, noted T1 in the following;

[0091] - a data packet carrying coded data representing audio or video content (i.e. here the coded representation C obtained by means of the coding module 4), designated by a second type identifier, noted T2 in the following;

[0092] - a data packet carrying parameters relating to the audio or video content, or to the decoding process to be used, designated by a third identifier, noted T3 in the following;

[0093] - a data packet carrying coded data representing audio or video content, this coded data being however obtained by a separate coding process (and consequently requiring a separate decoding process) from the aforementioned coded data contained in the T2 type data packets, this latter type of data packet being designated by a fourth identifier noted T4 in the following.

[0094] There are therefore data packets containing a coded representation of the content (data packets of types T2 and T4 in the example described here), data packets containing descriptive data of an artificial neural network (for example data coded according to a given format and representing this artificial neural network) - here data packets of type T1, and data packets containing parameters (here data packets of type T3).

[0095] In the embodiment described herein, data packets containing descriptive data of an artificial neural network (T1 type data packets) comprise:

[0096] - an M marker;

[0097] - a type identifier (forming the first data for this data packet), here with value T1;

[0098] - optionally, an NNI identifier associated with the artificial neural network concerned (the NNI identifier being part of a predetermined set of identifiers respectively associated with different artificial neural networks);

[0099] - optionally, an NNF format identifier indicating the format of the NNC descriptive data contained in the data packet;

[0100] - the descriptive NNC data of the artificial neural network concerned (these descriptive NNC data forming second data for this data packet).

[0101] The T1 type identifier and possibly the NNI identifier and / or the NNF format identifier are for example included in a header of the data packet; the NNC descriptive data can then form the useful data (in English "payload") of the data packet.

[0102] The description (or coding) format of the NNC descriptive data (identified where appropriate by the NNF format identifier) ​​may be, for example, the NNR format (MPEG-7 part 17), the NNEF format or the ONNX format. The NNF format identifier may alternatively designate a format accepted by a tool for manipulating artificial neural networks, or a format of an artificial neural network identifier from a predetermined set of artificial neural networks (the NNC data then comprising such an identifier).

[0103] The use of an NNF format identifier in the data packet is not necessary when the format used is agreed (predefined) by the coding device and the decoding device (or, in other words, only one format is used by the decoding device). When a part C of the coded representation has been produced by the coding module 4 on the basis of a data set B, the stream formation module 6 receives as already indicated (from the management module 2) the information i indicative of the decoding artificial neural network to be used for the decoding of the part C. The stream formation module 6 can thus determine on the basis of this information i which NNC descriptive data are to be placed in a given T1 type data packet, as will become apparent from the examples given below.

[0104] In embodiments where an NNI identifier is used, it is provided here that all data packets containing descriptive data of an artificial neural network (i.e. here all data packets of type T1) comprising a given NNI identifier comprise identical NNC descriptive data.

[0105] In the example described here, the data packets containing an encoded representation of the content (data packets of types T2 and T4) include:

[0106] - an M marker;

[0107] - a type identifier (forming the first data for this data packet), here of value T2 or T4 (the type being one of the types available for the coded representation of the content);

[0108] - optionally, an NNI identifier associated with an artificial neural network (according to the same association rule as mentioned above for T1 type data packets), this artificial neural network being the one to be used for decoding the coded representation of the content contained in this data packet;

[0109] - optionally, an NNL location identifier such as a file pointer that indicates the location in the data stream of a data packet containing descriptive data of the artificial neural network to be used for decoding the encoded representation of the content contained in this data packet;

[0110] - optionally, a DNN remote description indicator indicating whether a data packet containing descriptive data of the artificial neural network to be used is present in the part of the data stream relating to the current image sequence or in a part of the data stream relating to an image sequence different from the current image sequence; - the coded representation of the content C concerned (which is a part of the coded representation generated by the coding module 4 and forms the second data for this data packet).

[0111] An image sequence is here a set of images that can be obtained by decoding a part of the coded representation of the content (here of the video) without requiring access to another part of the coded representation of the content (here of the video).

[0112] The type identifier T2, T4 and possibly the identifier NNI and / or the location identifier NNL and / or the remote description indicator are for example included in a header of the data packet; the coded representation C can then form the useful data (in English "payload") of the data packet.

[0113] Among the data packets containing a coded representation of the content, some packets may have a particular type to identify an entry point in the data stream (here the type corresponding to the identifier T4 in the example described below). In this case, it can be provided for example that only packets of this type (corresponding to an entry point) can contain an identifier NNI or a location identifier NNL.

[0114] According to a possible variant, the NNI identifier and / or the NNL location identifier and / or the DNN remote description indicator could be contained in a data packet carrying parameters relating to an image or a sequence of images (T3 type data packet in the example described here).

[0115] It is also possible to provide that the flow formation module 6 constructs the data flow F so that any data packet containing a coded representation of the content (data packet of type T2 or T4) and a given NNI identifier is preceded (in the data flow F) by a data packet of type T1 also containing this given NNI identifier (and thus descriptive data of the artificial neural network designated by this given NNI identifier).

[0116] Various examples of possible data streams are now described with reference to Figures 4 to 9. The decoding of these possible data streams is described later.

[0117] A first example of a data flow is shown in Figure 4. In this example, the data flow comprises a data packet 12 of type T1 and a data packet 14 (later in the data flow than packet 12) of type T2.

[0118] The data packet 12 comprises the marker M, a type identifier having the value T1, an NNI identifier associated with a given artificial neural network, an NNF format identifier indicating the format of the NNC descriptive data (mentioned below) and these NNC data descriptive of the given artificial neural network.

[0119] The data packet 14 comprises the marker M, a type identifier having the value T2, the identifier NNI associated with the given artificial neural network (identical to the identifier contained in the data packet 12) and a part of the coded representation C generated by the coding module 4 and decodable using in particular the given artificial neural network.

[0120] A second example of a data flow is shown in Figure 5.

[0121] In this example, the data stream includes in this order data packet 16, data packet 18, and data packet 20. (Other data packets may be present in the data stream between data packet 16 and data packet 18, and / or between data packet 18 and data packet 20.)

[0122] The data packet 16 includes the marker M, a type identifier having the value T1, an NNI identifier associated with a given artificial neural network, and NNC data descriptive of the given artificial neural network.

[0123] The data packet 18 comprises the marker M, a type identifier having the value T3 (corresponding as indicated above to a data packet containing parameters, relating to a given image or to a given sequence of images) and (among these parameters) an identifier NNI associated with the given artificial neural network.

[0124] The data packet 20 comprises the marker M, a type identifier having the value T2 and a part of the coded representation C generated by the coding module 4 and decodable using in particular the given artificial neural network.

[0125] A third example of a data flow is shown in Figure 6.

[0126] In this example, the data stream includes a data packet 22 and, later in the data stream, a data packet 24. The data packet 22 includes the marker M, a type identifier having the value T1, possibly an NNI identifier associated with a given artificial neural network, and NNC data descriptive of the given artificial neural network.

[0127] The data packet 24 comprises the marker M, a type identifier having the value T2 and a part of the coded representation C generated by the coding module 4 and decodable using in particular the given artificial neural network.

[0128] A fourth example of a data flow is shown in Figure 7.

[0129] In this example, the data stream includes in this order data packet 26, data packet 28, and data packet 30. (Other data packets may be present in the data stream between data packet 26 and data packet 28, and / or between data packet 28 and data packet 30.)

[0130] The data packet 26 includes the marker M, a type identifier having the value T1, an NNI identifier associated with a given artificial neural network and NNC data descriptive of the given artificial neural network.

[0131] The data packet 28 comprises the marker M, a type identifier having the value T4 (corresponding as already indicated to an entry point in the flow), the identifier NNI associated with the given artificial neural network (identical to the identifier contained in the data packet 26) and a part C of the coded representation generated by the coding module 4 and decodable using in particular the given artificial neural network.

[0132] The data packet 30 comprises the marker M, a type identifier having the value T2 and another part C' of the coded representation generated by the coding module 4 and decodable using in particular the given artificial neural network.

[0133] A fifth example of a data flow is shown in Figure 8.

[0134] In this example, the data stream includes, in this order, data packet 32, data packet 34, data packet 36, and data packet 38. (Other data packets may be present in the data stream between these different packets.)

[0135] The data packet 32 ​​comprises the marker M, a type identifier having the value T4 (corresponding as already indicated to an entry point in the stream), a remote description indicator DNN, a location identifier NNL and a part C of the coded representation generated by the coding module 4.

[0136] The data packet 34 comprises the marker M, a type identifier having the value T2 and another part C' of the coded representation generated by the coding module 4.

[0137] The data packets 32 and 34 relate to the same sequence of images S, that is to say that the parts C, C' of the coded representation are part of a set of coded data allowing the decoding of a set of images without having recourse to coded data situated outside this set of coded data.

[0138] The data packet 36 includes the marker M, a type identifier having the value T1 and NNC data descriptive of an artificial neural network.

[0139] The data packet 38 comprises the marker M, a type identifier having the value T2 and a part C” of the coded representation generated by the coding module 4.

[0140] Data packets 36 and 38 relate to the same image sequence S' which is distinct from the image sequence S.

[0141] The remote description indicator DNN contained in the data packet 32 ​​indicates that the artificial neural network to be used for the decoding of the part C (and of the part C') of the coded representation is not described by descriptive data contained in the image sequence S, but outside this image sequence S (here in the image sequence S').

[0142] The data packet 32 ​​thus includes the location identifier NNL already mentioned, which is here a pointer to the data packet 36 (located in the image sequence S').

[0143] Such a pointer can be for example:

[0144] - a difference in number of bytes compared to the location of the data packet 32 ​​(this difference can be signed, i.e. have a positive value to indicate a number of bytes in one direction - within the data flow - or a negative value to indicate a number of bytes in the other direction);

[0145] - a difference in the number of bytes from the start of the file (or, in other words, of the data stream);

[0146] - a physical memory storage address (which would have been rewritten by the decoding device when processing the data stream during storage of the artificial neural network, and then in all NNL references to this artificial neural network during preprocessing).

[0147] A sixth example of a data flow is shown in Figure 9.

[0148] In this example, the data stream includes, in this order, data packet 40, data packet 42, data packet 44, and data packet 46. (Other data packets may be present in the data stream between these different packets.)

[0149] The data packet 40 includes the marker M, a type identifier having the value T1, an NNI identifier associated with a given artificial neural network and NNC data descriptive of the given artificial neural network.

[0150] The data packet 42 comprises the marker M, a type identifier having the value T2, the identifier NNI (identical to that contained in the data packet 40) and a part C of the coded representation generated by the coding module 4 and decodable using in particular the given artificial neural network.

[0151] The data packet 44 comprises the marker M, the type identifier having the value T1, the NNI identifier associated with the given artificial neural network and the NNC data descriptive of the given artificial neural network (this NNC data being identical to the NNC data contained in the data packet 40).

[0152] The data packet 46 comprises the marker M, the type identifier having the value T2, the identifier NNI (identical to that contained in the data packets 40, 42 and 44) ​​and another part C' of the coded representation generated by the coding module 4 and decodable using in particular the given artificial neural network.

[0153] The use of another data packet 44 of type T1 and containing the descriptive NNC data of the artificial neural network identified by the identifier NNI allows the decoding device to possibly read the data stream in an order different from that shown in FIG. 9, for example to start reading the data stream at a location different from the data packet 40 (random access). Other data packets identical to the data packets 40, 44 may thus, for example, be present at regular intervals in the data stream.

[0154] In the example described here, the data stream F constructed by the data formation module 6 is transmitted over a communication channel (possibly after other processing steps, for example an entropy coding step) by the stream transmission module 8. Alternatively, the data stream F could be stored (for example on a recording device, such as a hard disk, of the coding device) for subsequent reading and decoding (the coding device and the decoding device described below being for example in this case the same electronic device).

[0155] Figure 10 represents a decoding device according to the invention.

[0156] This decoding device comprises a stream receiving module 50, a stream analyzing module 52, a decoding module 54 and a configuration module 56.

[0157] Each of these modules can be implemented in practice by a processor programmed (for example by means of instructions stored in a memory associated with the processor) to implement the functionalities described below for the module concerned (in this example, due to the execution by the processor of some of the aforementioned instructions). Furthermore, several modules can in practice be implemented by means of the same processor, for example by means of the execution (by this processor) of several sets of instructions corresponding respectively to the different modules. Alternatively, one or other of the modules can be implemented by means of an application-specific integrated circuit.

[0158] The stream receiving module 50 receives (for example via a communication channel) a data stream such as the data stream F constructed by the stream forming module 6 and transmitted by the transmission module 8.

[0159] According to a variant already mentioned, this data stream is read from a recording medium such as a hard disk.

[0160] The data stream F (received by the stream reception module 50 or read from a recording medium) is analyzed by the stream analysis module 52 as described later, which makes it possible to identify, on the one hand, data C, C', C” forming part of the coded representation of the content and, on the other hand, an artificial neural network to be used for decoding these data C, C', C”.

[0161] The configuration module 56 is then designed to configure the decoding module 54 so that the decoding module decodes the data C, C', C” using the identified artificial neural network, in order to produce data B' representative of the audio or video content, as will be explained below in the context of different examples. According to a first possible embodiment of the decoding module 54 illustrated in FIG. 11, the data C, C', C” (coded representation of the content) are applied as input to the identified artificial neural network N”, which then generates as output the data B' representative of the audio or video content.

[0162] The data B' produced at the output of the artificial neural network N” correspond to the data B applied as input to the artificial neural network N and can thus represent a block of an image, or a block of a component of an image (for example a block of a luminance or chrominance component of this image, or a block of a color component of this image), or an image of a video sequence, or a component of an image of a video sequence (for example a luminance or chrominance component, or a color component), or even a series of images of the video sequence.

[0163] In this case, at least some of the neurons in the output layer of the artificial neural network N” each produce a pixel value of a component of an image, a value forming one of the data B'.

[0164] According to a second possible embodiment, the decoding module 54 applies to the data C, C', C” (denoted Cj in figure 12) a processing comprising several steps, at least one step of which is carried out by means of an artificial neural network N'.

[0165] Thus, as illustrated for example in figure 12, a part of the coded representation Cj-i previously received or read in the data stream is applied as input to the artificial neural network N', which makes it possible to generate at the output of the artificial neural network N' predicted data Pj which are combined (for example by addition) with the current part Cj of the coded representation so as to obtain (at the output of the decoding module 4) a part B'j of the data representative of the audio or video content.

[0166] We note that the artificial neural network N' used for decoding (as represented in figure 12) is here identical to the artificial neural network N' used for coding (see figure 3 described above).

[0167] In Figure 12, reference 60 represents a delay module to illustrate the fact that at the instant of processing the current part Cj of the coded representation to obtain the corresponding part B'j of the representative data, it is a previously received (or read) part Cj-i of the coded representation which is applied as input to the artificial neural network N'.

[0168] In practice, as already indicated for the coding, the part Cj-i is for example relative to a part B'j-i representative of an image preceding the image represented by the part B'j.

[0169] Alternatively, the previously received or read part, used as input to the artificial neural network N', may be a coded representation part corresponding to at least one block of the image neighboring the block whose pixel values ​​are represented by the data B'j.

[0170] Figure 13 shows steps of an example method for decoding the data stream F.

[0171] This decoding method can be used in particular for the examples of data flows described above and represented in Figures 4 and 9. The numerical references mentioned in Figures 4 and 9 will therefore be used to illustrate the description of this decoding method.

[0172] This method begins with a step E2 in which the analysis module 52 identifies (in the data flow F) the start of a data packet 12, 14, 40, 42, 44, 46, here thanks to the marker M with which any data packet begins.

[0173] Once the start of a data packet has been identified, the analysis module 54 can identify its type by reading (and possibly decoding) the type identifier T (or first data) of this data packet (step E4), for example within the header of this data packet.

[0174] The analysis module 54 then determines in step E6 whether the type indicated by the type identifier T is a predetermined type (corresponding here to the type T 1 ). As already indicated, this predetermined type (designated here T1 ) is associated with the data packets which contain data indicative of an artificial neural network.

[0175] In the event of a positive determination (arrow P) in step E6, the method continues to step E8. (This is the case in particular when processing data packets 12, 40, 44.)

[0176] In the event of a negative determination (arrow N) in step E6, the method continues in step E16. (This is the case in particular when processing data packets 14, 42, 46.) In step E8, the analysis module 52 reads in the data stream F (and possibly decodes) an identifier NNI which designates a particular artificial neural network (the identifier NNI being part of a predetermined set of identifiers respectively associated with different artificial neural networks).

[0177] The analysis module 52 then determines in step E10 (possibly by cooperation with the configuration module 56) whether the artificial neural network designated by the identifier NNI is stored within the decoding device, for example following the prior reception of a data packet which already contained data indicative of the artificial neural network (such as the data packet 40).

[0178] In the event of a positive determination (arrow P) at step E10 (as is the case during the processing of data packet 44 if data packet 40 has been previously processed), the artificial neural network previously received and stored can be reused (during a subsequent passage to step E22 described below) and it is therefore not necessary to continue processing the current data packet: the method then loops to step E2.

[0179] On the other hand, in the event of a negative determination (arrow N) in step E10 (as is the case during the processing of the data packet 12 or 40), the method continues in step E12 for reading in the data stream F and decoding of NNC data (second data of the current data packet) indicative of the artificial neural network associated with the identifier NNI, possibly taking into account the coding format of this NNC data indicated where appropriate by the format identifier NNF (case of the data packet 12), in order to obtain (for example to construct) the artificial neural network.

[0180] As already indicated, in the examples of figures 4 and 9 in particular, the indicative data NNC are descriptive data of the artificial neural network, which can be decoded (by the flow analysis module 52 or the configuration module 56) so as to obtain parameters of the artificial neural network, these parameters allowing the configuration module 56 to configure the decoding module 54 in such a way that this decoding module 54 implements in particular the artificial neural network (designated by the identifier NNI).

[0181] The parameters of the artificial neural network obtained by decoding the descriptive data NNC are then stored in a memory of the decoding device (for example a memory associated with the configuration module 56) in step E14 and the method loops in step E2 for processing a new data packet of the data stream F.

[0182] When it is determined in step E6 that the type of the current data packet does not correspond to the predetermined type T1, the method continues as already indicated in step E16 described now.

[0183] In step E 16, the flow analysis module 52 determines whether the type T designated by the type identifier (or first data) of the current data packet 12, 14, 40, 42, 44, 46 is part of the types associated with the data packets containing a coded representation of the content (which here correspond to the types T2 and T4).

[0184] In the event of a negative determination (arrow N) in step E16, the method continues in step E18 for decoding the data packet. This is for example the case where the data packet contains parameters relating to an image or to a sequence of images (data packet of type T3 in the example described here) and the decoding of the current data packet makes it possible in this case to obtain parameters relating to an image to be decoded or to the current image sequence (being decoded).

[0185] The method then loops to step E2 to process another data packet.

[0186] In the event of a positive determination (arrow P) in step E16, the method continues with a step E20 of reading (and possible decoding) an NNI identifier in the data stream F. This NNI identifier designates the artificial neural network to be used for decoding the coded representation C, C' contained in the current packet 14, 42, 46.

[0187] The parameters defining this artificial neural network have been previously obtained by means of the indicative data of this artificial neural network contained in a previously received data packet of type T1. In the example described here, these parameters have been previously decoded on the basis of the descriptive data NNC contained in a previously received data packet of type T1 (the data packet 12 in the case of FIG. 4 and, in the case of FIG. 9, the data packet 40, or, if the data packet 40 has not been read by the decoding device, the data packet 44). The parameters thus obtained (for example decoded) have also been stored as already explained in a memory of the decoding device (here a memory associated with the configuration device 56).

[0188] The configuration module 56 can then configure the decoding module 54 using the parameters of the artificial neural network designated by the identifier NNI read in the data stream in step E20 so that the decoding module 54 can perform a decoding of a coded representation using this artificial neural network.

[0189] The flow analysis module 52 then extracts the coded representation C, C' (second data) from the current packet and transmits this coded representation C, C' to the decoding module 54 for decoding of this coded representation C, C' (step E22) using the aforementioned artificial neural network (designated by the identifier NNI read in step E20) in order to obtain at the output of the decoding module 54 data B' representative of the audio or video content (this data being for example pixel values ​​of at least part of an image or of a component of an image).

[0190] Figure 14 shows steps of a possible method for decoding the data stream of Figure 5.

[0191] This method comprises a step E30 of identification and analysis of the data packet 16 by the flow analysis module 52.

[0192] This step here comprises the identification of the start of the data packet 16 by means of the marker M, the detection of the type identifier (first data) corresponding to the predetermined type T1 and the reading (in the data stream) of an identifier NNI associated with an artificial neural network and of data NNC (second data) descriptive of the artificial neural network corresponding to the identifier NNI.

[0193] The method can then comprise a step E32 of decoding the NNC data in order to obtain parameters of the artificial neural network, and of storing the parameters obtained in a memory of the decoding device (for example a memory associated with the configuration module 56).

[0194] Subsequently, during step E34, the flow analysis module 52 identifies and analyzes the data packet 18. Step E34 here comprises the identification of the start of the data packet 18 by means of the marker M, the detection of the type identifier which here indicates the type T3 corresponding to the data packets containing parameters relating to at least one image of the current image sequence, and the reading (in the data flow) of the parameters contained in the data packet 18, these parameters here comprising the identifier NNI.

[0195] According to one possible embodiment, the parameters contained in the data packet 18 of type T3 may relate only to the image currently being decoded (i.e. the image for which representative data will be obtained by the next decoding operation using the decoding module 54).

[0196] The presence of the identifier NNI in the data packet 18 indicates in this case that the artificial neural network associated with the identifier NNI will be used for the decoding of representative data C associated with the image being decoded (in order to obtain data B' relating to at least part of the image being decoded).

[0197] According to another possible embodiment, the parameters contained in the data packet 18 of type T3 can concern all the images of the sequence of images currently being decoded.

[0198] The presence of the identifier NNI in the data packet 18 indicates in this case that the artificial neural network associated with the identifier NNI will be used for the decoding of representative data C associated with the different images of the current image sequence (in order to obtain data B' relating to at least part of one of the images of the current image sequence).

[0199] During step E34, according to one possible embodiment, the configuration module 54 can then configure the decoding module 56 so that the decoding module 56 can decode representative data received in the data stream using the artificial neural network designated by the identifier NNI. This configuration can in practice be carried out by reading the parameters of the artificial neural network from the aforementioned memory of the coding device (see step E32 below).

[0200] Subsequently, during step E36, the flow analysis module 52 identifies and analyzes the data packet 20. It is considered here that this data packet 20 relates to an image to which the parameters contained in the aforementioned data packet 18 apply.

[0201] Step E36 here comprises the identification of the start of the data packet 20 by means of the marker M, the detection of the type identifier which here indicates the type T2 corresponding to the data packets containing a coded representation of the video, and the reading of a part C of the coded representation (second data of the data packet 20).

[0202] The method then comprises a step E38 of decoding the part C of the representation coded by the decoding module 54 using the artificial neural network designated by the identifier NNI.

[0203] Figure 15 shows steps of a possible method for decoding the data stream of Figure 6.

[0204] This method comprises a step E40 of identification and analysis of the data packet 22 by the flow analysis module 52.

[0205] This step here includes the identification of the start of the data packet 22 by means of the marker M, the detection of the type identifier (first data) corresponding to the predetermined type T1 and the reading of NNC data (second data) descriptive of an artificial neural network.

[0206] Step E40 may also comprise the reading and / or decoding of an NNI identifier associated with this artificial neural network. As explained in other embodiments, this makes it possible, in the event of subsequent detection of a data packet of type T1 and comprising the same NNI identifier, not to have to decode again the descriptive NNC data of the artificial neural network.

[0207] The method then comprises a step E42 of decoding the NNC data in order to obtain parameters of the artificial neural network, and of storing the parameters obtained in a memory of the decoding device (for example a memory associated with the configuration module 56). As indicated above, in the embodiments where an NNI identifier is used within the data packet 22 and a previous data packet of type T1 and carrying this NNI identifier has already been processed, this step can be omitted.

[0208] In step E42, the configuration module 54 can configure the decoding module 56 (by means of the parameters obtained as just mentioned) so that the decoding module 54 can decode the following representative data received in the data stream using the artificial neural network.

[0209] The method then comprises a step E44 of identification and analysis (by the flow analysis module 52) of the data packet 24. It is considered here that no data packet of type T1 is included between the data packet 22 and the data packet 24.

[0210] Step E44 here comprises the identification of the start of the data packet 24 by means of the marker M, the detection of the type identifier which here indicates the type T2 corresponding to the data packets containing a coded representation of the video, and the reading (in the data stream) of a part C of the coded representation (second data of the data packet 24).

[0211] The method then comprises a step E46 of decoding the part C of the representation coded by the decoding module 54 using the artificial neural network represented by the NNC data contained in the data packet 22.

[0212] Thus, in the present embodiment, the artificial neural network used for decoding the coded representation contained in a data packet 24 is defined (by indicative data, here NNC descriptive data, contained) in the last data packet of type T1 preceding this data packet 24.

[0213] Figure 16 shows steps of a possible method for decoding the data stream of Figure 7.

[0214] This method comprises a step E50 of identification and analysis of the data packet 26 by the flow analysis module 52.

[0215] This step here comprises the identification of the start of the data packet 26 by means of the marker M, the detection of the type identifier (first data) corresponding to the predetermined type T1 and the reading in the stream of an identifier NNI associated with an artificial neural network and of data NNC (second data) descriptive of the artificial neural network corresponding to the identifier NNI.

[0216] The method then comprises a step E52 of decoding the NNC data in order to obtain parameters of the artificial neural network, and of storing the parameters obtained in a memory of the decoding device (for example a memory associated with the configuration module 56). T1

[0217] Subsequently, during step E54, the flow analysis module 52 identifies and analyzes the data packet 28.

[0218] Step E54 here comprises the identification of the start of the data packet 28 by means of the marker M, the detection of the type identifier which here indicates the type T4 corresponding to the data packets containing a coded representation of the content and identifying an entry point in the data stream, and the reading (in the data stream) of the identifier NNI and a first part C of the coded representation of the content.

[0219] The method can then continue with a step E56 of decoding this first part C of the coded representation of the content, by the decoding module 54 and by means of the artificial neural network associated with this NNI identifier (as contained in the data packet 28). To do this, the step E56 can possibly comprise in practice a step of configuring the decoding module 54 by the configuration module 56 and by means of the parameters obtained (and stored) in the step E52.

[0220] Subsequently, during step E58, the flow analysis module 52 identifies and analyzes the data packet 30.

[0221] Step E58 here comprises the identification of the start of the data packet 30 by means of the marker M, the detection of the type identifier which here indicates the type T2 corresponding to data packets containing a coded representation of the content, and the reading (in the data stream) of a second part C' of the coded representation of the content.

[0222] Indeed, as already indicated, it is provided in this embodiment that only the data packets corresponding to a possible entry point in the data flow contain an identifier (here NNI) of the neural network to be used for decoding the coded representations of the content.

[0223] The method can then continue with a step E60 of decoding this second part C' of the coded representation of the content, by the decoding module 54 and by means of the artificial neural network associated with the identifier NNI contained in the data packet 28 designated as a possible entry point by the type identifier T4 contained in this data packet 28.

[0224] Figure 17 shows steps of a possible method for decoding the data stream of Figure 8. This method comprises a step E70 of identification and analysis of the data packet 32 ​​by the stream analysis module 52.

[0225] Step E70 here comprises the identification of the start of the data packet 32 ​​by means of the marker M, the detection of the type identifier which here indicates the type T4 corresponding to the data packets containing a coded representation of the content and identifying an entry point in the data stream, and the reading (in the data stream) of the remote description indicator DNN, the location identifier NNL and a first part C of the coded representation of the content.

[0226] Here, it is considered that the remote description indicator DNN has a value (for example the value 1) indicating that the artificial neural network to be used to decode the first part C is described outside the current sequence S; therefore, the flow analysis module 52 reads the location identifier NNL located after (here immediately after) the remote description indicator DNN in the data flow.

[0227] The flow analysis module 52 then scans the data flow according to the indications given by the location identifier NNL (for example by scanning the byte difference indicated by the location identifier NNL, or by jumping to the physical memory storage address indicated by the location identifier NNL) until reading and analyzing the data packet 36 (step E72).

[0228] This step E72 here comprises the identification of the start of the data packet 36 by means of the marker M, the detection of the type identifier (first data) corresponding to the predetermined type T1 and the reading in the data stream of NNC data (second data) descriptive of an artificial neural network.

[0229] The method then comprises a step E74 of decoding the NNC data in order to obtain parameters of the artificial neural network, and of storing the parameters obtained in a memory of the decoding device (for example a memory associated with the configuration module 56).

[0230] The method can then continue with a step E76 of decoding the first part C of the coded representation of the content, by the decoding module 54 and by means of the aforementioned artificial neural network (decoded from the descriptive data NNC contained in the data packet 36). To do this, the step E76 can possibly comprise in practice a step of configuring the decoding module 54 by the configuration module 56 and by means of the parameters obtained (and stored) in the step E74.

[0231] Subsequently, during step E78, the flow analysis module 52 identifies and analyzes the data packet 34.

[0232] Step E78 here comprises the identification of the start of the data packet 34 by means of the marker M, the detection of the type identifier which here indicates the type T2 corresponding to data packets containing a coded representation of the content, and the reading (in the data stream) of a second part C' of the coded representation of the content.

[0233] Indeed, as already indicated, it is provided in this embodiment that only the data packets corresponding to a possible entry point in the data flow contain an identifier (here NNI) of the neural network to be used for decoding the coded representations of the content.

[0234] The method can then continue with a step E80 of decoding this second part C' of the coded representation of the content, by the decoding module 54 and by means of the artificial neural network obtained as described above in steps E74 and 76 by means of the location identifier NNL contained in the data packet 32 ​​of type T4 preceding the present data packet 34.

[0235] Any further processing of data packet 38 (when decoding sequence S') is not described here.

Claims

Claims 1. Method for decoding a data stream comprising data packets each comprising at least first data and second data, characterized in that it comprises the following steps: - identification (E4, E6; E30; E40; E50; E72), among said data packets, of a first data packet (12; 16; 22; 26; 36; 40) whose first data includes information (T1) indicative of a predetermined type of data packet; - processing (E12; E32; E42; E52; E74) of the second data (NNC) of the first data packet to obtain an artificial neural network; - decoding (E22; E38; E46; E56; E60; E76; E80) the second data (C; C') included in a second data packet (14; 20; 24; 28; 30; 32; 34; 42; 46) among said data packets, using at least the artificial neural network obtained and so as to produce data representative of audio or video content.

2. Decoding method according to claim 1, wherein the second data of the first data packet comprises descriptive data (NNC) of the artificial neural network, and wherein the processing step is a step of decoding the descriptive data (NNC) to obtain parameters of the artificial neural network.

3. Decoding method according to claim 1 or 2, wherein the first data packet comprises an identifier (NNI) of the artificial neural network.

4. Decoding method according to claim 3, in which the identifier (NNI) is an element of a list of distinct identifiers associated respectively with distinct artificial neural networks.

5. Decoding method according to claim 4, comprising a step of receiving a third data packet (44) whose first data includes information (T1) indicative of said predetermined type of packet, and comprising said identifier (NNI), and a step of reusing said artificial neural network obtained to decode second data (C') included in a fourth data packet (46) among said data packets.

6. Decoding method according to one of claims 3 to 5, in which the second data packet comprises said identifier (NNI).

7. Decoding method according to one of claims 4 to 6, comprising a step of receiving another data packet (18) containing parameters relating to at least one image of said content, in which said parameters comprise said identifier (NNI).

8. Decoding method according to one of claims 1 to 5, in which the first data packet (22) is, among the data packets whose first data includes information (T1) indicative of said predetermined type of packet, the last packet preceding the second data packet (24) in the data stream.

9. Decoding method according to one of claims 1 to 3, in which the second data packet (32) comprises a pointer (NNL) to the first data packet (36).

10. Decoding method according to one of claims 1 to 3, comprising the following steps: - reading a flag (DNN) in the second data packet (32); - if the flag has a predefined value, reading, in the second data packet (32), a pointer (NNL) to the first data packet (36). 1 1. Decoding method according to one of claims 1 to 3, comprising the following steps: - receiving another data packet containing parameters relating to at least one image of said content - reading a flag among said parameters; - if the flag has a predefined value, reading, among said parameters, a pointer to the first data packet.

12. Decoding method according to one of claims 9 to 11, in which the pointer (NNL) designates a location in a part of the data stream relating to a sequence of images (S') distinct from the sequence of images (S) coded at least in part by the second data (C) of the second data packet (32).

13. Decoding method according to one of claims 1 to 11, in which the first data packet comprises information (NNF) indicative of the coding format of the second data (NNC) of the first data packet.

14. Device for decoding a data stream comprising data packets each comprising at least first data and second data, characterized in that it comprises a processor configured or programmed to: - identifying, among said data packets, a first data packet (12; 16; 22; 26; 36; 40) whose first data includes information (T1) indicative of a predetermined type of data packet; - process the second data (NNC) of the first data packet to obtain an artificial neural network; - decoding the second data (C; C') included in a second data packet (14; 20; 24; 28; 30; 32; 34; 42; 46) among said data packets, using at least the artificial neural network obtained and so as to produce data representative of audio or video content.

15. Computer program comprises instructions executable by a processor and designed to implement a method according to one of claims 1 to 13, when these instructions are executed by the processor.

16. Data stream comprising data packets each comprising at least first data and second data, characterized in that the data packets comprise: - a first data packet (12; 16; 22; 26; 36; 40) the first data of which includes information (T1) indicative of a predetermined type of data packet and the second data (NNC) of which defines an artificial neural network; - a second data packet (14; 20; 24; 28; 30; 32; 34; 42; 46) whose second data (C; C') can be decoded using at least said artificial neural network so as to produce data representative of audio or video content.

17. Data stream according to claim 16, wherein the first data packet (40) comprises an identifier (NNI), and wherein the data stream comprises another data packet (44) which comprises said identifier (NNI), the first data of which comprises said information (T1) indicative of the predetermined type of data packet, and the second data of which is identical to the second data (NNC) of the first data packet (40).

Images