Method and device for encoding and decoding images

Abstract

The invention relates to a method and device for encoding and decoding images comprising a plurality of samples to be encoded or decoded. The decoding method comprises the following steps: - decoding (E31) a group of feature maps (FMdi) representative of the current image, the group comprising at least one feature map having a lower resolution than the resolution of the input image, which map is referred to as a lower resolution map; - obtaining (E33) a set of parameters (WUck) representative of at least one neural network (UP1) referred to as an oversampling neural network; - oversampling (E34) the at least one lower resolution feature map using the oversampling network to obtain an oversampled map at the resolution of the at least one image; - generating (E35) the at least one image from the at least one oversampled map.

Description

Method and device for coding and decoding images.

[0001] The invention relates to the general field of coding digital image sequences. It relates more particularly to the compression of digital videos.

[0002] Digital videos are generally source-encoded to compress them in order to reduce the resources required for transmission and / or storage. There are many encoding standards, such as the ITU / MPEG standards (H.264 / AVC, H.265 / HEVC, H.266 / VVC, etc.) and their extensions (MVC, SVC, 3D-HEVC, etc.).

[0003] As video formats continue to evolve to compress more and more and adapt to the variety of expected formats and communication networks, the possibilities for prediction are becoming ever greater and traditional coding and decoding algorithms are becoming very complex.

[0004] In addition to these classic approaches proposed by compression standards (MPEG, ITU), approaches based on artificial intelligence, and in particular neural ones, tend to develop.

[0005] Some of these neural approaches can be seen as a simple extension of the competition notion of the aforementioned compression techniques, such as the prediction and transformation mode competition in video coding.

[0006] Other approaches use the concept of "autoencoder." Autoencoders are learning algorithms based on artificial neural networks, which allow the construction of a new representation of a dataset. The architecture of an autoencoder consists of two parts: the encoder and the decoder. The encoder consists of a set of layers of neurons, which process the data in order to construct new representations called "encoded," also called "latent representations." In turn, the layers of neurons in the decoder receive these representations and filter them in order to try to reconstruct the initial data. The differences between the reconstructed data and the initial data make it possible to measure the error made by the autoencoder. Training consists of modifying the parameters of the autoencoder in order to reduce the reconstruction error measured on the different samples of the dataset.The performance of such autoencoder-based systems comes at the cost of a considerable increase in memory footprint and complexity compared to conventional approaches as proposed by compression standards. They can have millions of parameters and may require on the order of a million MACs (multiply-accumulate) to decode a single pixel. This makes such decoders significantly more complex than conventional decoders, which could hamper the adoption of learning-based compression. In addition, the neural networks used at the encoder and decoder are different. US 2022 / 0385907 describes such an autoencoder system.

[0007] More recently, a simple neural network-based image encoding technique was described in the article "COmpression with Implicit Neural Representations" by Emilien Dupont et al. (arXiv:2103.03123). The proposed encoding technique consists of fitting a neural network to an image, quantizing the network weights, and transmitting them. At decoding time, the neural network is evaluated at each pixel position to reconstruct the image. However, such a technique remains inefficient in terms of compression.

[0008] Document US 2022 / 0385907 also describes a coding and decoding system based on the use of such implicit neural representations.

[0009] However, there is a need for a solution that can encode / compress a sequence of images in a simple and efficient way.

[0010] The invention relates to a method for coding at least one image comprising a plurality of samples to be coded, comprising the following steps: - a construction step, comprising the sub-steps of: - construction of a group of characteristic maps comprising at least one characteristic map of resolution lower than the resolution of the input image, called lower resolution map; - oversampling of said at least one characteristic map of lower resolution using an artificial oversampling neural network at least defined by a set of oversampling parameters to obtain an oversampled map at the resolution of said at least one image; - generation of said at least one image from said at least one oversampled map;- updating at least one value of one of said feature maps of said group and / or at least one oversampling parameter, as a function of a coding performance measurement,- a step of coding said group of feature maps and said set of oversampling parameters.;

[0011] The invention also relates to a method for decoding at least one image, from a binary stream comprising a plurality of samples to be decoded, comprising the following steps: - decoding a group of characteristic maps representative of the current image, said group comprising at least one characteristic map of resolution lower than the resolution of the input image, called lower resolution map; - obtaining a set of oversampling parameters representative of at least one neural network, called oversampling neural network; - oversampling said at least one characteristic map of lower resolution using said at least one oversampling network to obtain an oversampled map at the resolution of said at least one image. - generating said at least one image from said at least one oversampled map.

[0012] For the purposes of the invention, encoding, or "coding", means the operation which consists of representing a set of samples, or pixels, in a compact form carried for example by a digital binary train. Decoding means the operation which consists of processing a digital binary train to restore decoded samples, for example of a digital image or a plurality of digital images.

[0013] By "image" we mean a two-dimensional digital image comprising one or more components (color, depth, views, etc.)

[0014] By "at least one image" is meant a plurality of ordered two-dimensional images, for example temporally in the case of a video. According to another example, the images can be views of the same scene represented in multi-views. According to another example, the images can be a plurality of temporal and multi-view images (immersive video).

[0015] A "sample" is a value taken from an image. Sampling a signal produces a series of discrete values ​​called samples. In the case of an image signal, the sample is called a pixel, which can be, for example, a color pixel traditionally represented by a triplet of values, for example (R,G,B) or (Y,U,V). The position of the sample can be identified by its abscissa (x) and ordinate (y) coordinates in the image.

[0016] A "feature map" is an abstract representation of an image or a sequence of images, comprising a plurality of variable scalar data, also called characteristic values, for example real numbers, or integers, signed or unsigned. The abstract representation of an image or a sequence may therefore comprise several feature maps. The abstract representation of several images may also correspond to a single feature map.

[0017] By "up-sampling of feature maps" we mean an operation that transforms the values ​​of a first map into values ​​of a second map. The first map, intended for coding (respectively decoding), is arbitrary. The second map, called the up-sampled map, has a higher resolution. For example, it can have the same resolution as the input images, i.e. it has as many values ​​as an input image (respectively to be decoded) has samples (N).

[0018] By "neural network", or "upsampling neural network", we mean a neural network such as a convolutional neural network, a multi-layer perceptron, etc. The neural network is defined for example by a plurality of layers of artificial neurons and by a set of activation, weighting and addition functions (for example, a layer can calculate y = f(Ax+b), where y and b are vectors of dimension N, x is a vector of dimension M, A is a matrix of dimension MxN, and f is the activation function).

[0019] By "neural network parameter" we mean one of the values ​​that characterizes the neural network, for example a weight associated with one of the neurons (filter coefficient, value of a matrix, weighting, bias, value affecting the operation of non-linearity, etc.)

[0020] By “generation of said at least one image” is meant a step which aims to reconstruct the decoded samples of the image.

[0021] Samples are generated from upsampled feature maps. This simplifies sample generation, especially when the upsampled feature maps are image-sized, as extraction is easier. Furthermore, the upsampling neural network is able to automatically adapt to the nature of the maps and the image.

[0022] By "performance measurement" is meant a measurement between at least one value of a sample to be coded and a decoded value of said sample. The measurement can evaluate, for example, a distortion, or a perceptual error. It can be carried out on a sample or a plurality of samples (for example, the current samples, or the current images, etc.). The measurement can also include a measurement of the flow rate, in particular associated with the coding of the neural network and / or the coding of the group's characteristic maps. The measurement can be a joint measurement between the flow rate and the distortion through their weighting. As is well known in the state of the art, the value of this measurement is generally minimized until a target value is reached.

[0023] A "construction step" means a step that aims to construct the representative parameters of the image, including the values ​​of the feature maps and the parameters of the neural network, before their actual encoding. The construction sub-steps can be repeated as many times as necessary to obtain an acceptable performance measure.

[0024] Generally speaking, it is considered that the steps of an coding or decoding method should not be interpreted as being linked to a notion of temporal succession. In other words, the steps may be carried out in a different order than that indicated in the independent coding or decoding claim, or even in parallel.

[0025] The coding method according to the invention constructs the coding parameters, from an input image or sequence of images, by training a neural network on feature maps which may be at the resolution of the input images, or at a lower resolution. During the training, or construction, the parameters of the neural network and the values ​​of the feature maps are updated according to a performance measure, for example of the rate-distortion type. When the training is finished, that is to say when the performance measure obtained is satisfactory, the actual coding of the parameters of the upsampling neural network and the values ​​of the feature maps can be carried out and the result stored or transmitted to the decoder.

[0026] Advantageously, the training process makes it possible to refine the parameters of the upsampling neural network and / or the values ​​of the feature maps until an adequate representation in terms of performance is obtained, for example a desired balance between the generated bit rate and the distortion undergone by the input image or sequence of images being encoded. The training of the values ​​of the feature maps and the parameters of the upsampling neural network can be joint. Advantageously, the coding method according to the invention makes it possible to efficiently compress the signal.

[0027] Advantageously, the compression of the feature maps is efficient since at least one of the feature maps to be encoded (resp. decoded) has fewer values ​​than if it were at the resolution of the signal. For example, in the case of a digital image, one of the feature maps to be encoded (resp. decoded) may be at 1 / 2 resolution, that is to say it has half as many values ​​on the abscissa and ordinate as the input image has samples, or in total four times fewer values ​​than a feature map at the resolution of the image. On the other hand, the oversampled feature map has a higher resolution, for example the same resolution as the signal. Oversampling makes it possible to obtain the same number of values ​​in this transformed map as the input signal (resp. to be decoded) has samples.

[0028] Advantageously, the decoding method is simple since it is sufficient to decode the feature maps associated with an image or a group of images, and to obtain the oversampling neural network, to reconstruct a decoded version of the current image or the group of current images.

[0029] Such an upsampling neural network can advantageously be of very simple structure with few parameters and few operations per sample.

[0030] Moreover, decoding can be done sample by sample, in a progressive manner.

[0031] According to embodiments of the coding or decoding method:

[0032] – At least part of the set of oversampling parameters is decoded from data of said bitstream (resp. coded in the bitstream); advantageously according to this mode, the parameters of the neural network are transmitted in the stream. The module can be transmitted in a quantized form and coded in a compact form using any quantizer and entropy coder accessible to those skilled in the art. A format analogous to that of the MPEG-7 NNR standard can be used.

[0033] – At least part of the set of oversampling parameters is obtained from predetermined values; advantageously according to this mode, the parameters of the neural network are obtained, for example from a network, or from a storage memory.

[0034] – At least part of the set of upsampling parameters is decoded (resp. coded) as a function of reference parameters. Advantageously, according to this mode, the parameters of the neural network are decoded by taking into account reference parameters which can be stored in a memory accessible to the decoder, or obtained from a network. Thus, the storage space or the transmission rate on the network can be effectively reduced: certain parameters can be coded in a complementary manner, others not be coded at all, because they are available in the reference parameters. According to alternative embodiments, which can be combined with each other: the reference parameters can constitute a set of variants of an upsampling network: type of convolution, attention modules, etc. Thus, only these parameters need to be known to the decoder.They can be for example coded, inserted into the stream and decoded by the decoder which can reconstruct, from the reference parameters, the at least one upsampling network. The upsampling parameters are coded in a complementary manner to the parameters of a reference network. Thus, the decoder will only need to decode these residual parameters and then correct them by addition, multiplication (or combination in any other known manner) with those of the reference neural network. A part of the parameters of a reference network is reused for the upsampling network. For example, a complete layer of the upsampling network can be copied from the reference network, the parameters of this layer are therefore neither coded, nor transmitted, nor decoded. A reference network indicator is transmitted and decoded in the stream. It is thus possible to indicate a network to be used for decoding from among a plurality of possible networks known to the decoder.

[0035] – Said set of upsampling parameters is representative of a plurality of upsampling neural networks, one upsampling neural network being used to upsample at least one lower resolution feature map of the group of feature maps. Advantageously, according to this embodiment, a plurality of upsampling neural networks can be decoded, each of them being intended for decoding one or more maps. Thus, the upsampling of the different maps can be carried out separately. According to an alternative embodiment, at least two upsampling neural networks are used to upsample the same lower resolution feature map of the group of feature maps.

[0036] – An upsampling neural network is used to upsample at least two lower resolution feature maps from the feature map group.

[0037] – Said upsampling neural network is used to upsample at least one already upsampled map from the group of feature maps. Advantageously, according to this mode, it is possible to use the same upsampling network in cascade. For example, upsampling by a factor of four in each dimension can be achieved by cascading two upsampling networks by a factor of two. This limits the cost of data transmission and storage and makes it possible to limit the modules to be implemented by the encoder and the decoder, and therefore the algorithmic complexity, the space occupied in memory, etc.

[0038] – Said at least one lower resolution feature map is upsampled in at least two dimensions, and separate upsampling is used for each dimension. Advantageously in this mode, the transmission cost and complexity are reduced by the use of separable upsampling transformations (e.g., convolution).

[0039] – The generation of said at least one image uses a synthetic neural network capable of generating a sample of the image to be decoded from data extracted from the oversampled feature maps at positions defined by that of said sample in the image. Advantageously, according to this mode, the coding method produces the coded values ​​by training a neural network on vectors of values ​​extracted from the feature maps, associated with a position of a sample to be coded. When the training is complete, that is to say when the performance measurement obtained is satisfactory, the actual coding of the parameters of the synthetic neural network can be carried out and stored or transmitted to the decoder.The decoding method is simple since it is sufficient to decode the feature maps, the upsampling neural network and the synthesis neural network to reconstruct a decoded version of the image signal. According to variants of this embodiment: such a synthesis network comprises an MLP. Such a neural network is advantageously of very simple structure with few parameters. In addition, the decoding can be carried out sample by sample, in a progressive manner. The synthesis network comprises convolution type filtering elements. The construction of the data vector comprises a sub-step of extracting a value from said at least one upsampled feature map at a position identical to that of the current sample in the signal to be encoded (resp. decoded). Advantageously, this method is simple to implement.For example, if we have J oversampled feature maps of the same resolution as the signal, a simple extraction of the values ​​of the maps at the coordinates of the current sample (at the same abscissa and the same ordinate in the feature map) makes it possible to construct a data vector of J elements to be applied as input to the synthesis neural network to obtain a sample or a plurality of samples as output.

[0040] – At least one of said characteristic maps has a resolution identical to that of the signal to be coded (resp. decoded). Advantageously, according to this mode, at least one of the characteristic maps, at the same resolution as the input signal to be coded (resp. decoded), allows significant fidelity and respect for the details of the initial resolution of the signal. In this case, the oversampling preserves the number of values ​​of the characteristic map; it can be reduced to identity.

[0041] – At least one upsampled map is obtained from a plurality of lower resolution feature maps.

[0042] – Multiple decoded maps are used to produce multiple oversampled maps.

[0043] Correlatively, the invention also relates to a coding device and a decoding device.

[0044] The characteristics and advantages of the coding or decoding method apply in the same way to the coding or decoding device according to the invention and vice versa.

[0045] The invention also relates to a computer program on a recording medium, this program being capable of being implemented in a computer or a control device in accordance with the invention. This program comprises instructions adapted to the implementation of the corresponding method. This program can use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code, such as in a partially compiled form, or in any other desirable form.

[0046] The invention also relates to a computer-readable information carrier or recording medium comprising computer program instructions mentioned above. The information or recording media may be any entity or device capable of storing the programs. For example, the media may comprise a storage means, such as a ROM, for example a CD-ROM or a microelectronic circuit ROM, or a magnetic recording means, for example a floppy disk or a hard disk, a DNA sequence, or a flash memory. Furthermore, the information or recording media may be transmissible media such as an electrical or optical signal, which may be conveyed via an electrical or optical cable, by radio link, by wireless optical link or by other means.

[0047] The program according to the invention can in particular be downloaded from an Internet-type network.

[0048] Alternatively, each information or recording medium may be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of a method according to the invention.

[0049] Other characteristics and advantages of the present invention will emerge from the description given below, with reference to the appended drawings which illustrate exemplary embodiments thereof which are not limiting in nature.

[0050] schematically represents a coding device used in the context of the invention;

[0051] schematically represents a decoding device used in the context of the invention;

[0052] schematically represents an artificial neural network for oversampling used in the context of the invention.

[0053] Illustrates an example of an oversampling module used in the context of the invention.

[0054] presents other examples of implementation of the oversampling module used in the context of the invention.

[0055] is a flowchart representing an example of a coding process that can be implemented by the coding device of the.

[0056] illustrates an encoding method used in one embodiment of the invention.

[0057] is a flowchart representing an example of a decoding method that can be implemented by the decoding device of the.

[0058] illustrates a decoding method used in one embodiment of the invention.

[0059] It schematically represents a digital image coding device ENC.

[0060] The ENC coding device receives as input a succession of samples to be coded, noted P n , for example a temporal succession of sound samples, or a set of image data noted I(P n ). In this second case, the image signal I(P n ) can represent a two-dimensional image, or a plurality of two-dimensional images (video, color components, stereoscopic, multiscopic components, etc.). P ndenotes a sample n of the input signal comprising N samples. In one embodiment, the signal is a color image signal represented by means of at least one two-dimensional representation, such as a matrix of pixels, of width W and height H (with N = WxH) each pixel comprising a red (R), green (G), blue (B), or, alternatively, a luminance (Y) component and at least one chrominance component. The location of each pixel is defined by its abscissa and ordinate (x and y) coordinates in the image. In one embodiment, the image is in grayscale represented by means of a two-dimensional representation, such as a matrix of pixels, each pixel comprising a grayscale, or luminance, component.

[0061] This ENC coding device comprises an INIT module for generating initial characteristic maps, a Q quantization module, a UP module for oversampling the characteristic maps and a PROD signal synthesis module, according to the example shown a digital image I(P' n ), an FMC module for coding feature maps, an NNC module for coding neural networks, a PRC module for coding signals, an EVAL module for evaluating a performance function, and an MAJ update module.

[0062] An oversampling artificial neural network used for coding, contained in the UP module, and an oversampling artificial neural network used for decoding, contained in the UP' module, are defined by an identical structure, comprising for example a plurality of layers of artificial neurons, and by a set of weights and activation functions associated respectively with the artificial neurons of the network concerned.

[0063] During the process of training, or construction, of the coding, that is to say as long as the stage of evaluation of a performance is not satisfactory, the coding modules (FMC, NNC, optionally PRC) carry out a coding simulation, followed by a decoding, intended for the evaluation module. Subsequently, they carry out the actual coding of the values ​​of the FM maps i , parameters of the WU upsampling neural network k, and optionally coding parameters of the PROD synthesis module. As is known, the coding simulation can be identical to the actual coding, or an approximation thereof.

[0064] Each feature map contains scalar data, according to the definition of a feature map given above, and there are M feature maps. However, it is possible to organize the feature values ​​differently, for example by forming a single signal comprising, for each sample, a vector of M scalar values, without loss of generality. Even if the data are organized differently, we consider in the rest of the document that there are M feature maps. The maps coded by the FMC module for a current image (or a plurality of images) are denoted FMc iThe FMC encoding module quantizes the feature map values ​​if necessary using a quantizer to generate an ordered collection of quantized values. Then the encoding module compresses the quantized data, for example using an entropy encoder.

[0065] The Q module optionally performs quantization. The Q module produces the FMq maps i . During the coding step, quantization is essential for the proper functioning of the system if the feature maps contain, for example, floating point or real values. It is necessary to quantize them before coding them and / or providing them as input to the oversampling neural network. During decoding, however, inverse quantization is not necessary, depending on the embodiments.

[0066] The UP module performs oversampling of the FM(q) feature maps i. This module has WU oversampling parameters k representative of an upsampling neural network. This upsampling is carried out on at least one map, obtained after the Q module, with a resolution lower than the resolution of the image. Some maps may not require upsampling, in particular if their resolution is equal to that of the image to be coded. Such maps are not processed by the UP module (or the UP module is reduced to the identity function). At the output of the UP module, we obtain a set of FMU characteristic maps i including the actually upsampled maps and the non-upsampled maps for which FMU i = FMq i .

[0067] According to embodiments, the UP module may be a convolutional neural network, comprising at least one convolution layer. Each convolution layer may include non-linearities, such as rectification, arctangent, sigmoid, etc. It may also include a convolutional or convolutional with residual post-processing module or an attention module. The Up module may also include several neural networks.

[0068] The PROD module is a synthesis module, for example a neural network or an inverse linear transformation (wavelet, DCT, etc.) defined by its WS synthesis parameters k , capable of processing FMU characteristic maps i as input, to generate as output the synthesized image (coded then decoded) I(P' n ). The image samples decoded by the PROD module are denoted P' n.

[0069] According to one embodiment, the synthesis module is a filter bank, the WUk parameters are the parameters of the filters, and the output synthesized image results from the application of the filter bank to the FMU characteristic maps. i .

[0070] According to one embodiment, the synthesis module is a synthesis neural network and for each sample to be decoded, the synthesis parameters WSk are the parameters of the synthesis neural network.

[0071] According to another embodiment, the neural network is of the convolutional type or of the multilayer perceptron (MLP) type. A combination of these two types of topology is also possible. In order to limit the number of arithmetic operations, an MLP is placed upstream of a convolution module. The synthesis module may comprise a post-processing module of the convolutional type or convolutional with residual applied to a (R,G,B) or (Y,U,V) signal.

[0072] A neural network of the PROD module can behave like the implicit neural networks mentioned above: it can advantageously process vectors from co-located pixels from oversampled feature maps. In this case, the positional decoding of the samples can be carried out independently, which provides an advantage in terms of parallelization of the decoding. A vector (P' n ) representative of the sample P n to be coded is generated as output according to its position in the image. According to one embodiment, the output vector is the triplet (R,G,B) of the pixel P' n decoded, or the triplet (Y,U,V) of pixel P' n decoded when the image is in YUV 444 format, or a vector comprising four Y values, one U value and one V value, capable of representing four P' pixels n decoded when the image is in YUV 420 format.

[0073] WU Oversampling Settingsk of the oversampling neural network encoded by the NNC module are denoted WUc k .

[0074] WS synthesis parameters k of the synthesis module coded by the PRC module are noted WSc k.

[0075] When all samples P n of the signal have been coded then decoded, we have a reconstructed signal I(P' n), selon un exemple une image I comportant N échantillons décodés sous la forme de N vecteurs.

[0076] The EVAL module performs an evaluation and minimization of a coding performance. The evaluation function is for example of the rate-distortion type. It aims to minimize the differences between the input representation of the current image I(P n ) and its representation I(P' n ) at the output, while optimizing the amount of data to be encoded. The minimization can be carried out by gradient descent, or any other optimization method within the reach of those skilled in the art.

[0077] The MAJ module updates the values ​​of the FM maps i to encode from the current image, depending on the results of the performance function.

[0078] Once the encoder training is complete, the upsampling network parameters are encoded, either losslessly, in which case the decoder's UP' neural network is identical to UP, or lossily, in which case the UP' network may be slightly different from UP. The values ​​of the feature maps, FM i, are also encoded as FMc coded data i , for example via an entropy encoder. The parameters of the WSc synthesis module k are optionally encoded.

[0079] It schematically represents a DEC decoding device.

[0080] The decoding device DEC receives as input a first group of encoded data organized into M FMc characteristic maps iand WUc oversampling parameters k of an oversampling neural network. Optionally, the decoding device DEC receives the synthesis parameters WSc k of a synthesis module, as illustrated by the dotted arrow.

[0081] This DEC decoding device comprises a neural network decoding module NND, a feature map decoding module FMD, an optional parameter decoding module of the synthesis module PRD, an optional inverse quantization module DQ, a feature map oversampling module UP' and a signal synthesis module PROD', according to the example represented a digital image I(Pdn).

[0082] The cards decoded by the FMD module, numbering M, are noted FMd i. It is however possible to organize the feature maps differently, for example by forming a single signal comprising, for each sample, a vector of M scalar values ​​to be decoded, without loss of generality. Even if the data are organized differently, we consider in the rest of the document that there are M feature maps to be decoded.

[0083] The oversampling parameters decoded by the NND module are denoted WUd k . According to embodiments, certain WUd parameters k can be predetermined, or obtained by decoding the parameters WUcken utilisant des paramètres de référence connus du décodeur.

[0084] The synthesis parameters decoded by the PRD module are denoted WSd k .

[0085] The DQ module optionally performs an inverse quantization corresponding to the quantization performed at the encoder. The DQ module produces the FMdq maps i. Inverse quantization is not necessary if the encoder's Q quantizer has performed a simple rounding of the real values ​​submitted to it. Otherwise, the decoder performs the inverse operation of the Q quantizer. If quantization is not necessary, FMdq i =FMd i.

[0086] The UP' module performs oversampling of the feature maps. This module is configured by the WUd parameters k . This oversampling is carried out on at least one map, obtained after the DQ module, with a resolution lower than the resolution of the image. Some maps may not require oversampling, in particular if their resolution is equal to that of the image to be decoded. Such maps are not processed by the UP' module (i.e. for example it is reduced to identity). At the output of the UP' module, we obtain a set of FMUd characteristic maps iincluding the actually oversampled maps and the non-oversampled maps for which FMUd i = FMdq i . The embodiments described previously in support of the encoder apply. The WUd parameters k are identical to the parameters WUkdu codeur si le codage a été effectué sans pertes.

[0087] The PROD' module is a synthesis module, for example a neural network or an inverse linear transformation (wavelet, DCT, etc.) defined by its WSd parameters k , capable of processing FMUd characteristic maps i as input, to generate as output the synthesized (decoded) image I(Pd n ). The embodiments described previously in support of the encoder apply. The WSd parameters k are identical to the parameters WSkdu codeur si le codage a été effectué sans pertes.Les paramètres WSdksont optionnels, c'est-à-dire que le module PROD' peut utiliser des paramètres connus du décodeur.

[0088] As previously described in support of the, a positional decoding of the Pn samples of the signal can be carried out independently, which provides an advantage in terms of parallelization of the decoding. A vector (Pd n ) representative of the sample P n to be decoded is generated as output depending on its position in the image. According to one embodiment, the output vector is the triplet (R,G,B) of the pixel Pd n decoded, or the triplet (Y,U,V) of the pixel Pd n decoded when the image is in YUV 444 format, or a vector comprising four Y values, one U value and one V value, capable of representing four Pd pixels n decoded when the image is in YUV 420 format. When all P samples n of the signal have been decoded, we have a reconstructed signal I(Pd n), selon un exemple une image I comportant N échantillons décodés sous la forme de N vecteurs (Pdn).

[0089] The DEC decoding device may be implemented by means of an electronic device comprising a processor and a memory, not shown; each of the above-mentioned modules may then be realized by the cooperation of the processor and computer program instructions stored in the above-mentioned memory and designed to perform the functionalities of the module concerned, in particular as described below, when these instructions are executed by the processor.

[0090] Illustrates an oversampling module (UP, UP') used for encoding and decoding in embodiments of the invention.

[0091] The purpose of the upsampling module is to produce a feature map containing more pixels than the input feature map. The input feature map is denoted FM i , but may correspond to an FM(d)q map iwhere, as explained in support of Figures 1 and 2, d indicates that the index map i of the group of feature maps has been previously encoded and decoded, and q that it has been (de)quantized (if necessary) by the Q or DQ module. The output feature map is denoted FMU i . An FM card i W*H resolution comprising W*H values, where W indicates the map width and H its height, is upsampled to an FMU map i resolution s*W*s'*H. The scale parameters s and s' are positive and greater than or equal to one. If the scale parameters have a value greater than one, the FM card i intended for encoding contains fewer coded values ​​than a map with the same resolution as the image, which results in a lowering of the required bitrate.

[0092] In one embodiment s=s' to maintain a proportionality ratio.

[0093] In a general framework, the upsampling module corresponds to at least one artificial neural network. It includes weighting coefficients, biases and non-linearities specific to the operation of a neural network. This upsampling module can take the form of a simple convolution after inserting zero values. More generally, a neural network can take the form of a succession of layers, where a layer can calculate y = f(Ax+b), where y and b are vectors of dimension N, x is a vector of dimension M, A is a matrix of dimension MxN, and f is the activation function.

[0094] In one embodiment s and are written as integer powers of 2 (s=2k, s'=2k'), which allows for simpler implementation by inserting zero values ​​into the FM map i to be treated, as will be explained in support of the.

[0095] Illustrates an example of an oversampling module used in the context of the invention.

[0096] The example of represents a transposed convolution (CT) upsampling operating on an FM input feature map i of size 4x3 (W= 4 and H = 3 pixels) undergoing an interpolation of a factor of 2 in each direction (s=s'=2), therefore the size of the FMU output map i is 8x6.

[0097] The transposed convolution takes the form of a 3-dimensional convolution kernel, i.e. a 3*3 coefficient matrix.

[0098] In a first step, as illustrated in the middle of the figure, the FM card iis interpolated by inserting pixels of zero value between each pair of pixels of the input map. We therefore obtain 4 times more pixels on a grid noted GR of size 8x6 pixels. In a second step, we extend this resulting grid by a number of pixels (1 or 2 depending on the example) depending on the size kxk of the convolution filter (k=3 in this example). In a general and known manner, this extension, or padding, is of size (k-1) / 2 if k is odd. The padding appears on the figure in a hatched area. We obtain an extended grid FM' i size 10x8.

[0099] Then a filtering is performed using the convolution kernel, of size kxk (here k=3) operating on the extended grid FM' i The 3x3 kernel is applied to the different grid positions to obtain the output map of the desired dimension. On the right, two positions of the convolution kernel are shown in transparent gray.

[0100] Naturally, any alternative (transposed) convolution operation can be applied. Similarly, any complementary operation within the reach of a person skilled in the art can be added to the transposed convolution: bias, stride (shifting the inverse convolution kernel at each iteration), dilation (spacing the input values ​​supplied to the kernel), etc.

[0101] Similarly, any activation function within the reach of a person skilled in the art can be used (ReLU, sigmoid, Softmax, Softplus, Softsign, etc.)

[0102] According to the embodiment as presented in the, the convolution modules are composed of 2D convolution, that is to say that the convolution kernel is applied jointly on the rows and columns of the feature map.

[0103] According to another embodiment, they can be in a separable form. In this case, rather than performing the convolution on the two-dimensional feature map, a first one-dimensional (1D) convolution operator of length Lh is performed on the rows, then a second convolution of length Lv on the columns of the feature map obtained after the first operator, or vice versa. This implementation has the effect of reducing the storage of Lh*Lv for a given kernel to Lv+Lh and limits the complexity to Lh+Lw multiplications per feature map value versus Lh*Lv in the 2D case. The two 1D operators can be identical in length and have the same number of coefficients. The coefficients can be identical in the horizontal and vertical dimensions. As with non-separable convolution, in the case of transposed convolution the 1D operators are applied after interpolation of the signal.

[0104] The present examples of implementation of an oversampling module used in the context of the invention.

[0105] A simple module, corresponding to the description provided above in support of the, is presented in (a). It comprises a transposed convolution CT. Preferably, it is followed by a bias and a non-linearity.

[0106] A module comprising a convolution (C) followed by a transposed convolution (CT) is shown in (b). For example, module C performs filtering preprocessing on the feature map at its initial resolution.

[0107] A module using a residual connection is presented in (c). In this mode, the result of the transposed convolution (the oversampled map, for example by a factor of 2 in each dimension) is subjected to a new convolution whose result is added to or subtracted from the oversampled map. The result is a residual map which advantageously corrects the oversampled signal.

[0108] A module including an attention element is presented in (d). Instead of the simple convolution presented on the lower branch of the previous case (c), it includes a convolution function (C2) followed by a sigmoid operator SGM representing the sigmoid-type activation function. An attention module generally allows training a neural network to automatically associate each element of a set with a relevance relative to the other elements based on the context, resulting in the example from C2. This allows for more localized processing than a pure residual system which would not have this relevance element.

[0109] Naturally, the above elements, given as examples, can be combined.

[0110] In particular, in one embodiment, the oversampling module may take the form of a succession of sampling sub-modules. For example, oversampling by a factor s may be achieved by a succession of modules with smaller factors. An oversampling factor s=6 may be obtained by cascading two modules with respective factors s1=2 and s2=3. A factor 8 may be obtained by three modules s1=2, s2=2 and s2=2. Thus, a single oversampling sub-module may be replicated to obtain oversamplings adapted to the different resolutions of the FMd feature maps. i This embodiment has the advantage of limiting the number of modules and sharing the oversampling modules. Thus the number of parameters of this network, and consequently their storage and / or transmission volume is reduced, as well as the associated implementation complexity.

[0111] This is a flowchart representing an example of a coding process that can be implemented by the coding device of the.

[0112] According to this embodiment, the signal is a two-dimensional image, each sample to be coded is therefore a pixel P n of coordinates (x n , y n ).

[0113] Encoding takes place in two main phases:

[0114] In a first phase, called the construction phase, learning is carried out in order to determine, for an input signal I(P n ), the values ​​of FM cards i and WU parameters k of the oversampling neural network and optionally WS kof the production module to optimize a global cost function. Learning is for example carried out by gradient descent, followed by an update of the parameters of the UP oversampling neural network, optionally the parameters of the PROD synthesis module, the values ​​of the FM characteristic maps i . As known in the state of the art, the cost function can be of rate-distortion type, or rate, or distortion, or include a perceptual metric. To measure the rate R, it is necessary to simulate the coding of FM cards i , then measure the associated coding rate (the size of the B1 stream). According to one embodiment, the coding of the WU parameters is not simulated k and / or WS k, because their influence is less important than that of the feature maps. According to one embodiment, the coding of the WU parameters is also simulated k and / or WS kand the associated flow rate (the size of the flow B2) is measured. To measure the distortion D, it is necessary to simulate the coding then the decoding of at least part of the image I, to obtain at least one pixel P' n resulting from a simulation of coding then decoding, then measuring the difference between this part of the image or set of images I (P n ) as input and a corresponding part of the image or set of images I (P' n ) encoded then decoded.

[0115] Then during a second phase, called the coding phase, the FM cards i and WU parameters k and possibly WS k are encoded to produce the FMc encoded values i and WUc k (and possibly WSc k ) before transmission or storage. They constitute the compressed representation of the input signal I (P n ).

[0116] We will now describe the steps of a method according to one embodiment of the invention.

[0117] During a step E20, a set of images I (P n ) to be coded, comprising a plurality of N samples P n , is provided as input to the process.

[0118] During an E21 step, the M FM cards i are initialized. Subsequently, the WU parameters k of the UP upsampling neural network and the values ​​of the FM maps i , and possibly the WS parameters k of the PROD synthesis module must be optimized during the construction phase.

[0119] According to one embodiment, the FM cards i are of resolution less than or equal to that of the input signal I (P n ) and therefore include, for at least one of them, a number N' of values ​​to be coded less than N. According to a variant, the first FM card iis at the resolution of the images and each subsequent map is at half the resolution of the previous one.

[0120] According to one embodiment, several FM cards i are of the same resolution, lower than that of the input signal I (P n ). According to one embodiment, an FM card i unique has, for each sample, a vector of M scalar values ​​to decode, without loss of generality. Even if the data are organized differently, we consider that there are M feature maps to decode.

[0121] According to one embodiment, the FM cards i are initialized by the INIT module to predefined constant values.

[0122] According to another embodiment, the FM characteristic maps i are initialized by the INIT module to a set of random real values.

[0123] FM Feature Cards iare subsequently updated, or refined, during a step E22, by the encoder's MAJ update module during its learning.

[0124] During an E23 step, the FM cards i are quantified.

[0125] During an E24 stage the FM cards i are encoded by the encoder's FMC module. During the construction phase, this operation is a coding simulation. During the coding phase, this operation is an actual coding and the encoded values ​​constitute the B1 stream. The simulation can be identical to the actual coding, but it can also be different (for example, simplified). For this coding, for example, entropy coding of the quantized values ​​is used.

[0126] In one embodiment, the FM cards iare encoded in order (FM1, FM2,…, FM4), and the variables of each map in a predefined order, for example lexicographic. Each map undergoes entropy coding. Entropy coding produces a compressed stream B1 whose bitrate is subsequently measured during a step E28 or E29.

[0127] During an E25 stage, the M FM cards i are upsampled by the FMU module to generate FMU maps i to the resolution of the images in the input sequence. At least one FM card i is of lower resolution than the images to be encoded and the upsampling operation is performed so that the FMU card i oversampled has the same number of samples as the images to be encoded. Oversampling consists of adding values ​​to the FMU maps iand process them through a neural network to at least achieve the resolution of the input images. The oversampled maps are not encoded, they are only used to feed the PROD module.

[0128] The WU structure and parameters k of the upsampling neural network are initialized for example during the first iteration of this step. These parameters are subsequently updated, or refined, during the construction phase, during subsequent iterations of the process.

[0129] According to one embodiment, the parameters of the upsampling network are initialized by predefined values ​​known to give a satisfactory result (for example, following training on a corpus of images).

[0130] According to another embodiment, the parameters of the upsampling neural network are initialized by a set of random values.

[0131] According to one embodiment, M FMU cards i are generated.

[0132] According to one embodiment, each FM card i is transformed into an FMU card i .

[0133] During a step E26, the PROD module is called for a synthesis of the image I(P' n ). According to one embodiment, the PROD module is a neural network of the MLP or convolutional type, or a combination of the two, which transforms an input vector into an output vector representative of the sample being processed, for example a pixel. For example, the extraction is carried out in the oversampled maps according to the coordinates (x n , y n ) of sample P n of the input signal. In another example, the extraction is performed blockwise in the maps to produce a set of output pixels. In this case, the input vector corresponds to the block. The WS structure and parameters kof the synthetic neural network are initialized for example during the first iteration of this step. These parameters are subsequently updated, or refined, during the construction phase, during subsequent iterations of the process.

[0134] The samples to be coded are, for example, processed in sequential order, from n=1 to n=N.

[0135] During a step E27, the WS parameters k of the PROD module's synthesis neural network, if they exist, and the WU parameters kof the upsampling neural network, are quantized and encoded. During the construction phase, this operation is a coding simulation. During the coding phase, this operation is an actual coding and the encoded values ​​constitute the B2 stream. The simulation can be identical to the actual coding, but it can also be different (for example, simplified). Any known technique can be used for this purpose, for example the neural network coding standard proposed by the MPEG-7 part 17 standard, also called NNR (Neural Network Representation). Note that in this case, it is necessary to choose the amount of degradation that the coding brings to the WU weights k and optionally WS k .

[0136] According to one embodiment, the B2 stream is transmitted. According to another embodiment, the B2 stream is stored in memory, for example in ROM or RAM.

[0137] This storage space can contain a bank of parameters allowing to constitute variants of upsampling network. For example, several convolutions (simple interpolators such as cubic, linear interpolations or extended to two dimensions…) several residual convolutions or convolutions with attention adapted to specific contents (directional images, graphic images, satellites…) can be stored, an indicator can then be inserted in the stream to inform the decoder of the network to use.

[0138] According to one embodiment, certain WUc parameters kof the upsampling network UP can be expressed in relation to a reference network stored in memory. This has the advantage of reducing the space required for storage or transmission of the network. For example, a layer of the upsampling network can be taken from the reference network, the weights of a layer can be differentially encoded, that is, a corrective value is applied by addition or multiplication to each weight (convolution, bias) of the reference network. These corrective values ​​can be transmitted or stored efficiently using an entropy code. Thus, the upsampling network is adapted, with little additional information, to the nature of the signal contained in the feature map and to the image(s) being processed.

[0139] The two embodiments mentioned above can be combined: a reference indicator can then be inserted into the stream to inform the decoder which reference network to use, and the received parameters are used to reconstruct the upsampling network from the reference network to be used.

[0140] During step E28, a performance measure is evaluated.

[0141] For this purpose, the coding simulation rates associated with the feature maps (simulation of the B1 flow by coding the FM maps i ) and optionally to the parameters of the neural network(s) (simulation of the B2 flow by coding the WU parameters k and optionally WS k ) are measured.

[0142] According to one embodiment, the cost function is of the rate-distortion type, denoted (D+L*R), where D for example the quadratic error measured between the input signal and the decoded signal (or the error measured on a subset of samples of the signal). According to another example, D is calculated from a perceptual function such as the SSIM (for Structural SIMilarity), or MSSSIM (for Multi-scale Structural SIMilarity). According to one embodiment, R is the simulated rate of the stream B1; according to another embodiment, R is the total rate used to code this image, that is to say the sum of the simulated rates of B1 and B2. L is a parameter which regulates the rate-distortion trade-off. Other cost functions are possible.

[0143] As long as the cost function has not reached its minimum, the performance measurement is not satisfactory, and the method is repeated from step E22. This minimization can be carried out by a mechanism known as gradient descent with parameter updates during step E22 for the values ​​of the feature maps and E25, E26 for the parameters of the network(s).

[0144] At step E29, if the cost function has reached its minimum, training stops. If an encoded version corresponding to the last simulation of the parameters of the oversampling neural network (WU k ), optionally synthetic (WS k ) and feature maps (FM i ) is available, streams B1 and B2 can be made up of them. According to another embodiment, the effective coding of the updated parameters of the neural network(s) and the values ​​of the feature maps (FM i) is performed at this step to produce the WUc encoded parameters k (optionally WSc k ) and FMc iqui constituent des flux B1 etB2. Les flux B1 et B2 peuvent être concaténés pour produire un flux final. Selon un mode de réalisation, le flux B2 des paramètres codés du (des) réseau(x) de neurones est (sont) stocké(s) ou transmis avant le flux B1, afin de pouvoir être décodé(s) avant le flux B1.

[0145] The figure illustrates an encoding method used in one embodiment of the invention.

[0146] In this embodiment, the FM cards i generated are four in number. In a preferred mode, there are seven.

[0147] The first FM1 card has the same resolution as image I (P n), and therefore has WxH variable values, or scalars, where W represents the width of the image in pixels, and H its height. The second map FM2 has half the resolution (in each dimension) of the map FM1. Each additional map has half the resolution of the previous map. This structure allows the number of variables in the feature maps to be reduced, which facilitates coding and training while minimizing the coding cost. Other types of structure are possible, for example, a reduction ratio different from half can be used between the maps (a quarter, or a third, etc.).

[0148] According to this embodiment, the FM cards i are quantized by the Q module. Each card has its own Q quantization sub-module i . In another embodiment, the quantization module is common to all cards.

[0149] According to this embodiment, the FMq cards iare upsampled by the UP module, except for FMq1 which is already at the resolution of the output image. The UP1 module is reduced to the same (or non-existent) value.

[0150] In one embodiment, each card has its own UP oversampling sub-module i which allows it to switch from the resolution of the FM card i to the image resolution. Thus the FM2 map is oversampled by the UP2 module by a factor of 2 in each dimension, according to a process illustrated in Figures 3 to 5. The FM3 map is oversampled by a factor of 4, which can be achieved by two successive oversamplings by a factor of 2, in each dimension, and the FM4 map by a factor of 8 in each dimension.

[0151] FMU cards i resulting images are of the same resolution as the image I(P n ), and therefore each have WxH values, where W represents the width of the image in pixels and H its height (N=WxH).

[0152] In another embodiment, the oversampling module is common to all cards; for example, a sub-module of a factor of 2 can be used in cascade and / or shared by several cards.

[0153] In the illustrated embodiment, the synthesis PROD module is a synthesis neural network of the MLP or convolutional type, or a combination of both, and a vector Z n (z1...z4) is made up of the values ​​extracted from the FMU maps i located at the coordinates (x n , y n ) of the current pixel P n . The vector Z n consisting of the extracted (quantified) values ​​from the FMU maps i is processed by the synthesis neural network of the PROD module to generate a second vector as output, according to the example the output vector is the triplet (R,G,B) of the pixel P' nencoded (insofar as the feature maps used for extraction match those encoded in the stream) and then decoded (i.e. here generated, or produced, by the PROD neural network). The triplet is inserted into the decoded image I (P' n ) at positions (x n , y n ) of the color components (R', G', B') and can be used in the quality evaluation module, by comparison with the input image I (P n ).

[0154] This is a flowchart representing an example of a decoding method that can be implemented by the decoding device of the.

[0155] The decoding described concerns a sub-sequence of images comprising at least one image to be decoded.

[0156] During a step E30, the streams B1 and B2 are extracted from the encoded stream. They contain respectively, for a current subsequence of the sequence, the coded representations of the current maps FMc iand WUc oversampling parameters k of the UP upsampling neural network, optionally WSc synthesis parameters k of the synthesis module.

[0157] In one embodiment, the B2 stream is received. In another embodiment, the B2 stream is stored in memory, for example in ROM or RAM. As described for the encoder, this storage space may contain a bank, or plurality, of parameters for constituting oversampling network variants. A flag may then be read from the stream to inform the decoder of the network to use or to construct from a combination of parameters.

[0158] According to one embodiment, the WUc parameters kof the oversampling network can be decoded in relation to a reference network stored in memory. The examples described at the encoder apply. For example, the received parameters are correction values ​​relative to the reference network, which can be decoded using an entropy code and combined with the parameters of the reference network.

[0159] The two embodiments mentioned above can be combined: a reference flag can then be read from the stream to inform the decoder which reference network to use, and the received parameters are used to reconstruct the upsampling network from the reference network.

[0160] During a step E31, the M FMd cards i are generated by decoding FMc values i For this decoding, we use, for example, an entropy decoder.

[0161] According to embodiments as described for the encoder:

[0162] – FMd cards i are at a resolution lower than or equal to that of the signal I (Pd n ) to be reconstituted.

[0163] – Several FMd cards i are of the same resolution, lower than the signal resolution.

[0164] - An FMd card i unique has, for each sample, a vector of M scalar values ​​to decode. Even if the data are organized differently, we consider that there are M feature maps to decode

[0165] During a step E32, according to one embodiment, the FMd cards i are dequantized.

[0166] In step E33, the oversampling neural network of the UP module is decoded. The WUd parameters k of the UP' upsampling neural network and possibly the WSd parameters k of the PROD module's synthesis neural network are generated by decoding the WUc values k and WSck of the B2 stream. Any known decoding technique corresponding to the coding technique used by the encoder can be used for this purpose, for example a format similar to that of the MPEG-7 NNR standard. The synthesis neural network of the PROD' module is similar to the synthesis network of the PROD module, i.e. it has the same structure and includes the same parameters, except for the coding, which can be performed with or without losses. Similarly, the upsampling neural network UP' is similar to the upsampling network UP, i.e. it has the same structure and includes the same parameters, except for the coding, which can be performed with or without losses.

[0167] According to one embodiment, the stream B2 is decoded before the stream B1, in order to be able to have the upsampling neural network UP and possibly the synthesis neural network PROD before starting to decode the samples.

[0168] During an E34 step, the M FMd cards i decoded are oversampled by the UP' module to generate FMUd maps i to the resolution of the images in the input sequence. At least one FMd card i is of lower resolution than the images to be encoded and the upsampling operation is performed so that the upsampled FMUdi map has the same number of samples as the images to be decoded. Upsampling consists of adding values ​​to the FMd maps i and processing them by a neural network to achieve the resolution of the output images. This step is similar to step E25 which was described for the encoder in support of the, and the embodiments apply. In particular:

[0169] According to one embodiment, M FMUd cards i are generated.

[0170] According to one embodiment, each FMd card i is upsampled to an FMUd map i .

[0171] According to one embodiment, at least one FMd card i is of lower resolution than the images to be encoded and the upsampling operation is performed so that the FMUd card i oversampled has the same number of samples as the output image.

[0172] In a step E35, the PROD module is called to generate a synthesis of the decoded samples from the upsampled feature maps. According to one embodiment, the PROD module is a neural network of the MLP or convolutional type, or a combination of the two, which transforms an input vector into an output vector representative of the sample being processed, for example, a pixel. For example, the extraction is carried out in the upsampled maps according to the coordinates (x n , y n ) of sample P n of the input signal.

[0173] This step is similar to step E26 which was described for the encoder in support of the, and the embodiments apply.

[0174] The samples to be decoded are for example processed in sequential order, from n=1 to n=N.

[0175] When all the samples of the signal have been processed, the decoded signal corresponding for example to image I (Pd n ) is available.

[0176] Illustratively represents a decoding method used in one embodiment of the invention.

[0177] In this embodiment, the decoded FMd cards i are four in number. In a preferred mode, there are seven.

[0178] In this embodiment, the first map FMd1 has the same resolution as image I, and therefore has WxH variable values, where W represents the width of the image in pixels, and H its height. The second map FMd2 has half the resolution (in each dimension) of map FMd1. Each additional map has half the resolution of the previous map. This structure makes it possible to reduce the number of variables in the feature maps, which facilitates decoding while minimizing the coding cost. Other types of structure are possible, for example, a reduction rate different from half can be used between the maps (a quarter, or a third, etc.).

[0179] According to this embodiment, the FMd cards i are dequantized by the DQ module. Each card has its own DQ quantization sub-module i. In another embodiment, the inverse quantization module is common to all cards. In another embodiment, it does not exist.

[0180] According to this embodiment, the FMq cards i are oversampled by the UP' module, except for FMq1 which is already at the resolution of the output image. The UP1 module is reduced to the same (or non-existent) value.

[0181] In one embodiment, each card has its own UP oversampling sub-module i which allows it to move from the resolution of the FMdi map to the resolution of the image. The FMd2 map is oversampled by the UP'2 module by a factor of 2 in each dimension, according to a process illustrated in Figures 3 to 5. The FMd3 map is oversampled by a factor of 4, which can be achieved by two successive oversamplings by a factor of 2, in each dimension, and the FMd4 map by a factor of 8 in each dimension.

[0182] FMUd cards i resulting images are of the same resolution as image I (Pd n ), and therefore each have WxH values, where W represents the width of the image in pixels and H its height (N=WxH).

[0183] In another embodiment, as described to the encoder, the oversampling module is common to all cards; for example, a sub-module of a factor of 2 can be used in cascade and / or shared by several cards.

[0184] In the illustrated embodiment, the PROD' module is a synthetic neural network of the MLP or convolutional type, or a combination of the two, and a vector Zd n (zd1...zd4) is made up of the values ​​extracted from the FMU maps i located at the coordinates (x n , y n ) of the current pixel P n . The vector Zd nis processed by the synthesis neural network of the PROD' module to generate as output the triplet (R, G, B) representative of the sample Pd n to be decoded. The triplet (R, G, B) is inserted into the decoded image I (Pd n ) at coordinates (x n , y n ) in the color components (Rd, Gd, Bd).

Claims

Method of coding at least one image (I(P n )) comprising a plurality of samples (P n ) to be coded, comprising the following steps: - a construction step, comprising the sub-steps of: - construction (E21, E22) of a group of characteristic maps (FM i ) comprising at least one feature map of resolution lower than the resolution of the input image, called lower resolution map;- oversampling (E25) of said at least one feature map of lower resolution using an oversampling artificial neural network (UP) at least defined by a set of oversampling parameters (WUc k) to obtain an oversampled map at the resolution of said at least one image;- generation (E26) of said at least one image from said at least one oversampled map;- updating (E28, E22) of at least one value of one of said feature maps of said group and / or of at least one oversampling parameter, as a function of a coding performance measurement,- a coding step (E24, E27) of said group of feature maps (FMci) and of said set of oversampling parameters (WUc k ). Coding method according to claim 1, characterized in that said generation (E26) of said at least one image uses a synthesis module (PROD) defined by a set of synthesis parameters, capable of generating a sample of the image to be decoded from data extracted from said at least one oversampled characteristic map at positions defined by that of said sample in the image. Coding method according to claim 1 or 2, characterized in that the method comprises a step of coding said set of synthesis parameters. Method for decoding at least one image, from a binary stream comprising a plurality of samples (Pd n ) to be decoded, comprising the following steps: - decoding (E31) of a group of characteristic maps (FMd i) representative of the current image, said group comprising at least one map of characteristics of resolution lower than the resolution of the input image, called lower resolution map;- obtaining (E33) a set of oversampling parameters (WUck) representative of at least one neural network (UP'), called oversampling neural network;- oversampling (E34) of said at least one map of characteristics of lower resolution using said at least one oversampling network to obtain an oversampled map at the resolution of said at least one image;- generation (E35) of said at least one image from said at least one oversampled map. Decoding method according to claim 4, characterized in that at least part of the set of oversampling parameters (WUck) is decoded from data of said binary stream. Decoding method according to claim 4 or 5, characterized in that at least part of the set of oversampling parameters (WUck) is obtained from predetermined values. Decoding method according to one of claims 4 to 6, characterized in that at least part of the set of oversampling parameters (WUck) is decoded as a function of reference parameters. Decoding method according to one of claims 4 to 7, characterized in that the set of oversampling parameters (WUck) is representative of a plurality of neural networks (UP'1, UP'2, UP'3,UP'4) de suréchantillonnage, un réseau de neurones de suréchantillonnage étant utilisé pour suréchantillonner au moins une carte de caractéristiques de résolution inférieure du groupe de cartes de caractéristiques. Decoding method according to one of claims 4 to 8, characterized in that an oversampling neural network is used to oversample at least two lower resolution feature maps of the group of feature maps. Decoding method according to one of claims 4 to 9, characterized in that an oversampling neural network is used to oversample at least one already oversampled map of the group of feature maps. Decoding method according to one of claims 4 to 10, characterized in that said at least one lower resolution feature map is oversampled in at least two dimensions, and in that a separate oversampling is used for each dimension. Decoding method according to one of claims 4 to 11, characterized in that said generation of said at least one image uses a synthetic neural network (PROD') capable of generating a sample of the image to be decoded from data extracted from the oversampled characteristic maps at positions defined by that of said sample in the image. Device for coding at least one image (I(P n)) comprising a plurality of samples (P n ) to be coded, characterized in that said device is configured to implement:- a construction step, comprising the sub-steps of:- construction (INIT, MAJ) of a group of characteristic maps (FM i ) comprising at least one feature map of resolution lower than the resolution of the input image, called lower resolution map;- oversampling (UP) of said at least one feature map of lower resolution using an artificial neural network of at least oversampling (UP) defined by a set of oversampling parameters (WUc k) to obtain an oversampled map at the resolution of said at least one image;- generation (PROD) of said at least one image from said at least one oversampled map;- updating (EVAL, MAJ) of at least one value of one of said feature maps of said group and / or of at least one oversampling parameter, as a function of a coding performance measurement;- a coding step (FMC, NNC) of said group of feature maps (FMi) and of said set of oversampling parameters (WUc k ). Device for decoding at least one image, from a binary stream, comprising a plurality of samples (Pd n ) to be decoded, characterized in that said device is configured to implement:- decoding (FMD) of a group of characteristic maps (FMd i) representative of the current image, said group comprising at least one map of characteristics of resolution lower than the resolution of the input image, called lower resolution map;- obtaining (NND) a set of oversampling parameters (WUck) representative of at least one neural network (UP'), called oversampling neural network;- oversampling (UP') of said at least one map of characteristics of lower resolution using said oversampling network to obtain an oversampled map at the resolution of said at least one image;- generation (PROD') of said at least one image from said at least one oversampled map. Computer program comprising instructions for carrying out the steps of an encoding or decoding method according to claim 1 or 4 when said program is executed by a computer.

Images