Method and device for encoding and decoding an image sequence - Patents.com

Abstract

The present invention relates to a method and device for encoding and decoding a sequence of at least two images comprising a plurality of samples, the decoding method comprising the steps of: decoding a first group of feature maps; decoding a set of parameters representative of a neural network; for a sequence of samples, referred to as current samples, of each of the images of the sequence to be decoded, associated with a position in each of the images, constructing a feature vector from the feature maps of the first group based on the position of the current sample; processing the vectors by an artificial neural network defined by the decoded parameters to deliver a sequence of vectors each representing a current sample.

Description

[Technical Field]

[0001] The present invention relates to the general field of encoding sequences of digital images, and more particularly to the compression of digital video. [Background technology]

[0002] Digital video is typically source-encoded to be compressed in order to limit the resources required for its transmission and / or storage. There are many coding standards, such as those of the ITU / MPEG organization (H.264 / AVC, H.265 / HEVC, H.266 / VVC, etc.) and their extensions (MVC, SVC, 3D-HEVC, etc.).

[0003] An image is generally encoded by dividing the image into a number of rectangular blocks and encoding these blocks of pixels in a given processing sequence. In existing video compression techniques, the processing of a block typically involves prediction of the pixels of the block performed using previously encoded and then decoded pixels present in the image being encoded, in which case it is referred to as "intra-prediction," or prediction of the pixels of the block performed using a previously encoded image, in which case it is referred to as "inter-prediction." This use of any spatial and / or temporal redundancy avoids transmitting or storing values ​​of the pixels of each pixel block by representing at least some of the blocks using residuals that represent the difference between predicted values ​​of the pixels of the block and the actual values ​​of the pixels of the predicted block.

[0004] Video formats are constantly evolving in search of higher compression ratios and becoming more predictable to accommodate a wide variety of expected formats and communication networks, making conventional encoding and decoding algorithms very complex.

[0005] In addition to these traditional methods proposed by compression standards (MPEG, ITU), there is a trend towards the development of methods based on artificial intelligence, in particular neural intelligence.

[0006] Some of these neural approaches can be seen as simple extensions of the concepts of compression technique competition mentioned above, such as prediction mode competition and video coding transformation.

[0007] Another approach uses the "autoencoder" concept. An autoencoder is an artificial neural network-based learning algorithm that allows for the construction of new representations of a dataset. The autoencoder architecture consists of two parts: an encoder and a decoder. The encoder consists of a set of layers of neurons that process data to construct new representations, called "encoded" representations, also known as "latent representations." The decoder's neural layers then receive these representations and filter them to attempt to reconstruct the original data. The difference between the reconstructed data and the initial data allows for measurement of any errors introduced by the autoencoder. Training involves modifying the autoencoder's parameters to reduce the measurement error, measured on various samples of the dataset. The performance capabilities of such autoencoder-based systems come at the expense of a significant increase in memory footprint and complexity compared to traditional approaches, such as those proposed by compression standards. They can have millions of parameters and require up to a million MAC (multiply-accumulate) operations to decode a single pixel. This makes such decoders significantly more complex than traditional decoders and hinders the adoption of learning-based compression.

[0008] Recently, a simple neural network-based image coding technique was described in (Non-Patent Document 1). The proposed coding technique involves adjusting a neural network to an image and quantizing and transmitting the network weights. When decoding, the neural network is evaluated at each pixel location to reconstruct the image. Nevertheless, such techniques remain inefficient in terms of compression and require each image in the video to be coded independently. [Prior art documents] [Non-patent literature]

[0009] [Non-Patent Document 1] “Compression with Implicit Neural representations” by Emilien Dupont et al.,(arXiv:2103.03123) Summary of the Invention [Problem to be solved by the invention]

[0010] Therefore, there is a need for a solution to simply and efficiently encode / compress a sequence of images. [Means for solving the problem]

[0011] The object of the invention is a method for encoding a plurality of images according to claim 1 and a method for decoding a plurality of images according to claim 8.

[0012] Within the meaning of the present invention, the terms "decoding" or "encoding" are understood to mean operations that involve representing a set of samples or pixels in a compact form, for example carried by a digital bitstream. Decoding is understood to mean operations that involve processing a digital bitstream in order to render decoded samples.

[0013] The term "sequence of images" is understood to mean, for example in the case of video, a plurality of two-dimensional images ordered in time. According to another example, the images can be views of the same scene represented as multiple views. According to another example, the images can be a plurality of temporal multi-view images (immersive video).

[0014] The term "sample" is understood to mean a value sampled from an image of a sequence. Sampling a signal generates a sequence of discrete values ​​called samples. In the case of an image signal, a sample is called a pixel, which can be, for example, a color pixel conventionally represented by a triplet value, e.g., (R, G, B) or (Y, U, V). The location of a sample is identified by its coordinates on the horizontal (x) and vertical (y) axes in the image. A sequence of samples refers to a series of samples that are considered to be at the same coordinates in each series of images of an image sequence.

[0015] The term "feature map" is understood to mean a sequence of images containing multiple variable data, also called values, e.g., real numbers, i.e., an abstract representation of at least two images. These maps are also called "latent representations."

[0016] The expression "transforming a feature map" is understood to mean the application of a mathematical operation that allows the values ​​of a first map to be converted into values ​​of a second map. The first map, called a map of the first group, is intended for encoding and can be any type of map. The second map, called a transformed map or a map of the second group, has the same resolution as the input image, i.e., contains the same number (N) of values ​​as the input image (or the signal to be decoded) contains samples. The transformation can include, for example, interpolation, upsampling, filtering, quantization, Fourier transformation, etc.

[0017] The expression "data feature vector constructed from feature maps as a function of position" is understood to mean a vector composed of one or more, preferably discrete, elements or data, the data being constructed from feature maps at a position determined by the position of the sample being processed in the image. This feature vector is one input to the neural network. In the case of an image, such a vector may be constructed, for example, from multiple values ​​sampled in each of the feature maps at the same x and y coordinates as the sample being encoded (or decoded). Once sampled from the feature maps, these values ​​may be processed, for example by quantization, filtering, interpolation, etc., to form a feature vector before entering the neural network.

[0018] The term "neural network" or "synthetic neural network" is understood to mean neural networks such as convolutional neural networks, multi-layer perceptrons, LSTM (long short-term memory), etc. A neural network is defined, for example, by multiple layers of artificial neurons and a set of activation, weighting and summation functions (e.g., one layer can calculate y=f(Ax+b), where y and b are vectors of dimension N, x is a vector of dimension M, A is a matrix of dimension MxN, and f is the activation function).

[0019] The expression "parameter of a neural network" is understood to mean one of the values ​​characterizing a neural network, such as a weight (filter coefficient, weighting, bias, value influencing the operation of nonlinearities) associated with one of the neurons.

[0020] The expression "processing using a neural network" is understood to mean applying a function represented by a neural network to an input feature vector to generate an output vector representing the samples to be encoded (or decoded), which output vector may include one or more data representing the samples.

[0021] The term "performance measure" should be understood to mean a measure between at least one value of a sample to be coded and the decoded value of said sample. The measure can, for example, evaluate distortion or perceptual error. The performance measure can be performed on one sample or on multiple samples (e.g., the current sample or the current image, etc.). The measure can also include a measure of the bit rate, in particular the bit rate associated with the coding of the neural network and / or the coding of the first group of feature maps. The measure can also be a joint measure of bit rate and distortion, achieved by weighting them. As is well known in the art, the value of this measure is generally minimized until a target value is reached.

[0022] The term "construction step" is understood to mean a step aimed at constructing parameters that represent the image before it is actually coded. The construction sub-step can be repeated as many times as necessary to obtain an acceptable performance measure.

[0023] In general, the steps of the encoding or decoding methods should not be interpreted as relating to the notion of temporal succession, in other words the steps may be performed in a different order than indicated in the encoding or decoding independent claims or even simultaneously.

[0024] The encoding method according to the invention constructs parametric encoding parameters from a sequence of input images by training a neural network with feature vectors associated with the position of the samples to be encoded in each input image. These feature vectors are constructed from feature maps which may have the resolution of the input images or a lower resolution. During training or construction, the values ​​of the neural network parameters or feature maps are updated as a function of a performance measure, for example of bitrate distortion type. Once the training is over, i.e. the obtained performance measure is satisfactory, the actual encoding of the neural network parameters and feature map values ​​can be performed and stored or transmitted to a decoder.

[0025] Advantageously, the training process allows for the refinement of the neural network parameters and / or the feature map values ​​until a suitable representation in terms of performance is obtained, e.g., until a desired balance is achieved between the generated bit rate and the distortion suffered by the input image. The training of the feature map values ​​and the neural network parameters can be performed jointly. Advantageously, the coding method according to the invention allows for efficient compression of image signals by exploiting not only intra-image spatial redundancy, but also inter-image redundancy between images in a sequence, whether it be a sequence of video images, or a plurality of multi-view images, a series of medical images, etc.

[0026] Advantageously, the decoding method is simple, as it is only necessary to decode the feature maps and neural networks to reconstruct a decoded version of the sequence of images.

[0027] Such neural networks can advantageously have a very simple structure with a small number of parameters.

[0028] According to an embodiment of the encoding or decoding method, the method comprises a step of transforming the first group of feature maps to obtain a second group of feature maps having a resolution of the images of the sequence, the method being characterized in that the feature vector is constructed from the feature maps of the second group.

[0029] Advantageously, according to this embodiment, the feature maps are divided into two groups, one reserved for the extraction of feature vectors and the other reserved for encoding. It is therefore possible to separate two methods with different objectives: the first group of maps to be encoded (or decoded) must be compressed as efficiently as possible, while the second group of maps must be able to facilitate the process of extracting and building feature vectors.

[0030] According to one variant, at least one of the feature maps of the first group has a lower resolution than the image to be encoded (or decoded), and the transformation operation involves upsampling. Advantageously, according to this embodiment, at least one of the feature maps of the first group to be encoded (or decoded) contains fewer values ​​than it would have with the resolution of the image, so that the compression of the feature maps is more efficient. For example, one of the feature maps in the first group may have a resolution half the number of samples contained in the input image, i.e. it contains half the number of x and y values, i.e. it contains a quarter of the number of values ​​in total of a feature map with the resolution of the image. In contrast, the feature map of the second group, corresponding to the transformation of this map of the first group, has the same resolution as the image. The transformation in this case therefore involves at least one upsampling operation in order to obtain in this transformed map the same number of values ​​as the number of samples contained in one of the input images (or the image to be decoded).

[0031] At least one of said feature maps of the first group has the same resolution as the image to be encoded (or decoded).

[0032] Advantageously, according to this embodiment, at least one of the feature maps has the same resolution as the input image to be coded (respectively decoded), allowing high fidelity and matching to the details of the initial resolution of the images of the sequence. According to one embodiment, in this case, the transformation preserves the number of values ​​of the transformed feature map. In this case, according to one embodiment, the transformation preserves the number of values ​​of the transformed feature map and can be simplified to identity (no processing is performed on the values ​​of the first group of maps) or can include filtering operations, quantization operations, Fourier transforms, etc. In encoding, if the feature maps contain, for example, float or real values, quantization is essential for the correct operation of the system. They need to be quantized before being coded and / or input to the neural network. In contrast, in decoding, dequantization is not always necessary, depending on the embodiment.

[0033] - constructing said feature vector comprises the sub-step of extracting a value from said at least one feature map at the same position as the current sample in the encoded (or decoded) image.

[0034] Advantageously, to construct the elements of the feature vector, it is possible to extract values ​​from the feature maps of the first or second group at the same positions as the samples in the images of the sequence (input images in the case of encoding, decoded images in the case of decoding). This method is easy to implement. For example, if J input feature maps with the same resolution as the images of the sequence are available, a feature vector of J elements can be directly constructed by simple extraction of the map values ​​at the coordinates of the current sample (same abscissa and same coordinate in the feature maps).

[0035] said construction of the feature vector comprises the following sub-steps: - extracting a plurality of values ​​of the feature map of the first group as a function of the position of the current sample; - processing said extracted values ​​to obtain a feature vector.

[0036] Advantageously, according to this embodiment, the feature vectors are extracted from feature maps having a lower resolution than the image to be encoded (or decoded), in particular before undergoing processing, which may be optional. Such processing may correspond, for example, to quantization, scaling or filtering of the extracted data. In the case of encoding, quantization is essential for the proper operation of the system if the feature maps contain, for example, floating or real values, which need to be quantized before being encoded and / or input to the neural network. In contrast, in decoding, dequantization is not necessary depending on the embodiment.

[0037] The method includes constructing a group of additional feature maps, from which the feature vector is also constructed. Advantageously, these additional maps of a third group, constructed identically in the encoder and decoder, are not stored or transmitted to the decoder, nor are they decoded by the decoder, thus allowing additional data to be used to improve compression without reducing the bit rate. For example, the additional data may include coordinates, causal data available in the maps of the first or second group, data relating to other images already processed by the encoder or decoder, etc.

[0038] According to one variant, the additional map contains values ​​representative of the temporal distance between images in the video.

[0039] - encoding (respectively decoding) said first group of feature maps comprises the sub-step of entropy coding (respectively decoding), which advantageously allows to use any redundancy in the images that may remain in the feature maps, and therefore the images of the sequence are compressed more efficiently.

[0040] Correlatively, further objects of the invention are a device for encoding a sequence of images and a device for decoding a sequence of images.

[0041] The features and advantages of the encoding method or the decoding method apply equally to the encoding and decoding device according to the invention, and vice versa.

[0042] A further object of the invention is a computer program on a storage medium, which program is capable of implementing a computer or control device according to the invention, the program comprising instructions designed to implement the corresponding method, which program can use any programming language and can be in the form of source code, object code, an intermediate code between source code and object code, such as a partially compiled form, or any other desired form.

[0043] The present invention also relates to a computer-readable information medium or storage medium containing instructions for the computer program described above. The information or storage medium may be any entity or device capable of storing a program. For example, the medium may include a storage means such as a ROM, e.g., a CD-ROM or a microelectronic circuit ROM, or even a magnetic storage means, e.g., a floppy disk or hard disk, a DNA sequence, or a flash memory. Furthermore, the information or storage medium may be a transmissible medium, such as an electric or optical signal, which may be routed via an electric or optical cable, by a radio wave link, by a wireless optical link, or by other means.

[0044] The program according to the present invention can be downloaded via the Internet.

[0045] Alternatively, each information medium or storage medium may be an integrated circuit incorporating a program, the circuit being designed to perform or to be used for performing the method according to the invention.

[0046] Further characteristics and advantages of the invention will become apparent from the following description, with reference to the accompanying drawings, which show non-limiting embodiments. [Brief explanation of the drawings]

[0047] [Figure 1] 1 shows a schematic representation of a coding device for use within the scope of the present invention; [Figure 2] 1 shows a schematic representation of a decoding device for use within the scope of the present invention; [Figure 3] 1 shows an example of a synthetic artificial neural network used within the scope of the present invention during encoding and decoding. [Figure 4] 2 is a flowchart illustrating an example of an encoding method that may be implemented by the encoding device of FIG. 1. [Figure 5] 1 shows a diagram of an encoding method used in one embodiment of the present invention; [Figure 6]3 is a flowchart illustrating an example of a decoding method that may be implemented by the decoding device of FIG. 2. [Figure 7] 1 shows a diagram of a decoding method used in one embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0048] FIG. 1 shows diagrammatically the encoding device ENC.

[0049] This encoding device ENC comprises a feature map generation module GEN, an information transformation module SE, a data extraction module XTR, a processing and quantization module TT, a module corresponding to an artificial neural network MLP, a neural network coding module NNC, a feature map coding module FMC, a performance function evaluation module EVAL, an update module MAJ and an optional additional feature map generation module FME.

[0050] The encoding device ENC can be implemented by an electronic device including a processor and a memory (not shown), which can then generate each of said modules through the interaction of the processor with computer program instructions stored in said memory and designed, when executed by the processor, to carry out the functions of the module in question, in particular as described below.

[0051] The encoding device ENC shown in FIG. 1 receives as input a number of samples P Vn Including, I(P Vn ) to be coded. The index v denotes the image number in the sequence of V images. The index n denotes the pixel number in an image of N pixels. Vn) can represent two-dimensional images and sequences of multiple two-dimensional images (video sequences, stereoscopic components, multi-view components, medical image series, etc.). In one embodiment, the images of the sequence are represented by at least one two-dimensional representation, such as a matrix of pixels, where each pixel includes a lightness component and at least one saturation component, as a vector or a variant of red (R), green (G), and blue (B) components. The location of each pixel is determined by its x and y coordinates in the image (x n and y n ) In one embodiment, the sequence is a sequence of grayscale images represented by a two-dimensional representation such as a matrix of pixels, where each pixel has a grayscale or intensity component. In this case, the vector representing the pixel is simplified to one component.

[0052] As will be explained in more detail below with reference to FIGS. 4 to 7, the feature map generation module GEN generates a feature map from an input image I(P vn ) sequence, FM i The optional module FME is configured to generate a first group of M feature maps, denoted as FME. l One or more additional maps (L) denoted as

[0053] In one embodiment, the module SE transforms the first group of feature maps to generate a second group of feature maps having the same resolution as the images of the input sequence. The optional module SE can quantize the values ​​of the maps of the first group by using a quantizer Q to generate an ordered collection of quantized values. Note that quantizing a value refers to matching this value with a member of a discrete set of possible code symbols. For example, the set of possible code symbols may consist of integer values, and the quantization system simply rounds the actual value to an integer value. According to another example, quantization involves multiplication by a given value followed by rounding. The module SE then transforms at least one value of the feature maps, for example by upsampling, interpolation, filtering, etc. At the end of the transformation, the transformed feature maps of the second group have the same resolution as the images of the input sequence. Advantageously, according to this embodiment, the encoded feature maps may have a lower resolution than the resolution of the images to be encoded, while the maps of the second group used to construct the feature vector have the same resolution as the sequence of images, thereby facilitating the extraction of values.

[0054] In one embodiment, the module SE is not present, in which case the values ​​used to construct the feature vector are extracted from the first group of feature maps. The module XTR is n For a sequence of, the feature map FM i (and / or FMSi and / or FME according to one of the above-mentioned embodiments) l ) as a function of its coordinate in each input image. For example, the coordinate (x n ,y n ) sample P 1n and the coordinates of image number 2 (x n ,y n ) sample P 2n If you intend to encode the current pixel coordinate (x n ,y n) to extract the value at the position imposed by

[0055] In one embodiment, the extracted values ​​are represented by a vector Z n Form Z n is a J-tuple, i.e., a set of J elements or data z i The index n is the current sample or pixel P' vn refers to the feature vector of

[0056] In one embodiment, the optional module TT calculates the vector Z n The module TT processes the extracted values ​​to generate a set of feature maps. The module TT can quantize the data extracted from the set of feature maps. The processing can also include other operations such as filtering, scaling, etc. In particular, if the module SE is not used and the feature maps of the first group have a lower resolution than the images of the sequence, the module TT can take into account the coordinates of the values ​​in the maps with lower resolution.

[0057] Note that at least one of the modules SE or TT must quantize the feature maps.

[0058] The module MLP outputs the sample P' to be coded. vn To generate a sequence of vectors representing a sequence of k and as input, a vector Z n or a neural network capable of processing a J-tuple. According to one embodiment, the neural network is an MLP or Multi-Layer Perceptron, consisting of an input layer adapted to the input format (J-tuple), optionally one or more hidden layers, and an output layer adapted to the output format of an output vector, generally a plurality of vectors each containing A elements. According to one embodiment, A is equal to 3 and the output vector is a representation of N pixels P' of the image v to be coded and then decoded. vn is one (R,G,B) triplet.

[0059] The module NNC calculates the neural network, in particular its parameters W k During the coding training or construction process, i.e. as long as the step of evaluating the performance remains unsatisfactory, the module NNC simulates the coding, followed by the decoding, the results of which are sent to the evaluation module. The evaluation module updates the values ​​of the parameters as a function of the results of the performance measurements carried out by the module EVAL. Subsequently, the parameters W of the neural network are k The actual encoding of Wc is performed. k As is known, the coding simulation may be identical to or may approximate the actual coding.

[0060] Module FMC, Map FM i , i.e., encode the values ​​of the feature maps of the first group (any additional maps FME l and optionally excluding the second group of maps resulting from the upsampling by the module SE). During the coding training or construction process, i.e. as long as the step of evaluating the performance remains unsatisfactory, the module FMC simulates the coding, followed by the decoding, the results of which are sent to the evaluation module. Subsequently, the maps FM i The encoded map is then used to i As is known, the encoding simulation may be identical to or approximate the actual encoding. The encoding module optionally quantizes the latent representations of the values ​​of the first group of maps by generating an ordered set of quantized values ​​using a quantizer. The encoding module then compresses the quantized data, for example, using entropy coding.

[0061] The module EVAL performs the evaluation and minimization of the coding performance. The evaluation function is, for example, of the bit rate distortion type. The minimization can be performed via gradient descent or any other method within the capabilities of a person skilled in the art.

[0062] The module MAJ encodes the map FM as a function of the results of the performance function. i Update the value of

[0063] FIG. 2 shows diagrammatically the decoding device DEC.

[0064] The decoding device DEC in Fig. 2 receives as input M feature maps FMc i The first group of coded data organized in layers (also called layers FM) and the coded parameters Wc of the neural network. k The decoding device DEC receives the samples Pd of the V images of the sequence to be coded. vn Decode the N sequences of

[0065] This decoding device DEC comprises a neural network decoding module NND, a data extraction module XTR', an inverse transformation module SE', a processing and inverse quantization module TT', a module MLP' corresponding to the neural network, and an additional feature map generation module FME'. The decoding device DEC receives a plurality of decoded samples Pd vn Each contains I(Pd vn ) and outputs a sequence of decoded images denoted as

[0066] The map decoded by M modules FMD is FMd i The parameters decoded by the module NND are Wd k It is shown as follows.

[0067] The decoder is FME' l and the additional map FME generated by the encoder. l It is also possible to generate one or more additional maps, numbered L, that are identical to

[0068] In one embodiment, the module SE' is iThe decoded feature maps FMd are then used to generate a second group of feature maps with the same resolution as the image being decoded, denoted as i The decoder transforms the first group of feature maps. Module SE′ optionally performs an inverse quantization corresponding to the quantization performed in the encoder. Inverse quantization is not necessary if the quantizer Q of the encoder simply rounds the actual values ​​received. Inverse quantization is also not necessary if the neural network is able to take into account the quantization of its input data. In other cases, the decoder performs the inverse operation of quantizer Q. Module SE′ then transforms the values ​​of the feature maps in a manner similar to that performed by the encoder, including, for example, upsampling, interpolation, filtering, etc. Once the transformation is complete, the transformed feature maps of the second group have the same resolution as the images of the sequence to be decoded.

[0069] In one embodiment, module SE' is not present, in which case the values ​​used to construct the feature vector are extracted from the first group of feature maps.

[0070] The module XTR' is identical to the module XTR of Figure 1. The module XTR' receives the current sample Pd vn For a sequence of M feature maps FMd i (According to one of the above-mentioned embodiments, the FMS' i and / or FME' l ) as a function of the coordinates of the samples in each image being decoded. In one embodiment, J=M. In one embodiment, J=M+L.

[0071] In one embodiment, the extracted values ​​are represented by a vector Zd n Form Zd n is a J-tuple, i.e., J elements or data zd i Includes:

[0072] In one embodiment, the optional module TT' is nThe extracted values ​​are processed to generate a set of feature maps. The TT module can perform inverse quantization of the extracted data from the set of feature maps. The processing can also include other operations such as filtering, scaling, etc., similar to those performed by an encoder.

[0073] The module MLP' has K parameters Wd k and outputs a set of samples Pd each containing A elements. vn To generate a sequence of vectors representing a sequence of n or a neural network capable of processing J tuples. According to one embodiment, A=3 and the output vector is a vector of N pixels Pd vn The (R, G, B) triplet of the MLP module has the same structure as the MLP module, and its parameters are the parameters W k are identical if the encoding is lossless, or different if the encoding is lossy.

[0074] Sample Pd vn When all sequences of are decoded, V images I(Pd vn ) a sequence of reconstructed images is available.

[0075] The decoding device DEC may be implemented by an electronic device including a processor and a memory (not shown), which is then able to generate each of the above-mentioned modules through interaction of the processor with computer program instructions stored in said memory and designed, when executed by the processor, to carry out the functions of the module in question, in particular as described below.

[0076] FIG. 3 shows an example of a synthetic artificial neural network that may be used within the scope of the present invention during encoding and decoding.

[0077] The composite artificial neural network MLP used for encoding and the composite artificial neural network MLP' used for decoding are defined by the same structure, e.g., including multiple layers of artificial neurons, and a set of weights and activation functions associated with each of the artificial neurons of the network in question.

[0078] A vector representation of the current sequence of samples (feature map FM i / FMS i and FME l or FMd i / FMS' i and FME' l Vector Z obtained from n or Zd n ) is applied to the input (i.e., input layer) of the synthetic artificial neural network MLP or MLP'. The artificial neural network receives the reconstructed (during encoding) or decoded (during decoding) samples, according to one embodiment, a plurality of vectors P' representing the constituent color components (R, G, B) of the color pixels of the sequence of images. vn or Pd vn In Figure 3, the sequence of images corresponds to two images, each sequence of samples contains two samples, and the corresponding vector is Pd 1n and Pd 2n It is shown as follows.

[0079] All reconstructed pixels in each image in the output sequence are concatenated to form a sequence of decoded images (in a decoder) or reconstructed images (in an encoder).

[0080] In the encoder, a synthetic artificial neural network (MLP) optimizes the current image I(P vn ) and its output representation I(P' vn ) is trained on a sequence of images to minimize the difference between the

[0081] Once trained, the network's parameters are coded either reversibly (the neural network MLP' is identical to the MLP) or irreversibly (the network MLP' can be slightly different from the MLP).

[0082] FIG. 4 is a flowchart illustrating an example of an encoding method that can be implemented by the encoding device of FIG.

[0083] According to this embodiment, the sequences are sequences of two-dimensional images, and therefore each sequence of samples to be coded is coded with a respective image I(P vn ) at the coordinate (x n ,y n ) pixel P vn This is a set of:

[0084] The encoding is done in two main phases.

[0085] In the first phase, called the construction phase, the input sequence I(P vn ) to optimize the overall cost function. i and parameter W k Learning is performed to determine the values ​​of the parameters of the neural network MLP and the feature map FM. i This is followed by updating the value of . As known in the art, the cost function may be of the bit rate distortion type or of the bit rate, distortion or perceptual type. To measure the bit rate R, the map FM i The encoding of the parameter W must be simulated and then the associated encoding bitrate (size of stream B1) must be measured. k The encodings of the parameters W are not simulated because their influence is smaller than that of the feature maps. kis also simulated and the associated bit rate (size of stream B2) is measured. To measure the distortion D, it is necessary to simulate the encoding and decoding of at least a part of the sequence of images, so that the pixel P' resulting from the simulation of the encoding then decoding of the sample of index n is vn and then take an image I(P vn ) and the encoded and then decoded sequence I(P' vn ) and the corresponding part of

[0086] Then, during the second phase, called the encoding phase, the map FM i and parameter W k is encoded before being transmitted or stored, and the encoded value FMc i and Wc k They generate the input sequence I(P vn ) to form a compressed representation of

[0087] The steps of the method according to one embodiment of the present invention will now be described.

[0088] During step E20, a plurality of N samples P n The input sequence I(P) to be coded contains at least two images, vn ) are sent as input to the method. According to one embodiment, these images are temporal images of a video sequence. According to one embodiment, these images are images of a series of images, for example medical images. According to one embodiment, the images are multiview or 3D components of images or a sequence of multiview or 3D images.

[0089] During step E21, a first group of M maps FM i Next, the parameter W of the neural network MLP is initialized. k and Map FM i The value of must be optimized during the construction phase.

[0090] According to one embodiment, the map FM i is the input sequence I(P vn ) and therefore each has the same resolution as the encoded sample P vn Contains as many values ​​N as there are

[0091] According to one embodiment, the map FM i The resolution of the input sequence I(P vn ) or less and therefore contains, for at least one of them, a number N' of values ​​to be coded that is less than N. According to one variant, the first map FM i has the resolution of the image, and each subsequent map has half the resolution of the previous map.

[0092] According to one embodiment, the plurality of maps FMi are arranged to represent the input sequence I(P vn ) has the same resolution less than the resolution of

[0093] According to one embodiment, the map FM i is the transformed feature map FMS i The first group of maps is then transformed to provide a second group of maps. In this embodiment, the feature vector is preferably extracted from the transformed map of the second group, rather than directly from the map of the first group. Thus, in this embodiment, the feature vector is extracted indirectly from the map of the first group. The map of the second group is not encoded or transmitted, but is used only to construct the feature vector.

[0094] According to one embodiment, the map FM i is initialized to a predetermined constant value.

[0095] According to another embodiment, the feature map is initialized with a set of random real numbers.

[0096] According to one embodiment, one or more maps FME forming an additional group of L additional feature mapsl are generated and added to the first group. They are used to construct the feature vector but are not stored or transmitted.

[0097] First group feature map FM i is subsequently updated or improved during a step E22 by updating the module MAJ of the encoder during the learning phase.

[0098] During step E23, the first group of maps FMi l are coded by the module FMC of the encoder. During the construction phase, this operation is a coding simulation. During the coding phase, this operation is the actual coding, the coded values ​​forming the stream B1. The simulation may be identical to the actual coding, but may also be different (for example simplified). This coding may use any known technique aimed at compressing the values ​​of the map.

[0099] In one embodiment, the map FM i are coded in the order (FM1, FM2, ..., FM4), with the variables of each map coded in a predetermined order, for example lexicographical order. Each map is then subjected to entropy coding, which generates a compressed stream B1, the bit rate of which is subsequently measured during step E29.

[0100] During step E24, according to one embodiment, a first group of M maps FM i The second group is transformed by the module SE and maps FMS with the resolution of the images of the input sequence. i Generate.

[0101] According to one embodiment, M map FMSs i is generated.

[0102] According to one embodiment, each map FM i Map FMS i is converted to

[0103] According to one embodiment, at least one map FM i has a resolution lower than the resolution of the images of the sequence to be coded, and the conversion operation includes upsampling, so that the converted map FMS i contains the same number of samples as the number of images in the sequence. Upsampling is done by mapping FMS to achieve the image resolution of the input sequence. i It can be simple (by replicating the nearest neighbor) or it can involve interpolation (linear, polynomial, filtering, etc.).

[0104] During step E25, the converted map FM i or optionally FMS i and optionally additional maps FME i The value of is extracted by the module XTR. This extraction is performed for the current sample P of the input image. vn The sequence of coordinates (x n ,y n ) or may be performed as a function of the resolution of the map in question. The sequence of samples to be coded is processed sequentially, for example from n=1 to n=N.

[0105] According to one embodiment, the feature vector Z n is generated directly from this extraction.

[0106] According to one embodiment, during step E26, the feature vector Z n is the coordinate of the input image (x n ,y n ) for each sample P vn Regarding the sequence, map FM i or FMS i and optionally FME l The processing is carried out by the module TT from the extracted values ​​or the component vector Z nThe processing may include other operations such as filtering, scaling, application of any function, preferably a monotonic function, etc.

[0107] In one embodiment, Z n is the input map FM i or FMS i (and optionally FME l ) contains as many values ​​as there are in J, where J=M(+L).

[0108] In one embodiment, Z n is the current pixel P as explained with reference to Figure 5. n coordinates (x n ,y n ) Map FM i or FMS i (and optionally FME l ) values ​​form a J-tuple (z1,z2,...,z J )

[0109] In one embodiment, Z n is the map FM in coordinates which may vary depending on the map i (and optionally FME l ) is a J-tuple constructed from values ​​sampled from the map FM i (and / or FME l ) have different resolutions due to being downsampled, the coordinates are adapted (by scaling) to fit the resolution of each map.

[0110] In one embodiment, Z n is a method for generating a map FM by applying a process to one or more values ​​of the map, for example, by filtering neighboring values ​​of a targeted value in the map. i (and optionally FME l ) is a J-tuple constructed from values ​​sampled from the input signal. For example, a map FM i In this map FM i or FME lVector Z related to n The final value of element i of (z i To get the coordinates (x n ,y n ), (x n -1,y n ), (x n ,y n -1) and (x n -1,y n −1) and process these values ​​(filter, average, interpolate, etc.). According to another example, it is possible to map FM i So, this map FM i or FME l Vector Z related to n The final value of element i of (z i ) to get the coordinates (x n / 2,y n / 2), (x n / 2-1,y n / 2), (x n / 2,y n / 2-1) and (x n / 2-1,y n / 2-1) and can process these values ​​(filter, average, interpolate, etc.).

[0111] During step E27, vector Z n is processed by the neural network MLP and the output is the encoded sample P vn , i.e., according to one embodiment, the sample P' vn (encoded and then decoded sample P vn ) to generate an (R,G,B) triplet.

[0112] Neural network structure and parameters W k are initialized, for example, during the first iteration of this step. These parameters are then updated or refined during subsequent iterations of the method, during the construction phase.

[0113] According to one embodiment, the parameters of the neural network are initialized with predetermined values ​​that are known to produce satisfactory results (eg, after training on a corpus of images).

[0114] According to another embodiment, the parameter W of the neural network k is initialized with a set of random values.

[0115] During step E28, the parameters W of the neural network MLP are k is quantized and coded. During the construction phase, this operation is a coding simulation. During the coding phase, this operation is the actual coding, with the coded values ​​forming the stream B2. The simulation may be identical to the actual coding, but it may also be different (for example simplified). For this, any known technique can be used, for example the neural network coding standard proposed in part 17 of MPEG-7, also called neural network representation or NNR. In this case, the coding is performed using a parameter W k Note that the amount of downscaling due to

[0116] During step E29, the performance measures are evaluated.

[0117] For this purpose, the first group of feature maps and the associated coding simulation bit rate (map F i ) and optionally a coding simulation bit rate associated with the parameters of the neural network (parameters W k The measured values ​​are used to simulate the stream B2 by encoding the

[0118] According to one embodiment, the cost function is of the bitrate-distortion type, denoted (D+L*R), for example the squared error measured between the input signal and the decoded image (or the error measured on a subset of the image's samples). According to another example, D is calculated from a perceptual function such as SSIM (Structural Similarity) or MSSSIM (Multi-Scale Structural Similarity). According to one embodiment, R is the simulated bitrate of stream B1, and according to another embodiment, R is the overall bitrate used to code this image, i.e. the sum of the simulated bitrates of B1 and B2. L is a parameter that adjusts the rate-distortion compromise. Other cost functions are possible.

[0119] The performance measure remains unsatisfactory until the cost function reaches a minimum and the method is repeated from step E22. This minimization can be performed via known mechanisms such as gradient descent, and the parameters are updated in step E22 for the feature map values ​​and in step E27 for the network parameters.

[0120] During step EF, training stops when the cost function reaches a minimum. k ) and feature map (FM i ) is available, streams B1 and B2 can be formed from it. According to another embodiment, the updated parameters of the neural network (W k ) and feature map (FM i The actual encoding of the values ​​of Wc ) is performed during this step to generate the encoded parameters Wc forming the streams B1 and B2. k and FMc i Generate.

[0121] Streams B1 and B2 can be concatenated to generate the final stream. According to one embodiment, stream B2 of coded parameters of the neural network is stored or transmitted before stream B so that it can be decoded before stream B1.

[0122] FIG. 5 shows a diagram of the encoding method used in one embodiment of the present invention.

[0123] In this figure, two images I(P 1n ) and I(P 2n ) is coded. In general, a sequence of V images I(P vn ) is applied as input to the method and coding device. In this figure, samples or pixels are processed in two sequences, and two pixels are divided into (x n ,y n ) are sampled from each image, where n varies from 1 to N. (P 1n is the position (x n ,y n ) is sampled from image 1 at P 2n is the position (x n ,y n ) is sampled from image 2 at

[0124] In this embodiment, four generated maps FM i In the preferred embodiment, there are seven maps.

[0125] The first map FM1 has the same resolution as image I and therefore contains N=W×H variables, where W represents the image's width in pixels and H represents its height. The second map FM2 has half the resolution of map FM1 (in each dimension). Each additional map has half the resolution of the previous second map. This structure allows for a reduction in the number of variables in the feature maps, thus facilitating encoding and learning while minimizing encoding costs.

[0126] Map FM2 is upsampled by module SE by a factor of 2 in each dimension in the manner illustrated with reference to Figure 6. Map FM3 is upsampled by a factor of 4 in each dimension and map FM4 is upsampled by a factor of 8 in each dimension.

[0127] Generated Map FMS i is the image I(Pv n ), and therefore each contains W×H values, where W represents the width of the image in pixels and H represents its height (N=W×H).

[0128] According to this embodiment, the layer FM i is quantized by the module SE.

[0129] Other types of structures may be used between maps, for example reduction ratios other than half (such as 1 / 4 or 1 / 3).

[0130] In one variant, shown in dashed lines, there are five feature maps, and an additional map, FME0, is introduced, which is not coded or transmitted. This additional map typically contains data that can assist the network MLP in the task of reconstructing an image. The added map can therefore be one or more maps from the following non-limiting list: - a map containing, for each point, the x coordinate of this point, - a map containing, for each point, the y coordinate of this point, - a map containing the location coding of each point (for example, as explained on the following website: https: / / skosmos.loterre.fr / P66 / fr / page / -K0D65X2X-X), an image separate from the image being processed and capable of providing information about the image to be coded, for example a map representing a previously processed image or sequence of images, A map containing data representing the temporal difference between the images of the video being decoded. For example, if the first and last images of the video are 8 images apart, all samples in the map contain the value 8. a map representing a feature map of an image that is separate from the image being processed and that is capable of providing information about the image to be coded, for example a previously processed map; -Already coded samples of the same map, e.g. a map containing the values ​​of the previous samples in decoding order.

[0131] In this embodiment, the vector Z n is the current pixel P vn coordinates (x n ,y n ) Map FMS i The map FMS is a 4-tuple (z1···z4) formed from the values ​​extracted from i A vector Z consisting of the sampled (quantized) values ​​of n is processed by a neural network MLP to output a sequence of vectors, and according to this example, the samples P to be coded 1n and P 2n The output vector in this embodiment is the encoded and then decoded pixel P' 1n and P' 2n The triplet is the (R,G,B) triplet of the color components (R',G',B') of the two images at the positions (x n ,y n ), the decoded image I(P' 1n ) and I(P' 2n )

[0132] In another embodiment not shown, the vector Z n is the layer FM at the recalculated position as a function of the map size. i , and the extracted values ​​are then optionally processed and quantized after extraction.

[0133] According to a variant shown as a dashed line, the vector Z nis a 5-tuple (z0...z4), where the value z0 is extracted from the additional map FME0.

[0134] FIG. 6 is a flowchart illustrating an example of a decoding method that may be performed by the decoding device of FIG.

[0135] During a step E30, the streams B1 and B2 are extracted from the coded stream BS. The streams B1 and B2 are coded representations of the maps FMc of the first group. i and the coded representation of the parameters Wc k Each includes:

[0136] During step E31, M maps FMd i But the value FMc i This decoding can use any known technique, preferably entropy decoding, similar to that used in the encoder. In one embodiment, the map FMd i are decoded in the order (FMd1, FMd2, ..., FMd4), and the variables of each map are decoded in a predetermined order, for example, lexicographic order.

[0137] According to the embodiment as described for the encoder: -Map FMd i is the sequence I(Pd vn ), i.e. it contains N=W×H values. -Map FMd i is the sequence I(Pd vn ) has a resolution equal to or less than the resolution of the image. -Multiple maps FMd i has the same resolution, which is lower than the resolution of the images in the sequence.

[0138] During step E32, according to one embodiment, one or more maps FME' forming an additional group of L additional feature maps are l is generated to fill the first group. Map FME' lare not decoded, but are generated by the decoder in the same way as they are generated in the encoder. They may typically assist the network MLP' in the task of reconstructing the images of the sequence. The non-limiting list of possible additional feature maps described with reference to FIG. 5 for the encoder also applies in this case.

[0139] During step E33, according to one embodiment, the first group FMd i The second group of maps FMS' has the resolution of the images in the input sequence. i is transformed by module SE to generate

[0140] In particular, according to one embodiment, M maps FMS' i is generated.

[0141] According to one embodiment, each map FMd i Map FMS i is converted to

[0142] According to one embodiment, at least one map FMd i has a resolution lower than the resolution of the images of the sequence to be coded, and the transformation operation is performed to obtain a transformed map FMS' i contains the same number of samples as the images in the input sequence. As on the encoder side, upsampling involves mapping FMS' to achieve the resolution of the images in the input sequence. i This may be simple (nearest neighbor replication) or may involve interpolation (linear, polynomial, filtering, etc.).

[0143] The transformation may optionally include dequantization of the extracted values ​​if necessary, although dequantization is not required.

[0144] During step E34, the transformed map FMd i or optionally FMS' i and optionally an additional map FME'i The value of the current sample P to be decoded of the image of the sequence is extracted by the module XTR'. vn The sequence of coordinates (x n ,y n ) or may be performed as a function of the resolution of the map in question. The sequence of samples to be decoded is processed sequentially, for example from n=1 to n=N.

[0145] According to one embodiment, the feature vector Z n is generated directly from this extraction.

[0146] In particular, in one embodiment, Zd n is the current pixel Pd as shown in FIG. vn coordinates (x n ,y n ) Map FMd i or FMS' i (and optionally FME' l ) values ​​are used to form a J-tuple (z1, z2, , z J )

[0147] According to one embodiment, during step E35, the vector Zd n is the coordinate (x n ,y n ) as a function of the coordinates (x n ,y n ) sample Pd vn For each sequence of the first group, the map FMd i From or second group map FMS' i From and optionally additional groups map FME' i This step is identical to step E26 described for the encoder with reference to Figure 4 and applies to the described embodiment. The extraction is carried out, if necessary, by using the extracted values ​​or the formed vector Zd n This may include inverse quantization of

[0148] During step E36, the parameters Wd of the neural network MLP' k However, the value Wc of stream B2 k For this purpose, any known decoding technique corresponding to the encoding technique used by the encoder can be used. The neural network MLP' is similar to the network MLP, i.e., it has the same structure and the same parameters as the nearest neighbor coding, which may be lossy or lossless.

[0149] According to one embodiment, stream B2 is decoded before stream B1 in order to acquire the neural network before starting to decode the sequence of samples.

[0150] During step E37, vector Zd n is processed by the neural network MLP' to decode the current sample Pd vn According to one embodiment, the sample Pd vn The sample is generated by outputting the (R,G,B) triplet of color components (Rd,Gd,Bd) at the position (x n ,y n ) the decoded image I(Pd vn ) This step is identical to step E27 described for the encoder with reference to FIG.

[0151] When the entire sequence of samples has been processed, the corresponding sequence of decoded images is available.

[0152] FIG. 7 shows a diagram of the decoding method used in one embodiment of the present invention.

[0153] In this figure, two images I(Pd 1n ) and I(Pd 2n ) is decoded. In general, a sequence of V images I(Pd vn) is decoded by the method and the encoding device. In this figure, the samples or pixels are processed in two sequences, and the two pixels are decoded as (x n ,y n ) are decoded and reconstructed from each image at the positions indicated by n, where n varies from 1 to N. (Pd 1n is the position (x n ,y n ) inserted into Image 1, and Pd 2n is the position (x n ,y n ) and inserted into image 2).

[0154] In this embodiment, four maps FMd i In the preferred embodiment, there are seven maps.

[0155] In this embodiment, the first map FMd i has the same resolution as image I and therefore contains W × H variables, where W is the width of the image in pixels and H is its height. l (in each dimension). Each additional map has half the resolution of the previous map. This structure allows us to reduce the number of variables in the feature maps, thus facilitating decoding while minimizing decoding cost.

[0156] Map FM d2 is upsampled by the module SE′ by a factor of 2 in each dimension according to any upsampling method within the capabilities of a person skilled in the art. d3 is upsampled by a factor of 4 in each dimension and mapped to FM d4 is upsampled by a factor of 8 in each dimension.

[0157] Map FMS' i has the same resolution as the image to be decoded and therefore contains W×H values, where W is the width of the image in pixels and H is its height.

[0158] In this embodiment, the vector Zd n is the current pixel Pd vn The sequence of coordinates (x n ,y n ) Map FMS' i The vector Zd is a 4-tuple (z1···z4) formed from the values ​​of n is optionally dequantized and then processed by the neural network MLP′ to produce as output the two decoded samples Pd vn (Pd 1n and Pd 2n ) generates a (R,G,B) or (Y,U,V) triplet for each of the three color components of the image. The triplet (R,G,B or Y,U,V) is the coordinate (x,y) in the color components (Rd,Gd,Bd) or (Yd,Ud,Vd) of the image. n ,y n ) for each decoded image I(Pd 1n ) and I(Pd 2n )

[0159] According to one variant, shown as a dashed line, there are five maps and an additional map FME'0 is introduced. In this embodiment, the vector Zd n is a 5-tuple.

Claims

1. The samples to be coded (Pv ｎ , P １ｎ， , P ２ｎ ) and at least two images (I(P ｖｎ ), I(P １ｎ ), I(P ２ｎ )) a method for encoding a sequence of A construction step comprising: Feature Map (FM ｉ a substep (E21, E22) of constructing a first group of The position (x ｎ , y ｎ ) of each current sample (P ｖｎ , P １ｎ , P ２ｎ ) for a sequence of samples called The current sample (P １ｎ , P ２ｎ ) at the position (x ｎ , y ｎ ) of the first group as a function of the feature map (FM ｉ ) to the feature vector (Z ｎ ) and a substep (E25) of constructing The decoded value of the current sample (P' １ｎ , P' ２ｎ ) to provide a sequence of vectors representing the parameters (W ｋ ) is used to find the vector (Z ｎ a substep (E27) of processing a substep (E22, E27) of updating at least one value of one of the feature maps of the first group and / or at least one parameter of the network as a function of a coding performance measure; a construction step (E21, E22) comprising: The feature map (FM ｉ ) and the first group of parameters (W ｋ ) a step (E23, E28, EF) of encoding the set 10. An encoding method comprising:

2. The feature map (FM ｉ ) to generate feature maps (FMS ｉ ) and obtaining a second group of feature vectors (Z ｎ ) is the feature map (FM ｉ ) the transformed feature map (FMS ｉ 2. A method for encoding a sequence of images according to claim 1, characterized in that the method is constructed from:

3. The feature map (FM) of the first group ｉ 3. The method of claim 2, wherein at least one of the first and second inputs has a lower resolution than the image to be encoded and the transformation operation includes upsampling.

4. The feature vector (Z ｎ ) is constructed by the current sample (P ｖｎ ) and the same position (x ｎ , y ｎ ) in the at least one feature map (FM ｉ , FMS ｉ 4. The method according to claim 1, further comprising the sub-step of extracting the value of

5. The feature vector (Z ｎ ) the construction of - the current sample (P ｖｎ ) at the position (x ｎ , y ｎ ) of the first group as a function of the feature map (FM ｉ ) and a sub-step to extract multiple values ​​of a substep (E26) of processing said extracted values ​​in order to obtain said feature vectors; 2. The encoding method of claim 1, comprising:

6. Feature Map (FME) ｌ 6. The method according to claim 1, further comprising a step (E21, E22) of constructing additional groups of feature maps of said additional groups, and wherein said feature vector is also constructed from said feature maps of said additional groups.

7. Coding method according to any one of claims 1 to 6, characterized in that the coding of the first group of feature maps comprises a sub-step of entropy coding.

8. The decoded samples (P ｖｎ , P １ｎ , P ２ｎ At least two images (I(Pd ｖｎ ), I(Pd １ｎ ), I(Pd ２ｎ )) a method for decoding a sequence of Feature map (FMd ｉ a step (E31) of decoding a first group of The parameter (Wd ｋ ) and The position (x ｎ , y ｎ ) of each current sample (Pd ｖｎ ) for a sequence of samples called The position (x ｎ , y ｎ ) as a function of the feature map (FMd ｉ ) to the feature vector (Zd ｎ ) (E34), The current sample (Pd ｖｎ ) to provide a sequence of vectors each representing a ｋ ) is used to calculate the vector (Zd ｎ ) a step (E37) of processing A decoding method comprising:

9. The decoded feature map (FMd ｉ ) to generate a feature map (FMS′) with the resolution of the image of the input sequence. ｉ ) and obtaining a second group of feature vectors (Zd ｎ ) is the decoded feature map (FMd ｉ ) the transformed feature map (FMS′) of the second group obtained from ｉ 9. The method of claim 8, wherein the decoding is constructed from:

10. The feature map (FMd) of the first group ｉ 10. The decoding method of claim 9, wherein at least one of the first and second inputs has a lower resolution than the image to be decoded and the transformation operation includes upsampling.

11. The feature vector (Zd ｎ ) is constructed by the current sample (Pd ｖｎ ) and the same position (x ｎ , y ｎ ) in the at least one feature map (FMd ｉ , FMS' ｉ 11. A decoding method according to any one of claims 8 to 10, characterized in that it comprises a substep (E34) of extracting the value of .

12. The feature vector (Zd ｎ ) the construction of - the current sample (Pd ｖｎ ) at the position (x ｎ , y ｎ ) as a function of the feature map (FMd ｉ ) and a sub-step to extract multiple values ​​of a substep (E35) of processing said extracted values ​​in order to obtain said feature vectors; 9. The decoding method of claim 8, comprising:

13. Feature map (FME' ｌ 13. The decoding method according to claim 8, further comprising a step (E32) of constructing additional groups of feature maps, and wherein the feature vector is also constructed from the feature maps of the additional groups.

14. Decoding method according to any one of claims 8 to 13, characterized in that the decoding (E31) of the first group of feature maps comprises a sub-step of entropy decoding.

15. The samples to be coded (Pv ｎ , P １ｎ， , P ２ｎ ) and at least two images (I(P ｖｎ ), I(P １ｎ ), I(P ２ｎ 1. A device for encoding a sequence of Feature Map (FM ｉ ) to construct a first group (GEN, MAJ), The position (x ｎ , y ｎ ) of each current sample (P ｖｎ , P １ｎ , P ２ｎ ) for a sequence of samples called The current sample (P １ｎ , P ２ｎ ) at the position (x ｎ , y ｎ ) as a function of the feature map (FMS ｉ ) to the feature vector (Z ｎ ) (XTR), The decoded value of the current sample (P' １ｎ , P' ２ｎ ) to provide a sequence of vectors representing the parameters (W ｋ ) is used to find the vector (Z ｎ ) (MLP), updating at least one value of one of said feature maps and / or at least one parameter of said network as a function of the coding performance measure (MAJ, NND); The feature map (FM ｉ ) and the first group of parameters (W ｋ ) encoding the set (FMC, NNC) A device configured to perform the steps of:

16. The sample to be decoded (Pd ｖｎ , Pd １ｎ , Pd ２ｎ At least two images (I(Pd ｖｎ ), I(Pd １ｎ ), I(Pd ２ｎ 1. A device for decoding a sequence of Feature map (FMd ｉ ) decoding a first group (FMD); The parameter (Wd ｋ ) (NND), and The position (x ｎ , y ｎ ) of each current sample (Pd ｖｎ ) for a sequence of samples called The position (x ｎ , y ｎ ) as a function of the feature map (FMd ｉ ) to the feature vector (Zd ｎ ) (E34, XTR'), and The current sample (Pd ｖｎ ) to provide a sequence of vectors each representing a ｋ ) is used to calculate the vector (Zd ｎ ) (E36, MLP'); A device configured to perform the steps of:

17. A computer program comprising instructions for carrying out the steps of the encoding or decoding method according to claim 1 or 8, when said program is run by a computer.

Images