Method and apparatus for encoding and decoding image sequences

Abstract

The present invention relates to a method and an apparatus for encoding and decoding a sequence of at least two images comprising a plurality of samples. The decoding method comprises the following steps: decoding the first set of feature maps; decoding a set of parameters representing the neural network; for said respective image in the sequence to be decoded, a sequence of samples, referred to as the current sample, associated with a position in said respective image: constructing a feature vector from the feature maps in said first group on the basis of said position of said current sample; -processing said vectors by means of an artificial neural network defined by decoded parameters in order to provide sequences of vectors respectively representing said current sample points.

Description

Technical Field

[0001] The present invention relates to the general field of encoding digital image sequences. More particularly, the present invention relates to compressing digital video. Background Art

[0002] Digital video is typically source-encoded to achieve compression, thereby limiting the resources required for its transmission and / or storage. Numerous coding standards exist, such as those developed by the ITU / MPEG organizations (H.264 / AVC, H.265 / HEVC, H.266 / VVC, etc.) and their extensions (MVC, SVC, 3D-HEVC, etc.).

[0003] An image is typically encoded by dividing it into a number of rectangular blocks and encoding these blocks of pixels in a given processing sequence. In existing video compression techniques, the processing of the blocks typically involves block pixel prediction, which is performed using previously encoded and then decoded pixels present in the image being encoded (in this case, called "intra-frame prediction") or using previously encoded images (in this case, called "inter-frame prediction"). This exploitation of any spatial and / or temporal redundancy avoids the need to transmit or store the pixel values ​​of each pixel block by representing at least some of the blocks using a residual (which represents the difference between the predicted values ​​of the pixels in the block and the actual values ​​of the pixels in the predicted block).

[0004] As video formats continue to evolve to achieve higher compression rates and accommodate a diverse range of expected formats and communication networks, the prediction possibilities continue to grow, and conventional encoding and decoding algorithms become very complex.

[0005] In addition to these conventional methods proposed by compression standards (MPEG, ITU), methods based on artificial intelligence (especially neural network intelligence) are also emerging.

[0006] Some of these neural network methods can be viewed as simple extensions of the competition concept in previous compression techniques (such as prediction mode competition, video coding transformation, etc.).

[0007] Other approaches use the concept of "autoencoders." Autoencoders are learning algorithms based on artificial neural networks that allow for the construction of new representations of a dataset. The autoencoder architecture consists of two parts: an encoder and a decoder. The encoder consists of multiple layers of neurons that process the data to construct new representations, called "encoded" representations, also known as "latent representations." The decoder's neural network layers, in turn, receive these representations and filter them to attempt to reconstruct the original data. The difference between the reconstructed and original data allows for the measurement of any error introduced by the autoencoder. Training involves modifying the autoencoder's parameters to reduce the reconstruction error measured at each sample point in the dataset. While these autoencoder-based systems offer superior performance, they come at the expense of significantly increased memory usage and complexity compared to conventional approaches, such as those proposed by compression standards. Such systems can have millions of parameters, and decoding a single pixel can require up to a million MAC (multiply-add) operations. This makes these decoders significantly more complex than conventional decoders, potentially hindering the application of learning-based compression.

[0008] A simple neural network-based image coding technique was recently described in Emilien Dupont et al.'s paper, "Compression with Implicit Neural Representations" (arXiv:2103.03123). The proposed encoding technique involves adapting a neural network to an image, quantizing the network's weights, and transmitting these quantized weights. During decoding, the neural network is evaluated at each pixel location to reconstruct the image. However, this technique still suffers from compression inefficiencies and requires separate encoding of each image in the video.

[0009] Therefore, there is a need for a solution for simply and efficiently encoding / compressing image sequences. Summary of the Invention

[0010] The object of the invention is a method for encoding a plurality of images as claimed in claim 1 and a method for decoding a plurality of images as claimed in claim 8 .

[0011] Within the meaning of the present invention, the term "encoding" is understood to mean an operation involving representing a set of samples (or pixels) in a compact form, such as for transmission via a digital bit stream. Decoding is understood to mean an operation involving processing a digital bit stream to restore decoded samples.

[0012] The term "image sequence" should be understood to mean a plurality of two-dimensional images that are temporally ordered, for example in the case of a video. As another example, the images may be views of a multi-perspective representation of the same scene. As another example, the images may be multi-perspective images at multiple times (immersive video).

[0013] The term "sample" should be understood to mean a value sampled from an image in a sequence. Sampling a signal produces a sequence of discrete values, called samples. In the case of image signals, a sample is called a pixel, which can be, for example, a color pixel traditionally represented by a triplet of values, such as (R, G, B) or (Y, U, V). The location of a sample is identified by its coordinates in the image, on the horizontal (x) and vertical (y) axes. A sample sequence is a series of samples considered at the same coordinates in a series of corresponding images in the image sequence.

[0014] The term "feature map" should be understood to mean an abstract representation of an image sequence (i.e., at least two images) that includes a plurality of variable data (also called values, such as real numbers). These maps are also called "latent representations."

[0015] The expression "feature map transformation" should be understood to mean the application of a mathematical operation to transform the values ​​of a first map into the values ​​of a second map. The first map used for encoding (referred to as the first set of maps) can be any type of map. The second map (referred to as the transformed map or map in the second set) has the same resolution as the input image, meaning it contains the same number of values ​​as the number of samples (N) in the input image (to be decoded). Transformations can involve, for example, interpolation, upsampling, filtering, quantization, Fourier transforms, and so on.

[0016] The expression "a feature vector of data constructed from a feature map according to a position" should be understood to mean a vector consisting of one or more (preferably discrete) elements or data, where the data is constructed from a feature map at a position determined by the position of the sample being processed in the image. This feature vector is an input to the neural network. For an image, such a vector can be constructed, for example, from a plurality of values ​​sampled from each of the feature maps at the same x and y coordinates as the sample to be encoded (or respectively decoded). Once these values ​​are sampled from the feature maps, they can be processed (e.g., by quantization, filtering, interpolation, etc.) to form a feature vector, which is then input to the neural network.

[0017] The term "neural network" or "synthetic neural network" should be understood to mean a neural network, such as a convolutional neural network, a multilayer perceptron, an LSTM (long short-term memory), etc. A neural network is defined, for example, by multiple layers of artificial neurons and a set of activation functions, weighting functions, and summation functions (for example, a layer may compute y = f (Ax + b), where y and b are N-dimensional vectors, x is an M-dimensional vector, A is an M × N-dimensional matrix, and f is an activation function).

[0018] The expression “parameter of a neural network” is understood to mean one of the values ​​that characterize a neural network, for example a weight associated with one of the neurons (filter coefficients, weights, biases, values ​​that influence nonlinear operations), etc.

[0019] The expression "processing using a neural network" should be understood to mean applying a function expressed by the neural network to an input feature vector in order to produce an output vector representing the samples to be encoded (or respectively decoded). The output vector may include one or more data representing the samples.

[0020] The term "performance measure" should be understood to mean a measurement between at least one value of a sample to be encoded and the decoded value of that sample. This measurement can assess, for example, distortion or perceptual error. This measurement can be performed for one sample or for multiple samples (e.g., the current sample or the current image, etc.). This measurement can also include a measurement of bitrate (particularly the bitrate associated with encoding the neural network and / or encoding the feature maps in the first group). This measurement can be a joint measure of bitrate and distortion achieved by weighting the bitrate and distortion. As is well known in the art, the value of this measure is typically minimized until a target value is reached.

[0021] The term "construction step" is understood to mean a step aimed at constructing the parameters representing the image before actually encoding them. The construction sub-step may be repeated as many times as necessary to obtain an acceptable performance measure.

[0022] Generally, the steps of the encoding or decoding method should not be interpreted as being associated with the concept of a chronological order. In other words, these steps can be performed in a different order than indicated in the independent encoding or decoding claims, or even performed simultaneously.

[0023] The encoding method according to the present invention constructs encoding parameters based on a sequence of input images by training a neural network on feature vectors associated with the locations of the samples to be encoded in each input image. These feature vectors are constructed from feature maps, which can have the resolution of the input images or a lower resolution. During training or construction, the parameters of the neural network and the values ​​of the feature maps are updated based on performance measures (e.g., bitrate distortion type). When training is complete (i.e., the performance measures obtained are satisfactory), the actual encoding of the neural network parameters and feature map values ​​can be performed and stored or transmitted to a decoder.

[0024] Advantageously, the training process allows for refining the parameters of the neural network and / or the values ​​of the feature maps until a representation that satisfies performance requirements is achieved, for example, until a desired balance between the generated bitrate and the distortion experienced by the input image is achieved. The training of the values ​​of the feature maps and the parameters of the neural network can be performed jointly. Advantageously, the encoding method according to the present invention allows for efficient compression of image signals by exploiting both intra-image spatial redundancy and inter-image redundancy between images in a sequence (whether a sequence of video images, a plurality of multi-view images, a series of medical images, etc.).

[0025] Advantageously, the decoding method is simple since only the feature maps and the neural network need to be decoded to reconstruct a decoded version of the image sequence.

[0026] Advantageously, such a neural network can have a very simple structure, with very few parameters.

[0027] According to an embodiment of the encoding or decoding method:

[0028] - The method comprises a step of transforming said first set of feature maps so as to obtain a second set of feature maps having the resolution of an image in the sequence, the method being characterized in that said feature vector is constructed from feature maps of said second set.

[0029] Advantageously, according to this embodiment, the feature maps are divided into two groups, one of which is reserved for extracting feature vectors and the other for encoding. Thus, two methods with different objectives can be separated: the maps in the first group to be encoded (and correspondingly decoded) must be compressed as efficiently as possible, while the maps in the second group must facilitate the process of extracting and constructing feature vectors.

[0030] According to one variant, at least one of the feature maps in the first set has a lower resolution than the resolution of the image to be encoded (or respectively decoded), and the transformation operation involves upsampling. Advantageously, according to this embodiment, compression of the feature maps is more efficient because at least one of the feature maps in the first set to be encoded (or respectively decoded) contains fewer values ​​than if it had the resolution of the image. For example, one of the feature maps in the first set may have a resolution of 1 / 2, i.e., it contains half the number of x and y values ​​as the number of samples contained in the input image (i.e., a total of 1 / 4 the number of values ​​of a feature map with the resolution of the image). In contrast, a feature map in the second set (corresponding to the transformation of this map in the first set) has the same resolution as the image. Therefore, in this case, the transformation includes at least one upsampling operation to obtain in the transformed map the same number of values ​​as the number of samples contained in one of the input images (or respectively decoded).

[0031] - At least one of said feature maps in the first group has the same resolution as a resolution of an image to be encoded (respectively to be decoded).

[0032] Advantageously, according to this embodiment, at least one of the feature maps has the same resolution as the input image to be encoded (and respectively decoded), thereby achieving high fidelity and preserving the details of the original resolution of the images in the sequence. According to one embodiment, in this case, the transformation preserves the number of values ​​in the transformed feature map. This transformation can be simplified to an identity transformation (no manipulation of the values ​​in the maps of the first group) or include filtering operations, quantization operations, Fourier transforms, etc. During encoding, if the feature maps include, for example, floating-point values ​​or real values, quantization is crucial for the correct operation of the system. These feature maps need to be quantized before encoding and / or input to the neural network. In contrast, during decoding, depending on the embodiment, an inverse quantization operation is not always necessary.

[0033] - Constructing said feature vector comprises the sub-step of extracting the value of said at least one feature map in the same position as the position of the current sample in the image to be encoded (respectively to be decoded).

[0034] Advantageously, the elements of the feature vector can be constructed by extracting values ​​from the feature maps in the first or second group at the same locations as the samples in the images of the sequence (the input images for encoding, the images to be decoded for decoding). This approach is easy to implement. For example, if J input feature maps with the same resolution as the images of the sequence are available, simply extracting the values ​​from these maps at the coordinates of the current sample (at the same horizontal and vertical coordinates in the feature maps) allows for the direct construction of a feature vector of J elements.

[0035] - The construction of the feature vector includes the following sub-steps:

[0036] - extracting a plurality of values ​​of the feature map in the first group according to the position of the current sample point;

[0037] - processing said extracted values ​​in order to obtain a feature vector.

[0038] Advantageously, according to this embodiment, feature vectors are extracted from feature maps before processing. These feature maps can be arbitrary, in particular having a resolution lower than that of the image to be encoded (or decoded). This processing can correspond, for example, to quantization, scaling, or filtering of the extracted data. For encoding, for example, if the feature maps include floating-point or real values, quantization is crucial for the correct operation of the system. These feature maps need to be quantized before encoding and / or input into the neural network. In contrast, during decoding, depending on the embodiment, an inverse quantization operation is not necessary.

[0039] The method includes the steps of constructing a set of additional feature maps, and also constructing feature vectors based on these additional feature maps. Advantageously, these third set of additional maps, constructed identically in the encoder and decoder, are neither stored in nor transmitted to the decoder, nor decoded in the decoder. Thus, these additional maps allow for improved compression using additional data without reducing the bitrate. This additional data may include, for example, coordinates, causal data available in the maps of the first or second set, data about other images already processed by the encoder or decoder, etc.

[0040] According to one variant, the additional map contains values ​​representing the temporal distances between images in the video.

[0041] - encoding (respectively decoding) said first set of feature maps comprises a sub-step of entropy encoding (respectively decoding). Advantageously, entropy encoding allows exploiting any redundancy in the image that may remain in the feature maps, thereby enabling more efficient compression of the images in the sequence.

[0042] Relatedly, another object of the invention is a device for encoding a sequence of images and a device for decoding a sequence of images.

[0043] The features and advantages of the encoding or decoding method are also applicable to the encoding or decoding device according to the present invention, and vice versa.

[0044] Another object of the present invention is a computer program on a storage medium, wherein the program can be implemented in a computer or control device according to the present invention. The program comprises instructions designed to implement the corresponding method. The program can use any programming language and can be in the form of source code, object code, or an intermediate code between source code and object code, such as in a partially compiled format, or in any other desired format.

[0045] The present invention also relates to a computer-readable information medium or storage medium comprising instructions for the aforementioned computer program. The information or storage medium may be any entity or device capable of storing a program. For example, the medium may include a storage device such as a ROM (e.g., a CD-ROM or a microelectronic circuit ROM), or even a magnetic storage device (e.g., a floppy disk, a hard disk, a DNA sequence, or a flash memory). Furthermore, the information or storage medium may be a transmissible medium such as an electrical or optical signal, which may be routed via an electrical or optical cable, a radio link, a wireless optical link, or other means.

[0046] The program according to the invention can in particular be downloaded via the Internet.

[0047] Alternatively, each information medium or storage medium may be an integrated circuit incorporating the program, the circuit being designed to execute or being used to execute the method according to the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Other characteristics and advantages of the invention will become apparent from the following description given with reference to the accompanying drawings, which illustrate embodiments of the invention that are by no means limiting.

[0049]

Figure 1

[0050]

Figure 2

[0051]

Figure 3

[0052]

Figure 4

[0053]

Figure 5

[0054]

Figure 6

[0055]

Figure 7

[0056] Figure 1 An encoding device ENC is schematically shown.

[0057] The encoding device ENC includes a feature map generation module GEN, a transformation module SE, a data extraction module XTR, a processing and quantization module TT, a module MLP corresponding to an artificial neural network, a neural network encoding module NNC, a feature map encoding module FMC, a performance function evaluation module EVAL, an update module MAJ, and an optional additional feature map generation module FME.

[0058] The encoding device ENC may be implemented by an electronic device comprising a processor and a memory (not shown); each of the aforementioned modules may then be generated via the interaction of the processor with computer program instructions stored in the aforementioned memory and designed, in particular as described below, to perform the functions of the module in question when these instructions are executed by the processor.

[0059] Figure 1 The encoding device ENC shown receives the signal denoted as I(P Vn ) as input, each of the images to be coded includes a plurality of sample points P Vn The index v indicates the image number in a sequence of V images. The index n indicates the pixel number in an image of N pixels. Image I(P Vn ) can represent a two-dimensional image or a sequence of multiple two-dimensional images (video sequence, stereoscopic component, multi-view component, medical image series, etc.). In one embodiment, the images in the sequence are represented by at least one two-dimensional representation (such as a pixel matrix), where each pixel includes a vector of red (R), green (G), blue (B) components, or as a variant, a luminance component and at least one chrominance component. The position of each pixel is represented by its x and y coordinates in the image (x n and y n) is defined. In one embodiment, the sequence is a sequence of grayscale images represented by a two-dimensional representation (such as a pixel matrix), where each pixel has a grayscale component or a brightness component. In this case, the vector representing the pixel is reduced to a single component.

[0060] As will be referred to below Figures 4 to 7 Described in more detail:

[0061] The feature map generation module GEN is configured to generate a feature map according to the input image I(P vn ) sequence generates multiple (M) feature maps in the first group, denoted as FM i The optional module FME can also generate one or more additional maps (number L) that are neither encoded nor transmitted and are represented as FME l .

[0062] In one embodiment, module SE transforms the first set of feature maps to generate a second set of feature maps having the same resolution as the images of the input sequence. Module SE may optionally quantize the values ​​of the first set of maps using a quantizer Q to generate an ordered set of quantized values. It should be noted that quantization of a value refers to matching that value to a member of a discrete set of possible code symbols. For example, this set of possible code symbols may consist of integer values, and the quantization system simply rounds the actual value to an integer value. According to another example, quantization involves multiplying by a given value and then rounding. Next, module SE transforms the values ​​of at least one feature map, for example by upsampling, interpolation, filtering, etc. At the end of the transformation, the transformed feature maps in the second set have the same resolution as the images of the input sequence. Advantageously, according to this embodiment, the encoded feature maps may have a lower resolution than the resolution of the images to be encoded, while the maps in the second set used to construct the feature vector have the same resolution as the image sequence, thereby facilitating the extraction of values.

[0063] In one embodiment, the module SE is not present, in which case the values ​​to be used to construct the feature vector are extracted from the first set of feature maps. The module XTR generates the current sample Pv to be encoded. n The sequence of features FM is extracted according to its coordinates in the corresponding input image. i (and / or FMS i and / or FME l , according to one of the previously described embodiments). For example, when the coordinates (x n ,y n ) is used to encode the sample point P1n and the coordinates (x n ,y n ) is used to encode the sample point P2n, the module XTR is based on the coordinates of the current pixel (xn ,y n ) to extract the values ​​in the graph at the position determined by

[0064] In one embodiment, the extracted values ​​form a vector Z n . Z n is a J-tuple, i.e. it contains J elements or data z i The index n refers to the current sample or pixel P' vn The feature vector of the sequence.

[0065] In one embodiment, the optional module TT processes the extracted values ​​to generate a vector Z n Module TT can quantize the data extracted from the set of feature maps. The processing can include other operations such as filtering, scaling, etc. In particular, if module SE is not used and if the feature maps in the first set have a lower resolution than the images in the sequence, module TT can take into account the coordinates of the values ​​in the map with lower resolution.

[0066] It should be noted that at least one of the modules SE or TT has to quantize the feature maps.

[0067] The module MLP consists of K parameters W k The neural network defined here can process vector Z n (or J-tuple) as input to generate the sample point P' to be encoded vn According to one embodiment, the neural network is an MLP (Multi-layer Perceptron) consisting of an input layer adapted to the input format (J-tuples), optionally one or more hidden layers, and an output layer adapted to the output format of the output vectors (typically multiple vectors, each containing A elements). According to one embodiment, A is equal to 3, and the output vector is the N pixels P' of the encoded and then decoded image v vn One of the (R, G, B) triplets.

[0068] The module NNC affects the neural network (especially its parameters W k ) is encoded. During the encoding training or construction process, that is, as long as the step of evaluating the performance is still not satisfactory, the module NNC simulates encoding and then decoding, the results of which are sent to the evaluation module. The evaluation module updates the values ​​of the parameters according to the results of the performance measurements performed by the module EVAL. Subsequently, the parameters W of the neural network are adjusted. k The actual encoding of the coded parameters is represented by Wc k In a known manner, the coding simulation can be identical to or similar to the actual coding.

[0069] Module FMC to FM i, i.e., the values ​​of the feature maps in the first group (excluding any additional maps FME optionally upsampled by SE) l and the second group of graphs). During the encoding training or construction process, i.e., as long as the step of evaluating the performance is still unsatisfactory, the module FMC simulates encoding and then decoding, the result of which is sent to the evaluation module. Subsequently, the execution of the graph FM i The actual encoding of the value of . The encoded graph is represented as FMc i In a known manner, the encoding simulation can be identical to or similar to the actual encoding. If necessary, the encoding module quantizes the potential representation of the values ​​of the graphs in the first group using a quantizer to generate an ordered set of quantized values. Next, the encoding module compresses the quantized data, for example using entropy coding.

[0070] The module EVAL performs evaluation and minimization of coding performance. For example, the evaluation function is of the bit rate distortion type. Minimization can be performed via gradient descent or any other method within the capabilities of those skilled in the art.

[0071] The module MAJ updates the graph FM to be encoded according to the results of the performance function i value.

[0072] Figure 2 A decoding device DEC is schematically shown.

[0073] Figure 2 The decoding device DEC receives as input a first set of encoded data organized into M feature maps FMc i (also called layer FM) and the encoded parameters Wc of the neural network k The device samples Pd of the V images in the sequence to be decoded. vn N sequences are decoded.

[0074] The decoding device DEC comprises a neural network decoding module NND, a feature map decoding module FMD, a data extraction module XTR', an inverse transformation module SE', a processing and dequantization module TT', a module MLP' corresponding to the neural network, and an additional optional feature map generation module FME'. The device outputs a decoded image (denoted as I(Pd vn )) sequence, each decoded image includes a plurality of decoded sample points Pd vn .

[0075] The graph (number M) decoded by the module FMD is denoted as FMd i The parameters decoded by the module NND are denoted as Wd k .

[0076] The decoder may also generate one or more additional graphs represented as FME's l , and the number is L, with the additional graph FME generated by the encoder l The same number.

[0077] In one embodiment, the module SE′ processes the first set of decoded feature maps FMd i Transform to generate a second set of feature maps with the same resolution as the image to be decoded, denoted as FMS' i . The module SE' optionally performs an inverse quantization corresponding to the quantization performed on the encoder. If the quantizer Q of the encoder only rounds the actual values ​​it receives, then there is no need to perform inverse quantization. If the neural network is able to take into account the quantization of its input data, then there is no need to perform inverse quantization. Otherwise, the decoder performs the inverse operation of the quantizer Q. The module SE' then performs a transformation on the values ​​of the feature maps similar to that performed by the encoder (including, for example, upsampling, interpolation, filtering, etc.). Once the transformation is completed, the transformed feature maps in the second group have the same resolution as the images of the sequence to be decoded.

[0078] In one embodiment, the module SE' is not present, in which case the values ​​to be used to construct the feature vector are extracted from the first set of feature maps.

[0079] Module XTR' with Figure 1 The module is the same as the module XTR. This module is for the current sample point Pd to be decoded vn Sequence, according to the coordinates of the sample points in the corresponding image to be decoded, extract M feature maps FMd i (and / or FMS' i and / or FME' l , according to one of the previously described embodiments). In one embodiment, J = M. In one embodiment, J = M + L.

[0080] In one embodiment, the extracted values ​​form a vector Zd n . Zd n It is a J-tuple, that is, it contains J elements or data. i .

[0081] In one embodiment, the optional module TT' processes the extracted values ​​to generate a vector Zd n The module TT may perform inverse quantization on the data extracted from the set of feature maps. The processing may include other operations similar to those performed by the encoder, such as filtering, scaling, etc.

[0082] The module MLP' is composed of K parameters Wd k The neural network defined here can process the vector Zdn (or J-tuple) as input to generate the sample point Pd vn As output, a sequence of vectors of a sequence of A elements is output, each vector comprising A elements. According to one embodiment, A = 3, and the output vector is the N pixels Pd of the image v to be decoded vn The module MLP' has the same structure as the module MLP, and if the parameter W k If the encoding is lossless, its parameters are the same, or if the encoding is lossy, its parameters are different.

[0083] When all sample points Pd vn When all the sequences have been decoded, a sequence of reconstructed images is obtained, for example, V images I(Pd vn ), each image contains N samples.

[0084] The decoding device DEC may be implemented by an electronic device comprising a processor and a memory (not shown); each of the aforementioned modules may then be generated via the interaction of the processor with computer program instructions stored in the aforementioned memory and designed, in particular as described below, to perform the functions of the module in question when these instructions are executed by the processor.

[0085] Figure 3 Examples of synthetic artificial neural networks used within the scope of the present invention during encoding and decoding are presented.

[0086] The synthetic artificial neural network MLP for encoding and the synthetic artificial neural network MLP' for decoding are defined by the same structure (eg, comprising multiple layers of artificial neurons) and a set of weights and activation functions respectively associated with the artificial neurons of the networks in question.

[0087] The vector representation of the current sample sequence (from the feature map FM i / FMS i and FME l or FMd i / FMS' i and FME' l The vector Z obtained n or Zd n ) is applied to the input (i.e., input layer) of the synthetic artificial neural network MLP or MLP'. The artificial neural network produces a plurality of vectors P' representing the reconstructed (during encoding) or decoded (during decoding) samples vn or Pd vn As output, according to one embodiment, are the constituent color components (R, G, B or Y, U, V) of the color pixels of the image sequence. Figure 3In the example, the image sequence corresponds to two images, each sample sequence contains two samples, and the corresponding vector is represented as Pd 1n and Pd 2n .

[0088] The concatenation of all reconstructed pixels in each image in the output sequence forms a sequence of decoded images (at the decoder) or a sequence of reconstructed images (at the encoder).

[0089] On the encoder, a synthetic artificial neural network MLP is trained on the image sequence to minimize the input representation I(P vn ) and its output representation I(P' vn ), while also minimizing the amount of data to be encoded. In this sense, the module EVAL performs a performance measurement.

[0090] Once trained, the network's parameters are either encoded losslessly (in which case the neural network MLP' is identical to the MLP) or lossily (in which case the network MLP' may be slightly different from the MLP).

[0091] Figure 4 It shows that Figure 1 Flowchart of an example of an encoding method implemented by an encoding device.

[0092] According to this embodiment, the sequence is a two-dimensional image sequence, so that each sequence of samples to be coded is the corresponding image I(P vn ) in the coordinates (x n ,y n ) a set of pixels P at vn .

[0093] Coding is done in two main stages:

[0094] In the first phase, called the construction phase, learning is performed so that for an input sequence I(P vn ) Determine the FM i The value and parameter W k , to optimize the total cost function. For example, via gradient descent, the parameters of the neural network MLP and the feature map FM are then updated i As is known in the art, the cost function can be of the bitrate-distortion type, or of the bitrate, or distortion, or perceptual type. To measure the bitrate R, it is necessary to simulate the FM i The encoding of , then the associated encoding bit rate (the size of stream B1) needs to be measured. According to one embodiment, the parameter W is not simulated. k The encoding of , because their influence is smaller than that of the feature map. According to one embodiment, the parameter W is also simulated. kTo measure the distortion D, it is necessary to simulate the encoding and then decoding of at least a portion of the image sequence in order to obtain at least one pixel sequence P' resulting from the simulation of encoding and then decoding the sample indexed n. vn , then, we need to measure the image sequence I(P vn ) as input and the sequence I(P' vn ) between the corresponding parts.

[0095] Next, during the second phase, called the encoding phase, the graph FM i and parameter W k Encoded to produce the encoded value FMc i and WC k , and then transmitted or stored. They form the input sequence I(P vn ) is a compressed representation of .

[0096] The steps of a method according to one embodiment of the present invention will now be described.

[0097] During a step E20, an input sequence to be coded I(P vn ) is passed to the method as input, each image together includes multiple (N) sample points P n According to one embodiment, the images are temporal images of a video sequence. According to one embodiment, the images are images of a series of images, such as medical images. According to one embodiment, the images are multi-view or 3D components of a single image or a sequence of multi-view or 3D images.

[0098] During a step E21 , a first set of M graphs FM is initialized i Subsequently, the parameters W of the neural network MLP must be optimized during the construction phase k Hetu FM i value.

[0099] According to one embodiment, FIG. FM i With the same input sequence I(P vn ), and therefore each picture includes the same resolution as the samples P to be coded in each picture v vn The number of values ​​is the same as the number N.

[0100] According to one embodiment, FIG. FM i The resolution is less than or equal to the input sequence I(P vn ) and therefore for at least one of the images the number N' of values ​​to be coded it comprises is less than N. According to one variant, the first image FM ihas the resolution of the image, and each subsequent image has the Figure 1 Half resolution.

[0101] According to one embodiment, a plurality of graphs FM i With the same resolution, it is smaller than the input sequence I(P vn ) resolution.

[0102] According to one embodiment, FIGFM i Transformed to provide a second set of transformed feature maps FMS i In this embodiment, the feature vectors are preferably extracted from the transformed graphs in the second group, rather than directly from the graphs in the first group. Thus, in this embodiment, the feature vectors are indirectly extracted from the graphs in the first group. The graphs in the second group are neither encoded nor transmitted; they are only used to construct the feature vectors.

[0103] According to one embodiment, the map FM is initialized using predefined constant values. i .

[0104] According to another embodiment, the feature map is initialized using a set of random real values.

[0105] According to one embodiment, FME generates one or more graphs l And added to the first set, these one or more maps form another set of L additional feature maps. They are used to construct the feature vector but are not stored or transmitted.

[0106] Subsequently, during a step E22 , the updating module MAJ of the encoder updates or refines the feature maps FM of the first group during its learning phase i .

[0107] During a step E23, the map FMi of the first group is processed by the module FMC of the coder. i The encoding is performed. During the construction phase, this operation is a simulation of the encoding. During the encoding phase, this operation is the actual encoding, and the encoded values ​​form the stream B1. The simulation can be identical to the actual encoding, but can also be different (e.g., simplified). For this encoding, any known technique for compressing graph values ​​can be used.

[0108] In one embodiment, FIGFM i The encoding is performed in the order (FM1, FM2, ..., FM4), with the variables of each graph encoded in a predefined order (e.g., lexicographic order). Each graph undergoes entropy encoding. Entropy encoding produces a compressed stream B1, the bit rate of which is subsequently measured during step E29.

[0109] During a step E24, according to one embodiment, the module SE processes the M maps FM of the first groupi Transformed to generate a second set of images FMS with the image resolution of the input sequence i .

[0110] According to one embodiment, M graphs FMS are generated i .

[0111] According to one embodiment, each map FM i Transformed into graph FMS i .

[0112] According to one embodiment, at least one map FM i has a lower resolution than the resolution of the image of the sequence to be coded, and the transform operation includes upsampling so that the transformed image FMS i Includes the same number of samples as the image in the sequence. Upsampling involves adding values ​​to the image FMS i , so as to achieve the resolution of the images of the input sequence. This operation can be simple (by nearest neighbor replication) or include interpolation (linear, polynomial, filtering, etc.).

[0113] During a step E25, the transformed map FM is extracted by the module XTR i , or optionally FMS i , and optionally additional diagrams FME i The extraction is based on the current sample sequence P of the input image. vn The coordinates (x n ,y n ) is performed. This extraction can also be performed according to the resolution of the image in question. The sequence of samples to be encoded is processed, for example, in the order from n=1 to n=N.

[0114] According to one embodiment, the feature vector Z n directly from this extraction.

[0115] According to one embodiment, during a step E26, the module TT determines the coordinates (x n ,y n ) of each sample sequence P vn , according to Figure FM i or FMS i and optionally FME l Extract the values ​​to construct the feature vector Z n If desired, this processing may involve performing a calculation on the extracted values ​​or constructing the vector Z n This processing may include other operations such as filtering, scaling, applying any function (preferably a monotonic function), etc.

[0116] In one embodiment, Zn Included with input graph FM i or FMS i (and optionally FME l ) is the same number of values ​​as the number of values ​​in the median. In this case, J = M(+L).

[0117] In one embodiment, Z n is located at the current pixel P vn The coordinates (x n ,y n ) at Figure FM i or FMS i (and optionally FME l ) values ​​form a J-tuple (z1, z2, ..., z J ), such as referring to Figure 5 What is shown.

[0118] In one embodiment, Z n It is from Figure FM i (and optionally FME l ) is constructed from the values ​​sampled in the graph, where the sampling coordinates of these values ​​vary depending on the graph. For example, if the graph FM i (and / or FME l ) Since they have been downsampled and have different resolutions, the coordinates are adapted (by scaling) to match the resolution of each figure.

[0119] In one embodiment, Z n is a J-tuple constructed from values ​​that are retrieved from the graph FM by applying a process to one or more values ​​of the graph (e.g., filtering the neighbors of the target value in the graph). i (and optionally FME l ) is obtained by sampling. For example, in a graph FM with the same resolution as the input signal i In the example, we can extract the coordinates (x n ,y n )、(x n -1,y n )、(x n ,y n -1) and (x n -1,y n -1) and process these values ​​(filtering, averaging, interpolation, etc.) to obtain the value of FM i or FME l Related vector Z n The final value of element i (z i According to another example, in the image FM having half the resolution of the input signal iIn the example, we can consider the coordinates (x n / 2,y n / 2)、(x n / 2-1,y n / 2)、(x n / 2,y n / 2-1) and (x n / 2-1,y n / 2-1) and process these values ​​(filtering, averaging, interpolation, etc.) to obtain the value of FM in this figure. i or FME l Related vector Z n The final value of element i (z i ).

[0120] During step E27, the vector Z n Processed by the neural network MLP to generate the sample sequence P to be encoded vn As output, that is, according to one embodiment, (sample P vn The sample point P' is encoded and then decoded vn The (R, G, B) triplet.

[0121] The structure and parameters W of the neural network k For example, they are initialized in the first iteration of this step. These parameters are then updated or refined during subsequent iterations of the method during the construction phase.

[0122] According to one embodiment, the parameters of the neural network are initialized with predefined values ​​that are known to produce satisfactory results (e.g., after training on a library of images).

[0123] According to another embodiment, the parameters W of the neural network are initialized with a set of random values. k .

[0124] During a step E28 , the parameters W of the neural network MLP are adjusted k quantized and encoded. During the construction phase, this operation is a simulation of encoding. During the encoding phase, this operation is the actual encoding, and the encoded values ​​form the stream B2. The simulation can be identical to the actual encoding, but can also be different (e.g., simplified). For this purpose, any known technique can be used, such as the Neural Network Coding standard proposed in Part 17 of the MPEG-7 standard, also known as Neural Network Representation or NNR. It should be noted that in this case, the encoding pair parameter W needs to be selected k The amount of degradation caused.

[0125] During a step E29 , the performance measures are evaluated.

[0126] To this end, the measurements are made on the feature maps in the first group (by comparing the FM i is encoded to simulate the flow B1) and optionally with the parameters of the neural network (via the parameters W k The encoding is performed to simulate the encoding simulation bit rate associated with stream B2).

[0127] According to one embodiment, the cost function is of the bitrate-distortion type, expressed as (D + L * R), for example, the squared error measured between the input signal and the decoded image (or the error measured over a subset of the image's samples). According to another example, D is calculated from a perceptual function such as SSIM (Structural Similarity) or MSSSIM (Multi-Scale Structural Similarity). According to one embodiment, R is the simulated bitrate of stream B1; according to another embodiment, R is the total bitrate used to encode the image, i.e., the sum of the simulated bitrates of B1 and B2. L is a parameter that adjusts the rate-distortion tradeoff. Other cost functions may also be used.

[0128] As long as the cost function has not reached its minimum, the performance measure is not satisfactory and the method is repeated starting from step E22. This minimization can be performed via a known mechanism such as gradient descent, wherein the parameters are updated by updating the values ​​of the feature maps in step E22 and the parameters of the network in step E27.

[0129] During step EF, if the cost function has reached its minimum, the training stops. k ) and feature maps (FM i ) is available, the streams B1 and B2 can be formed from them. According to another embodiment, the neural network (W) is processed in this step. k )’s updated parameters and feature maps (FM i ) to produce the coded parameters Wc that form the streams B1 and B2 k and FMc i .

[0130] Streams B1 and B2 can be concatenated to produce the final stream. According to one embodiment, stream B2 of encoded parameters of the neural network is stored or transmitted before stream B1 so as to be able to be decoded before stream B1.

[0131] Figure 5 A schematic diagram showing an encoding method used in an embodiment of the present invention is shown.

[0132] In this illustration, there are two images I(P 1n ) and I(P 2n ) will be encoded. Usually, V images I(P vn) is applied as input to the method and encoding device. In this diagram, samples (or pixels) are processed in pairs, where in the sequence represented by (x n ,y n ) where n varies from 1 to N. 1n At position (x n ,y n ) is sampled from image 1, and P 2n At position (x n ,y n ) is sampled from image 2).

[0133] In this embodiment, there are 4 generated graphs FM i In the preferred embodiment, there are 7 graphs.

[0134] The first map FM1 has the same resolution as image I and therefore contains N = W × H variables, where W represents the width of the image (in pixels) and H represents its height. The second map FM2 has half the resolution (in each dimension) of map FM1. The resolution of each additional map is half that of the previous one. This structure allows reducing the number of variables in the feature map, thereby facilitating encoding and learning while minimizing encoding costs.

[0135] According to the reference Figure 6 In the presented method, module SE upsamples the image FM2 by a factor of 2 in each dimension, upsamples the image FM3 by a factor of 4 in each dimension, and upsamples the image FM4 by a factor of 8 in each dimension.

[0136] The obtained graph FMS i With the same image I(Pv n ) and therefore each image contains W × H values, where W is the width of the image in pixels and H is its height (N = W × H).

[0137] According to this embodiment, the layer FM i Quantization is performed by module SE.

[0138] Other types of structures are possible, for example, using a drop rate other than half between graphs (such as one quarter, one third, etc.).

[0139] In the variant shown with dashed lines, there are five feature maps: an additional map FME0 has been introduced that is neither encoded nor transmitted. This additional map typically contains data that can help the network MLP perform the task of reconstructing the image. Thus, the additional map can be one or more maps from the following non-limiting list:

[0140] - A plot that contains at each point the x-coordinate of that point.

[0141] - A plot that contains at each point the y-coordinate of that point.

[0142] - A diagram containing a positional code at each point (e.g. as described on the following website: https: / / skosmos.loterre.fr / P66 / fr / page / -K0D65X2X-X).

[0143] - A map that represents an image different from the image being processed and that can provide information about the image to be encoded, such as a previously processed image or sequence of images.

[0144] - A map containing data representing the time differences between pictures of the video being decoded. For example, if the first and last pictures of the video are 8 pictures apart, all samples in the map will contain the value 8.

[0145] - A map representing a feature map of an image different from the one being processed and that is able to provide information about the image to be encoded, such as a previously processed map.

[0146] - a picture containing the values ​​of an already decoded sample of the same picture (i.e. the previous sample in decoding order).

[0147] In this embodiment, vector Z n is obtained from the pixel sequence P vn The coordinates (x n ,y n ) at the FMS i The extracted values ​​form a 4-tuple (z1...z4). From the graph FMS i The vector Z of the (quantized) extracted values n Processed by the neural network MLP to output a sequence of vectors, for example, representing the sample point P to be encoded 1n and P 2n The output vector in this embodiment is the pixel P' that is encoded and then decoded 1n and P' 2n The (R, G, B) triplet is inserted into the decoded image I(P' 1n ) and I(P' 2n ) the positions (x n ,y n ) place.

[0148] In another embodiment (not shown), the image is directly retrieved from the layer FM at a position recalculated according to the size of the image. i Extract vector Z n, and then optionally after extraction, the extracted values ​​are processed and quantized.

[0149] According to the variant shown as the dotted line, the vector Z n It is a 5-tuple (z0...z4), where the value z0 is extracted from the attached map fme0.

[0150] Figure 6 It shows that Figure 2 A flowchart of an example of a decoding method performed by a decoding device.

[0151] During a step E30, the streams B1 and B2 are extracted from the coded stream BS. These streams respectively contain a first set of images FMc i and parameter Wc k The encoded representation of .

[0152] During step E31, by i Decode to generate M graphs FMd i For this decoding, any known technique similar to that used in the encoder may be used, and preferably entropy decoding is used. In one embodiment, FIGFMd i Decoding is performed in the order of (FMd1, FMd2, ..., FMd4), and the variables of each graph are decoded in a predefined order (e.g., lexicographic order).

[0153] According to the embodiment as described for the encoder:

[0154] - Figure FMd i The image I(Pd vn ) the same resolution, i.e., the figures contain N = W × H values.

[0155] - Figure FMd i The resolution of the image I(Pd vn ) resolution.

[0156] - Multiple FMd i have the same resolution, which is lower than the resolution of the images in the sequence.

[0157] During a step E32, according to one embodiment, one or more graphs FME' are generated l And added to the first set, this one or more maps form another set of L additional feature maps. These maps are not decoded, but are generated by the decoder in the same way as they were generated in the encoder. These maps usually include data that can assist the network MLP's task of reconstructing the image of the sequence. Reference Figure 5The non-limiting list of possible additional feature maps described for the encoder applies in this case as well.

[0158] During a step E33, according to one embodiment, the module SE processes the M maps FMd of the first group i Transformed to generate a second set of images FMS' with the image resolution of the input sequence i .

[0159] According to one embodiment, M graphs FMS' are generated. i .

[0160] According to one embodiment, each map FMd i Transformed into graph FMS i .

[0161] According to one embodiment, at least one map FMd i has a lower resolution than the resolution of the images of the sequence to be coded, and the transform operation comprises upsampling, so that the transformed image FMS' i Contains the same number of samples as the input sequence of images. As in the encoder, upsampling involves adding values ​​to the image FMS' i , so as to achieve the resolution of the images of the input sequence. This operation can be simple (nearest neighbor copying) or include interpolation (linear, polynomial, filtering, etc.).

[0162] If necessary, the transform may optionally include inverse quantization performed on the extracted values. However, inverse quantization is not mandatory.

[0163] During a step E34, the transformed map FMd is extracted by the module XTR' i , or alternatively FMS' i , and optionally additional diagrams FME' i The extraction is based on the current sample sequence P to be decoded of the sequence image. vn The coordinates (x n ,y n ). This extraction can also be performed according to the resolution of the image in question. The sequence of samples to be decoded is processed, for example, in the order from n=1 to n=N.

[0164] According to one embodiment, the feature vector Zd n directly from this extraction.

[0165] In particular, in one embodiment, Zd n is located at the current pixel Pd vn The coordinates (x n ,y n ) at the FMd ior FMS' i (and optionally FME' l ) values ​​form a J-tuple (z1, z2, ..., z J ), such as referring to Figure 7 What is shown.

[0166] According to one embodiment, during step E35, for the coordinates (x n ,y n ) of each sample sequence Pd vn , module TT' is based on the coordinate (x n ,y n ) According to Figure FMd from the first group i Or from the second group of Figure FMS' i Extract the values ​​and optionally append the group from the FME diagram i Extract the values ​​to construct the vector Zd n This step is similar to the reference Figure 4 The same step E26 as described for the encoder is used and the described embodiments are applicable. If necessary, the extraction may include performing a calculation on the extracted values ​​or the formed vector Zd n Perform dequantization.

[0167] During step E36, the value Wc of the convection flow B2 is k Decode to generate the parameters Wd of the neural network MLP' k To this end, any known decoding technique corresponding to the encoding technique used by the encoder can be used. The neural network MLP' is similar to the network MLP, ie it has the same structure and the same parameters, except for the encoding, which can be lossy or lossless.

[0168] According to one embodiment, stream B2 is decoded before stream B1 in order to obtain the neural network before starting to decode the sequence of samples.

[0169] During step E37, the vector Zd n Processed by the neural network MLP' to generate the current sample sequence Pd to be decoded vn As output, that is, according to one embodiment, the sample point Pd vn The position (x) of the color component (Rd, Gd, Bd) of the corresponding image in the sequence is n ,y n ) is inserted into the decoded image I(Pd vn ). This step is the same as reference Figure 4 Step E27 is identical to that described for the encoder.

[0170] When all sequences of samples have been processed, a corresponding sequence of decoded pictures is obtained.

[0171] Figure 7 A schematic diagram showing a decoding method used in an embodiment of the present invention is shown.

[0172] In this illustration, there are two images I(Pd 1n ) and I(Pd 2n ) will be decoded. Usually, V images I(Pd vn ) is decoded by the method and encoding device. In this diagram, the samples (or pixels) are processed in pairs, where the sequence represented by (x n ,y n ) where n varies from 1 to N. (Pd 1n At position (x n ,y n ) is inserted into image 1, and Pd 2n At position (x n ,y n ) is inserted into image 2).

[0173] In this embodiment, there are 4 graphs FMd i In the preferred embodiment, there are 7 graphs.

[0174] In this embodiment, the first map FMd1 has the same resolution as image I and therefore contains W × H variables, where W is the width of the image (in pixels) and H is its height. The second map FMd2 has half the resolution (in each dimension) of FMd1. The resolution of each additional map is half that of the previous one. This structure allows the number of variables in the feature map to be reduced, thereby facilitating decoding while minimizing decoding cost.

[0175] Module SE' upsamples the image FMd2 by a factor of 2 in each dimension, the image FMd3 by a factor of 4 in each dimension, and the image FMd4 by a factor of 8 in each dimension, according to any upsampling method within the capabilities of a person skilled in the art.

[0176] Figure FMS' i Has the same resolution as the image to be decoded, and therefore contains W × H values, where W is the width of the image in pixels and H is its height.

[0177] In this embodiment, the vector Zd n is obtained from the pixel sequence Pd vn The coordinates (x n ,y n ) at the FMS'i The value of the vector Zd forms a 4-tuple (z1...z4). n is optionally dequantized and then processed by the neural network MLP' to generate the two samples to be decoded Pd vn (Pd 1n and Pd 2n ) is output as the corresponding (R, G, B) or (Y, U, V) triplet of the image. The coordinates (x) of the triplet (R, G, B or Y, U, V) in the color components (Rd, Gd, Bd) or (yd, Ud, Vd) of the image are n ,y n ) is inserted into the corresponding decoded image I(Pd 1n ) and I(Pd 2n )middle.

[0178] According to the variant shown in dashed lines, there are 5 graphs: an additional graph FME'0 has been introduced. In this embodiment, the vector Zd n It is a 5-tuple.

Claims

1. A method for encoding a sample point (Pv n , P 1n , P 2n ) of at least two images (I(P vn ), I(P 1n ), I(P 2n )) is a method for encoding a sequence, the method comprising the following steps: - Build step, which includes the following sub-steps: - Construct (E21, E22) the first set of feature maps (FM i ); - for the corresponding image in the sequence and the position in the corresponding image (x n ,y n ) is associated with the current sample point (P vn , P 1n , P 2n ) of the sample sequence: - According to the current sample point (P 1n , P 2n ) of the position (x n ,y n ) according to the feature map (FM) in the first group i ) Construct (E25) feature vector (Z n );as well as - Use a set of parameters (W k ) defines an artificial neural network (MLP) to process the vector (Z) in (E27) n ), in order to provide decoded values ​​(P') representing the current sample points respectively. 1n , P' 2n ) vector sequence; - updating (E22, E27) at least one value of one of said feature maps of said first group and / or at least one parameter of said network as a function of a coding performance measure; - For the first set of feature maps (FM i ) and the set of parameters (W k ) are encoded in steps (E23, E28, EF).

2. Method for encoding a sequence of images according to claim 1, characterized in that The method comprises transforming the first set of feature maps (FM i ) to obtain a second set of feature maps (FMS) with the resolution of the image of the input sequence i ) step (E24), and is characterized in that the characteristic vector (Z n ) is based on the feature map (FM) from the first group i ) obtained in the second group of the transformed feature maps (FMS i ) constructed.

3. The encoding method according to claim 2, wherein: The feature map (FM) in the first group i ) has a lower resolution than the resolution of the image to be encoded, and is characterized in that the transformation operation involves upsampling.

4. The encoding method according to any one of the preceding claims, characterized in that The eigenvector (Z n ) consists of extracting the current sample point (P vn ) is the same position as (x n ,y n ) in the at least one feature map (FM i , FMS i ) value of the substep.

5. The encoding method according to claim 1, wherein: The eigenvector (Z n ) consists of the following sub-steps: - According to the current sample point (P vn ) of the position (x n ,y n ) extract the feature map (FM) in the first group i ) multiple values; - Processing (E26) the extracted values ​​to obtain the feature vector.

6. The encoding method according to any one of the preceding claims, characterized in that The method involves constructing another set of feature maps (FME l ) steps (E21, E22), and is characterized in that the feature vector is also constructed based on the feature map in the other group.

7. The encoding method according to any one of the preceding claims, characterized in that Encoding the first set of feature maps includes an entropy encoding sub-step.

8. A method for decoding a sample point (P vn , P 1n , P 2n ) of at least two images (I(Pd vn ), I(Pd 1n ), I(Pd 2n )) is a method for decoding a sequence, the method comprising the following steps: - For the first set of feature maps (FMd i ) to decode (E31); - A set of parameters (Wd k ) to decode (E36); - for the corresponding image in the sequence to be decoded and the position in the corresponding image (x n ,y n ) is associated with the current sample point (Pd vn ) sample sequence: - According to the position of the current sample point (x n ,y n ) According to the feature map (FMd) in the first group i ) Construct (E34) feature vector (Zd n ); as well as: - Using the decoded parameters (Wd k ) defines an artificial neural network (MLP') to process the vector (Zd n ), so as to provide the current sample point (Pd vn ) is a vector sequence.

9. The decoding method according to claim 8, wherein: The method comprises transforming the first set of decoded feature maps (FMd i ) to obtain a second set of feature maps (FMS') with the resolution of the image of the input sequence i ) step (E33), and is characterized in that the characteristic vector (Zd n ) is based on the decoded feature map (FMd i ) obtained in the second group of the transformed feature maps (FMS' i ) constructed.

10. The decoding method according to claim 9, wherein: The characteristic map (FMd in the first group i ) has a lower resolution than the resolution of the image to be decoded, and is characterized in that the transformation operation involves upsampling.

11. The decoding method according to any one of claims 8 to 10, characterized in that: The characteristic vector (Zd n ) consists of extracting the current sample point (Pd vn ) is the same position as (x n ,y n ) in the at least one feature map (FMd i , FMS' i ) value of the sub-step (E34).

12. The decoding method according to claim 8, wherein: The characteristic vector (Zd n ) consists of the following sub-steps: - According to the current sample point (Pd vn ) of the position (x n ,y n ) extract the feature map (FMd) in the first group i ) multiple values; - Processing (E35) the extracted values ​​to obtain the feature vector.

13. The decoding method according to any one of claims 8 to 12, characterized in that: The method involves constructing another set of feature maps (FME' l ) step (E32), and is characterized in that the feature vector is also constructed based on the feature map in the other group.

14. The decoding method according to any one of claims 8 to 13, characterized in that: Decoding (E31) said first set of feature maps comprises an entropy decoding sub-step.

15. A method for encoding a sample point (Pv n , P 1n , P 2n ) of at least two images (I(P vn ), I(P 1n ), I(P 2n )) is a device for encoding a sequence, characterized in that The device is configured to perform the following operations: - Construct (GEN, MAJ) the first set of feature maps (FM i ); - for the corresponding image in the sequence and the position in the corresponding image (x n ,y n ) is associated with the current sample point (P vn , P 1n , P 2n ) of the sample sequence: - According to the current sample point (P 1n , P 2n ) of the position (x n ,y n ) according to the feature map (FMS) in the first group i ) Construct (XTR) feature vector (Z n ); - Use a set of parameters (W k ) defines an artificial neural network (MLP) to process the vector (Z n ), in order to provide decoded values ​​(P') representing the current sample points respectively. 1n , P' 2n ) vector sequence; - updating (MAJ, NND) at least one value of one of said feature maps and / or at least one parameter of said network according to a coding performance measure; - For the first set of feature maps (FM i ) and the set of parameters (W k ) to perform encoding steps (FMC, NNC).

16. A method for decoding a sample point (Pd vn , Pd 1n , Pd 2n ) of at least two images (I(Pd vn ), I(Pd 1n ), I(Pd 2n )) is a device for decoding a sequence, characterized in that The device is configured to perform the following operations: - For the first set of feature maps (FMd i ) for decoding (FMD); - A set of parameters (Wd k ) for decoding (NND); - for the corresponding image in the sequence to be decoded and the position in the corresponding image (x n ,y n ) is associated with the current sample point (Pd vn ) sample sequence: - According to the position of the current sample point (x n ,y n ) According to the feature map (FMd) in the first group i ) Construct (E34, XTR') feature vector (Zd n ); as well as: - Using the decoded parameters (Wd k ) defines an artificial neural network (MLP') to process the vector (Zd n ), so as to provide the current sample point (Pd vn ) is a vector sequence.

17. A computer program comprising instructions for executing the steps of the encoding method according to claim 1 or the decoding method according to claim 8 when said program is executed by a computer.

Images