Method and device for encoding and decoding images

Abstract

The invention relates to a device for decoding a region of a signal, called the decoding region, said decoding region comprising a plurality of samples to be decoded, characterized in that said decoding device is configured to perform: the decoding (F21) of a first mask representing a region in the domain of said signal, the decoding of a group of at least one feature map representing at least said decoding region, said decoding comprising: the determination (F22) of a dilated feature area in a group of at least one feature map, said dilated feature area being co-located with said first mask, and the decoding (F23) of the values ​​of the points of said dilated feature area, the decoding (F21) of an erosion rule, and the synthesis (F24-F29) of said decoding region from the points of said dilated feature area and the erosion rule to obtain the signal Decoded. Figure for the abbreviation: Fig. 7.

Description

Title of the invention: Method and device for encoding and decoding images. Prior art.

[0001] The invention relates to the general field of coding one-dimensional or multidimensional signals. It relates more particularly to the compression of digital images or videos.

[0002] Digital videos are generally subject to source coding aimed at compressing them in order to limit the resources required for their transmission and / or storage. Numerous coding standards exist, such as the ITU / MPEG standards (H.264 / AVC, H.265 / HEVC, H.266 / VVC, etc.) and their extensions (MVC, SVC, 3D-HEVC, etc.). In these approaches, image encoding is generally performed by predicting pixels using previously encoded and then decoded pixels present in the image being encoded, in which case it is called "Intra prediction," or previously encoded images, in which case it is called "Inter prediction."

[0003] In addition to these classic approaches, approaches based on artificial intelligence, and in particular neural networks, tend to develop.

[0004] Some neural approaches, starting from an input signal, for example an image, train a so-called synthetic neural network on characteristic vectors associated with a position of a sample of the input signal to be encoded. These characteristic vectors are constructed from feature maps that may be at the resolution of the input signal, or at a lower resolution. During training, or construction, the parameters of the neural network and the values ​​of the feature maps are updated according to a performance measure, for example, a data rate-distortion type. When training is complete, i.e., when the performance measure obtained is satisfactory, the actual encoding of the synthetic neural network parameters and the values ​​of the feature maps can be performed and stored or transmitted to the decoder.The decoding of the coded signal is then carried out by applying the synthesis neural network to the feature maps.

[0005] One drawback of the classical and neural approaches described above is that they do not allow the creation of independently decodable areas in the coded signal, which limits not only the ability to interact with the semantic content of the coded signal but also the possibility of limiting the memory required at the decoder level or of parallelizing the decoding of this coded signal.

[0006] There is therefore a need for a solution to improve upon the classical and neural approaches described above. Summary of the invention

[0007] The invention relates to a method for encoding a region of a signal, called the encoding region, said encoding region comprising a plurality of samples to be encoded, said encoding method comprising the following steps: - obtaining a group of at least one characteristic map representative of said signal, - obtaining a mask, called the first mask, in the signal of the zone to be coded, - obtaining a rule for the expansion of said first mask, - obtaining a dilated mask, called the second mask, by applying the said dilation rule to the said first mask, - the determination of a characteristic area expanded within said group of at least one feature map as a function of said second mask, - the coding of the value of the points of said dilated characteristic zone, - the determination of an erosion rule of said second mask, and - the coding of said second mask and of said erosion rule.

[0008] The invention also relates to a method for decoding a region of a signal, called the region to be decoded, said region to be decoded comprising a plurality of samples to be decoded, said decoding method comprising the following steps: - the decoding of a first mask representing an area within the domain of said signal, - the decoding of a group of at least one characteristic map representative of at least the said area to be decoded, said decoding comprising: • the determination of a dilated characteristic area within a group of at least one feature map based on said first mask, and • the decoding of the value of the points in said dilated characteristic zone, - the decoding of an erosion rule, and - the synthesis of said area to be decoded from the points of said dilated characteristic area and the erosion rule to obtain the decoded signal.

[0009] For the purposes of the invention, encoding, or "coding", means the operation of representing a set of samples in a compact form. for example by a digital binary stream. Decoding is understood to be the operation of processing a digital binary stream to produce decoded samples.

[0010] By "sample" of the signal, we mean a value taken from the signal. Sampling the signal produces a sequence of discrete values ​​called samples. In the case of an image signal, the sample is called a pixel, which can be, for example, a color pixel traditionally represented by a triplet of values, for example (R, G, B) or (Y, U, V). The position of the sample is located by its abscissa (x) and ordinate (y) coordinates in the image.

[0011] By "signal comprising a plurality of samples" is meant a signal with one (audio, sound), two (image) or more than two (stereoscopic image, multiscopic image, image associated with a depth map, video, etc.) dimensions. Depending on this dimensionality, the sample has one, two, or more coordinates in the signal. In the case of an image signal, the position of the sample is located by its abscissa (x) and ordinate (y) coordinates.

[0012] By "feature maps" is meant an abstract representation of the signal comprising a plurality of variable scalar data, discrete or not, also called values, for example real or integer numbers, signed or unsigned. These maps are also known as "latent representations" or "representations in the transformed domain".

[0013] “By region expansion rule”, we mean a procedure allowing to Transforming a region involves increasing its size (one-dimensional region), area (two-dimensional region), volume (three-dimensional region), or space (higher-dimensional region). A region expansion rule, for example, is the Minkowski sum, which defines the morphological expansion of that region by adding a structuring element or a number of pixels to its boundary.

[0014] “By a region erosion rule”, we mean a procedure allowing to Transforming a region involves decreasing its size (one-dimensional region), area (two-dimensional region), volume (three-dimensional region), or space (higher-dimensional region). An example of a region erosion rule is the morphological erosion of that region by a structuring element, or a number of pixels to be removed from the region's edge.

[0015] Generally speaking, the steps of an encoding or decoding process should not be interpreted as being linked to a notion of temporal succession. In other words, the steps may be carried out in a different order than that indicated in the independent encoding or decoding claim, or even in parallel.

[0016] The coding method according to the invention encodes a region of a signal from a representation of that signal in the form of characteristic maps by identifying in these characteristic maps the set of values ​​necessary to subsequently decode said region. In other words, the encoding of this set of values, called the expanded characteristic region, makes it possible to obtain a coded representation of the region that can subsequently be decoded independently of any other part of the signal.

[0017] Furthermore, this dilated characteristic area can be determined by using a mask encompassing the area to be encoded in the signal domain whose minimum area depends predictably on the type of encoding and / or decoding used to encode and / or decode the characteristic maps.

[0018] Thus, such a coding / decoding process improves the ability to interact with the semantic content of the coded signal and offers the possibility of parallelizing the decoding process of the entire signal.

[0019] According to one embodiment of the decoding process, the synthesis includes the determination of a mask of the area to be decoded in the domain of said signal, called second mask, by application of said erosion rule to said first mask.

[0020] According to one embodiment of the decoding process, the synthesis comprises: - the processing of said dilated characteristic area to obtain an intermediate signal, and - the application of said second mask to said intermediate signal to obtain said plurality of samples from said area to be decoded.

[0021] According to one embodiment of the decoding process, the synthesis includes processing said dilated characteristic area to obtain only the decoded signal corresponding to the second mask.

[0022] According to one embodiment of the decoding process, the synthesis includes processing said dilated characteristic area taking into account said second mask to obtain the decoded signal corresponding to the second mask.

[0023] According to one embodiment of the decoding process, the erosion rule is a morphological erosion by a structuring element.

[0024] According to one embodiment of the decoding process, the erosion rule is defined by a maximum distance, the second mask comprising the points of the first mask located at a distance from an edge of the first mask in at least one direction inwards of the first mask greater than said maximum distance.

[0025] According to one embodiment of the decoding process, the erosion rule is defined by a number of parameters equal to twice the dimension of said signal, each direction of one of the dimensions of said signal being associated with a (possibly) different parameter, said second mask being obtained by removing said first mask, in at least one direction of a dimension of said signal, a number of points equal to said parameter associated with said at least one direction of a dimension of said signal.

[0026] According to one embodiment of the decoding process, the points of said dilated characteristic area are representative of said area to be decoded in a latent domain and the decoding process further comprises the decoding of the parameters of a neural network, called a synthesis neural network, in which the synthesis comprises, for at least one sample, called the current sample, of the signal to be decoded, associated with a position in the signal to be decoded: - the construction of a characteristic vector from said characteristic values ​​as a function of said position of said current sample, and - the processing of said characteristic vector by the synthesis neural network defined by the decoded parameters to provide a decoded value of the current sample.

[0027] According to one embodiment of the decoding process, the characteristic values ​​are representative of said area to be decoded in a transformed domain, for example associated with a direct discrete cosine transform or a wavelet transform, and the synthesis includes the application of said characteristic values ​​of an inverse transform associated with said direct transform.

[0028] Correspondingly, the invention also relates to a device for encoding a region of a signal, called the encoding region, said encoding region comprising a plurality of samples to be encoded, characterized in that said encoding device is configured to implement: - obtaining a group of at least one characteristic map representative of said signal, - obtaining a mask, called the first mask, in the signal of the zone to be coded, - obtaining a rule for the expansion of said first mask, - obtaining a dilated mask, called the second mask, by applying the said dilation rule to the said first mask, - the determination of a characteristic area expanded within said group of at least one feature map as a function of said second mask, - the coding of the value of the points of said dilated characteristic zone, - the determination of an erosion rule of said second mask, and - the coding of said second mask and of said erosion rule.

[0029] The invention also relates to a device for decoding a region of a signal, called the region to be decoded, said region to be decoded comprising a plurality of samples to be decoded, characterized in that said decoding device is configured to implement: - the decoding of a first mask representing an area within the domain of said signal, - the decoding of a group of at least one characteristic map representative of at least the said area to be decoded, said decoding comprising: • the determination of a dilated characteristic area within a group of at least one feature map based on said first mask, and • the decoding of the value of the points in said dilated characteristic zone, - the decoding of an erosion rule, and - the synthesis of said zone to be decoded from the points of said dilated characteristic zone and the erosion rule to obtain the decoded signal.

[0030] The characteristics and advantages of the coding or decoding process apply in the same way to the coding or decoding device according to the invention and vice versa.

[0031] The invention also relates to a computer program on a recording medium, this program being capable of being implemented in a computer or an encoding or decoding device according to the invention. This program includes instructions adapted to the implementation of the corresponding method. This program can use any programming language and be in the form of source code, object code, or code intermediate between source code and object code, such as in a partially compiled form, or in any other desirable form.

[0032] The invention also relates to a computer-readable information or recording medium containing the aforementioned computer program instructions. The information or recording medium may be any entity or device capable of storing programs. For example, the medium may include a storage means, such as a ROM, for example a CD-ROM or a microelectronic circuit ROM, or a magnetic recording means, for example a floppy disk or a hard drive, a DNA sequence, or flash memory. Furthermore, the information or recording medium may be a transmissible medium such as an electrical or optical signal, which may be transmitted via an electrical or optical cable, by radio link, by wireless optical link, or by other means.

[0033] The program according to the invention can in particular be downloaded onto an Internet-type network.

[0034] Alternatively, each information or recording medium may be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of a process according to the invention. Brief description of the figures

[0035] The invention will be better understood with the aid of the following description, given solely by way of example and made with reference to the accompanying drawings in which: - Figure [1] schematically represents a coding device according to a first embodiment of the invention, - Figure [Fig. 2] schematically represents a decoding device according to the first embodiment of the invention, - Figure [Fig. 3] illustrates an example of a synthetic artificial neural network used in the context of the invention, - [Fig.4] schematically represents an example of determining a characteristic area dilated by the encoding device of [Fig.1] or by the decoding device of [Fig.2], - [Fig.5] is a logic diagram representing an example of a coding process that can be implemented by the coding device of [Fig.1], - [Fig.6] illustrates a coding method used by the coding device of [Fig.1], - [Fig.7] is a flowchart representing an example of a decoding process that can be implemented by the decoding device of [Fig.2], - [Fig.8] illustrates a decoding process used by the decoding device of [Fig.2], - [Fig.9] is a flowchart representing a method for encoding feature cards that can be implemented by the encoding device of [Fig.1] and by the encoding process of [Fig.5], - [Fig. 10] is a logic diagram representing a method for decoding feature cards that can be implemented by the decoding device of [Fig. 2] and by the decoding process of [Fig. 7], - Figure

[11] schematically represents a transcoding device according to a first embodiment of the invention, Detailed description of the invention

[0036] Figure 1 schematically represents, according to a first embodiment, an ENC coding device for at least one zone of a signal (I(Pn)). In the example described here, the signal I(Pn) is segmented into a set of J zones, and all zones are coded independently. Alternatively, only one zone or only a few zones may be coded independently.

[0037] This ENC coding device comprises a GEN module for generating feature maps, an SE module for transformation, and an XTR module for extraction. data, a TT processing and quantification module, an NNSYN module corresponding to a synthetic artificial neural network, an NNC neural network coding module capable of coding the synthetic neural network, an FMC feature map coding module, an EVAL coding performance evaluation module, and an MAJ update module.

[0038] The ENC coding device also includes an FMD decoding module for feature maps, a SEG module for obtaining zone segmentation, an ORD module for obtaining a dilation rule, an ORE module for obtaining an erosion rule, a CRE module for coding the erosion rule, a DMZ module for obtaining a dilated zone mask, a CDMZ module for coding this dilated zone mask, a DZCD module for determining dilated feature zones in feature maps, and a CE module for coding the value of the points of the dilated feature zones.

[0039] The ENC coding device can be implemented by means of an electronic device comprising a processor and a memory, not shown; each of the modules mentioned above can then be realized by the cooperation of the processor and computer program instructions stored in the aforementioned memory and designed to perform the functionalities of the module concerned, in particular as described below, when these instructions are executed by the processor.

[0040] The ENC coding device of [Fig. 1] receives as input a sequence of samples to be coded, denoted Pn, for example, a temporal sequence of sound samples, or a set of image data denoted I(Pn). In this second case, the image signal I(Pn) can represent a two-dimensional image, or a plurality of two-dimensional images (video, color components, stereoscopic components, multiscopic components, etc.). Pn designates a sample n of the input signal comprising N samples. In one embodiment, the signal is a color image signal represented by means of at least one two-dimensional representation, such as a pixel matrix, each pixel having a red (R), green (G), blue (B) component, or, alternatively, a luminance (Y) component and at least one chrominance (U, V) component. The location of each pixel is defined by its abscissa and ordinate (x and y) coordinates in the image.In one embodiment, the image is in greyscale represented by means of a two-dimensional representation, such as a pixel matrix, each pixel having a greyscale component, or luminance.

[0041] The GEN feature map generation module is configured to generate a plurality of M feature maps denoted FM;.

[0042] In one embodiment, the SE module performs a transformation of the first group of FM characteristic maps; to generate a second group of FMS characteristic maps; at the same resolution as the input signal.

[0043] The optional SE module can perform quantization of the data extracted from this set of M FM maps, or of the vector Zn constructed from this data. Recall that quantizing a value refers to mapping that value to a member of a discrete set of possible code symbols. For example, the set of possible code symbols may consist of integer values, and the quantization system performs a simple rounding of a real value to an integer. In another example, quantization consists of multiplication by a given value followed by rounding. The SE module then performs a transformation of the values ​​of at least one of the feature maps, for example, oversampling, interpolation, filtering, etc. After the transformation, a transformed feature map from the second group has the same resolution as the images in the input sequence.Advantageously, in this method, the feature maps that are coded can be of lower resolution than the images to be coded, while the maps of the second group, which are used to construct the feature vectors, are at the same resolution as the image sequence, which facilitates the extraction of values.

[0044] In one embodiment, the SE module is absent; in this case, the values ​​that will be used to construct the characteristic vector are extracted from the first group of characteristic maps.

[0045] The XTR module performs value extraction from the FM (and / or FMSi) feature maps for a current sample Pn to be encoded, based on its coordinates in the input signal. For example, if one seeks to encode the sample Pn at the coordinates (xn, yn) of an input image, the XTR module performs value extraction from the maps at positions imposed by the coordinates (xn, yn).

[0046] In one embodiment, the extracted values ​​constitute the vector Zn. Zn is an L-tuple, meaning it contains L elements, or data z;. For example, in one embodiment, L=M, meaning that only one value is extracted for each feature map FM;. In another embodiment, L>M, meaning that several values ​​are extracted for at least one feature map FM;. The dimension L of the vector depends on the topology of the NNSYN synthesis neural network and, more particularly, on the topology of the input layer of this NNSYN synthesis neural network. The vector Zn with index n refers to the characteristic vector of pixel P'n.

[0047] In one embodiment, the optional TT module processes the extracted values ​​to generate the vector Zn. The TT module can quantify the data extracted from the set of feature maps. The processing may include other operations, such as filtering, scaling, etc. In particular, if the SE module is not used and if the feature maps in the first group have lower resolutions than the images in the sequence, the TT module can take into account the coordinates of the values ​​in the lower-resolution maps.

[0048] It should be noted that at least one of the SE or TT modules must perform a quantification of the characteristic maps.

[0049] The NNSYN module is a synthetic neural network defined by K parameters Wk, capable of processing the input vector Zn, or L-Tuple, to generate as output a second vector representative of the sample Pn to be coded.

[0050] An example of a synthetic neural network is presented later with reference to [Fig.3].

[0051] The NNC module performs the coding of the synthetic neural network, specifically its parameters Wk. During the coding training, or construction, process—that is, as long as the performance evaluation step is not satisfactory—the NNC module performs a coding simulation, followed by decoding, for the evaluation module. Subsequently, it performs the actual coding of the synthetic neural network parameters Wk. The coded parameters are denoted Wck. As is known, the coding simulation can be identical to the actual coding, or it can approximate it.

[0052] The FMC module performs the encoding of the FM maps; that is, the values ​​of the feature maps of the first group (excluding the maps of the second group, which optionally result from oversampling by the SE module). During the encoding training, or construction, process—that is, as long as the performance evaluation step is not satisfactory—the FMC module performs a coding simulation, followed by decoding, for the evaluation module. Subsequently, it performs the actual encoding of the FM map values. The encoded maps are denoted FMc;. In a known manner, the coding simulation can be identical to the actual encoding, or it can approximate it. The encoding module quantifies, if necessary, the latent representation of the values ​​of the maps of the first group using a quantifier to generate an ordered collection of quantized values.Then the coding module compresses the quantized data, using coding that takes into account the proximity of a value to be coded in the feature map.

[0053] The EVAL module performs an evaluation and minimization of coding performance. The evaluation function is, for example, of the rate-distortion type. Minimization can be performed by gradient descent, or any other method within the grasp of a person skilled in the art.

[0054] The MAJ module performs an update of the values ​​of the FM cards; to be encoded and / or the parameters of the Wk synthesis neural network, according to the results of the performance function.

[0055] When the evaluation function is minimized, or a predefined number of iterations have been performed, the coded values ​​FMc and Wck constitute the compressed representation of the input signal I(Pn). At this stage, it is not possible to decode a specific area of ​​the coded signal from this compressed representation without decoding the entire coded signal.

[0056] To achieve this functionality, the FMD module performs the decoding of the FMc encoded values. The cards decoded by the FMD module, numbering M, are denoted FMdi.

[0057] The SEG module performs a segmentation into J (greater than or equal to 2) zones ZoJ of the sequence of samples to be encoded Pn. This segmentation operation allows the samples to be grouped into J different homogeneous sets according to one or more predefined criteria. For example, in the case of a temporal sequence of sound samples, the segmentation operation makes it possible to obtain sequences of units corresponding to silences, noises, phonemes, words, etc. Similarly, in the case of an image signal I(Pn), the segmentation operation makes it possible to group the pixels Pn of the image signal into J homogeneous zones according to criteria, notably intensity or spatial criteria. For example, the segmentation operation can make it possible to identify in the image signal I(Pn) two different zones (J=2), one corresponding to the background of the image and the other to the foreground of the image.Segmentation is, for example, represented as a set of J masks MZo', each allowing the identification of a zone Zo', each mask being associated with a value different from that associated with the other masks.

[0058] The ORD module obtains a dilation rule Rd of a region of the signal I(Pn) to obtain a dilated region.

[0059] This dilation rule is applied by the DMZ module to each mask MZo' to obtain a dilated mask MZD'.

[0060] These expanded masks MZD' are encoded by the CDMZ module. This encoding can be achieved by independently encoding each expanded mask MZD', or, in yet another variant, by independently encoding the contours of the J expanded masks MZD'. The encoding of the expanded masks MZD' is denoted MZD'c.

[0061] For each dilated mask MZD', a dilated characteristic area ZCD' of the FMdi maps is determined by the DZCD module. This dilated characteristic area ZCD' comprises all the points of the FMd maps; co-located (possibly up to a scale factor) with the points of a dilated mask MZD'.

[0062] The ORE module obtains an erosion rule Re from the dilation rule Rd. This erosion rule makes it possible to obtain the mask MZo' or an approximation thereof from the dilated mask MZD'.

[0063] The CRE module performs the coding of the erosion rule Re obtained by the ORE module in a coded form denoted Rec.

[0064] The CE module performs the entropic coding for each zone Zo* of the value of the points of the dilated characteristic zones ZCD* in the form of different coded data for each zone Zo'.

[0065] In a first embodiment, the CE module only codes the values ​​of the points in the expanded characteristic zone ZCD* to the exclusion of any other point in the FMd characteristic maps;.

[0066] Alternatively, the CE module creates a secondary feature map p\42d' for each expanded characteristic zone ZCD'. The value of a point on the secondary feature map PA42d' is equal to the value of the point on the FMd map; if this point belongs to the expanded characteristic zone ZCD*, it is a predetermined value, for example, zero, otherwise. The CE module then encodes the entire FM2d feature map to obtain different encoded data for each zone Zo'.

[0067] Thus, the coded values ​​Wck, Rec, and MZD'c constitute the compressed representation of the Zo* region of the input signal I(Pn). These coded values ​​are represented as binary streams Bl, B2j, and B3j. Therefore, according to this example, the binary stream Bl comprises the coded values ​​Wck, the binary stream B2j the coded values, and the binary stream B3j the coded values ​​Rec and MZD'c.

[0068] Thus, thanks to the invention, each zone Zo* is coded independently of the other zones, which allows their subsequent decoding to be carried out independently and possibly in parallel.

[0069] It should be noted that the ENC coding device for an area of ​​a signal I(Pn) described above comprises the successive application of a coding device for the signal I(Pn) as a whole in order to produce a stream B0 comprising the coded values ​​FMc; and of a transcoding device for this coded signal as a whole into a code of one or more zone(s) of the signal I(Pn), the said zone(s) being decodable independently of each other.

[0070] According to this embodiment, the transcoding device includes the FMD decoding module for feature maps, the SEG module for obtaining a segmentation into zones, the ORD module for obtaining a dilation rule, the ORE module for obtaining an erosion rule, the CRE module for encoding the erosion rule, the DMZ module for obtaining a dilated zone mask, the CDMZ module for encoding this dilated zone mask, the DZCD module for determining dilated feature zones in the feature maps and the CE module for encoding the value of the points of the dilated feature zones.

[0071] Fig. 2 schematically represents a DEC decoding device for a Zodj zone of a signal, called the decoding zone, said decoding zone Zodj comprising a plurality of Pdn samples to be decoded.

[0072] The DEC decoding device of Figure 2 receives as input data streams Bl1, B2j, and B3j. Stream B1 includes the encoded parameters Wck of the synthesis neural network NNSYN'. Stream B2j includes a group of encoded data gcJ corresponding to the M feature maps representative of the decoded area Zodj, and stream B3j includes a coded form Rec of an erosion rule and the encoded parameters of an expanded mask MZD'c of the decoded area Zodj.

[0073] This decoding device DEC includes an NND module for decoding neural network(s) capable of decoding the synthesis neural network NNSYN', a DRE module capable of decoding an erosion rule Re, a DDMZ module capable of decoding a dilated mask MZD' of the area to be decoded in the signal domain, an EMZ module for obtaining a mask MZo' of the area to be decoded in the signal domain, a DZCD module capable of determining a dilated characteristic area ZCD' in the M feature maps, a DE module for decoding feature maps from the encoded data group pcÿ, an XTR' module for data extraction, an SE' module for inverse transformation, a TT' module for inverse processing and quantization, an NNSYN' module corresponding to a synthesis neural network and an optional EXTR module for extracting samples belonging to the Zodj(Pn) area.

[0074] According to one embodiment, the DEC decoder produces at output a decoded area of ​​the image Zodj(Pdn) comprising a plurality of decoded samples Pdn.

[0075] The DRE module obtains an erosion rule by decoding the Rec encoded parameters.

[0076] The DDMZ module performs the decoding of an expanded mask MZD' of the area to be decoded from the MZD'c data of the B3 stream.

[0077] From the erosion rule Re and the expanded mask MZDj, the EMZ module determines the mask MZodj of the area to be decoded ZoJ. The mask MZodj is identical to the mask MZo' if the erosion rule is the inverse function of the expansion rule used by the ENC encoder, otherwise it is different.

[0078] The DZCD module performs the determination of the dilated characteristic zone ZCD' in the characteristic maps. This characteristic zone ZCD' includes all the points of the FMd maps; co-located (possibly up to a scale factor) with the points of the expanded mask MZD'.

[0079] The DE module decodes the values ​​of the points in the dilated characteristic zone ZCD* from the coded data pcJ in order to obtain the characteristic maps FM^-

[0080] According to embodiments as described for the encoder: - The p\Id- maps are decoded independently of the ZCD* mask and then possibly only the values ​​corresponding to points of the characteristic zone ZCD* are kept. - pM^ j maps are decoded taking into account the ZCD* mask when the coded data pc,i only include values ​​corresponding to points in the characteristic zone ZCDj.

[0081] The NND module decoded the parameters of the NNSYN' synthesis neural network. These parameters are denoted Wdk.

[0082] In one embodiment, the SE' module performs a transformation of the first set of decoded feature maps p^ji to generate a second set of feature maps with the same resolution as the signal to be decoded, denoted p]\ / [$'i. The SE' module optionally performs inverse quantization corresponding to the quantization performed by the encoder. Inverse quantization is not necessary if the encoder's quantizer Q has simply rounded the real values ​​submitted to it. Inverse quantization is also unnecessary if the neural network is capable of handling quantization of its input data. Otherwise, the decoder performs the inverse operation of the quantizer Q. Then, the SE' module performs a transformation of the feature map values, including, for example, oversampling, interpolation, filtering, etc., similar to that performed by the encoder.At the end of the transformation, a characteristics map. The transformed image of the second group has the same resolution as the images in the sequence to be decoded.

[0083] In one embodiment, the SE' module is absent; in this case, the values ​​that will be used to construct the characteristic vector are extracted from the first group of characteristic maps.

[0084] The XTR' module is identical to the XTR module of Figure 1. It performs an extraction of values ​​from the M characteristic maps (and / or pMS'-! according to one of the embodiments described above), for a sample Pdn to be decoded as a function of its coordinates in the signal to be decoded.

[0085] In one embodiment, the extracted values ​​constitute the vector Zdn. Zdn is an L-tuple, that is to say, it comprises L elements, or data zd;.

[0086] In one embodiment, the optional TT' module processes the extracted values ​​to generate the Zdn vector. The TT' module can perform inverse quantization of the data extracted from the feature set. The processing may include other operations, such as filtering, scaling, etc., similar to those performed by the encoder.

[0087] The NNSYN' module is a so-called synthetic neural network, defined by K parameters Wdk, capable of processing the input vector Zdn, or L-tuple, to generate as output a second vector representing a decoded sample, generally a vector containing A elements. In one embodiment, K=3 and the output vector is the triplet (R, G, B) of the decoded sample. The NNSYN' module has the same structure as the NNSYN module, and its parameters are either identical if the encoding of its parameters Wk is lossless, or different if the encoding is lossy.

[0088] The processing of the decoded pjyjjj maps described above generates an intermediate image I;(Pn) which contains samples that may not belong to the decoded area Zod(Pdn). To obtain this decoded area, the EXTR module transforms the intermediate image I;(Pn) into a signal corresponding to the Zodj(Pn) area of ​​the image I(Pdn). To do this, the EXTR module retains the samples from the intermediate image I;(Pn) belonging to the mask MZodj (provided to the EXTR module by the EMZ module) while the others are removed, the resulting signal then corresponding to the Zodj(Pn) area of ​​the image I(Pdn).

[0089] Alternatively, the NNSYN' module only processes the input vectors Zdn capable of generating as output a second vector representative of a sample Pdn to be decoded. This selection of the Zdn vectors is performed by the NNSYN' module taking into account the mask MZodj of the area to be decoded. In this variant, the intermediate image I;(Pn) corresponding directly to the zone to be decoded Zodj(Pn) without the DEC decoder including the EXTR module.

[0090] The DEC decoding device can successively be implemented to decode all the Zodj areas in order to reconstruct all the samples to be decoded of the signal, i.e. of the image I(Pdn).

[0091] Alternatively, a plurality of DEC decoding devices can be implemented in parallel, so as to decode the different areas of the image in parallel. Once all the areas are decoded, the image can be reconstructed by compositing the different areas.

[0092] The DEC decoding device can be implemented by means of an electronic device comprising a processor and a memory, not shown; each of the modules mentioned above can then be implemented through the cooperation of the processor and computer program instructions stored in the aforementioned memory and designed to perform the functionalities of the module concerned, in particular as described below, when these instructions are executed by the processor. The DEC device can also comprise a plurality of processors, the processors being dedicated to the parallel decoding of image areas.

[0093] Figure 3 illustrates an example of a synthetic artificial neural network used for encoding and decoding in the context of embodiments of the invention.

[0094] The synthetic artificial neural network used for encoding, NNSYN, and the synthetic artificial neural network used for decoding, NNSYN', are defined by an identical structure, comprising for example a plurality of layers of artificial neurons, and by a set of weights and activation functions associated respectively with the artificial neurons of the network concerned.

[0095] The synthetic neural network is, according to one embodiment, an MLP, or Multi Layer Perceptron, followed by one or more convolutional neural network(s) Ci, ... Cn, each of the convolutional neural networks being associated with a filtering mask of predefined size, for example of size 3x3.

[0096] The MLP consists of an input layer adapted to the input format (the L-tuple), optionally one or more hidden layers, and an output layer providing an intermediate output vector Vsn, for example a vector containing A' elements. The intermediate output vector is then processed successively by the convolutional neural network(s) to provide an output vector also containing A elements.

[0097] Thus, a vector representation of a current sample (a vector Zn or Zdn from the FM / FMS feature maps, at the encoder or FMd^FMS'- at the decoder) is applied to the input (i.e., to the input layer) of the neural network Artificial synthesis NNSYN or NNSYN' produces the output vector. In one embodiment, A is equal to 3 and the output vector is the triplet (R, G, B) of the pixel P'n encoded and then decoded or of the pixel Pdnd decoded by the decoder.

[0098] The concatenation of all these reconstructed pixels in an image constitutes the reconstructed signal Zodj (Pdn), according to an example an image I containing only the Zodj area.

[0099] At the encoder, the NNSYN synthetic artificial neural network is trained on the image so as to minimize the differences between the input representation of the current image I(Pn) and its output representation I(P'n), while also minimizing the amount of data to be encoded. The EVAL module performs a performance measurement in this regard.

[0100] Once the encoder training is complete, the network parameters are encoded, either losslessly, in which case the NNSYN' neural network is identical to NNSYN, or lossily, in which case the NNSYN' network may be slightly different from NNSYN.

[0101] With reference to [Fig. 4], we will now describe an example of determining a dilated characteristic area ZCD1 associated with an area Zo1 in the case of an image signal I(Pn) segmented into three areas Zo1, Zo2, Zo3 identified by their masks MZo1, MZo2 and MZo3. [Fig. 4] thus presents two steps (1)-(2) for determining the dilated characteristic area corresponding to the area Zo1.

[0102] In the example described, the dilated characteristic area is determined in four input maps FMdB, FMd2, FMd3, and FMd4 of the NNSYN / NNSYN' synthesis network. It is further assumed in this example that the largest convolutional kernel used by the convolutional neural networks of the NNSYN / NNSYN' synthesis neural network is a 3x3 convolutional kernel.

[0103] Thus, in the example presented, decoding a sample Pdn[ typically requires access not only to the point values ​​of the FMdB, FMd2, FMd3, and FMd4 maps co-located with this sample Pdn[, but also to the point values ​​of the FMdi, FMd2, FMd3, and FMd4 maps co-located with the Pdn2 samples belonging to the neighborhood used during filtering of this point Pdni. In other words, when the sample Pdn[ is on the edge of zone Zo1, its decoding requires access to points of the FMdi, FMd2, FMd3, and FMd4 maps co-located with a Pdn2 sample belonging to a Zo2 zone different from the Zo1 zone. The set of points of the FMdB, FMd2, FMd3, and FMd4 maps necessary to decode zone Zo1 defines the dilated characteristic zone ZCD1 of these FMdi, FMd2, FMd3, and FMd4 maps.

[0104] This determination includes, in a step (1), obtaining a dilated mask MZD1 by applying a dilation rule to the mask MZo1.

[0105] In the example described here, the dilation rule is defined by a structuring element ES, namely a 3x3 square corresponding to a convolution kernel used within the NNSYN / NNSYN' synthesis neural network. In this example, the dilated mask MZD1 is thus obtained by a morphological dilation of the mask MZol by this 3x3 square structuring element.

[0106] Alternatively, the dilation rule can be defined in the case of an image I(Pn) by four values ​​indicating the number of points from the maps FMdb, FMd2, FMd3, and FMd4 to be added for each point on the edge of the region R1 in both the horizontal and vertical directions. In the example given, all four values ​​are equal to one.

[0107] In another variant, the structuring element can be defined by a scale factor indicating proportionally to the size of the Zo1 zone the number of points of the signal I(Pn) to be added to the edge of the mask MZo1 to obtain the dilated mask MZD1.

[0108] In a step (2), the dilated characteristic area ZCD1 of these FMdB, FMd2, FMd3, and FMd4 maps is obtained. This dilated characteristic area ZCD1 includes all the points of the FMdB, FMd2, FMd3, and FMd4 maps co-located with samples belonging to the dilated mask MZD1 of the Zo1 area.

[0109] It should be noted that according to the dilation rule chosen, the dilated characteristic area ZCD1 of the FMdB FMd2, FMd3 FMd4 cards may possibly include, depending on the shape of the contour of the Zo1 zone, more points than the set of points of the FMdB FMd2, FMd3 FMd4 cards strictly necessary to decode the Zo1 zone.

[0110] When the FMdi, FMd2, FMd3 FMd4 feature maps have the same resolution as the I(Pn) image, as is the case in the example shown, the boundary of this dilated feature area ZCD1 is in each FMdb FMd2, FMd3 FMd4 feature map identical to the boundary of the dilated mask MZD1.

[0111] In the preceding example, the FMdB, FMd2, FMd3, and FMd4 feature maps have the same resolution as the I(Pn) image. However, it should be noted that the determination of the dilated feature area associated with the Zo1 area is similar when the FMdB, FMd2, FMd3, and FMd4 feature maps have different resolutions than the I(Pn) image. Simply put, the determination of the boundary of the dilated feature area takes into account the transformation function from the resolution of the I(Pn) image to that of the FMdB, FMd2, FMd3, and FMd4 feature maps. For example, if the FMdB, FMd2, FMd3, and FMd4 feature maps have a lower resolution, the boundary of the dilated feature area is located between the points whose co-located samples in the I(Pn) image lie on either side of the Zo1 area boundary.

[0112] [Fig.5] is a logic diagram representing an example of a method for encoding at least one area of ​​a signal (I(Pn)) which can be implemented by the encoding device of [Fig.1], for example when the synthesis neural network NNSYN is that shown in [Fig.3].

[0113] According to this embodiment, the signal is a two-dimensional image, each sample to be coded is therefore a pixel Pn with coordinates (xn, yn).

[0114] The encoding takes place in several phases.

[0115] In a first phase, called the construction phase, a learning process is performed to determine, for an input signal I(Pn), the values ​​of the FM maps and the parameters Wk to optimize an overall cost function. The learning is, for example, performed by gradient descent, followed by an update of the parameters of the NNSYN synthesis neural network and the values ​​of the FM feature maps. As is known in the prior art, the cost function can be of the rate-distortion type, or rate-distortion type, or perceptual type. To measure the rate R, it is necessary to simulate the encoding of the FM maps and then measure the associated encoding rate (the size of the flow B0). According to one embodiment, the encoding of the parameters Wk is not simulated because their influence is less significant than that of the feature maps.According to one embodiment, the encoding of the parameters Wk is also simulated and the associated throughput (the size of the flux Bl) is measured. To measure the distortion D, it is necessary to simulate the encoding and then the decoding of at least a part of the image I, to obtain at least one pixel P'n resulting from a simulation of encoding and then decoding, and then to measure the difference between this part of the input image I(Pn) and a corresponding part of the encoded and then decoded image I(P'n).

[0116] Then, during a second phase, called the coding phase, the parameters Wk, the maps, the expanded masks MZDj, and the erosion rule Re are encoded to produce the coded values ​​Wck, pc.i, MZD'c, and Rec before transmission or storage. They constitute the compressed representation of the ZoJ region of the input signal I(Pn).

[0117] Naturally, the first phase and the second phase correspond to two independent processes which can be carried out on different devices.

[0118] We will now describe the steps of a method for encoding a ZoJ region of an input signal I(Pn) according to the first embodiment of the invention.

[0119] During a step E20, a signal I(Pn) to be coded, comprising a plurality of N samples Pn, is provided as input to the process.

[0120] During step E21, the M FM maps of the first group and the Wk parameters of the synthesis neural network are initialized. Subsequently, the Wk parameters of the NNSYN synthesis neural network and FM map values; must be optimized during the construction phase.

[0121] According to one embodiment, the FM cards; are of the same resolution as the input signal I(Pn) and therefore each have the same number of values ​​N as there are samples Pn to be coded.

[0122] According to one embodiment, the FM cards; have a resolution less than or equal to that of the input signal I(Pn) and therefore include, for at least one of them, a number N' of values ​​to be coded less than N. According to a variant, the first FMi card is at the resolution of the images and each subsequent card is at half the resolution of the previous one.

[0123] According to one embodiment, several FMi cards have the same resolution, lower than that of the input signal I(Pn).

[0124] According to one embodiment, the FM maps are transformed to provide a second group of transformed feature maps FMS. In this embodiment, the feature vectors are preferably extracted from the transformed maps of the second group, and not directly from the maps of the first group. Thus, in this embodiment, the feature vectors are indirectly extracted from the maps of the first group. The maps of the second group are not coded; they serve only for the construction of the feature vectors.

[0125] According to one embodiment, the FM cards are initialized with predefined constant values.

[0126] According to another embodiment, the feature cards are initialized by a set of random real values.

[0127] The FM feature cards; of the first group are subsequently updated, or refined, during an E22 step, by the encoder update module during its learning.

[0128] During step E23, the FM cards of the first group are encoded by the FMC module of the encoder. During the build phase, this operation is a coding simulation. During the encoding phase, this operation is the actual encoding, and the encoded values ​​constitute the B0 stream. The simulation may be identical to the actual encoding, but it may also be different (for example, simplified). For this encoding, a technique for predicting a feature card value based on its neighborhood is used, as will be described, for example, in support of [Fig. 9]. These parameters are subsequently updated, or refined, during the build phase, in later iterations of the process.

[0129] In one embodiment, the FM cards are coded in the order (FMb FM2, ..., FM4), and the variables of each card in a predefined order, for example lexicographic. Each card undergoes entropic coding. The entropic coding produces a compressed BO stream whose throughput is subsequently measured during an E29 step.

[0130] During a step E24, according to one embodiment, the M cards of the first group FM; are transformed by the module SE to generate cards of the second group FMSi at the resolution of the images of the input sequence.

[0131] According to one embodiment, M FMS cards are generated.

[0132] According to one embodiment, each FM card; is transformed into an FMS card;.

[0133] According to one embodiment, at least one FM card; is of lower resolution to that of the images in the sequence to be encoded, and the transformation operation includes oversampling so that the transformed FMS map contains the same number of samples as the images in the sequence. Oversampling consists of adding values ​​to the FMS maps to achieve the resolution of the images in the input sequence. It can be simple (by nearest neighbor replication) or involve interpolation (linear, polynomial, filtered, etc.).

[0134] During step E25, values ​​are extracted by the XTR module from the FM cards; or possibly FMS cards; transformed. This extraction is performed according to the coordinates (xn, yn) of the sample Pn of the input signal. It can also be performed according to the resolution of the card considered.

[0135] According to one embodiment, the characteristic Zn vector results directly from this extraction.

[0136] The samples to be coded are, for example, processed sequentially, from n=1 to n=N.

[0137] According to one embodiment, during a step E26, the characteristic vector Zn is constructed by the TT module from the values ​​extracted from the FM or FMS maps for each sample Pn with coordinates (xn, yn) of the input signal. The processing may include quantization of the values ​​extracted from the FM maps or of the resulting vector Zn, if necessary. The processing may include other operations, such as filtering, scaling, the application of any function, preferably monotonic, etc.

[0138] In one embodiment, Zn has as many values ​​as there are FMi or FMS cards; in input. In this case, L=M.

[0139] In one embodiment, Zn is an L-tuple (zb z2,..., Zj), consisting of the values ​​of the FM or FMS maps located at the coordinates (xn, yn) of the current pixel Pn, as will be illustrated in Figure 6. In another embodiment, Zdn is an L-tuple (zb z2,..., Zj), consisting of the values ​​located in a neighborhood around these coordinates.

[0140] In one embodiment, Zn is an L-tuple constructed from values ​​taken from the FMi maps at coordinates that may be different for each map. For example, if the FMi maps are at different resolutions because they have been undersampled, the coordinates are adjusted (by scaling) to match the resolution of each map.

[0141] In one embodiment, Zn is an L-tuple constructed from values ​​taken from FM maps; by applying processing to one or more values ​​from the maps, for example, filtering out values ​​close to the target value in a map. For example, for a current sample Pn in an FMi map that is at the same resolution as the input signal, one can extract the values ​​located at coordinates (xn, yn), (xn-l, yn), (xn, yn-l), and (xn-l, yn-1) and apply processing to these values ​​(filtering, averaging, interpolation, etc.) to obtain the final value (¾) of element i of the vector Zn relative to that FM map; According to another example, in an FM map; which is at half the resolution of the input signal, we can consider the values ​​located at coordinates (xn / 2, yn / 2), (xn / 2-l, yn / 2), (xn / 2, yn / 2-l) and (xn / 2-l, yn / 2-l) and apply processing to these values ​​(filtering, averaging, interpolation, etc.) to obtain the final value (¾) of element i of the vector Zn relative to this FM map;. .

[0142] During a step E27, the vector Zn is processed by the NNSYN synthesis neural network to generate as output a vector representative of the sample Pn to be coded, according to an embodiment the triplet (R, G, B) of the sample P'n (the sample Pn coded then decoded).

[0143] The structure and parameters Wk of the synthesis neural network are initialized, for example, during the first iteration of this step. These parameters are subsequently updated, or refined, during the construction phase, in subsequent iterations of the process.

[0144] According to one embodiment, the parameters of the synthesis neural network are initialized by predefined values ​​known to give a satisfactory result (for example, following training on a corpus of images).

[0145] According to another embodiment, the parameters of the synthesis neural network are initialized by a set of random values.

[0146] During step E28, the parameters Wk of the NNSYN synthesis neural network are quantized and encoded. During the construction phase, this operation is a coding simulation. During the encoding phase, this operation is the actual encoding, and the encoded values ​​constitute the Bl stream. The simulation may be identical to the actual encoding, but it may also be different (for example, simplified). Any known technique may be used for this purpose, for example, the neural network coding standard proposed by the MPEG-7 Part 17 standard, also called NNR (Neural Network). Network Representation). Note that in this case, it is necessary to choose the amount of degradation that the coding brings to the Wk weights.

[0147] During an E29 step, a performance measure is evaluated.

[0148] To this end, the coding simulation rates associated with the feature maps of the first group (simulation of the B0 flow by coding the FM maps;) and optionally with the parameters of the neural network (simulation of the B1 flow by coding the Wk parameters) are measured.

[0149] In one embodiment, the cost function is of the rate-distortion type, denoted (D+L*R), where D, for example, is the root mean square error measured between the input signal and the decoded signal (or the error measured on a subset of the signal samples). In another example, D is calculated from a perceptual function such as SSIM (for Structural SIMilarity) or MSSSIM (for Multi-scale Structural SIMilarity). In one embodiment, R is the simulated rate of the B0 stream. In another embodiment, R is the total rate used to encode this image, i.e., the sum of the simulated rates of B0 and B1. L is a parameter that controls the rate-distortion trade-off. Other cost functions are possible.

[0150] As long as the cost function has not reached its minimum, or a predefined number of iterations has not been reached, the performance measurement is not satisfactory, and the process is repeated from step E22. This minimization can be performed by a mechanism known as gradient descent with updating of the parameters during step E22 for the values ​​of the feature maps and E23, E27 for the network parameters.

[0151] During step E30, if the cost function has reached its minimum, or if the desired number of iterations is reached, the training stops. If a coded version corresponding to the last simulation of the synthesis neural network parameters Wk and the feature maps FM; is available, the streams B0 and Bl can be constructed from it. According to another embodiment, the actual coding of the updated synthesis neural network parameters Wk and the values ​​of the feature maps FM; is performed at this step to produce the encoded parameters Wcket FMc; which constitute the streams B0 and Bl.

[0152] At this stage, it is not possible from the B0 and Bl streams to decode a specific area of ​​the coded signal without decoding the entirety of this coded signal.

[0153] To achieve this functionality, according to one embodiment, the coding process performs the steps as described below. These steps correspond to a transcoding, or re-encoding, of the B0 stream into a new zone-decodable stream.

[0154] During an E31 step, the FMc coded values ​​are decoded by the FMD module to generate the FMd maps.

[0155] During an E32 step, the segmentation of the signal I(Pn) into J (greater than or equal to 2) zones Zo' is carried out by the SEG module.

[0156] According to a first example of segmentation, the image is divided into J regular zones, for example of identical size and shape (except possibly at the edges of the image). For example, the image can be divided into rectangular zones whose boundaries are horizontal and vertical. Such a division corresponds to the concept of "tiles" implemented by coding standards such as H.264 / AVC, H.265 / HEVC, and H.266 / VVC.

[0157] According to a second example of segmentation, the image is divided into J zones corresponding to samples (or blocks) traversed in a lexicographical order. Such a division corresponds to the concept of "slices" implemented by coding standards such as H.264 / AVC, H.265 / HEVC and H.266 / VVC.

[0158] According to a third example of segmentation, the image is divided into semantic zones, for example, a background and a foreground. This can be done manually by an operator. Segmentation can also be automatic or semi-automatic, depending on the segmentation algorithm used. It should be noted that there are no restrictions related to the type of segmentation algorithm used.

[0159] Advantageously, such a division allows, during the subsequent decoding of the signal I(Pdn), to parallelize the decoding of the areas to be decoded Zodj by distributing the decoding load equally on each of the decoders and / or the decoder processors.

[0160] At the end of step E32, this segmentation is represented as a set of J masks MZo' each allowing to identify a zone Zo'.

[0161] For a given zone Zo*, the coding process then performs steps E33 to E39.

[0162] During a step E33, a dilation rule Rd is obtained by the ORD module.

[0163] During step E34, a dilated mask MZD' is obtained by applying the rule dilation Rd to the MZo' mask.

[0164] According to one embodiment, the dilation rule Rd is defined by a structuring element ES defined as a square whose dimension is a function of the size of the convolutional kernels used by the convolutional neural networks of the NNSYN synthesis neural network. The dilated mask MZD' is obtained, for example, by a morphological dilation mask MZo' by the structuring element ES.

[0165] According to another embodiment, the dilation rule is defined by a number of numerical values ​​equal to twice the dimension of the signal. Thus, there are four numerical values ​​for a 2-dimensional signal such that an image I(P n) corresponds to a dilation in each of the two vertical and J. The coded values ​​gçJ constitute the B2J flow associated with zone Zo1. On horizontal. The dilated mask MZD' is obtained by adding a number of points to the edge of the mask MZo' in the two directions associated with each dimension of the signal. In other words, for a 2-dimensional signal such as an image I(Pn), pixels are added in both horizontal and both vertical directions.

[0166] According to another embodiment, the dilation rule is defined by a scale factor. The dilated mask MZD' is obtained by adding to the points of the mask MZo' a number of points at the edge of the mask MZo' proportionally to the size of the mask MZo'.

[0167] During a step E35, a dilated characteristic zone ZCD' of the FMd maps; comprising all points of the FMd maps; co-located with samples belonging to the dilated mask MZD' is obtained.

[0168] During step E36, the CE module performs the encoding for each zone Zo' of the value of the points of the dilated characteristic zones ZCD* in the form of coded data Note that this coding is entropic in nature, and preferably lossless.

[0169] During an E37 step, the expanded mask MZD' of the Zo* zone is encoded as an MZD'c stream.

[0170] During step E38, an erosion rule Re is obtained from the dilation rule Rd. This erosion rule makes it possible to obtain the mask MZo' or an approximation thereof from the dilated mask MZD'.

[0171] For example, according to an embodiment as described to the encoder, the erosion rule Re is defined by a structuring element obtained by a morphological erosion corresponding to the morphological dilation element defined previously.

[0172] According to another embodiment, the erosion rule Re is defined by a number of numerical values ​​equal to twice the signal dimension. Thus, there are four numerical values ​​for a 2-dimensional signal such that an image I(Pn) corresponds to erosion in each of the two vertical and horizontal directions. The mask MZo' is obtained by subtracting a number of points from the boundary of the dilated mask MZD' in the two directions associated with each signal dimension corresponding to the points that were added during the dilation step.

[0173] According to another embodiment, the erosion rule is defined by a scaling factor inverse to that used in the encoder. The mask MZo' is obtained by subtracting a number of points from the edge of the expanded mask MZD' proportionally to the size of the mask MZD'.

[0174] During step E39, the erosion rule is coded in a form denoted Rec

[0175] The coded values ​​MZDc and Rec constitute the B3j stream associated with the Zo' zone.

[0176] Thus, the coded values ​​Rec, MZD'c and Wck constitute the compressed representation of the Zo* zone of the input signal I(Pn).

[0177] It should be noted that steps E31 to E39 can be analyzed as steps in a transcoding process of a stream B0 representing a coded image (obtained during a preliminary step EP) into two streams B2j and B3j representing the encoding of a Zo' region of the coded image. The preliminary step EP can correspond to the encoding steps (E20-E30) described previously, or to any other encoding process resulting in a coded representation of a set of feature maps (and a synthetic neural network).

[0178] Figure 6 illustrates a method for encoding an I(Pn) signal used in an embodiment of the invention.

[0179] In this embodiment, there are 4 FM cards generated. In a preferred mode, there are 7.

[0180] The first FMi map has the same resolution as the I(Pn) image, and therefore contains WxH variables, where W represents the image width in pixels, and H its height. The second FM2 map has half the resolution (in each dimension) of the FMi map. Each additional map has half the resolution of the previous map. This structure reduces the number of variables in the feature maps, which facilitates coding and learning while minimizing coding costs.

[0181] The FM2 map is oversampled by the SE module by a factor of 2 in each dimension, according to a method illustrated in support of [Fig. 6]. The FM3 map is oversampled by a factor of 4 in each dimension, and the FM4 map by a factor of 8 in each dimension. The FMi map is not affected by the oversampling. (FMS^FMj).

[0182] The resulting FMS maps are of the same resolution as the image I(Pn), and therefore each have WxH values, where W represents the width of the image in pixels and H its height (N=WxH).

[0183] Other types of structure are possible, for example one can use a different reduction rate of one half between the cards (one quarter, or one third, etc.).

[0184] In this embodiment, the vector Zn is a 4-tuple (zi...z4) consisting of the values ​​extracted from the FMS maps located at the coordinates (xn, yn) of the current pixel Pn. The vector Zn, consisting of the extracted (quantized) values ​​from the FMS maps, is processed by the NNSYN synthesis neural network to generate a second output vector. In this example, the output vector is the triplet (R, G, B) of the encoded and then decoded pixel P'n. The triplet is inserted into the decoded image I(P'n) at the positions (xn, yn) of the color components (R', G', B').

[0185] In another embodiment, not shown, the Znest vector is extracted directly from the FMi layers, with positions recalculated according to the size of the maps, and then the extracted values ​​are optionally processed and quantified after extraction.

[0186] The [Fig.7] is a logic diagram representing an example of a decoding process for a decoded area (Zodj) of a signal (I(Pn)) which can be implemented by the decoding device of the [Fig.2] when the synthesis neural network NNSYN' is for example that presented in the [Fig.3].

[0187] During a step F20, the Bl, B2j and B3j streams are extracted from the encoded stream. They contain respectively the parameters Wck, the coded representations pc.i of the points of the dilated characteristic zones ZCD' and the coded values ​​MZD'c and Rec.

[0188] During a step F21, the erosion rule and the dilated mask MZD' of the Zodj zone are generated by decoding the coded values ​​Rec and MZDjc, then an MZodj mask is generated by applying the erosion rule to the dilated mask MZD'.

[0189] During a step F22, p^ji maps are optionally initialized (for example with null values) and a dilated characteristic zone ZCD* comprising all the points of the p^ji maps co-located (possibly taking into account a scale factor) with samples belonging to the dilated mask MZD' of the Zodj zone is obtained.

[0190] During a step F23, the M maps pM^i are generated by decoding the values ​​Pqi of the points of the dilated characteristic zones ZCD'. In one embodiment, the maps pj / L]j are decoded in order (FMdi, FMd2, ... FMd4), and the values ​​of each map in a predefined order, for example lexicographic.

[0191] According to embodiments as described for the encoder: - The p^ji cards have the same resolution as the coded I(Pn) signal, that is to say they have N=WxH values. - The p^f^J cards have a resolution less than or equal to that of the coded I(Pn) signal. - Several p^^J maps have the same resolution, lower than the resolution of the coded I(Pn) signal.

[0192] During a step F24, according to one embodiment, the M cards of the first group FMd- are transformed by the module SE' to generate cards of the second group FMS'^ at the resolution of the input images.

[0193] This step is similar to step E24, which was described for the encoder in support of [Fig. 5], and the embodiments apply. In particular: - According to one embodiment, M FMS'- cards are generated. - According to one embodiment, each FMd' card is transformed into a FMS'j card. - In one embodiment, at least one pMd- map has a lower resolution than the images of the image to be encoded, and the transformation operation includes oversampling so that the transformed FMS'- map has the same number of samples as the input image. Oversampling consists of adding values ​​to the FMS'- maps to achieve the resolution of the input image. It can be simple (by nearest neighbor replication) or involve interpolation (linear, polynomial, filtered, etc.).

[0194] The transformation may optionally include inverse quantization of the extracted values, if necessary. However, inverse quantization is not mandatory.

[0195] During step F25, values ​​are extracted by the XTR' module from the FMd^ or possibly FMS'' transformed cards. This extraction is performed based on the coordinates (xn, yn) of a sample Pn of the input signal. It can also be performed based on the resolution of the card in question. This step is similar to step E25, which was described for the encoder in support of [Fig. 5], and the embodiments apply. In particular:

[0196] According to one embodiment, the characteristic vector Zdn results directly from this extraction.

[0197] In one embodiment, Zdn is an L-tuple (zb z2,..., Zj), consisting of the values ​​of the maps pMd'î or pMS'J located at the coordinates (xn, yn) of a current pixel Pdn, as will be illustrated in Figure 8. In another embodiment, Zdn is an L-tuple (zb z2,..., Zj), consisting of the values ​​located in a neighborhood around these coordinates.

[0198] The samples to be decoded are, for example, processed in sequential order.

[0199] According to one embodiment, during a step F26, a vector Zdn is constructed by the TT' module from the values ​​extracted from the FMd'* first group cards or the FMS'- second group cards, for each Pdn sample of coordinates (xn, yn) to be decoded, as a function of the coordinates (xn, yn). This step is similar to the step E26, which was described for the encoder in support of [Fig. 5], and the described embodiments apply. The extraction may include inverse quantization of the extracted values ​​or of the resulting vector Zdn, if necessary.

[0200] During step F27, the Wdk parameters of the NNSYN' synthesis neural network are generated by decoding the Wck values ​​of the Bl stream. Any known decoding technique corresponding to the encoding technique used by the encoder can be used for this purpose. The NNSYN' synthesis neural network is similar to the NNSYN synthesis network, that is, it has the same structure and the same parameters, except for the encoding, which can be performed with or without loss.

[0201] According to one embodiment, the B2 stream is decoded before the Bl stream, in order to have the NNSYN' synthesis neural network available before starting to decode the samples.

[0202] During step F28, the vector Zdn is processed by the NNSYN' synthesis neural network to generate as output a second vector representing the sample Pdn to be decoded, according to an embodiment a triplet which is injected into the intermediate image I;(Pdn) at the positions (xn, yn) of the color components (Rd, Gd, Bd). This step is similar to step E27 which was described for the encoder in support of [Fig. 5].

[0203] During a step F29, the intermediate image I;(Pn) is transformed into a signal corresponding to the Zodj area of ​​the image I(Pdn) by the EXTR module.

[0204] Alternatively, steps F28 and F29 described above are replaced by a step F30. During step F30, the NNSYN' module only processes the input Zdn vectors capable of generating as output a second vector representative of a Pdn sample to be decoded. This selection of the Zdn vectors is performed by the NNSYN' module taking into account the MZodj mask of the area to be decoded.

[0205] Figure 8 illustrates a method for decoding an I(Pdn) signal used in an embodiment of the invention.

[0206] In this embodiment, there are 4 decoded cards pjyjçjj. In a • 1 Their preferred mode of operation is 7 in number.

[0207] In this embodiment, the first card J has the same resolution as Image I consists of WxH variables, where W represents the image width in pixels, and H its height. The second map, p^jj, has half the resolution (in each dimension) of map pMdj. Each additional map has half the resolution of the previous map. This structure reduces the number of variables in the feature maps, which facilitates decoding while minimizing the encoding cost.

[0208] The map is oversampled by a factor of 2 in each dimension, using any oversampling method available to a person skilled in the art. The map is oversampled by a factor of 4 in each dimension, and the pMd^ map by a factor of 8 in each dimension.

[0209] The pfyfS'J cards have the same resolution as the image to be decoded, and therefore include WxH values, where W represents the width of the image in pixels, and H its height.

[0210] In this embodiment, the vector Zdn is a 4-tuple (zi...z4) consisting of the values ​​of the pM$'î maps located at the coordinates (xn, yn) of the current pixel Pdn. The vector Zdn is optionally dequantized and then processed by the NNSYN' synthesis neural network (and optionally by the EXTR module) to generate as output the triplet (R, G, B) representative of the sample Pdn to be decoded. The triplet (R, G, B) is inserted into the decoded image Zodj(Pdn) at the coordinates (xn, yn) in the color components (Rd, Gd, Bd).

[0211] The [Fig.9] is a flowchart representing a method of coding feature cards which can be implemented by the coding device of the [Fig.1] and by the coding process of the [Fig.5].

[0212] These steps constitute sub-steps of step E23 described previously in support of Figure 5. They aim to encode a current value Vd' of a point of a dilated characteristic zone ZCD' of a feature map of the first group being processed using values ​​from the neighborhood.

[0213] In a substep E231, a neighborhood vector (Cn) is established, comprising values ​​close to the value Vn. These neighboring values ​​may be located in the same map and / or in a different map from the plurality M of maps FMd'Cc. The neighborhood vector consists of a number C of values, or data, corresponding to neighborhood values ​​(for example, C=10). These values ​​must be known to the encoder and the decoder; therefore, they must be located in a causal neighborhood of the value Vn.

[0214] According to a first embodiment, these values ​​are used to determine the context of an entropy encoder for encoding the current value during an E233 step. This encoder can be a CAB AC (Context-adaptive binary arithmetic coding) type encoder. This type of encoder is well known to those skilled in the art. It is notably used in the H.265 / HEVC video compression standard. It is an arithmetic encoder with lossless compression. It decomposes all non-binary symbols into binary symbols. Then, for each bit, the encoder selects the most suitable probability model and uses a context to optimize the probability estimation. This context can be defined by information from neighboring elements. Adaptive or non-adaptive arithmetic coding is then applied to compress the resulting data. As is known to those skilled in the art, there are several ways to use the neighborhood vector to produce contextual information. For example, one can count the number of non-zero neighboring values ​​and associate a context with each number. Alternatively, one can perform comparisons between several neighboring values ​​and associate a given context with an ordering configuration among the neighboring values, for example, by sorting the neighboring values ​​in ascending order and associating a context with each possible order.

[0215] In a second embodiment, the neighborhood is used to predict, during a step E232, the current value from an autoregressive model. It is recalled that an autoregressive model predicts a sample from a series based on its past values. In this embodiment, the past values ​​are constituted by the context, and the difference between the predicted variable and the actual value is quantified and then entropically coded during step E233.

[0216] At the end of the process, the current coded value Vcnde of the card p^ji being processed is coded.

[0217] The [Fig. 10] is a logic diagram representing a method for decoding feature cards which can be implemented by the decoding device of [Fig. 2] and by the decoding process of [Fig. 7]

[0218] These steps constitute sub-steps of step F23 described previously in support of Figure 7. They aim to decode a current value of a point of a dilated characteristic zone ZCD' of a feature map pj^i of the first group being processed using values ​​from the neighborhood.

[0219] In a substep F231, a neighborhood vector (Cdn) is established, comprising values ​​close to the value Vdn. This step is similar to the previously described step E231, and the same embodiments apply. This neighborhood vector consists of a number C of values, or data, corresponding to neighborhood values ​​(for example, C=10) located in the same map and / or in a different map from the plurality M of maps of a point in an expanded characteristic zone ZCD* of a feature map pjjji. These values, being in a causal neighborhood of the value Vdn, are known to the decoder.

[0220] According to a first embodiment, these values ​​are used to determine the context of an entropy decoder for decoding the current value during an F233 step. This decoding is similar to that used in the encoder, for example, CAB AC. The use of the neighborhood to produce context information is similar to that chosen in the encoder. For example, one can count the number of non-zero neighboring values ​​and associate a context with each number. Alternatively, one can perform comparisons between several neighboring values, and associate a given context with an order configuration between neighboring values, for example by ranking the neighboring values ​​in ascending order, and associating a context with each possible order.

[0221] In a second embodiment, the neighborhood is used to predict the current value from an autoregressive model during a step F232. In this embodiment, the past values ​​are constituted by the context, and the difference between the predicted variable and the actual value is quantified and then entropically coded during step F233.

[0222] At the end of the process, the current decoded value Vdnde of the FMd card being processed is decoded.

[0223] Fig. 11 schematically represents a TRANSC transcoding device of a coded representation of a signal in the form of J independent streams B2j and B3j into a single stream B0.

[0224] The TRANSC transcoding device of [Fig.1 1] receives as input the J data streams B2j and B3j corresponding to the J zones ZoJ segmenting the signal I(Pn).

[0225] This TRANSC decoding device comprises a DDMZ module capable of decoding an expanded mask MZD', a DZCD module capable of determining an expanded characteristic zone ZCD' in the M characteristic maps, a DE module for decoding the characteristic maps from the encoded data group, a COMP module for composing the characteristic maps p]\jdj, and an FMC module for encoding the characteristic maps FMdb

[0226] With the exception of the COMP module, the other modules of the TRANSC transcoding device correspond to the modules of the decoding device of [Fig.2] referenced in a similar way and are not described again here.

[0227] The COMP module receives as input the J sets of characteristic maps p\4dJ associated with the Zo1 zones of the signal I as well as associated dilated characteristic zones (these dilated characteristic zones are provided by the DR module). From this data, the COMP module will construct a set of characteristic maps FMd; by assigning to each point of a region of a characteristic map FMdj colocated with a dilated characteristic zone the value of the corresponding point in the FMd.i map.

[0228] The cards are then coded by the FMC module to produce the FMc data; constituting the B0 stream.

[0229] It should also be noted that the invention is not limited to the embodiments described above. It will indeed be apparent to those skilled in the art that various Modifications may be made to the embodiments described above, in light of the teaching which has just been disclosed to him.

[0230] For example, NNSYN / NNSYN' synthesis neural networks can be recurrent neural networks.

[0231] In another example, the synthetic neural networks can consist of one or more convolutional neural networks, followed by an MLP, and then followed by one or more more convolutional neural networks, each of the convolutional neural networks being associated with a convolution kernel of predefined size, for example, size 3x3. In these examples, the acquisition of the vectors Zn / Zdn is adapted to the topology of the NNSYN / NNSYN' synthetic neural networks; that is, several values ​​are extracted from at least one FM feature map. In other words, Zn is an L-tuple with L>M, constructed from values ​​taken from the FM maps at coordinates close to those of the current pixel, for example, on the support of the kernel of the convolution filter of one of the convolutional neural networks.

[0232] For example, the encoding and decoding processes described above may use a wavelet transform or a discrete cosine transform instead of a neural coding method. In this case, without loss of generality, the NNSYN encoding neural network can be replaced by a direct transform followed by a quantizer, and the synthesis-to-decoder network can take the form, for example, of an (inverse) quantizer and an inverse transform followed by filtering (at the block boundary). The transform is, for example, applied to a block of pixels. The inverse transform is, for example, applied to a characteristic vector of values ​​from the characteristic value maps corresponding to a block.

[0233] In the detailed presentation of the invention given above, the terms used shall not be interpreted as limiting the invention to the embodiments set forth in this description, but shall be interpreted as including all equivalents which can be foreseen by a person skilled in the art by applying their general knowledge to the implementation of the teaching which has just been disclosed to them.

Claims

1.

2. Demands A method for encoding a region (Zo^) of a signal (I(Pn)), called the region to be encoded, said region to be encoded comprising a plurality of samples (Pn) to be encoded, said encoding method comprising the following steps: - obtaining (E32) a group of at least one characteristic map (FM;) representative of said signal (I(Pn)), - obtaining (E32) a mask (MZoJ), called the first mask, in said signal (I(Pn)) of said zone to be coded (Zo*), - obtaining E(33) a dilation rule (Rd) of said first mask (MZoj), - obtaining (E34) a dilated mask (MZD), called the second mask, by applying said dilation rule (Rd) to said first mask (MZoj), - the determination (E35) of a dilated characteristic zone (ZCD') in said group of at least one feature map (FM;), as a function of said second mask (MZEP), - the coding (E36) of the value of the points of said dilated characteristic zone (ZCD'), - the determination of an erosion rule (Re) for said second mask (MZD), and - the coding (E37, E39) of said second mask (MZD) and of said erosion rule (Re). A method for decoding a region (Zodj) of a signal (I(Pdn)), called the region to be decoded, said region to be decoded comprising a plurality of samples (Pdn) to be decoded, said decoding method comprising the following steps: - the decoding (F21) of a first mask (MZD) representative of an area in the domain of said signal (I(Pdn)), - the decoding of a group of at least one characteristic map (pMD^) representative of at least said area to be decoded, said decoding comprising: • the determination (F22) of a characteristic dilated zone (CDZ) in a group of at least one feature map as a function of said first mask (MZD'), and • the decoding (F23) of the value of the points of said dilated characteristic zone (ZCDj), - the decoding (F21) of an erosion rule (Re), and - the synthesis (F24-F29, F30) of said zone to be decoded (Zodj) from the points of said dilated characteristic zone (ZCD') and the erosion rule (Re) to obtain the decoded signal (Zodj).

3. A method for decoding a zone (Zodj) according to the preceding claim in which the synthesis (F24-F29) comprises the determination (F21) of a mask (MZodj) of the zone to be decoded in the domain of said signal, said second mask, by application of said erosion rule (Re) to said first mask (MZD').

4. Decoding method according to the preceding claim wherein the synthesis (F24-F29) comprises: - the processing (F28) of said dilated characteristic area (ZCDj) to obtain an intermediate signal (I;(Pn)), and - the application (F29) of said second mask (MZodj) to said intermediate signal (I;(Pn)) to obtain said plurality of samples (Pdn) of said area to be decoded (Zodj).

5. Decoding method according to claim 3 wherein the synthesis includes processing (F24-F27, F30) of said dilated characteristic zone (ZCD*) taking into account said second mask (MZodj) to obtain the decoded signal (Zodj) corresponding to the second mask (MZodj).

6. Decoding method according to any one of claims 2 to 5 wherein said erosion rule (Re) is a morphological erosion by a structuring element (ES).

7. A decoding method according to any one of claims 2 to 5 wherein said erosion rule (Re) is defined by a maximum distance, the second mask (MZodj) comprising the points of the first mask (MZDj) located at a distance from an edge of the first mask (MZD') in at least one direction inwards of the first mask (MZDj) greater than said maximum distance.

8. A decoding method according to any one of claims 2 to 5 wherein said erosion rule (Re) is defined by a number of parameters equal to twice the dimension of said signal (I(Pdn)), each direction of one of the dimensions of said signal being associated with a parameter, said second mask (MZodj) being obtained by removing from said first mask (MZD'), in at least one direction of a dimension of said signal, a number of points equal to said parameter associated with said at least one direction of a dimension of said signal.

9. A decoding method according to any one of claims 2 to 8 wherein the points of said dilated characteristic area are representative of said area to be decoded in a latent domain, wherein said decoding method further comprises the decoding of the parameters (Wdk) of a neural network (NNSYN'), said synthesis neural network, and wherein the synthesis comprises, for at least one sample, said current sample (Pdn), of the signal to be decoded, associated with a position (xn, yn) in the signal to be decoded: - the construction of a characteristic vector (Zdn) from said characteristic values ​​as a function of said position (xn, yn) of said current sample, and - the processing of said characteristic vector (Zdn) by the synthesis neural network (NNSYN') defined by the decoded parameters (Wdk) to provide a decoded value of the current sample (Pdn).

10. A decoding method according to any one of claims 2 to 8 wherein said characteristic values ​​are representative of said area to be decoded in a transformed domain, for example associated with a direct discrete cosine transform or a wavelet transform and wherein the synthesis comprises applying said characteristic values ​​of an inverse transform associated with said direct transform.

11. Encoding device for an area (Zo1) of a signal (I(Pn)), said area to be encoded, said area to be encoded comprising a plurality of samples (Pn) to be encoded, characterized in that said encoding device is configured to implement: - obtaining (E32) a group of at least one characteristic map (FM;) representative of said signal (I(Pn)),

12. - obtaining (E32) a mask (M / o*), called the first mask, in said signal (I(Pn)) of said zone to be coded (Zo^, - obtaining E(33) a dilation rule (Rd) of said first mask (MZoj), - obtaining (E34) a dilated mask (MZD'), called the second mask, by applying said dilation rule (Rd) to said first mask (MZoj), - the determination (E35) of a dilated characteristic zone (ZCDO) in said group of at least one feature map (FM;) as a function of said second mask (Mziy), - the coding (E36) of the value of the points of said dilated characteristic zone (ZCD*), - the determination of an erosion rule (Re) for said second mask (MZD'), and - the coding (E37, E39) of said second mask (MZD) and of said erosion rule (Re). A decoding device for a region (Zodj) of a signal (I(Pdn)), called the region to be decoded, said region to be decoded comprising a plurality of samples (Pdn) to be decoded, characterized in that said decoding device is configured to implement: - the decoding (F21) of a first mask (MZD') representative of an area in the domain of said signal (I(Pdn )), - the decoding of a group of at least one characteristic map (pMD^) representative of at least said area to be decoded, said decoding comprising: • the determination (F22) of a dilated characteristic zone (ZCD1) in a group of at least one fpMD^ feature map as a function of said first mask (MZD'), and • the decoding (F23) of the value of the points of said dilated characteristic zone (ZCDj), the decoding (F21) of an erosion rule (Re), and

13. - the synthesis (F24-F29) of said zone to be decoded (Zodj) from the points of said dilated characteristic zone (ZCDj) and the erosion rule (Re) to obtain the decoded signal (Zodj). Computer program comprising instructions for carrying out the steps of an encoding process according to claim 1 or a decoding process according to any one of claims 2 to 10 when said program is executed by a computer.

Images