Method and device for encoding and decoding a signal
By segmenting signals into zones and using neural networks to process characteristic vectors based on proximity, the method allows independent decoding and reduces memory requirements, enhancing parallel processing and interaction with semantic content.
Patent Information
- Application Number
- PCT/EP2025/068305
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-06-27
- Filing Date
- 2025-06-27
- Publication Date
- 2026-01-02
AI Technical Summary
Existing encoding methods, including classical and neural network approaches, do not allow for the creation of independently decodable areas in coded signals, limiting interaction with semantic content and memory requirements at the decoder level, as well as parallel decoding possibilities.
A method and device for encoding and decoding signals by segmenting them into zones, using neural networks to process characteristic vectors based on proximity to boundaries, allowing independent decoding of each zone, and encoding feature maps entropically.
Enables independent decoding of signal areas, reducing memory requirements and facilitating parallel processing, while improving interaction with semantic content.
Smart Images

Figure EP2025068305_02012026_PF_FP_ABST
Abstract
Description
Description TITLE: Method and device for encoding and decoding a signal Previous technique The invention relates to the general field of coding one-dimensional or multidimensional signals. It relates more particularly to the compression of digital images or videos. Digital videos are generally encoded using source code to compress them and reduce the resources required for transmission and / or storage. Numerous encoding standards exist, such as those of the ITU / MPEG organizations (H.264 / AVC, H.265 / HEVC, H.266 / VVC, etc.) and their extensions (MVC, SVC, 3D-HEVC, etc.). In these approaches, image encoding is typically performed by predicting pixels using previously encoded and decoded pixels present in the image being encoded—in which case it is called "intra-prediction"—or previously encoded images—in which case it is called "inter-prediction." In addition to these traditional approaches, approaches based on artificial intelligence, and in particular neural networks, are tending to develop. Some neural network approaches use an input signal, such as an image, to train a synthetic neural network on characteristic vectors associated with a position of a sample of the input signal to be encoded. These characteristic vectors are constructed from feature maps that may have the same resolution as the input signal or a lower resolution. During training, or construction, the neural network parameters and the feature map values are updated based on a performance metric, such as bitrate-distortion. When training is complete, meaning the performance metric is satisfactory, the actual encoding of the synthetic neural network parameters and feature map values can be performed and stored or transmitted to the decoder.The decoding of the coded signal is then carried out by applying the synthesis neural network to the feature maps. One drawback of the classical and neural approaches described above is that they do not allow the creation of independently decodable areas in the coded signal, which limits not only the ability to interact with the semantic content of the coded signal but also the possibility of limiting the memory required at the decoder level or parallelizing the decoding of this coded signal. Therefore, there is a need for a solution that improves upon the classical and neural approaches described above. Summary of the invention The invention relates to a method for encoding a signal segmented into a plurality of zones, said encoding method comprising the following steps, for a zone comprising a plurality of samples to be encoded: obtaining at least one first characteristic map representative of said signal, for at least one sample of said zone to be encoded, said current sample, associated with a position in said signal to be encoded: • the construction of a characteristic vector from the aforementioned initial characteristic maps, as a function of the aforementioned position of the current sample, • the processing of said characteristic vector by a neural network defined by a set of parameters, to generate a vector representing a decoded value of the current sample, said processing comprising the processing, applied to an input of at least one neural layer of said neural network, of a point of said input, associated with a point of said zone, according to its proximity to a boundary associated with the zone, • the updating of at least one value of one of said first feature maps and / or at least one parameter of said neural network, according to a coding performance measure, the obtaining of at least one second feature map representative of at least said area to be coded from segmentation information, the coding of at least one second feature map and of said set of parameters of said neural network. According to a particular embodiment, the coding process is such that said processing of a point of said input comprises: obtaining, from said segmentation information, a characteristic region corresponding to said area in an input (CCE) of at least one neuronal layer, said characteristic region comprising the points associated with said area, for at least one point of said area: • obtaining an associated point in said entry, • the evaluation of a proximity criterion of said associated point in relation to a boundary of said characteristic region, • the construction of an input vector of said at least one neuronal layer from a neighborhood of said associated point and said evaluation of said proximity criterion, and the processing of said input vector by said at least one neuronal layer to obtain an output point of the at least one neuronal layer. The invention also relates to a method for decoding a data stream representative of a signal segmented into a plurality of zones, the method comprising the following steps, for a zone: the decoding of at least one feature map representative of said zone, the processing of the at least one feature map decoded by a neural network comprising the processing, applied to an input of at least one neural layer of said neural network, of a point of said input, associated with a point of said zone, according to its proximity to a boundary associated with the zone. For the purposes of this invention, encoding, or "coding," refers to the operation of representing a set of samples or values in a compact form, for example, using a digital binary stream. Decoding refers to the operation of processing a digital binary stream to recover the decoded samples or values. A "sample" of the signal refers to a value taken from the signal. Sampling the signal produces a series of discrete values called samples. In the case of an image signal, the sample is called a pixel, which could be, for example, a color pixel. traditionally represented by a triplet of values, for example (R, G, B) or (Y, U, V). The position of the sample is located by its abscissa (x) and ordinate (y) coordinates in the image. A "signal comprising a plurality of samples" is defined as a signal with one (audio, sound), two (image), or more than two (stereoscopic, multiscopic, depth-mapped, video, etc.) dimensions. Depending on its dimensionality, the sample has one, two, or more coordinates within the signal. In the case of an image signal, the sample's position is identified by its x-coordinate (abscissa) and y-coordinate (ordinate). By "signal area," we mean a set of signal samples. For example, in the case of an audio signal, an area might correspond to a succession of units representing silences, noises, phonemes, words, etc. Similarly, in the case of an image signal, an area might correspond to a set of pixels, grouped according to criteria such as intensity, spatial distribution, etc. For example, a segmentation operation can identify two different areas within the image signal, one corresponding to the background and the other to the foreground. By "feature maps" or equivalently by "latent value maps", we mean an abstract representation of the signal comprising a plurality of variable data, discrete or not, which are also called values, for example real or integer numbers. By "characteristic data vector constructed from feature maps based on a position" we mean a vector consisting of one or more elements, or data, preferably discrete, the data being constructed from the feature maps at a position determined by that of the sample being processed in the signal. This characteristic vector is the one that is applied to the input of the synthesis neural network. For example, in the case of a one-dimensional audio signal, such a vector can be constructed from a plurality of values taken from each of the feature maps at the same coordinate as the sample to be encoded or in a neighborhood of it. In the case of an image, such a vector can be constructed from a plurality of values taken from each of the feature maps at the same x- and y-coordinates as the sample to be encoded (respectively, x, y, and z).to decode) or in a neighborhood of it. Once these values are taken from the feature maps, they can be processed to constitute the feature vector, before entering the synthesis neural network, for example by quantization, filtering, interpolation, etc. By "synthetic neural network", we mean a neural network such as a convolutional neural network, a multilayer perceptron, an LSTM (for "Long Short Term Memory"), etc. The neural network is defined for example by a plurality of layers of artificial neurons and by a set of activation, weighting and addition functions (for example, a layer can calculate y = f (Ax+b), where y and b are vectors of dimension N, x a vector of dimension M, A is a matrix of dimension MxN, and f is the activation function). By "neural network parameter" we mean one of the values that characterizes the neural network, for example a weight associated with one of the neurons (filter coefficient, weighting, bias, value affecting the functioning of non-linearity, etc.) "Processing by a synthetic neural network" refers to the application of a function expressed by a synthetic neural network to the input characteristic vector to produce an output vector representative of the sample to be encoded (or decoded). This output vector may contain one or more data points representative of the sample. A "performance measure" is defined as a measurement between at least one value of a sample to be encoded and a decoded value of that sample. The measure might assess, for example, distortion or perceptual error. It can be performed on one or more samples (e.g., a running sample, the current image, etc.). The measure may also include a measurement of throughput, particularly associated with the encoding of the synthetic neural network and / or the encoding of the feature maps in the first group. The measure may be a combined measurement of throughput and distortion through their weighting. As is well known in the state of the art, the value of this measure is generally minimized until a target value is reached. Segmentation information refers to any information that identifies the different areas requiring specific synthesis during decoding (or encoding). For example, in the case of an image signal, segmentation information indicates a division of the image into a plurality of regular areas of identical size and shape: the image can be divided into rectangular areas with horizontal and vertical boundaries, or into areas corresponding to samples (or blocks) processed in a lexicographical order. In this case, the segmentation information can indicate the beginning and / or end of the set. As another example, a region can be of arbitrary size and shape. In this case, the segmentation information can indicate the shape and size of the region, in the form of a segmentation mask. Generally, the steps of an encoding or decoding process should not be interpreted as being linked to a notion of temporal succession. In other words, the steps can be carried out in a different order than that indicated in the independent encoding or decoding claim, or even in parallel. The coding method according to the invention encodes a region of a signal from a representation of that signal in the form of feature maps. These feature maps are segmented into characteristic regions whose values are subsequently encoded entropically, independently of one another. Furthermore, a synthetic neural network is trained on all of these regions, taking into account the segmentation into characteristic regions. Thus, it is possible to obtain a coded representation of a region of the original signal that can subsequently be decoded by the synthetic neural network independently of any other part of that signal. The decoding process (and symmetrically the encoding process) may further include one or more of the following optional features, taken individually or in any technically possible combination. According to a first characteristic, the processing step applied to an input, called an input map, of at least one neuronal layer of said neural network comprises: obtaining a feature region corresponding to said area in the at least one input map from segmentation information, said feature region comprising the points associated with said area, and for at least one point of said area: obtaining an associated point in the at least one input feature map, evaluating a proximity criterion for said associated point with respect to a boundary of said feature region, constructing an input vector of said neuronal layer from a neighborhood of said associated point and said evaluation of said proximity criterion, and processing said input vector by said neuronal layer to obtain a point of at least one feature map as output of the at least one neuronal layer. According to another characteristic, the neighborhood of the associated point is independent of the evaluation of the proximity criterion. According to another characteristic, the neighborhood of the associated point is selected based on the evaluation of the proximity criterion. According to another characteristic, the neighborhood of the associated point is selected based on the associated point. According to another feature, during the construction of the input vector, a component of the input vector is associated with a point in the neighborhood, the value of the component being equal to the value of the associated point if the associated point is a point in the characteristic region, and to a replacement value otherwise. According to another characteristic, the replacement value is dependent on points in the characteristic region or on points associated with the neighborhood belonging to the characteristic region. According to another characteristic, the replacement value does not depend on the points of the characteristic region. According to another characteristic, the proximity criterion is a distance, for example a Euclidean distance. According to another characteristic, the neuronal layer is a convolutional neuronal layer. Correspondingly, the invention also relates to a device for encoding a signal segmented into a plurality of zones comprising a plurality of samples to be encoded, the device comprising a processor coupled to a memory in which instructions configured to implement the following steps are stored, for a zone comprising a plurality of samples to be encoded: - obtaining at least one initial characteristic map representative of said signal, - for at least one sample of said area to be coded, called the current sample, associated with a position in said signal to be coded: the construction of a characteristic vector from said characteristic maps of said first group, as a function of said position of said current sample, • the processing of said characteristic vector by a neural network defined by a set of parameters, to generate a vector representing a decoded value of the current sample, said processing comprising the processing, applied to an input of at least one neural layer of said neural network, of a point of said input, associated with a point of said zone, according to its proximity to a boundary associated with the zone, • the updating of at least one value of one of the aforementioned first feature maps and / or at least one parameter of said neural network, based on a coding performance measure, - obtaining at least a second feature map, representative of at least said zone, from segmentation information (MZo 1 ). - the coding of at least a second feature map and said parameter set of said neural network. The invention also relates to a device for decoding a data stream representative of a signal segmented into zones, the device comprising a processor coupled to a memory in which instructions are stored, configured to implement the following steps for a zone: - the decoding of at least one characteristic map representative of said area, - the processing of at least one feature map decoded by a neural network comprising the processing, applied to an input of at least one neural layer of said neural network, of a point of said input, associated with a point of said zone, according to its proximity to a boundary associated with the zone. The characteristics and advantages of the coding or decoding process apply in the same way to the coding or decoding device according to the invention and vice versa. The invention also relates to a computer program on a recording medium, this program being capable of being implemented in a computer or an encoding or decoding device according to the invention. This program includes instructions adapted to the implementation of the corresponding method. This program can use any programming language and be in the form of source code, object code, or code intermediate between source code and object code, such as in a partially compiled form, or in any other desirable form. Thus, the invention relates to a computer program comprising instructions for executing the steps of an encoding or decoding process as described above when said program is executed by a computer. The invention also relates to a computer-readable information or recording medium containing the aforementioned computer program instructions. The information or recording medium can be any entity or device capable of storing programs. For example, the medium may include a storage means, such as a ROM (e.g., a CD-ROM or a microelectronic circuit ROM), or a magnetic recording means, such as a floppy disk or hard drive, a DNA sequence, or flash memory. Furthermore, the information or recording medium may be a transmissible medium, such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, by radio link, by wireless optical link, or by other means. The program according to the invention can in particular be downloaded onto an Internet-type network. Alternatively, each information or recording medium can be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of a process according to the invention. Brief description of the figures The invention will be better understood with the aid of the following description, given solely by way of example and made with reference to the accompanying drawings in which: Figure 1 schematically represents an encoding device according to an embodiment of the invention, Figure 2 schematically represents a decoding device according to an embodiment of the invention, Figure 3 illustrates an example of a synthetic artificial neural network used in the context of the invention, Figure 4 schematically illustrates an example of processing performed by a synthetic neural layer of the neural network of Figure 3, Figure 5 schematically illustrates a second example of processing performed by a neural synthesis layer of the neural network of Figure 3. Figure 6 schematically illustrates a third example of processing performed by a neural synthesis layer of the neural network of Figure 3.Figure 7 is a flowchart representing an example of a coding method that can be implemented by the coding device of Figure 1; Figure 8 illustrates a coding method used in an embodiment of the invention; Figure 9 is a flowchart representing an example of a decoding method that can be implemented by the decoding device of Figure 2; Figure 10 illustrates a decoding method used in an embodiment of the invention; Figure 11 is a flowchart representing a method for coding feature cards that can be implemented by the coding device of Figure 1 and by the coding method of Figure 7; Figure 12 is a flowchart representing a method for decoding feature cards that can be implemented by the decoding device of Figure 2 and by the decoding method of Figure 9. Detailed description of the invention Figure 1 schematically represents, according to a first embodiment, an ENC coding device for at least one region of a signal (I(P n )). In the example described here, the signal I(P n The system is segmented into a set of J zones, and all zones are coded independently. Alternatively, only one zone or only a few zones can be coded independently. This ENC coding device includes a SEG segmentation module, an SEGC encoding module for the segmentation provided by the SEG segmentation module, a GEN module for generating feature maps, an SE module for transforming feature maps, an XTR module for extracting data from feature maps, a TT module for processing and quantization, an NNSYN module corresponding to a synthetic artificial neural network, an NNC module for encoding a neural network capable of encoding the synthetic neural network, and an FMC module for encoding the feature maps. features, an EVAL module for evaluating coding performance, an MAJ module for updating. The ENC coding device can be implemented by means of an electronic device comprising a processor and a memory, not shown; each of the modules mentioned above can then be realized by the cooperation of the processor and computer program instructions stored in the aforementioned memory and designed to perform the functionalities of the module concerned, in particular as described below, when these instructions are executed by the processor. The ENC coding device in Figure 1 receives as input a signal consisting of a succession of samples to be coded, denoted P n , for example a temporal sequence of sound samples, or a set of image data denoted I(P n ). In this second case, the image signal I(P n ) can represent a two-dimensional image, or a plurality of two-dimensional images (video, color components, stereoscopic components, multiscopic components, etc.). Pn denotes a sample n of the input signal comprising N samples. In one embodiment, the signal is a color image signal represented by at least one two-dimensional representation, such as a pixel matrix, each pixel having a red component (R), a green component (G), a blue component (B), or, alternatively, a luminance component (Y) and at least one chrominance component (U, V). The location of each pixel is defined by its abscissa x and ordinate y coordinates in the image. In another embodiment, the image is grayscale represented by a two-dimensional representation, such as a pixel matrix, each pixel having a grayscale component, or luminance component. In this case, the pixel's representative vector is reduced to a single component. The SEG module performs a segmentation Sg of the sequence of samples to be coded P n in J (greater than or equal to 2) Zo zones 1This segmentation operation allows the samples to be grouped into different homogeneous sets based on one or more predefined criteria. For example, in the case of a temporal sequence of sound samples, the segmentation operation allows us to obtain sequences of units corresponding to silences, noises, phonemes, words, etc. Similarly, in the case of an image signal I(P n ), the segmentation operation allows the pixels P to be grouped together n of the image signal in J homogeneous zones according to criteria, notably intensity or spatial. For example, the segmentation operation can make it possible to identify in the image signal I(P n ) two different zones (J=2) correspond, one to the background of the image and the other to the foreground of the image. The segmentation Sg is, for example, represented as a set of J MZo masks J allowing everyone to identify a Zo zone J, each of the masks being associated with a value different from that associated with the other masks. Optionally, the SEGC module performs lossless encoding of the Sg segmentation information. This encoding can be achieved by encoding the segmentation map corresponding to the J MZo masks. J (for example, by using the JPEG-LS algorithm defined by the international standard ISO / IEC 14495-1) or, alternatively, by independently encoding each MZoj mask, or, in yet another alternative, by independently encoding the contours of the J MZo masks J , or in yet another variant by encoding a size or position data for an area, for example a rectangle. The coded segmentation is denoted Sgc. The NNC module performs a coding simulation, followed by a decoding, for the evaluation module. The GEN feature map generation module is configured to generate a plurality of M feature maps, denoted FM!. In one embodiment, the SE module performs a transformation of the first group of FM! characteristic maps to generate a second group of FMSi characteristic maps at the same resolution as the input signal. The optional SE module can perform quantization of the data extracted from this set of M FMi maps. Recall that quantizing a value refers to mapping that value to a member of a discrete set of possible code symbols. For example, the set of possible code symbols might consist of integers, and the quantization system would simply round a real value to an integer. Alternatively, quantization could involve multiplying by a given value and then rounding. The SE module then performs a transformation of the values in at least one of the feature maps, such as oversampling, interpolation, filtering, etc. After the transformation, a transformed feature map from the second group has the same resolution as the images in the input sequence.Advantageously, according to this method, the feature maps that are coded can be of lower resolution than the images to be coded, while the maps of the second group, which are used to construct the feature vectors, are at the same resolution as the image sequence, which facilitates the extraction of values. In one embodiment, the SE module is absent; in this case, the values that will be used to construct the characteristic vector are extracted from the first group of characteristic maps. The XTR module performs value extraction from the FMSi (or FM!, depending on one of the previously described embodiments) feature maps, for a current sample P n to be encoded, depending on its coordinates in the input signal and possibly the segmentation Sg performed by the SEG module. For example, if we want to encode sample P n at coordinates (x n , y n) in a Zo zone 1 From an input image, the XTR module performs an extraction of values in the maps at positions imposed by the coordinates (x n , y n ) and by the MZo mask 1 of the Zo zone 1 . In one embodiment, the extracted values constitute the vector Z n . Z n is an L-tuple, meaning it contains L elements, or data Zi. For example, in one embodiment, L=M, meaning that only one value is extracted for each feature map FM!. In another embodiment, L>M, meaning that several values are extracted for at least one feature map FMi. The dimension L of the vector depends on the topology of the NNSYN synthesis neural network and, more specifically, on the topology of the input layer of this NNSYN synthesis neural network. The vector Z nindex n refers to the characteristic vector of pixel P' n . In one embodiment, the optional TT module processes the extracted values to generate the vector Z n The TT module can perform quantification of the data extracted from the feature set. The processing may include other operations, such as filtering, scaling, etc. In particular, if the SE module is not used and if the feature maps in the first group have lower resolutions than the images in the sequence, the TT module can take into account the coordinates of the values in the lower-resolution maps. Note that at least one of the SE or TT modules must perform a quantification of the feature maps. The NNSYN module is a synthetic neural network defined by K parameters Wk, capable of processing the vector Z n, or L-Tuple, as input, to generate as output a second vector representative of the sample P n to code. An example of a synthetic neural network is presented later with reference to Figure 3. It should be noted that the behavior of the synthesis neural network may depend on the Sg segmentation as will be presented in more detail later with reference to figures 4 to 6. The NNC module performs the encoding of the synthetic neural network, specifically its parameters Wk. During the training, or construction, process of the encoding—that is, until the performance evaluation stage is satisfactory—the NNC module performs a coding simulation, followed by decoding, for the evaluation module. Subsequently, it performs the actual encoding of the synthetic neural network parameters Wk. The encoded parameters are denoted Wck. As is known, the coding simulation can be identical to the actual encoding, or it can approximate it. The FMC module performs the encoding of the FM maps, that is, the values of the characteristic maps in the first group (excluding the maps in the second group, which may optionally result from oversampling by the SE module). During the encoding training or construction process—that is, until the performance evaluation step is satisfactory—the FMC module performs a coding simulation, followed by decoding, for the evaluation module. Subsequently, it performs the actual encoding of the FM map values. The FMC module determines, by taking into account the Sg segmentation performed by the SEG module, an SgL segmentation of each of the FM! feature maps into J ZoL zones. Thus, the FMC module uses the Sg segmentation defined in the domain of the signal to be encoded to obtain a segmentation in the latent domain of the FM! feature maps. Obtaining this segmentation in the latent domain depends on the transformation transform used to go from the domain of the signal to be encoded to the latent domain. Therefore, the SgL segmentation in the latent domain results from a calculation that simultaneously considers the location of the Sg segmentation zones in the signal to be encoded and the transformation transform. For example, if the latent domain and the signal to be encoded have the same resolution, then the segmentation in the latent domain is identical to the segmentation in the domain of the signal to be encoded.According to another example, if a latent latent has a resolution lower than that of the signal to be encoded, then the boundary between two regions of the segmentation in the domain of this latent latent lies between the points in the latent domain whose co-located samples in the signal to be encoded belong to two different regions of the Sg segmentation. Each of the FM maps is represented as a set of J characteristic regions ZoL, each of the characteristic regions ZoL corresponding in the FM map to region j of the SgL segmentation. It is possible that some characteristic regions ZoL are empty. The encoding of the FM map values is performed taking into account the SgL segmentation. More precisely, each of the J characteristic regions ZoL... J r i is coded successively, for example in the order of their Indexing. Thus, each of the characteristic zones (ZoL) is coded independently of the other characteristic zones. The coded data of each coded zone (ZoL) J r i Each FM! characteristic map is marked FMcJ. Each ZoL zone J r i undergoes entropic coding. Thus, the coding of the M FM! cards corresponds to the independent coding of the J zones characteristic of the M FMi cards. In a first example implementation, the FMC module only codes the values of the points in the characteristic zone ZoL J with the exception of any other point on the FMi feature maps. Thus, the coding of a characteristic zone ZoL includes the successive coding for each FMi map of the values of the points of the characteristic zone ZoL'j in the FMi map. Alternatively, the FMC module creates a secondary FM2J feature map for each feature zone ZoL JThe value of a point on the FM2J secondary feature map is equal to the value of the point on the FMi map if that point belongs to the ZoL feature zone J and to a predetermined value, for example zero otherwise. The FMC module then encodes the entire FM2 feature map ■ to obtain the different EcJ-encoded data for each feature zone ZoL J . As is known, the coding simulation can be identical to the actual coding, or it can approximate it. The coding module quantizes, if necessary, the latent representation of the values in the first group of maps using a quantifier to generate an ordered collection of quantized values. The coding module then compresses the quantized data, using a coding method that takes into account the neighborhood of a value to be coded in the feature map. The EVAL module performs an evaluation and minimization of coding performance. The evaluation function is, for example, of the rate-distortion type. Minimization can be performed by gradient descent, or any other method within the grasp of a person skilled in the art. The MAJ module performs an update of the values of the FMi cards and / or the parameters of the neural network to be encoded, according to the results of the performance function. Figure 2 schematically represents a DEC decoding device for a zone to be decoded Zod J of a signal, called the decoding zone, said decoding zone Zod J comprising a plurality of Pd samples n to decode. This DEC decoding device includes an NND module for decoding neural network(s) capable of decoding the NNSYN' synthesis neural network, a SEGD module for decoding a segmentation, an FMD module for decoding feature maps, an XTR' module for data extraction, an SE' module for inverse transformation, and a TT' module for inverse processing and quantization. The DEC decoding device produces an output decoded image including at least the decoded area, denoted Zod. J (Pd n ), comprising a plurality of decoded samples Pd n . The DEC decoding device in Figure 2 receives as input the Sgc encoded segmentation and a group of FMcJ encoded data. The DEC decoding device in Figure 2 also receives as input the Wck encoded parameters from the NNSYN' synthesis neural network. The parameters of the NNSYN' synthesis neural network decoded by the NND module are noted Wdk. The SEGD module optionally decodes segmentation information, for example the MZo'' mask from the Sgc encoded data. According to embodiments as described for the encoder: The segmentation map corresponding to the J masks MZo 1 is decoded from the Sgc data and the MZo mask 1 is extracted from the segmentation map. The MZo mask J is decoded directly when it has been previously encoded in Sgc-encoded data independently of other masks. If the segmentation information is not decoded, it can be inferred by the decoder (for example, because it is static, or because the decoder knows the size of the rectangular areas corresponding to the tiles or the number of blocks in the slices, etc.). The FMD module constructs M decoded FMdJ maps using the MZo mask 1 and the FMcJ encoded data. Depending on the FMc data encoding implementation method used by the ENC encoder and based on the MZo mask 1 In the domain of the signal to be decoded, the FMD module determines the characteristic area ZoL J corresponding in the latent domain, then the FMD module decodes the FMcJ data to obtain the point values of this characteristic zone ZoL J in each of the FMdJ maps. In one embodiment, the SE' module performs a transformation of the first group of decoded FMdJ characteristic maps to generate a second group of maps. Features at the same resolution as the signal to be decoded, denoted FMS'J. The SE' module optionally performs inverse quantization corresponding to the quantization performed by the encoder. Inverse quantization is not necessary if the encoder's quantizer Q has simply rounded the real values submitted to it. Inverse quantization is also unnecessary if the neural network is capable of handling quantization of its input data. Otherwise, the decoder performs the inverse operation of quantizer Q. The SE' module then performs a transformation of the feature map values, including, for example, oversampling, interpolation, filtering, etc., similar to that performed by the encoder. After the transformation, a transformed feature map from the second group has the same resolution as the images of the sequence to be decoded. In one embodiment, the SE' module is absent; in this case, the values that will be used to construct the characteristic vector are extracted from the first group of characteristic maps. The XTR' module is identical to the XTR module in Figure 1. It performs an extraction of values from the M FMdJ (or FMS'J, depending on one of the embodiments described previously) feature maps, for a sample Pd n to decode, depending on its coordinates in the signal to be decoded and the MZo mask 1 . In one embodiment, the extracted values constitute the vector Zd n . Zd n is an L-tuple, that is to say, it contains L elements, or zd data, In one embodiment, the optional TT' module processes the extracted values to generate the vector Zd nThe TT module can perform inverse quantization of the data extracted from the feature set. The processing can include other operations, such as filtering, scaling, etc., similar to those performed by the encoder. The NNSYN' module is a so-called synthetic neural network, defined by K parameters Wdk, capable of processing the vector Zd n , or L-Tuple, as input, to generate as output a second vector representative of the sample Pd n The decoding process generally involves a vector containing A elements. In one embodiment, K=3 and the output vector is the triplet (R, G, B) of the decoded pixel Pdn. The NNSYN' module has the same structure as the NNSYN module, and its parameters are either identical if the encoding of its parameters Wk is lossy, or different if the encoding is lossy. When all the samples Pd n The signal has been decoded, we have a reconstructed Zod signalJ (Pd n ) of the Zo zone J . Furthermore, the DEC decoding system can be successively implemented to decode all Zod zones J in order to reconstruct all the samples to be decoded from the signal, i.e., from the image I(Pd n ). The DEC decoding device can be implemented using an electronic device comprising a processor and memory, not shown. Each of the modules mentioned above can then be implemented through the cooperation of the processor and computer program instructions stored in the aforementioned memory and designed to perform the functionalities of the module in question, particularly as described below, when these instructions are executed by the processor. The DEC device can also include multiple processors, with each processor dedicated to the parallel decoding of image areas. Figure 3 illustrates an example of a synthetic artificial neural network used for encoding and decoding in embodiments of the invention. The synthetic artificial neural network used for encoding, NNSYN, and the synthetic artificial neural network used for decoding, NNSYN', are defined by an identical structure, including for example a plurality of layers of artificial neurons, and by a set of weights and activation functions associated respectively with the artificial neurons of the network concerned. The synthetic neural network is, in one embodiment, an MLP, or Multi-Layer Perceptron, followed by one or more layers of convolutional neurons Ci, ... C n , each of the convolutional neuron layers being associated with a filtering mask of predefined size, for example size 3x3. MLP consists of an input layer adapted to the input format (the L-tuple), optionally one or more hidden layers, and an output layer providing an intermediate output vector Vs n , generally a vector containing A' elements. Thus, a vector representation of a current sample (a vector Z) n or Zd n derived from the FMi / FMSi characteristic maps at the encoder or FMdJ / FMS'J at the decoder) is applied to the input (i.e., to the input layer) of the MLP which produces the intermediate output vector Vs n . The concatenation of all these intermediate output vectors constitutes intermediate latent value maps which are then processed successively by the layer(s) of convolutional neurons to provide as output a set of output vectors also containing A elements. According to one embodiment, A is equal to 3, the intermediate output vector is a triplet, and the output vector is the triplet (R, G, B) of pixel P' n encoded then decoded. The concatenation of all these triplets in an image constitutes the reconstructed signal I(Pdn), for example an I image, or Zod J (Pd n ), according to an example the decoded zone Zod J . At the encoder, the NNSYN synthetic artificial neural network is trained on the image, so as to minimize the differences between the input representation of the current image I(P n ) and its representation I(P' n ) at the output, while also minimizing the amount of data to be encoded. The EVAL module performs a performance measurement in this regard. Once the encoder training is complete, the network parameters are encoded, either losslessly, in which case the NNSYN' neural network is identical to NNSYN, or lossily, in which case the NNSYN' network may be slightly different from NNSYN. With reference to Figure 4 and Figure 5, we will now present the application of a convolutional neural layer Ck of the NNSYN / NNSYN' synthesis neural network to a group of input latent value maps CCEi, CCE2, CCE3, to obtain at the output of the convolutional neural layer Ck a group of output latent value maps CCSi, CCS2, CCS3. In the example described here, the convolutional neural layer Ck is associated with a convolutional kernel of predefined size, for example size 3x3. This convolutional neural layer Ck transforms three input latent value maps CCEi, CCE2, CCE3 into three output latent value maps CCSi, CCS2, CCS3. The application of this convolutional neural layer Ck to the three input latent value maps CCEi, CCE2, CCE3 includes, in a step (1), obtaining a characteristic region R J including all points of the CCEi, CCE2, CCE3 maps co-located with samples belonging to the MZo mask 1 of the Zod zone J The points in this characteristic region R are called determined points. Then, during step (2), for a Ps point n CCSi, CCS2, CCS3 maps colocalized with a Pt sample n belonging to the MZo mask 1 from the Zod decoding zone J , an input vector Vcon of the convolutional neural layer Ck is constructed. This input vector Vco n depends on point Pa n CCEi, CCE2, CCE3 maps co-located with point Ps n CCSi, CCS2, CCS3 maps and contextual information from this point Pa n . This contextual information is represented by a neighborhood of point Pa n , this neighborhood including by definition the point Pa n itself. In the example presented here, the neighborhood used comprises 9 points associated with a 3x3 square mask centered on point Pa n . This contextual information is also conditioned by the characteristic region R J in the CCEi, CCE2, and CCE3 input maps. More specifically, the contextual information depends on the location of point Pa n relative to the boundary of zone R J This location is obtained by evaluating a proximity criterion Cp(Pa n ) from point Pa n with respect to the boundary of the characteristic region R. In the example described here, the proximity criterion Cp(Pa n ) is the Euclidean distance between the point Pa nand the boundary of the characteristic region R. Alternatively, the proximity criterion Cp(Pa n ) can be a Manhattan, Minkowski, or Chebyshev distance. In another variant, the distance can simply be measured in pixels, for example, by counting the number of pixels along the two directions of a coordinate system associated with the current map. In yet another variant, the proximity criterion Cp(Pa n ) perhaps also a quasi-distance or gap or any other relevant measure indicating the proximity of point Pa n at the border of the characteristic region R. Two cases must then be distinguished: In the first case, illustrated in Figure 4, the proximity criterion Cp(Pa n ) is greater than (or possibly equal to) a distance d (for example, predefined) and the neighborhood values of the input maps CCEi, CCE2, CCE3 are used to form the input vector Vco n . In the second case, illustrated in Figure 5, the proximity criterion Cp(Pa n ) is less than this distance d (in the example described here, the point Pa n is located at zero distance from the boundary of the characteristic region R). In this case, the input vector Vco n is defined component by component. The value of a component of the input vector Vco n is equal to the value of the associated point on the input map if that point belongs to the characteristic region R and to a replacement value otherwise. In step (3), the input vector is processed by the convolutional neural layer Ck to generate the output point Ps n CCSi, CCS2, CC3 cards In Figure 4 and Figure 5, the values shown in gray are part of the characteristic region R and are used to determine components of the input vector Vco nThe values shown in white (Figure 5) do not belong to the characteristic region R but to another region R x and cannot be used to define the input vector Vco n , they are replaced by a replacement value Rp. In one embodiment, the replacement value is a predetermined constant value, for example equal to 0. In another embodiment, the replacement value is a function of the values of the neighborhood points belonging to the characteristic region R; for example, the replacement value is equal to the value of point Pa nIn another embodiment, a set of replacement values is calculated based on the values of neighborhood points belonging to the characteristic region R. A neural network can be implemented to calculate a given number of replacement values based on neighborhood points of the characteristic region R. In the previous embodiments, the definition of the neighborhood of point Pa n is independent of the proximity criterion Cp(Pa n ). In another embodiment, and as shown in Figure 6, the definition of the neighborhood of point Pa n may depend on the proximity criterion Cp(Pa n ). Two cases must again be distinguished: In the first case, the proximity criterion Cp(Pa n ) is greater than (possibly equal to) the distance d and the values associated with a first neighborhood of point Pa n(for example identical to that defined previously in connection with figure 4) input cards CCEi, CCE2, CCE3 are used to form the input vector VC0n. In the second case, the proximity criterion Cp(Pa n ) is less than this distance d. In this case, a second neighborhood of point Pa n is used. Preferably, to respect the topology of the convolutional neural layer Ck, the number of points in the first neighborhood is equal to the number of points in the second neighborhood. In the example shown in Figure 6, the second neighborhood is obtained from the first neighborhood by adapting its shape to the boundary of the characteristic region R so that the points in the second neighborhood all belong to the characteristic region R. Once determined, the input vector is processed, in step (3), by the convolutional neural layer Ck to generate the output point Ps nCCSi, CCS2, CC3 cards Figure 7 is a flowchart representing an example of a coding process that can be implemented by the coding device of Figure 1 when the NNSYN synthesis neural network is, for example, the one shown in Figure 3. According to this embodiment, the signal is a two-dimensional image; each sample to be coded is therefore a pixel P n coordinates (x n , y n ). The encoding process takes place in three main phases: In a first phase, called the segmentation phase, the segmentation Sg of the input signal I(P n ) in J (greater than or equal to 2) Zo zones J is done. In a first example of segmentation, the image is divided into J regular zones, for example, of identical size and shape (except possibly at the edges of the image). For example, the image can be divided into rectangular zones with horizontal and vertical boundaries. Such a division corresponds to the concept of "tiles" implemented by encoding standards such as H.264 / AVC, H.265 / HEVC, and H.266 / VVC. The segmentation information allows these rectangular zones to be identified, for example, by specifying their vertical and horizontal dimensions. In a second example of segmentation, the image is divided into J zones corresponding to samples (or blocks) scanned in a lexicographical order. This division corresponds to the concept of "slices" implemented by encoding standards such as H.264 / AVC, H.265 / HEVC, and H.266 / VVC. Segmentation information allows these zones to be identified, for example, by indicating the number of blocks. In a third segmentation example, the image is divided into semantic zones, such as a background and a foreground. This can be done manually by an operator. Segmentation can also be automatic or semi-automatic, depending on the segmentation algorithm used. It's important to note that there are no restrictions related to the type of segmentation algorithm used. The segmentation information allows these zones to be identified, for example, through a segmentation mask. Advantageously, such a division into zones allows, during the subsequent decoding of the I(Pdn) signal, for the decoding of the zones to be decoded Zod to be parallelized. J by distributing the decoding load evenly across each of the decoders and / or processors. In a second phase, called the construction phase, a learning process is carried out in order to determine, for an input signal I(P n), the values of the FM! maps and the Wk parameters to optimize an overall cost function. Learning is, for example, performed by gradient descent, followed by an update of the neural network parameters of NNSYN synthesis of FM characteristic map values. As is known in the state of the art, the cost function can be of the rate-distortion type, or rate-distortion, or perceptual. To measure the rate R, it is necessary to simulate the encoding of the J-zones ZoL JFM-" cards, then measuring the associated coding rate (the size of the B2 stream). In one embodiment, the coding of the Wk parameters is not simulated because their influence is less significant than that of the characteristic cards. In another embodiment, the coding of the Wk parameters is also simulated, and the associated rate (the size of the B1 stream) is measured. To measure the distortion D, it is necessary to simulate the coding and then the decoding of at least a portion of the image I, to obtain at least one pixel P' n resulting from a simulation of encoding and then decoding, then measuring the difference between this part of the image I(P n ) as input and a corresponding part of the image I(P' n ) encoded then decoded. Then, during a third phase, called the coding phase, the data from each ZoL zone J r iand optionally the Sg segmentation, the Wk parameters are encoded to produce the encoded values Sgc, FMc and Wck before transmission or storage. They constitute the compressed representation of the input signal I(P n ), this compressed signal can be decoded zone by zone. We will now describe the steps of a coding process according to one embodiment of the invention. During an E20 step, a signal I(P n ) to be coded, comprising a plurality of N samples P n , is provided as input to the process. During step E21, the segmentation Sg of the input signal I (P n ) in J (greater than or equal to 2) Zo zones 1 is done. During step E22, the M FM! maps of the first group are initialized. Subsequently, the Wk parameters of the NNSYN synthesis neural network and the values of the FM! maps must be optimized during the construction phase. According to one embodiment, the FMi cards have the same resolution as the input signal I (P n ) and therefore each have the same number of values N as there are samples P n to code. According to one embodiment, the FMi cards have a resolution less than or equal to that of the input signal I(P n ) and therefore include, for at least one of them, a number N' of values to be coded less than N. According to a variant, the first FMi card is at the resolution of the images and each subsequent card is at half the resolution of the previous one. According to one embodiment, several FM cards have the same resolution, lower than that of the input signal I(P n ). In one embodiment, the FM! maps are transformed to provide a second group of FMSi transformed feature maps. In this embodiment, the feature vectors are preferably extracted from the transformed maps of the second group, and not directly from the maps of the first group. Thus, in this embodiment, the feature vectors are indirectly extracted from the maps of the first group. The maps of the second group are not coded; they are used solely for constructing the feature vectors. According to one embodiment, FMi cards are initialized with predefined constant values. According to another embodiment, the feature maps are initialized by a set of random real values. The FMi feature maps of the first group are subsequently updated, or refined, during an E23 step, by the encoder's MAJ update module during its learning. During step E24, the J zones ZoL of the FMi maps in the first group are encoded by the FMC module of the encoder. During the construction phase, this operation is a coding simulation. During the encoding phase, this operation is the actual encoding, and the encoded values constitute the B2 stream. The simulation can be identical to the actual encoding, but it can also be different (for example, simplified). For this encoding, a technique for predicting a feature map value based on its neighborhood is used, as will be described, for example, in support of Figure 11. In one embodiment, the J zone ZoL J FMi maps are coded in the indexing order of the J zones and in a predefined order of associated characteristic maps (i.e., FMi, FM^, FM|, FM4, FMi,...) and the values of each map, for example, lexicographically. Each ZoL zone J FMi cards undergo entropic coding. Entropic coding of all ZoL areas of all FMi maps produces a compressed B2 stream whose throughput is subsequently measured during an E29 step. In an E25 step, according to one embodiment, the M cards of the first group FMi are transformed by the SE module to generate cards of the second group FMSi at the resolution of the images of the input sequence. According to one embodiment, M FMSi cards are generated. According to one embodiment, each FM! card is transformed into an FMSi card. In one embodiment, at least one FM! map has a lower resolution than the images in the sequence to be encoded, and the transformation operation includes oversampling so that the transformed FMSi map has the same number of samples as the images in the sequence. Oversampling consists of adding values to the FMSi maps to achieve the resolution of the input sequence images. It can be simple (by nearest neighbor replication) or involve interpolation (linear, polynomial, filtered, etc.). During step E26 and taking into account the segmentation Sg of the input signal I(P n In J zones Zo, values are extracted by the XTR module from the FM! or possibly FMSi transformed maps. This extraction is performed based on the coordinates (x n , y n ) of sample P nof the input signal and possibly the MZcL mask. It can also be done according to the resolution of the card in question. According to one embodiment, the vector Z n This characteristic results directly from this extraction. The samples to be coded are, for example, processed in sequential order, from n=l to n=N. According to one embodiment, during step E27, the vector Z n The characteristic is constructed by the TT module from the values extracted from the FMi or FMSi maps for each sample P n coordinates (x n , y n ) of the input signal. The processing may involve quantizing the values extracted from the FM cards or the Z vector n constituted, if necessary. The processing may include other operations, such as filtering, scaling, the application of any function, preferably monotonic, etc. In one embodiment, Zn is an M-tuple (zi, Z2, ..., zj), consisting of the values of the FM! or FMSi cards located at the coordinates (x n , y n ) of the current pixel P n as will be illustrated in support of figure 8.
[0002] In one embodiment, Z n is an M-tuple constructed from values taken from FM maps at coordinates that may differ between maps. For example, if the FM maps have different resolutions because they were undersampled, the coordinates are adjusted (by scaling) to match the resolution of each map. During step E28, the vector Z n is processed by the NNSYN synthesis neural network to generate as output a vector representative of the sample P n to code, according to one embodiment, the triplet (R, G, B) of the sample P' n (sample P) n (encoded then decoded). The structure and Wk parameters of the synthesis neural network are initialized, for example, during the first iteration of this step. These parameters are subsequently updated, or refined, during the construction phase, in later iterations of the process. According to one embodiment, the parameters of the synthesis neural network and / or the prediction neural network are initialized by predefined values known to give a satisfactory result (for example, following training on a corpus of images). According to another embodiment, the parameters of the synthesis neural network and / or the prediction neural network are initialized by a set of random values. During step E29, the Wk parameters of the NNSYN synthetic neural network are quantized and encoded. During the construction phase, this operation is a coding simulation. During the encoding phase, this operation is the actual encoding, and the encoded values constitute the B1 stream. The simulation can be identical to the actual encoding, but it can also be different (for example, simplified). Any known technique can be used for this purpose, for example, the neural network coding standard proposed by MPEG-7 Part 17, also called NNR (Neural Network Representation). Note that in this case, the amount of degradation that the encoding introduces to the Wk weights must be chosen. During an E30 step, a performance measure is evaluated. To this end, the coding simulation rates associated with the feature maps of the first group (simulation of the B2 flow by coding the J FM J zones of the FM maps,) and optionally with the parameters of the neural network(s) (simulation of the B1 flow by coding the Wk parameters) are measured. In one embodiment, the cost function is of the rate-distortion type, denoted (D+L*R), where D, for example, is the root mean square error measured between the input signal and the decoded signal (or the error measured on a subset of the signal samples). In another example, D is calculated from a perceptual function such as SSIM (for Structural SIMilarity) or MSSSIM (for Multi-scale Structural SIMilarity). In one embodiment, R is the simulated rate of stream B1; in another embodiment, R is the total rate used to encode this image, i.e., the sum of the simulated rates of B1 and B2. L is a parameter that controls the rate-distortion trade-off. Other cost functions are possible. As long as the cost function has not reached its minimum, or a maximum number of iterations of the cost function minimization algorithm has been reached, the measure of performance is not satisfactory, and the process is repeated from step E23. This minimization can be performed by a mechanism known as gradient descent with parameter updates during step E23 for feature map values and E29 for network(s) parameters. During an EF step, if the cost function has reached its minimum, or if a maximum number of iterations of the cost function minimization algorithm has been reached, training stops. If a coded version corresponding to the last simulation of the synthetic neural network parameters (Wk) and feature maps (FMj) is available, the B1 and B2 streams can be constructed from it. In another embodiment, the actual coding of the updated synthetic neural network parameters (Wk) and feature map values (FMj) is performed at this step to produce the encoded parameters Wcket FMc that constitute the B1 and B2 streams. Furthermore, at this step, the Sg segmentation is encoded to produce an encoded segmentation of the J masks MZoc J which constitutes the B3 flow. The B2 stream comprises sub-streams corresponding to each area of the feature maps. During the encoding of each latent area, it is possible, in a preferred embodiment, to place information identifying the location of the sub-streams within the overall stream at an identifiable point in the stream. This location information can be: a stream pointer (which indicates the address of the beginning of each sub-stream in the overall stream), or a marker (a series of bits otherwise prohibited, which allows traversing the stream to find the beginning of each sub-stream), or any other means of identifying a sub-part of an encoded stream. The B1 and B2 streams can be multiplexed and / or concatenated to produce a final stream. In one embodiment, the B3 stream of the coded segmentation and the B1 stream of the coded parameters of the neural network(s) are stored or transmitted before the B2 stream, so that they can be decoded before the B2 stream. It should be noted that only one NNSYN neural network is used for encoding each Zcf area. Figure 8 illustrates a coding method used in one embodiment of the invention. In this embodiment, 4 FM! cards are generated. In a preferred mode, there are 7. The first FMi card has the same resolution as the I(P) image n ), and therefore includes WxH variables, where W represents the image width in pixels, and H its height. The second FM2 map has half the resolution (in each dimension) of the FM1 map. Each additional map has half the resolution of the previous map. This structure reduces the number of variables in the feature maps, which simplifies coding and learning while minimizing coding costs. The FM2 map is oversampled by the SE module by a factor of 2 in each dimension, according to a procedure illustrated in Figure 6. The FM3 map is oversampled by a factor of 4 in each dimension, and the FM4 map by a factor of 8 in each dimension. The FMi map is not affected by the oversampling. (FMSi=FMi). The resulting FMSi maps have the same resolution as the I(P) image. n ), and therefore each have WxH values, where W represents the width of the image in pixels and H its height (N=WxH). Other types of structure are possible, for example one can use a different reduction rate than half between cards (a quarter, or a third, etc.). In this embodiment, the vector Z n is a 4-tuple (Z1...Z4) consisting of the values extracted from the FMSi maps located at the coordinates (x n , y n ) of the current pixel P n The vector Z nconsisting of the extracted (quantified) values from the FMSi maps is processed by the NNSYN synthesis neural network to generate a second output vector; in this example, the output vector is the triplet (R, G, B) of pixel P'. n encoded then decoded. The triplet is inserted into the decoded image I(P' n ) at positions (x n , y n ) of the color components (R', G', B'). In another embodiment, not shown, the vector Z n is extracted directly from the FM! layers, with positions recalculated according to map size, then the extracted values are possibly processed and quantified after extraction. Figure 9 is a flowchart representing an example of a decoding process for a Zod zone. J which can be implemented by the DEC decoding device in Figure 2 when the NNSYN' synthesis neural network is, for example, the one shown in Figure 3. During an F20 step, the B1 flow, a part of the B2 flow (that corresponding to the Zod area characteristic maps) J ) and the B3 stream are extracted from the encoded stream. They contain respectively, Wck parameters, an Sgc segmentation, and FMc coded representations of the M maps of the first group representing the area to be decoded Zod J . During an F21 step, an MZo segmentation mask 1 is generated by decoding the values of the encoded segmentation information, or if this information is not encoded, obtained from decoder parameters (size or known position of slices or tiles, etc.). During step F22, the M FMdJ cards are generated by decoding the FMcJ values. In one embodiment, the FMdJ cards are decoded in the order (FMd^, FMd J2,... FMd^), and the values of each card in a predefined order, for example lexicographical, possibly taking into account (depending on the coding technique implemented by the ENC encoder) the MZo mask J . According to embodiments as described for the encoder: The FMdJ cards have the same resolution as the I(Pd) signal. n ) to be reconstructed, that is to say that they include N=WxH values. The FMdJ cards have a resolution less than or equal to that of the I(Pd) signal. n ) to be reconstructed. Several FMdJ cards have the same resolution, which is lower than the signal resolution. In step F23, according to one embodiment, the M maps of the first group FMdJ are transformed by the SE' module to generate maps of the second group FMS'J at the resolution of the input images. This step is similar to step E25, which was described for the encoder in support of Figure 5, and the embodiments apply. In particular: According to one embodiment, M FMS'Jsont cards are generated. According to one embodiment, each FMdJ card is transformed into an FMS'J card. In one embodiment, at least one FMdJ map has a lower resolution than the images of the image to be encoded, and the transformation operation includes oversampling so that the transformed FMS'J map has the same number of samples as the input image. Oversampling consists of adding values to the FMS'J maps to achieve the resolution of the input image. It can be simple (by nearest neighbor replication) or involve interpolation (linear, polynomial, filtered, etc.). The transformation may optionally include inverse quantization of the extracted values, if necessary. However, inverse quantization is not mandatory. During step F24, values are extracted by the XTR' module from the transformed FMdJ or possibly FMS'J maps. This extraction is performed based on the coordinates (x n , y n ) of a sample P nof the input signal. It can also be implemented depending on the resolution of the card in question. This step is similar to step E26, which was described for the encoder in support of Figure 7, and the embodiments apply. In one embodiment, Zd n is an M-tuple (zi, Z2, ..., zj), consisting of the values of the FMdj or FMS'j maps located at the coordinates (x n , y n ) of a current pixel Pd n as will be illustrated in support of figure 10. The samples to be decoded are, for example, processed in sequential order relative to the MZo mask. J . According to one embodiment, during a step F35, a vector Zd n is constructed by the TT' module from the values extracted from the FMdJ maps of the first group or the FMS'J maps of the second group, for each sample Pd n coordinates (x n , y n) to decode, based on the coordinates (x n , y n This step is similar to step E27, which was described for the encoder in support of Figure 7, and the described embodiments apply. The extraction may involve inverse quantization of the extracted values or of the vector Zd n constituted, if necessary. During step F26, the Wdk parameters of the NNSYN' synthesis neural network are generated by decoding the Wck values of the B1 stream. Any known decoding technique corresponding to the encoding technique used by the encoder can be used for this purpose. The NNSYN' synthesis neural network is similar to the NNSYN synthesis network; that is, it has the same structure and parameters, except for the encoding, which can be performed with or without loss. In one embodiment, stream B1 is decoded before streams B2 and B3, so that the NNSYN' synthesis neural network is available before decoding the samples. Similarly, stream B3 is decoded before stream B2 so that the mask of the region to be decoded is available before decoding the samples. During step F27, the vector Zd n is processed by the NNSYN synthesis neural network 1 to generate as output a second vector representing the sample Pd n to decode, according to one embodiment, a triplet that is injected into the image of the decoded zone Zod J (Pd n ) to positions (x n , y n ) of the color components (Rd, Gd, Bd). This step is similar to step E28 which was described for the encoder in support of Figure 5. When all the signal samples have been processed, the decoded signal corresponds, for example, to the Zod zone J (Pd n ), is available. It should be noted that only one NNSYN' neural network is used during decoding regardless of the Zod area J decoded. Figure 10 illustrates a method for decoding a Zod zone. J used in one embodiment of the invention. In this embodiment, there are 4 decoded F Md cards. In a preferred mode, there are 7. In this embodiment, the decoded FMdJ maps are representative only of the Zod zone J In other words, only data from the ZoL area J (corresponding in the latent domain to the Zod zone) J ) have been decoded. In this embodiment, the first FMd card has the same resolution as image I, and therefore has variable WxH, where W represents the width of the image in pixels, and H its height. The second FMd card J 2 is half the resolution (in each dimension) of the FMd map J r Each additional map has half the resolution of the previous map. This structure reduces the number of variables in the feature maps, which facilitates decoding while minimizing coding costs. The FMdJ map is oversampled by a factor of 2 in each dimension, using any oversampling method available to a person skilled in the art. The FMd map J 3est is oversampled by a factor of 4 in each dimension, and the FMd^ map by a factor of 8 in each dimension. The FMS'J cards have the same resolution as the image to be decoded, and therefore include WxH values, where W represents the width of the image in pixels, and H its height. In this embodiment, the vector Zd n is a 4-tuple (Z1...Z4) consisting of the values of the FM S' cards, located at the coordinates (x n , y n ) of the current pixel Pd n The vector Zd n is optionally dequantized and then processed by the NNSYN' synthesis neural network to generate the output triplet (R, G, B) representative of the Pd sample n to be decoded. The triplet (R, G, B) is inserted into the decoded image I(Pd n ) at the coordinates (x n , y n ) in the color components (Rd, Gd, Bd). Figure 11 is a flowchart representing a method for coding feature cards which can be implemented by the coding device of Figure 1 and by the coding process of Figure 5. These steps are substeps of step E30 described previously in Figure 7. Their purpose is to encode a current value V n from a point in a characteristic zone ZoL; from a feature map FMdJ of the first group being processed using neighborhood values. During substep E301, a neighborhood vector C n is established, including values close to the value V n These neighboring values can be located in the same map and / or in a different map from the plurality M of FMdJ maps. This neighborhood vector consists of a number C of values, or data, corresponding to neighborhood values (for example, C=10). These values must be known to the encoder and the decoder; therefore, they must be located in a causal neighborhood of the value V. n . According to a first embodiment, these values are used to determine the context of an entropy encoder to encode the current value during an E303 step. This encoder can be a CAB AC (Context-Adaptive Binary Arithmetic Coding) encoder. This type of encoder is well known to those skilled in the art. It is notably used in the H.265 / HEVC video compression standard. It is an arithmetic encoder with lossless compression. It decomposes all non-binary symbols into binary symbols. Then, for each bit, the encoder selects the most suitable probability model and uses a context to optimize the probability estimation. This context can be defined by information from neighboring elements. Arithmetic coding is then applied to compress the resulting data. As is known to those skilled in the art, there are several ways to use the neighborhood vector to produce context information.For example, one can count the number of neighboring values that are not zero and associate a context with each number. Alternatively, one can perform comparisons between several neighboring values and associate a given context with a configuration of order between the neighboring values, for example by ranking the neighboring values in ascending order and associating a context with each possible order. In a second embodiment, the neighborhood is used to predict, during step E302, the current value from an autoregressive model. Recall that an autoregressive model predicts a sample from a series based on its past values. In this embodiment, the past values are constituted by the context, and the difference between the predicted variable and the actual value is quantified and then entropically coded during step E303. At the end of the process, the current coded value Vc nThe FMdJ card currently being processed is coded. Figure 12 is a flowchart representing a method for decoding feature cards that can be implemented by the decoding device in Figure 2 and by the decoding process in Figure 7. These steps are sub-steps of step F22 described previously in support of figure 7. They aim to decode a current value of a point of a characteristic zone ZoLjd'a feature map FMdJ of the first group being processed using neighborhood values. During a substep F221, a neighborhood vector Cd n is established, including values close to the value Vd nThis step is similar to step E301 described earlier, and the same embodiments apply. This neighborhood vector consists of a number C of values, or data, corresponding to neighborhood values (e.g., C=10) located in the same map and / or in a different map from the plurality M of maps of a point in a characteristic zone ZoL; of a feature map FMdJ. These values lie in a causal neighborhood of the value Vd n are known to the decoder. According to a first embodiment, these values are used to determine the context of an entropy decoder for decoding the current value during an F223 step. This decoding is similar to that used in the encoder, for example, CABAC. The use of the neighborhood to produce context information is similar to that chosen for the encoder. For example, one can count the number of non-zero neighboring values and associate a context with each number. Alternatively, one can perform comparisons between several neighboring values and associate a given context with an ordering configuration among the neighboring values, for example, by sorting the neighboring values in ascending order and associating a context with each possible order. In a second embodiment, the neighborhood is used to predict the current value from an autoregressive model during step F222. In this mode, the past values are constituted by the context, and the difference between the predicted variable and the actual value is quantified and then entropically coded during step F223. At the end of the process, the current decoded value Vd n The FMd card, currently being processed, is being decoded. It should also be noted that the invention is not limited to the embodiments described above. Indeed, it will be apparent to those skilled in the art that various modifications can be made to the embodiments described above, in light of the information just disclosed to them. For example, NNSYN / NNSYN' synthetic neural networks can be recurrent neural networks. In another example, synthetic neural networks can consist of one or more convolutional neural networks, followed by a machine learning program (MLP), and then followed by one or more convolutional neural networks, each convolutional neural network being associated with a convolutional kernel of predefined size, for example, 3x3. In these examples, obtaining the Z vectors n / Zd n is adapted to the topology of the NNSYN / NNSYN' synthesis neural networks in a similar way to the adaptation of the input vector Vcon shown with reference to figures 4 to 6. In the detailed presentation of the invention given above, the terms used shall not be interpreted as limiting the invention to the embodiments set forth in this description, but shall be interpreted as including all equivalents which can be foreseen by a person skilled in the art by applying their general knowledge to the implementation of the teaching which has just been disclosed to them. Other implementation methods of the coding process are presented below. In one embodiment, a method for coding a zone (Zo) is proposed. 1 ) of a signal (I(P n )), said coding area, said coding area comprising a plurality of samples (P n ) to be encoded, said encoding process comprising the following steps: obtaining a first group of at least one latent value map (FM) representative of said signal (I(P n )), obtaining a mask (MZo 1) in said signal (I(P n )) of said area to be coded for at least one sample of said area to be coded (Zo 1 ). said current sample (P n ), associated with a position (x n , y n ) in said signal (I(P n )) to be coded: the construction of a characteristic vector (Z n ) from said latent value maps (FM,) of said first group, as a function of said position (x n , y n ) of said current sample (P n ), • the processing of said characteristic vector (Z n ) by an artificial neural network, called a synthetic neural network (SNN), said synthetic neural network being defined by a set of parameters (Wk) and comprising at least one synthetic neural layer (ck) to obtain, at the output of said synthetic neural network, a vector (P' n) representative of a decoded value from the current sample (P n ), • the updating of at least one value of one of said latent value maps of said first group and / or of at least one parameter of said synthetic neural network, as a function of a coding performance measure, obtaining a second group of latent value maps representative of at least said area to be coded (Zo J ) from said mask (MZo 1). the encoding of the second group of latent value maps (F Mc J). the encoding of said mask (MZoi), and the encoding of said parameter set (Wk) of said synthetic neural network. said processing comprising the application of said at least one synthetic neural layer to transform at least one group of input latent maps (CCE), referred to as input maps, into at least one group of output latent maps (CCS), referred to as output maps, said application comprising: obtaining a characteristic region (R J ) of said zone to be decoded in said entry maps from said mask (MZo 1 ). said characteristic region (R J ) including the points associated with the mask, for at least one point (Pt n ) of said area to be decoded, called point to be decoded: • obtaining an associated point (Pa n ) audit point to decode (Pt n ) in at least one of the latent input maps (CCE), • the evaluation of a proximity criterion for said associated point (Pa n ) with respect to a boundary of said characteristic region, the construction of an input vector (Vco n ) of said neural synthesis layer from a neighborhood of said associated point (Pa n ) and of said assessment of said proximity criterion, and the processing of said input vector by said neural synthesis layer to obtain a point (Ps n ) of at least one of the output cards. Here we present other ways of implementing the decoding process. According to a specific implementation, a method for decoding a zone (Zod) is proposed. J ), called the decoding area, of a signal comprising at least two areas, said decoding area comprising a plurality of samples (Pd n) to be decoded, said decoding process comprising the following steps: the decoding (F22) of a group of latent value maps (FMdJ) representative of said area to be decoded (Zod J ) of said signal, obtaining (F20) a mask (MZo 1 ) of said decoding area in said signal (I(Pn)), the decoding (F26) of a set of parameters (Wdk) representative of a neural network (NNSYN'), said synthetic neural network comprising at least one synthetic neural layer, the processing (F27) of said group of decoded latent maps (FMdJ) by said synthetic neural network (NNSYN') to produce at the output of said synthetic neural network at least said decoding area (Zod J), said processing comprising the application of said at least one neural synthesis layer to transform at least one group of input latent maps (ILMs), referred to as input maps, into at least one group of output latent maps (ILMs), referred to as output maps, said application comprising obtaining a characteristic region (R J ) of said zone to be decoded in said entry maps from said mask (MZo J ), said characteristic region (R J ) including the points associated with said zone to be decoded (Zod J ) and for at least one point (Pt n ) of said area to be decoded, called point to be decoded, the processing of an associated point (Pa n ) audit point to be decoded in at least one of the latent input maps (CCE) by said neural layer as a function of its proximity to a boundary of said characteristic region. In a particular embodiment, the processing step of said associated point (Pan ) includes: obtaining an associated point (Pan) to be decoded in at least one of the latent input maps (CCE), the evaluation of a proximity criterion of said associated point (Pan) with respect to a boundary of said characteristic region, the construction of an input vector (Vcon) of said neural synthesis layer from a neighborhood of said associated point (Pan) and said evaluation of said proximity criterion, and the processing of said input vector by said neural synthesis layer to obtain a point (Psn) of at least one of the output maps. In a particular embodiment, said neighborhood of said associated point (Pa n ) is independent of said evaluation of the proximity criterion. In a particular embodiment, said neighborhood of said associated point (Pa n ) is selected based on said evaluation of the proximity criterion. In a particular embodiment, said neighborhood of said associated point (Pa n ) is selected based on said associated point (Pa n ). In a particular embodiment, during the construction of the input vector (Vco n ), a component of the input vector (Vco n ) is associated with a point of said neighborhood, the value of said component being equal to the value of said associated point if said associated point is a point of said characteristic region (R J ) and a replacement value, otherwise. In a particular embodiment, said replacement value is dependent on the points of said characteristic region (R J ) or points associated with said neighborhood belonging to said characteristic region (R). In a particular embodiment, said replacement value does not depend on the points of said characteristic region (R). In a particular embodiment, the said proximity criterion is a distance. In a particular embodiment, said neural synthesis layer is a convolutional neural layer.
Claims
Demands 1. Method for encoding a signal (I(P n )) segmented into a plurality of zones, said coding process comprising the following steps, for a zone (Zo 1 ) comprising a plurality of samples (P n ) to be coded: obtaining at least one first characteristic map (FM,) representative of said signal (I(P n )), for at least one sample of said area to be coded (Zc / ), called current sample (P n ), associated with a position (x n , y n ) in said signal (I(P n )) to code: • the construction of a characteristic vector (Z n ) from the aforementioned first characteristic cards (FM,), depending on the aforementioned position (x n , y n ) of said current sample (P n ), • the processing of said characteristic vector (Z n) by a neural network (NNSYN) defined by a set of parameters (Wk), to generate a vector (P' n ) representative of a decoded value from the current sample (P n said processing comprising the processing, applied to an input of at least one neuronal layer of said neural network, of a point of said input (Pa n ), associated with a point (Pt n ) of said zone (Zod J ), depending on its proximity to a border associated with the zone (Zod J ), • the updating of at least one value of one of the said first feature maps and / or at least one parameter of said neural network, based on a coding performance measure, obtaining at least a second feature map (FMcJ) representative of at least said zone (Zo 1 ) from segmentation information (MZo J), the coding of at least a second feature map (FMc) and said parameter set (Wk) of said neural network.
2. A coding method according to claim 1, wherein said processing of a point (Pa n ) of said entry includes: obtaining, from said segmentation information (MZo J ), of a characteristic region (R J ) corresponding to said zone (Zo 1 ) in an entry (CCE) of at least one neuronal layer, the so-called characteristic region (R J ) including the points associated with said zone (Zo>), for at least one point (Pt n ) of said zone (Zo 1 ): • obtaining an associated point (Pa n ) in said entry (CCE), • the evaluation of a proximity criterion for said associated point (Pa n ) with respect to a boundary of said characteristic region (R J ), • the construction of an input vector (Vco n ) of said at least one neuronal layer from a neighborhood of said associated point (Pa n ) and of said assessment of said proximity criterion, and • the processing of said input vector (Vco n ) by said at least one neuronal layer to obtain a point (Ps n ) at the output (CCS) of at least one neuronal layer.
3. A method for decoding a data stream representative of a signal (I(Pn)) segmented into a plurality of zones, the method comprising the following steps, for a zone (Zod J ): the decoding (F22) of at least one characteristic map (FMcJ) representative of said zone (Zod J ), the processing (F27) of at least one decoded feature map (FMdJ) by a neural network (NNSYN') comprising the processing, applied to an input (CCE) of at least one neuronal layer of said neural network, of a point of said input (Pan ), associated with a point (Pt n ) of said zone (Zod J ), depending on its proximity to a boundary associated with the zone (Zod J ).
4. A method according to claim 3, wherein the processing step applied to an input, referred to as an input map (IM), of at least one neuronal layer of said neural network comprises: obtaining a characteristic region (R) corresponding to said area (Zod J ) in at least one input map (CCE) from segmentation information (MZo 1 ). said characteristic region (R) comprising the points associated with said zone (Zod J ), and for at least one point (Pt n ) of said zone (Zod J ) : obtaining an associated point (Pa n ) in at least one year an input characteristics map (ICM), the evaluation of a proximity criterion of said associated point (Pa n) with respect to a boundary of said characteristic region (R J ), the construction of an input vector (Vco n ) of said neuronal layer from a neighborhood of said associated point (Pa n ) and said evaluation of said proximity criterion, and the processing of said input vector by said neural layer to obtain a point (Ps n ) of at least one feature map (CCS) output from at least one neural layer.
5. A decoding method according to the preceding claim, wherein said neighborhood of said associated point (Pa n ) is independent of said evaluation of the proximity criterion.
6. A decoding method according to claim 4, wherein said neighborhood of said associated point (Pa n ) is selected based on said evaluation of the proximity criterion.
7. A decoding method according to any one of claims 4 to 6, wherein said neighborhood of said associated point (Pa n ) is selected based on said associated point (Pa n ).
8. A decoding method according to any one of claims 4 to 7, wherein, during the construction of the input vector (Vco n ), a component of the input vector (Vcon) is associated with a point in said neighborhood, the value of said component being equal to the value of said associated point if said associated point is a point in said characteristic region (R J ), and to a replacement value otherwise.
9. Decoding method according to the preceding claim wherein said replacement value is dependent on the points of said characteristic region (R) or of the points associated with said neighborhood belonging to said characteristic region (R).
10. Decoding method according to claim 8 wherein said replacement value does not depend on the points of said characteristic region (R).
11. Decoding method according to any one of claims 3 to 10 wherein said proximity criterion is a distance.
12. Decoding method according to any one of claims 3 to 11 wherein said neural layer is a convolutional neural layer.
13. Encoding device (ENC) of a signal (I(P n )) segmented into a plurality of zones comprising a plurality of samples (P n ) to be coded, the device comprising a processor coupled to a memory in which instructions configured to implement the following steps are stored, for an area (Zo 1 ) comprising a plurality of samples (P n ) to be coded: obtaining at least one first characteristic map (FM,) representative of said signal (I(Pn )), for at least one sample of said area to be coded ( / o'), called current sample (P n ), associated with a position (x n , y n ) in said signal (I(P n )) to code: o the construction of a characteristic vector (Z n ) from the aforementioned first characteristic cards (FM,). depending on said position (x n , y n ) of said current sample (P n ), or the processing of said characteristic vector (Z n ) by a neural network (NNSYN) defined by a set of parameters (Wk), to generate a vector (P' n ) representative of a decoded value from the current sample (P n said processing comprising the processing, applied to an input of at least one neuronal layer of said neural network, of a point of said input (Pa n ), associated with a point (Pt n ) of said zone (Zod J), depending on its proximity to a border associated with the zone (Zod J ), or the updating of at least one value of one of said first feature maps and / or at least one parameter of said neural network, based on a coding performance measure, obtaining at least a second feature map (FMcJ) representative of at least said zone (Zo 1 ) from segmentation information (MZo 1 ). the coding of at least a second feature map (F Mc J) and said parameter set (Wk) of said neural network.
14. Decoding device (DEC) for a data stream representing a signal segmented into zones, the device comprising a processor coupled to a memory in which Instructions are recorded that are configured to implement the following steps for a zone (Zod). J): the decoding (F22) of at least one characteristic map (FMc) representative of said zone (Zod J ), - the processing (F27) of at least one decoded feature map (FMdJ) by a neural network (NNSYN') comprising the processing, applied to an input (CCS) of at least one neuronal layer of said neural network, of a point of said input (Pa n ), associated with a point (Pt n ) of said zone (Zod J ), depending on its proximity to a boundary associated with the zone (Zod J ).
15. Computer program comprising instructions for carrying out the steps of an encoding process according to any one of claims 1 to 2 or of a decoding process according to any one of claims 3 to 13 when said program is executed by a computer.
Citation Information
Patent Citations
Coding concept allowing efficient multi-view / layer coding
EP2984839B1
Method and device for encoding and decoding images.
FR3143245A1
Method and apparatus for decoding with signaling of feature map data
US20230353764A1
AU2016259446A1