Multiple frequency fourier mapping for implicit neural representation based compression

Abstract

A method for encoding comprising: obtaining (41) a Fourier mapping function defined by a plurality of sets of Fourier basis frequencies, each Fourier basis frequency of a set of the plurality of sets being randomly sampled from a Gaussian distribution using a same scale parameter, the scale parameter being different for each set; applying (42) the Fourier mapping function to picture coordinates of samples of a picture unit to obtain mapped picture coordinates; applying (43) a learning phase allowing learning parameters of an implicit neural representation, the learning phase comprising a minimization of a loss function depending on the parameters of the implicit neural representation, the implicit neural representation being applied to the mapped picture coordinates; and, signaling (44, 45) the scale parameters and the parameters of the implicit neural representation in picture data.

Description

[0001] 2023PF01036 MULTIPLE FREQUENCY FOURIER MAPPING FOR IMPLICIT NEURAL REPRESENTATION BASED COMPRESSION 1. CROSS REFERENCE TO RELATED APPLICATIONS This application claims priority to European Application No.23307413.7, filed December 28, 2023, which is incorporated herein by reference in its entirety. 2. TECHNICAL FIELD At least one of the present embodiments generally relates to a method and a device for encoding and decoding picture or video data based on an Implicit Neural Representation. 3. BACKGROUND Implicit Neural Representation (INR) based compression techniques are relatively new compression techniques that can be applied to 2D picture, video, 3D scenes or objects. These techniques have a far lower computational complexity than end-to-end neural network-based compression approaches. An INR network is typically a neural network, composed of multiple neural layers, such as fully connected layers. Each neural layer can be described as a function that first multiplies an input signal by a tensor, adds a vector called the bias and then applies a nonlinear function on the resulting values. The shape (and other characteristics) of the tensor and the type of non-linear functions are called the architecture of the network. The input signal may be modified by a transformation before being used as input for the neural network. This transformation can be a Fourier mapping, coordinate transformation, normalization etc. Document Tancik, M. S.-K. (2020). Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems, (pp. 7537-7547) shown that a Fourier mapping enables a Multi- layer Perceptron (MLP) learn high-frequency components of an input signal. Otherwise, the MLP has a spectral bias and is unable to learn the high frequencies of the input signal, which degrades considerably a quality when reconstructing the input 2023PF01036 signal from an encoded signal. It has been shown that random selection of Fourier mapping is optimal. When using a Fourier mapping, a reconstruction quality of a signal or an ability to learn all frequency components of the signal is entirely dependent on a number of Fourier basis frequencies sampled from a Gaussian distribution but also on a scale parameter of the Gaussian distribution. Generally, only a single scale parameter of the Gaussian distribution is used to sample the Fourier basis frequencies. Therefore, using a single scale parameter might not be optimal to learn all frequency components of the signal. It is desirable to propose solutions allowing to overcome the above issues. In particular, it is desirable to propose solutions improving the learning of frequency components of a signal. 4. BRIEF SUMMARY In a first aspect, one or more of the present embodiments provide a method for encoding comprising: obtaining a Fourier mapping function defined by a plurality of sets of Fourier basis frequencies, each Fourier basis frequency of a set of the plurality of sets being randomly sampled from a Gaussian distribution using a same scale parameter, the scale parameter being different for each set; applying the Fourier mapping function to picture coordinates of samples of a picture unit to obtain mapped picture coordinates; applying a learning phase allowing learning parameters of an implicit neural representation, the learning phase comprising a minimization of a loss function depending on the parameters of the implicit neural representation, the implicit neural representation being applied to the mapped picture coordinates; and, signaling the scale parameters and the parameters of the implicit neural representation parameters in picture data. In an embodiment the method comprises signaling in the picture data each seed used to randomly sample a Fourier basis frequency from the Gaussian distribution. In an embodiment, a same seed is used for all Fourier basis frequencies. 2023PF01036 In an embodiment, each seed is selected randomly in a plurality of seeds based on a criterion of reconstruction quality of the samples of the picture unit. In an embodiment, at least two sets of the plurality of sets comprise a different number of Fourier basis frequencies and the method comprises signaling the number of Fourier basis frequencies of each set in the picture data. In a second aspect, one or more of the present embodiments provide a method for decoding comprising: decoding scale parameters and implicit neural representation parameters from picture data; applying a Fourier mapping function to coordinates of picture samples of a picture unit to obtain mapped coordinates, the Fourier mapping function being defined by a plurality of sets of Fourier basis frequencies, each Fourier basis frequencies of a set of the plurality of sets being randomly sampled from a Gaussian distribution using one of the scale parameters, the scale parameter being different for each set; obtaining an implicit neural representation from the implicit neural representation parameters; and, applying the implicit neural representation to the mapped coordinates to obtain a reconstructed picture unit. In an embodiment, the method comprises decoding from the picture data each seed used to randomly sample a Fourier basis frequency from the Gaussian distribution. In an embodiment, a same seed is used for all Fourier basis frequencies. In an embodiment, each seed is selected randomly in a plurality of seeds based on a criterion of reconstruction quality of the samples of the picture unit. In an embodiment, at least two sets of the plurality of sets comprise a different number of Fourier basis frequencies and the method comprises decoding the number of Fourier basis frequencies of each set. In a third aspect, one or more of the present embodiments provide a device for encoding comprising electronic circuitry configured for: 2023PF01036 obtaining a Fourier mapping function defined by a plurality of sets of Fourier basis frequencies, each Fourier basis frequency of a set of the plurality of sets being randomly sampled from a Gaussian distribution using a same scale parameter, the scale parameter being different for each set; applying the Fourier mapping function to picture coordinates of samples of a picture unit to obtain mapped picture coordinates; applying a learning phase allowing learning parameters of an implicit neural representation, the learning phase comprising a minimization of a loss function depending on the parameters of the implicit neural representation, the implicit neural representation being applied to the mapped picture coordinates; and, signaling the scale parameters and the parameters of the implicit neural representation in picture data. In an embodiment, the electronic circuitry is further configured for signaling in the picture data each seed used to randomly sample a Fourier basis frequency from the Gaussian distribution. In an embodiment, a same seed is used for all Fourier basis frequencies. In an embodiment, each seed is selected randomly in a plurality of seeds based on a criterion of reconstruction quality of the samples of the picture unit. In an embodiment, at least two sets of the plurality of sets comprise a different number of Fourier basis frequencies and the electronic circuitry is further configured for signaling the number of Fourier basis frequencies of each set in the picture data. In a fourth aspect, one or more of the present embodiments provide a device for decoding comprising electronic circuitry configured for: Decoding scale parameters and implicit neural representation parameters from picture data; applying a Fourier mapping function to coordinates of picture samples of a picture unit to obtain mapped coordinates, the Fourier mapping function being defined by a plurality of sets of Fourier basis frequencies, each Fourier basis frequencies of a set of the plurality of sets being randomly sampled from a Gaussian distribution using one of the scale parameters, the scale parameter being different for each set; 2023PF01036 obtaining an implicit neural representation from the implicit neural representation parameters; and, applying the implicit neural representation to the mapped coordinates to obtain a reconstructed picture unit. In an embodiment, the electronic circuitry is further configured for decoding from the picture data each seed used to randomly sample a Fourier basis frequency from the Gaussian distribution. In an embodiment, a same seed is used for all Fourier basis frequencies. In an embodiment, each seed is selected randomly in a plurality of seeds based on a criterion of reconstruction quality of the samples of the picture unit. In an embodiment, at least two sets of the plurality of sets comprise a different number of Fourier basis frequencies and the electronic circuitry is further configured for decoding the number of Fourier basis frequencies of each set. In a fifth aspect, one or more of the present embodiments provide an output signal generated by the method of the first aspect or by the device of the second aspect. In a sixth aspect, one or more of the present embodiments provide a Non- transitory information storage medium storing program code instructions for implementing the methods according to the first or the second aspect. In a sixth aspect, one or more of the present embodiments provide a computer program comprising program code instructions for implementing the methods according to the first or the second aspect. 5. BRIEF SUMMARY OF THE DRAWINGS Fig. 1 illustrates an example of context in which various embodiments may be implemented; 2023PF01036 Fig. 2A illustrates schematically an example of hardware architecture of a processing module able to implement an encoding module or a decoding module in which various aspects and embodiments are implemented; Fig. 2B illustrates a block diagram of an example of a first system in which various aspects and embodiments are implemented; Fig.2C illustrates a block diagram of an example of a second system in which various aspects and embodiments are implemented; Fig.3 illustrates a simple neural network used for implicit neural representation; Fig.4A illustrates a process to encode a signal using an implicit neural representation; Fig.4B illustrates a process to decode a signal using an implicit neural representation; Fig.5A illustrates schematically a method for encoding INR parameters; Fig.5B illustrates schematically a method for decoding INR parameters; Fig. 6A illustrates a process to encode a signal using an INR comprising a Fourier mapping based on Fourier basis frequencies randomly sampled from a Gaussian distribution using multiple scale parameters; Fig. 6B illustrates a process to decode a signal using an INR comprising a Fourier mapping based on Fourier basis frequencies randomly sampled from a Gaussian distribution using multiple scale parameters; and, Fig.7 illustrates an example of batch gradient descent method. 6. DETAILED DESCRIPTION In the following, various embodiments are applied to a 2D signal such as picture or video data. One can note that these various embodiments can also be applied identically to other types of signals such as 3D signals representing 3D scenes or objects. Fig.1 describes an example of a context in which following embodiments can be implemented. In Fig. 1, a system 11, that could be a camera, a storage device, a computer, a server or any device capable of delivering a video stream (i.e. video data), transmits a video stream to a system 13 using a communication channel 12. The video stream is either encoded and transmitted by the system 11 or received and / or stored by the system 11 and then transmitted. The communication channel 12 is a wired (for example 2023PF01036 Internet or Ethernet) or a wireless (for example WiFi, 3G, 4G or 5G) network link. The system 13, that could be for example a set top box, receives and decodes the video stream to generate a sequence of decoded pictures. The obtained sequence of decoded pictures is then transmitted to a display system 15 using a communication channel 14, that could be a wired or wireless network. The display system 15 then displays said pictures. In an embodiment, the system 13 is comprised in the display system 15. In that case, the system 13 and display system 15 are comprised in a TV, a computer, a tablet, a smartphone, a head-mounted display, a vehicle entertainment system, a medical device, etc. Fig. 2A illustrates schematically an example of hardware architecture of a processing module 200 able to implement an encoding module or a decoding module capable of implementing respectively a method for encoding of Fig.6A and a method for decoding of Fig.6B. The encoding module is for example comprised in the system 11 when this apparatus is in charge of encoding the video stream. The decoding module is for example comprised in the system 13. The processing module 200 comprises, connected by a communication bus 2005: a processor or CPU (central processing unit) 2000 encompassing one or more microprocessors, general purpose computers, special purpose computers, and processors based on a multi-core architecture, as non-limiting examples; a random access memory (RAM) 2001; a read only memory (ROM) 2002; a storage unit 2003, which can include non-volatile memory and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), flash, magnetic disk drive, and / or optical disk drive, or a storage medium reader, such as a SD (secure digital) card reader and / or a hard disc drive (HDD) and / or a network accessible storage device; at least one communication interface 2004 for exchanging data with other modules, devices or equipment. The communication interface 2004 can include, but is not limited to, a transceiver configured to transmit and to receive data over a communication channel. The communication interface 2004 can include, but is not limited to, a modem or network card. If the processing module 200 implements a decoding module, the 2023PF01036 communication interface 2004 enables for instance the processing module 200 to receive encoded video streams and to provide a sequence of decoded pictures. If the processing module 200 implements an encoding module, the communication interface 2004 enables for instance the processing module 200 to receive a sequence of original picture data to encode and to provide an encoded video stream. The processor 2000 is capable of executing instructions loaded into the RAM 2001 from the ROM 2002, from an external memory (not shown), from a storage medium, or from a communication network. When the processing module 200 is powered up, the processor 2000 is capable of reading instructions from the RAM 2001 and executing them. These instructions form a computer program causing, for example, the implementation by the processor 2000 of a decoding method as described in relation with Fig. 6B, an encoding method described in relation to Fig. 6A, these methods comprising various aspects and embodiments described below in this document. All or some of the algorithms and steps of the methods of Figs.6A and 6B may be implemented in software form by the execution of a set of instructions by a programmable machine such as a DSP (digital signal processor) or a microcontroller, or be implemented in hardware form by a machine or a dedicated component such as a FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit). As can be seen, microprocessors, general purpose computers, special purpose computers, processors based or not on a multi-core architecture, DSP, microcontroller, FPGA and ASIC are electronic circuitry adapted to implement (i.e., configured for implementing) at least partially the methods of Figs.6A and 6B. Fig. 2C illustrates a block diagram of an example of the system 13 in which various aspects and embodiments are implemented. The system 13 can be embodied as a device including the various components described below and is configured to perform one or more of the aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set top boxes, digital television receivers, personal video recording systems, connected home appliances, vehicle entertainment systems, a medical devices and head mounted display. Elements of system 13, singly or in combination, can be embodied in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, 2023PF01036 in at least one embodiment, the system 13 comprises one processing module 200 that implements a decoding module. In various embodiments, the system 13 is communicatively coupled to one or more other systems, or other electronic devices, via, for example, a communications bus or through dedicated input and / or output ports. In various embodiments, the system 13 is configured to implement one or more of the aspects described in this document. The input to the processing module 200 can be provided through various input modules as indicated in block 231. Such input modules include, but are not limited to, (i) a radio frequency (RF) module that receives an RF signal transmitted, for example, over the air by a broadcaster, (ii) a component (COMP) input module (or a set of COMP input modules), (iii) a Universal Serial Bus (USB) input module, and / or (iv) a High Definition Multimedia Interface (HDMI) input module. Other examples, not shown in Fig.2C, include composite video. In various embodiments, the input modules of block 231 have associated respective input processing elements as known in the art. For example, the RF module can be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) band-limiting again to a narrower band of frequencies to select (for example) a signal frequency band which can be referred to as a channel in certain embodiments, (iv) demodulating the down-converted and band- limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired stream of data packets. The RF module of various embodiments includes one or more elements to perform these functions, for example, frequency selectors, signal selectors, band-limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF portion can include a tuner that performs various of these functions, including, for example, down-converting the received signal to a lower frequency (for example, an intermediate frequency or a near-baseband frequency) or to baseband. In one set-top box embodiment, the RF module and its associated input processing element receives an RF signal transmitted over a wired (for example, cable) medium, and performs frequency selection by filtering, down- converting, and filtering again to a desired frequency band. Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements performing similar or different functions. Adding 2023PF01036 elements can include inserting elements in between existing elements, such as, for example, inserting amplifiers and an analog-to-digital converter. In various embodiments, the RF module includes an antenna. Additionally, the USB and / or HDMI modules can include respective interface processors for connecting system 13 to other electronic devices across USB and / or HDMI connections. It is to be understood that various aspects of input processing, for example, Reed-Solomon error correction, can be implemented, for example, within a separate input processing IC or within the processing module 200 as necessary. Similarly, aspects of USB or HDMI interface processing can be implemented within separate interface ICs or within the processing module 200 as necessary. The demodulated, error corrected, and demultiplexed stream is provided to the processing module 200. Various elements of system 13 can be provided within an integrated housing. Within the integrated housing, the various elements can be interconnected and transmit data therebetween using suitable connection arrangements, for example, an internal bus as known in the art, including the Inter-IC (I2C) bus, wiring, and printed circuit boards. For example, in the system 13, the processing module 200 is interconnected to other elements of said system 13 by the bus 2005. The communication interface 2004 of the processing module 200 allows the system 13 to communicate on the communication channel 12. As already mentioned above, the communication channel 12 can be implemented, for example, within a wired and / or a wireless medium. Data is streamed, or otherwise provided, to the system 13, in various embodiments, using a wireless network such as a Wi-Fi network, for example IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi- Fi signal of these embodiments is received over the communications channel 12 and the communications interface 2004 which are adapted for Wi-Fi communications. The communications channel 12 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet for allowing streaming applications and other over-the-top communications. Other embodiments provide streamed data to the system 13 using the RF connection of the input block 231. As indicated above, various embodiments provide data in a non- streaming manner. Additionally, various embodiments use wireless networks other than 2023PF01036 Wi-Fi, for example a cellular network or a Bluetooth network. The system 13 can provide an output signal to various output devices, including the display system 15, speakers 26, and other peripheral devices 27. The display system 15 of various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display system 15 can be for a television, a tablet, a laptop, a cell phone (mobile phone), a head mounted display or other devices. The display system 15 can also be integrated with other components (for example, as in a smart phone), or separate (for example, an external monitor for a laptop). The other peripheral devices 27 include, in various examples of embodiments, one or more of a stand-alone digital video disc (or digital versatile disc) (DVR, for both terms), a disk player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 27 that provide a function based on the output of the system 13. For example, a disk player performs the function of playing an output of the system 13. In various embodiments, control signals are communicated between the system 13 and the display system 15, speakers 26, or other peripheral devices 27 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communications protocols that enable device-to-device control with or without user intervention. The output devices can be communicatively coupled to system 13 via dedicated connections through respective interfaces 232, 233, and 234. Alternatively, the output devices can be connected to system 13 using the communications channel 12 via the communications interface 2004 or a dedicated communication channel corresponding to the communication channel 14 in Fig. 1 via the communication interface 2004. The display system 15 and speakers 26 can be integrated in a single unit with the other components of system 13 in an electronic device such as, for example, a television. In various embodiments, the display interface 232 includes a display driver, such as, for example, a timing controller (T Con) chip. The display system 15 and speaker 26 can alternatively be separate from one or more of the other components. In various embodiments in which the display system 15 and speakers 26 are external components, the output signal can be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs. Fig. 2B illustrates a block diagram of an example of the system 11 in which 2023PF01036 various aspects and embodiments are implemented. System 11 is very similar to system 13. The system 11 can be embodied as a device including the various components described below and is configured to perform one or more of the aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, a camera and a server. Elements of system 11, singly or in combination, can be embodied in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the system 11 comprises one processing module 200 that implements an encoding module. In various embodiments, the system 11 is communicatively coupled to one or more other systems, or other electronic devices, via, for example, a communications bus or through dedicated input and / or output ports. In various embodiments, the system 11 is configured to implement one or more of the aspects described in this document. The input to the processing module 200 can be provided through various input modules as indicated in block 231 already described in relation to Fig.2C. Various elements of system 11 can be provided within an integrated housing. Within the integrated housing, the various elements can be interconnected and transmit data therebetween using suitable connection arrangements, for example, an internal bus as known in the art, including the Inter-IC (I2C) bus, wiring, and printed circuit boards. For example, in the system 11, the processing module 200 is interconnected to other elements of said system 11 by the bus 2005. The communication interface 2004 of the processing module 200 allows the system 11 to communicate on the communication channel 12. Data is streamed, or otherwise provided, to the system 11, in various embodiments, using a wireless network such as a Wi-Fi network, for example IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi- Fi signal of these embodiments is received over the communications channel 12 and the communications interface 2004 which are adapted for Wi-Fi communications. The communications channel 12 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet for allowing streaming applications and other over-the-top communications. Other embodiments provide streamed data to the system 11 using the RF connection of the input block 231. 2023PF01036 As indicated above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, for example a cellular network or a Bluetooth network. The data provided to the system 11 can be provided in different format. In various embodiments, these data are raw data provided for example by a picture acquisition module connected to the system 11 or comprised in the system 11. In that case, the processing module 200 take in charge the encoding of these data. The system 11 can provide an output signal to various output devices capable of storing and / or decoding the output signal such as the system 13. Various implementations involve decoding. “Decoding”, as used in this application, can encompass all or part of the processes performed, for example, on a received encoded video stream (i.e., received video data) in order to produce a final output suitable for display. In various embodiments, such processes include processes performed by a decoder of various implementations described in this application in relation to Fig.6B. Various implementations involve encoding. In an analogous way to the above discussion about “decoding”, “encoding” as used in this application can encompass all or part of the processes performed, for example, on an input video sequence in order to produce an encoded video stream. In various embodiments, such processes include processes performed by an encoder of various implementations described in this application in relation to Fig.6A. When a figure is presented as a flow diagram, it should be understood that it also provides a block diagram of a corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow diagram of a corresponding method / process. The implementations and aspects described herein can be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed can also be implemented in other forms (for example, an apparatus or program). An apparatus can be implemented in, for example, appropriate hardware, software, and firmware. The methods can be implemented, for example, in a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, 2023PF01036 an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, smartphones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end-users. Reference to “one embodiment” or “an embodiment” or “one implementation” or “an implementation”, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment” or “in one implementation” or “in an implementation”, as well any other variations, appearing in various places throughout this application are not necessarily all referring to the same embodiment. Additionally, this application may refer to “determining” various pieces of information. Determining the information can include one or more of, for example, estimating the information, calculating the information, predicting the information, retrieving the information from memory or obtaining the information for example from another device, module or from user. Further, this application may refer to “accessing” various pieces of information. Accessing the information can include one or more of, for example, receiving the information, retrieving the information (for example, from memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information. Additionally, this application may refer to “receiving” various pieces of information. Receiving is, as with “accessing”, intended to be a broad term. Receiving the information can include one or more of, for example, accessing the information, or retrieving the information (for example, from memory). Further, “receiving” is typically involved, in one way or another, during operations such as, for example, storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information. It is to be appreciated that the use of any of the following “ / ”, “and / or”, and “at least one of”, “one or more of” for example, in the cases of “A and / or B” and 2023PF01036 “at least one of A and B”, “one or more of A and B” is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, “one or more of A, B and C” such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended, as is clear to one of ordinary skill in this and related arts, for as many items as are listed. Also, as used herein, the word “signal” refers to, among other things, indicating something to a corresponding decoder. For example, in certain embodiments the encoder signals a use of some INR parameters and parameters of a Fourier mapping. In this way, in an embodiment the same parameters can be used at both the encoder side and the decoder side. Thus, for example, an encoder can transmit (explicit signaling) a particular parameter to the decoder so that the decoder can use the same particular parameter. Conversely, if the decoder already has the particular parameter as well as others, then signaling can be used without transmitting (implicit signaling) to simply allow the decoder to know and select the particular parameter. By avoiding transmission of any actual functions, a bit savings is realized in various embodiments. It is to be appreciated that signaling can be accomplished in a variety of ways. For example, one or more syntax elements, flags, and so forth are used to signal information to a corresponding decoder in various embodiments. While the preceding relates to the verb form of the word “signal”, the word “signal” can also be used herein as a noun. As will be evident to one of ordinary skill in the art, implementations can produce a variety of signals formatted to carry information that can be, for example, stored or transmitted. The information can include, for example, instructions for performing a method, or data produced by one of the described implementations. For example, a signal can be formatted to carry the encoded video stream (i.e. encoded data). Such a signal can be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting can include, for example, encoding an encoded video stream and modulating 2023PF01036 a carrier with the encoded video stream. The information that the signal carries can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is known. The signal can be stored on a processor-readable medium. Fig.3 illustrates a simple neural network used for implicit neural representation (INR). Such a neural network used for INR can be referred to as an INR network. For clarity, we use for illustration a 2D signal such as a picture, but, as already mentioned above, an INR can be used for signals of any dimension. An INR parameterizes a signal as a function (300) which takes coordinates (310) as input and outputs potentially approximated signal values (360) at these coordinates. When the signal processed by the INR is a picture, the input coordinates (310) can be sample coordinates (x,y) of picture samples and the INR outputs (360) are the picture sample values. Picture samples values can be original sample values of an original picture or residual values representative of a difference between predictor samples and the original samples. A picture sample can be a single component signal (such as a grey scale picture) or a multi-component signal comprising a plurality of components such as for example a RGB, YUV or YUV+d picture where d represents a depth component. In the video case, the output is similar, but the input can include a picture index t in addition to the sample coordinates. The INR can be used to reconstruct a signal by computing picture sample values for some or each sample coordinates (x,y). An INR network is typically a neural network composed of multiple neural layers, such as fully connected layers. In Fig.3, the network has four neural layers 320, 330, 340 and 350. Intermediate outputs are represented by circles. Each neural layer can be described as a function that first multiplies the input (for example S1, S2, S3 and S4) by a tensor, adds a vector called the bias and then applies a nonlinear function on the resulting values. In the present document, we may also refer to a neural layer simply as a layer. Tensors shapes (and other characteristics of the tensors) and non-linear functions types of the neural network defines an architecture of the neural network. In the following, tensor values and bias values are denoted by the term weights. The weights and, if applicable, the parameters of the non-linear functions, are called parameters ^^ of the neural network. The architecture and the parameters ^^ define a model. In the following we use ^^ఏto denote an INR function parameterized by ^^. In some variants of the process of Fig. 3, the input coordinates (x,y) may be 2023PF01036 modified in an optional preprocessing step by a transformation before being used as input in step 41. This transformation can be a Fourier mapping, a coordinate transformation, a normalization etc. Document Tancik, M. S.-K. (2020). Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems, (pp. 7537-7547) showed that a mapping into Fourier features (i.e., a Fourier mapping) enables a Multi-layer Perceptron (MLP) learn high-frequency components of an input signal. Otherwise, the MLP has a spectral bias and is unable to learn the high frequencies of the input signal, which degrades considerably a visual quality when reconstructing the encoded signal. Technically, the Fourier mapping of an input coordinate ^^ ൌ ^^^, ^^^ is definedby a Fourier mapping function as follows: ^^^^^^ ൌ^^^ ^^^^^^^2^^^^ ^^^,^^ ^^^^^^^2^^^^ ^^^,   … ,    ^^ ^^^^^^^2^^^^ ^ ^ ^^்^ ^ ^ ^ ெ ெ^^ ,^^ெ ^^^^^^ 2^^^^ெ^^ (Eq.1) Hence, the Fourier mapping depends on the coefficients a୧and ^^^where the coefficients ^^^represent Fourier basis frequencies when the mapping is seen as a Fourier approximation of a kernel function. Each Fourier basis frequency ^^^is randomly sampled from a Gaussian distribution with scale parameter ^^, and typically the coefficients ^^^=1, that is ^^^ ∼ ^^^0,^^^, ^^ ൌ 1, … , ^^The scale parameter (or bandwidth parameter) ^^  is a hyperparameter, which should be adapted with respect to the frequency content of the underlying signal. For instance, a choice of a scaling parameter σ with a small value leads to smooth reconstruction, while a scaling parameter σ with a large value leads to overfitting and the reconstructions are noisy. Furthermore, Fourier basis frequencies sampled from a Gaussian distribution using a single scale parameter σ might not be sufficient to represent all the frequency content of the underlying signal. Furthermore, using one INR network globally for a whole signal makes learning difficult, as all parameters contribute to all values and lead to a large network as it must encode all details of the signal. A solution to address this issue is to divide the signal in 2023PF01036 partitions and to define a local INR network for each partition. When the signal is a picture, the partition of a picture could be a slice, a tile, a coding unit, etc. Therefore, a signal processed by an INR network may be a picture unit, a picture unit being either a full picture or any partition of a picture. Fig.4A illustrates a typical process to encode a signal using an INR. The process of Fig. 4A is executed for example by the processing module 200 of the system 11. In a step 40, the processing module 200 obtains an input signal represented by a set of samples. The input signal is for example a picture unit. In a step 42, the processing module 200 applies a Fourier mapping to the spatial coordinates of samples of the picture unit to obtain mapped coordinates. For instance, in step 42, the processing module 200 applies the Fourier mapping function of Equation Eq.1. In a step 43, the processing module 200 applies a learning phase during which the INR parameters ^^ (or a subset of them) of the INR network allowing reconstructing values representative of the input signal from the mapped coordinates are learned. The learning of the INR parameters (or weights) ^^ is typically performed by an iterative optimization process such as a batch gradient descent method based on a loss function. Fig.7 illustrates an example of batch gradient descent method. In this example, the batch gradient descent method is executed during step 43. In a step 4301, the processing module 200 initialize a variable j to zero. The variable j is used to count a number of iterations in the batch gradient descent method. In a step 4302, the processing module 200 computes a distortion ^^ெௌாthe picture unit PU: ^^ ^^^^ ൌ ^^^^^^,^ ^ ^ଶெௌா ^ ^ െ ^^ఏ^^^, ^^^where I is an output of an INR function ^^ఏwith parameters ^^. Here the distortion is a mean square error. However, other metrics such as LPIPS (Learned Perceptual Image Patch Similarity) can also be used in this case. In a step 4303, the processing module 200 computes a value loss of the loss function: 2023PF01036 loss^= ^^ெௌா^^^^ ^ ^^ ^^(^^^where β is a trade-off parameter representing a trade-off between the distortion ^^ெௌா^^^^ and a bitrate ^^(^^^, ^^(^^^ being a bitrate value representing the bitrate for encoding the INR parameters ^^: In a step 4304, the processing module updates the INR parameters ^^ using for example a stochastic gradient descent method. In a step 4305, the processing module 200 determines if a end condition for stopping the batch gradient descent method is fulfilled. For instance, a end condition is fulfilled when the number of iteration j is equal to a value J_MAX, or if a difference between the values loss obtained in two successive iterations is below a value DIFF_LOSS. If the end condition is not fulfilled, step 4305 is followed by a step 4306. In step 4306, the processing module 200 increases the variable j of one unit. Otherwise, if the end condition is fulfilled, the processing module 200 stops the process of Fig. 7 in a step 4307. Back to Fig. 4A, in a step 45, the processing module 200 signals the INR parameters ^^ (or a subset of them) in an output bitstream (i.e., in output data or in a data set). When the signal is a picture unit, the processing module 200 also adds information representative of the picture unit such as the width and the height of the picture unit. Fig.5A illustrates schematically a method for encoding the INR parameters ^^. The method of Fig.5A represents a detail of step 45. In a step 450, the processing module 200 quantize the INR parameters ^^. In a step 451, the quantized INR parameters are entropy coded. Fig.4B illustrates a typical process to decode data using INR. The process of Fig. 4B is executed for example by the processing module 200 of the system 13. In a step 47, the processing module 200 obtains input data, for instance, corresponding to the output data generated by the processing module 200 of the system 11 when applying the method of Fig. 4A. The input data comprises encoded INR parameters ^^. During step 47, the processing module 200 decodes the INR parameters ^^ from the input data and regenerates the INR networks applying the INR functions ^^ఏ 2023PF01036 for each partition. When the signal is a picture, the processing module 200 also decodes the information representative of the picture. Fig.5B illustrates schematically a method for decoding the INR parameters ^^. The method of Fig.5B represents a detail of step 47. In a step 470, the processing module 200 entropy decodes the INR parameters ^^. In a step 471, the processing module 200 inverse quantize the entropy decoded INR parameters to obtain reconstructed INR parameters ^^^. Due to the quantization, thereconstructed INR parameters ^^^ are an approximation of the INR parameters ^^determined in step 43. Back to Fig. 4B, in a step 49, the processing module 200 applies the Fourier mapping to the spatial coordinates of samples of the picture unit to obtain mapped coordinates. It is supposed here that the parameters a୧and ^^^describing the Fourier mapping are known by the system 11 and the system 13. In a step 50, the processing module 200 applies the regenerated INR network (i.e., the processing module 200 applies the INR function ^^ఏ^ .) to the mapped coordinates to generate a reconstructed version of the input signal obtained by the system 11 in step 40. If the input signal is a picture unit, the processing module 200 applies the regenerated INR network to at least a sub-part of the mapped coordinates of the picture unit. As an example, for a 256x256 samples picture unit, these coordinates could be all pairs (x,y) for all x∈{0,1,…,255} and y∈{0,1,…,255}. Other choices are possible, for example to generate an up-sampled, down-sampled or extended version of the input picture unit. The Fourier basis frequencies used in the Fourier mapping function of eq. 1 applied in step 42 and 49, are sampled from a Gaussian distribution using a single scale parameter σ. As already mentioned before, such Fourier basis frequencies might not be sufficient to represent all the frequency content of the underlying signal. In the following, we propose to randomly sample Fourier basis frequencies using multiple scale parameters. Such Fourier basis frequencies allow obtaining a better representation of the underlying signal. Fig. 6A illustrates a process to encode a signal using an INR comprising a Fourier mapping based on Fourier basis frequencies randomly sampled from a Gaussian 2023PF01036 distribution using multiple scale parameters. The process of Fig. 6A is executed for example by the processing module 200 of the system 11. The process of Fig. 6A comprises steps 40, 42, 43 and 45 identical to steps 40, 42, 43 and 45 described in relation to Fig.4A. In a step 41, the processing module obtains the following mapping function: γ^^^^ൌ ^^^ cos^2π^^^^^^,^^ sin^2^ ^ ^^^ ^^ ^^்^ ^ ^π^^^^^ ,   … ,    ^^ெ cos 2π^^^^^ ,^^ெsin 2π^^^ ^^ ^^^^^.2^with ^^^^ ∼ ^^^0,^^^^, ^^ ൌ 1, …^^, ^^ ൌ 1, … ,^^ Where P=M / K, M is a total number of Fourier basis frequencies ^^^^, K is a number of different scale parameter ^^^used to sample the different Fourier basis frequencies ^^^^from a Gaussian distribution, P is a number of Fourier basis frequencies ^^^^in a set of Fourier basis frequencies sampled using a same scale parameters ^^^. Therefore, in step 41, the processing module 200 obtains a Fourier mapping function defined by a plurality of sets of Fourier basis frequencies, each Fourier basis frequency of a set of the plurality of sets being randomly sampled from a Gaussian distribution using a same scale parameter, the scale parameter being different for each set. Step 41 is followed by steps 43 and 44 already described in relation to Fig.4A. In a step 44, the processing module signals information used to re-generate the Fourier mapping function on the decoder side (i.e., on the system 13 side). The Fourier mapping function is for example based on equation Eq.2. In the context of the Fourier mapping based on Fourier basis frequencies randomly sampled from a Gaussian distribution using multiple scale parameters, the different scale parameters ^^^used to sample the different Fourier basis frequencies ^^^^from a Gaussian distribution are signaled in the bitstream. In each set of Fourier basis frequencies sampled using a same scale parameter ^^^, a same single seed can be used to randomly sample the Fourier basis frequencies. In an embodiment, the seed is signaled in the bitstream in addition to the scale parameters ^^^. In addition, to further improve the encoding performances, the optimization of 2023PF01036 INR can be performed with (stochastically) generating random Fourier mapping by using various random seeds, and to select the random seed which produced the best reconstruction quality. In that case, the loss function used in step 43 not only depends on the INR parameters ^^ but also on a parameter ξ representing a seed (i.e. loss^= ^^ெௌா^^^, ^^^ ^ ^^ ^^(^^, ^^^).a universal seed known by both the encoder and the decoder is used. In that case, the seed is not signaled in the bitstream. Until now, each set of Fourier basis frequencies sampled using a same scale parameters ^^^comprises a same number P of Fourier basis frequencies ^^^^. In an embodiment, the sets of Fourier basis frequencies comprise different numbers of Fourier basis frequencies ^^^^. In that case, an information representing the number of Fourier basis frequencies ^^^^in each set is encoded in the bitstream. The number of Fourier basis frequencies in each set could depend on the frequency content of the picture unit. Step 44 is followed by step 45 described in relation to Fig.4A. Fig. 6B illustrates a process to encode a signal using an INR comprising a Fourier mapping based on Fourier basis frequencies randomly sampled from a Gaussian distribution using multiple scale parameters. The process of Fig. 6B is executed for example by the processing module 200 of the system 13 on the output data generated by the processing module 200 of the system 11 when applying the method of Fig.6A. The process of Fig. 6B comprises steps 47, 49 and 50 identical to steps 47, 49 and 50 described in relation to Fig.4B. In a step 46, the processing module 200 decodes information used to re-generate the Fourier mapping function on the decoder side. In the context of the Fourier mapping based on Fourier basis frequencies randomly sampled from a Gaussian distribution using multiple scale parameters, the processing module decodes the different scale parameters ^^^. In an embodiment, if a seed was signaled in the bitstream, the processing module 200 decodes the seed from the bitstream. In an embodiment, if a universal seed known by the encoder and the decoder was used, the processing module 200 sets the value of the seed to the value of the universal seed. 2023PF01036 In an embodiment, if an information representing the number of Fourier basis frequencies ^^^^in each set was encoded in the bitstream, the processing module decodes this information from the bitstream. Step 46 is followed by step 47 already described in relation to Fig.4B. In a step 48, the processing module 200 obtains the Fourier basis frequencies ^^^^by randomly sampling a Gaussian distribution using the decoded scale parameters ^^^, the seed and the number of Fourier basis frequencies ^^^^in each set. The obtained Fourier basis frequencies ^^^^allow re-generating the Fourier mapping function (for example based on equation Eq.2). Step 48 is followed by steps 49 and 50 already described in relation to Fig.4B. In order to evaluate the coding efficiency of the encoding process of Fig. 6A, experiments were conducted to compare the encoding process of Fig. 4A with the encoding process of Fig.6A. The INR uses a MLP with a ReLU activation function. We conducted two sets of experiments: In a first experiment, the INR uses two layers (instead of “4” in Fig.3), the size of each layer being “16” (instead of four for layers 1, 2, 3 and 4 in Fig.3) and the size of the Fourier mapping is M=16. In a second experiment, the INR uses two layers, the size of each layer being “64” ) and the size of the Fourier mapping is M=32. The scale parameter σ is set to 1 in the process of Fig. 4A because it gave the best results. The scale parameters ^^^are set to [0.5,1,2,5] in the process of Fig. 6A, with K=4 for the first experiment and K=8 for the second experiment. When M=16 and K= 4, then first four mapping dimensions correspond to ^^^ൌ0.5, the four second mapping dimensions corresponds to ^^ଶ=1, the third four mapping dimensions correspond to ^^ଷൌ2, and so on. Results are shown in table TAB1 for the two different bitrates, and it shows that the process of Fig.6A brings about 1.7 dB improvement in RGB PSNR over the process of Fig. 4A for the first experiment. For the second experiment, the process of Fig. 6A brings about 1.13 dB in RGB PSNR over the process of Fig.4A. These improvements are significant in the compression literature. Further, it is noted that the process of Fig. 6A (and the process of Fig.6B) bring(s) this improvement with a negligible increase in 2023PF01036 terms of the bitstream size, and also without increasing the encoder and decoder complexity.

[0002] 2023PF01036 Methods Rate (bpp) PSNR (dB) Mapping size M = Fourier Mapping with 0.02 19.45 16 single scale No of layers =2parameter σ ൌ 1Layer size = 16 Multiple frequency 0.02016 21.21 Fourier mapping (4 scale parameters ) ^^^ ൌ ^0.5, 1, 2, 5^Mapping size M = Fourier Mapping with 0.26 25.52 32 single scale No of layers =2parameter ^^ ൌ 1Layer size = 64 Multiple frequency 0.26016 26.65 Fourier mapping (4 scale parameters) ^^^ ൌ ^0.5, 1, 2, 5^Table TAB1 We described above a number of embodiments. Features of these embodiments can be provided alone or in any combination. Further, embodiments can include one or more of the following features, devices, or aspects, alone or in any combination, across various claim categories and types: ^ A bitstream or signal or video data or picture data that includes information representative of one or more of the described INR parameters and information representing a mapping function, or variations thereof. ^ Creating and / or transmitting and / or receiving and / or decoding a bitstream or signal that includes information representative of one or more of the described INR parameters and information representing a mapping function, or variations thereof. ^ A TV, set-top box, cell phone, tablet, or other electronic device that performs at least one of the embodiments described. ^ A TV, set-top box, cell phone, tablet, or other electronic device that performs at least one of the embodiments described, and that displays (e.g. using a monitor, screen, or other type of display) a resulting picture. 2023PF01036 ^ A TV, set-top box, cell phone, tablet, or other electronic device that tunes (e.g. using a tuner) a channel to receive a signal including an encoded video stream, and performs at least one of the embodiments described. ^ A TV, set-top box, cell phone, tablet, or other electronic device that receives (e.g. using an antenna) a signal over the air that includes an encoded video stream, and performs at least one of the embodiments described. ^ A server, camera, cell phone, tablet or other electronic device that transmits (e.g. using an antenna) a signal over the air that includes an encoded video stream, and performs at least one of the embodiments described. ^ A server, camera, cell phone, tablet or other electronic device that tunes (e.g. using a tuner) a channel to transmit a signal including an encoded video stream, and performs at least one of the embodiments described.

Claims

2023PF01036 Claims 1. A method for encoding comprising: obtaining (41) a Fourier mapping function defined by a plurality of sets of Fourier basis frequencies, each Fourier basis frequency of a set of the plurality of sets being randomly sampled from a Gaussian distribution using a same scale parameter, the scale parameter being different for each set; applying (42) the Fourier mapping function to picture coordinates of samples of a picture unit to obtain mapped picture coordinates; applying (43) a learning phase allowing learning parameters of an implicit neural representation, the learning phase comprising a minimization of a loss function depending on the parameters of the implicit neural representation, the implicit neural representation being applied to the mapped picture coordinates; and, signaling (44, 45) the scale parameters and the parameters of the implicit neural representation in picture data.

2. The method of claim 1 comprising signaling in the picture data each seed used to randomly sample a Fourier basis frequency from the Gaussian distribution.

3. The method of claim 2 wherein a same seed is used for all Fourier basis frequencies.

4. The method of claim 2 or 3 wherein, each seed is selected randomly in a plurality of seeds based on a criterion of reconstruction quality of the samples of the picture unit.

5. The method of claim 1 wherein at least two sets of the plurality of sets comprise a different number of Fourier basis frequencies and the method comprises signaling the number of Fourier basis frequencies of each set in the picture data.

6. A method for decoding comprising: decoding (46, 47) scale parameters and implicit neural representation parameters from picture data;2023PF01036 applying (49) a Fourier mapping function to coordinates of picture samples of a picture unit to obtain mapped coordinates, the Fourier mapping function being defined by a plurality of sets of Fourier basis frequencies, each Fourier basis frequencies of a set of the plurality of sets being randomly sampled from a Gaussian distribution using one of the scale parameters, the scale parameter being different for each set; obtaining an implicit neural representation from the implicit neural representation parameters; and, applying (50) the implicit neural representation to the mapped coordinates to obtain a reconstructed picture unit.

7. The method of claim 6 comprising decoding from the picture data each seed used to randomly sample a Fourier basis frequency from the Gaussian distribution.

8. The method of claim 7 wherein a same seed is used for all Fourier basis frequencies.

9. The method of claim 7 or 8 wherein, each seed is selected randomly in a plurality of seeds based on a criterion of reconstruction quality of the samples of the picture unit.

10. The method of claim 6 wherein at least two sets of the plurality of sets comprise a different number of Fourier basis frequencies and the method comprises decoding the number of Fourier basis frequencies of each set.

11. A device for encoding comprising electronic circuitry configured for: obtaining (41) a Fourier mapping function defined by a plurality of sets of Fourier basis frequencies, each Fourier basis frequency of a set of the plurality of sets being randomly sampled from a Gaussian distribution using a same scale parameter, the scale parameter being different for each set; applying (42) the Fourier mapping function to picture coordinates of samples of a picture unit to obtain mapped picture coordinates; applying (43) a learning phase allowing learning parameters of an implicit neural representation, the learning phase comprising a minimization of a loss function2023PF01036 depending on the parameters of the implicit neural representation, the implicit neural representation being applied to the mapped picture coordinates; and, signaling (44, 45) the scale parameters and the parameters of the implicit neural representation in picture data.

12. The device of claim 11 wherein the electronic circuitry is further configured for signaling in the picture data each seed used to randomly sample a Fourier basis frequency from the Gaussian distribution.

13. The device of claim 12 wherein a same seed is used for all Fourier basis frequencies.

14. A device for decoding comprising electronic circuitry configured for: decoding (46, 47) scale parameters and implicit neural representation parameters from picture data; applying (49) a Fourier mapping function to coordinates of picture samples of a picture unit to obtain mapped coordinates, the Fourier mapping function being defined by a plurality of sets of Fourier basis frequencies, each Fourier basis frequencies of a set of the plurality of sets being randomly sampled from a Gaussian distribution using one of the scale parameters, the scale parameter being different for each set; obtaining an implicit neural representation from the implicit neural representation parameters; and, applying (50) the implicit neural representation to the mapped coordinates to obtain a reconstructed picture unit.

15. The device of claim 14 wherein the electronic circuitry is further configured for decoding from the picture data each seed used to randomly sample a Fourier basis frequency from the Gaussian distribution.

Images