Film grain synthesis using coded information

Abstract

6. A method comprising: prior to reconstructing a current sample of a picture, obtaining (601) at least one characteristic of the current sample; determining (602) parameters of a film grain compositing process to be applied to the current sample from the at least one characteristic; storing (603) the determined parameters together with information representing a position of the current sample; reconstructing (604) the current sample; and applying (605) the film grain compositing process to the current sample using the determined parameters.

Description

[Technical Field]

[0001] At least one of the present embodiments generally relates to a method or device for film grain synthesis. [Background technology]

[0002] To achieve high compression efficiency, video coding schemes typically employ prediction and transformation to exploit spatial and temporal redundancy in the video content. During encoding, a picture of the video content is divided into blocks of samples (i.e., pixels), which are then divided into one or more sub-blocks, hereinafter referred to as original sub-blocks. Intra- or inter-prediction is then applied to each sub-block to exploit intra- or inter-image correlation. Regardless of the prediction method (intra- or inter-) used, a predictor sub-block is determined for each original sub-block. Sub-blocks representing the difference between the original and predictor sub-blocks, often referred to as prediction error sub-blocks, prediction residual sub-blocks, or simply residual sub-blocks, are then transformed, quantized, and entropy coded to generate an encoded video stream. To reconstruct the video, the compressed data is decoded by the inverse process corresponding to the transform, quantization, and entropy coding.

[0003] In the entertainment industry, film grain is widely present in motion picture and TV material and is considered part of the creative intent. This grain is inherent in analog motion picture film due to the process of exposure and development of silver halide crystals dispersed in the photographic emulsion, as randomly distributed grain appears where silver crystals formed. Digital cameras do not produce film grain, but in post-production, film grain is often added to captured material to create a "cinematic" look. Therefore, when encoding motion picture and TV content, it is important to preserve film grain to maintain the creative intent of the content creator.

[0004] The randomness of this film grain, which can be considered random noise, makes it difficult to compress using conventional encoding tools. Common encoding tool parameters, such as those selected for low bit rates, can remove the film grain. Maintaining and reconstructing the film grain with sufficient quality requires a high bit rate, which contradicts the compression tool's goal of saving bits. To overcome this problem, film grain is typically modeled before the encoding stage and then added at the decoding stage during a so-called compositing step.

[0005] In some implementations, once estimated during the encoding stage, film grain parameters are sent along with the compressed video data in the form of metadata. After decoding, the film grain is synthesized and added to the reconstructed video picture. Film grain synthesis generally involves extracting information from the reconstructed picture to adapt the added film grain to the picture content. Extracting this information increases the complexity of the decoding stage.

[0006] It would be desirable to propose a solution that makes it possible to overcome the above problems, and in particular to propose a solution that reduces the complexity of the film grain synthesis process. Summary of the Invention

[0007] In a first aspect, one or more of the present embodiments provide a method including: before reconstructing a current sample of a picture, obtaining at least one characteristic of the current sample; determining parameters of a film grain synthesis process to be applied to the current sample from the at least one characteristic; storing the determined parameters together with information representing a position of the current sample; reconstructing the current sample; and applying the film grain synthesis process to the current sample using the determined parameters.

[0008] In an embodiment, the at least one characteristic of the current sample is at least one characteristic of the current block that includes the current sample.

[0009] In an embodiment, the at least one characteristic of the current block includes at least one of: a shape of the current block; a position of the current block; information representing the sum of bits used to code all components of the current block; information representing a surface bounded by the current block; and information indicating that the prediction residual of the current block includes at least one non-zero transform coefficient.

[0010] In an embodiment, in response to at least one characteristic of the current block including information representing the shape and position of the current block, the parameters of the film grain synthesis process are the shape and position of a block of film grain samples including the film grain sample to be applied to the current sample.

[0011] In an embodiment, applying the film grain synthesis process to the current sample comprises adding a film grain value to the value of the current sample, wherein the film grain value is obtained by weighting the value obtained using the film grain model by a first weighting factor that depends on the value of the current sample or the value of the average value of the samples of the current block.

[0012] In an embodiment, in response to at least one characteristic of the current block including the sum of bits used to code all components of the current block, the value obtained using the film grain model is further weighted by a weighting factor that depends on the sum of bits used to code all components of the current block.

[0013] In an embodiment, in response to at least one characteristic of the current block including information representative of a surface bounded by the current block, the value obtained using the film grain model is further weighted by a weighting factor that depends on the surface bounded by the current block.

[0014] In an embodiment, in response to at least one characteristic of the current block including information indicating that the prediction residual of the current block includes at least one non-zero transform coefficient, the value obtained using the film grain model is further weighted by a weighting factor that depends on the information indicating that the prediction residual of the current block includes at least one non-zero transform coefficient.

[0015] In a second aspect, one or more of the present embodiments provide a device including electronic circuitry configured to: before reconstructing a current sample of a picture, obtain at least one characteristic of the current sample; determine parameters of a film grain synthesis process to be applied to the current sample from the at least one characteristic; store the determined parameters together with information representing a position of the current sample; reconstruct the current sample; and apply the film grain synthesis process to the current sample using the determined parameters.

[0016] In an embodiment, the at least one characteristic of the current sample is at least one characteristic of the current block that includes the current sample.

[0017] In an embodiment, the at least one characteristic of the current block includes at least one of: a shape of the current block; a position of the current block; information representing the sum of bits used to code all components of the current block; information representing a surface bounded by the current block; and information indicating that the prediction residual of the current block includes at least one non-zero transform coefficient.

[0018] In an embodiment, in response to at least one characteristic of the current block including information representing the shape and position of the current block, the parameters of the film grain synthesis process are the shape and position of a block of film grain samples including the film grain sample to be applied to the current sample.

[0019] In an embodiment, applying the film grain synthesis process to the current sample comprises adding a film grain value to the value of the current sample, wherein the film grain value is obtained by weighting the value obtained using the film grain model by a first weighting factor that depends on the value of the current sample or the value of the average value of the samples of the current block.

[0020] In an embodiment, in response to at least one characteristic of the current block including the sum of bits used to code all components of the current block, the value obtained using the film grain model is further weighted by a weighting factor that depends on the sum of bits used to code all components of the current block.

[0021] In an embodiment, in response to at least one characteristic of the current block including information representative of a surface bounded by the current block, the value obtained using the film grain model is further weighted by a weighting factor that depends on the surface bounded by the current block.

[0022] In an embodiment, in response to at least one characteristic of the current block including information indicating that the prediction residual of the current block includes at least one non-zero transform coefficient, the value obtained using the film grain model is further weighted by a weighting factor that depends on the information indicating that the prediction residual of the current block includes at least one non-zero transform coefficient.

[0023] In a third aspect, one or more of the present embodiments provide a computer program comprising program code instructions for implementing a method according to the first aspect.

[0024] In a fourth aspect, one or more of the present embodiments provide a non-transitory information storage medium storing program code instructions for implementing a method according to the first aspect. 4. [Brief explanation of the drawings]

[0025] [Figure 1] 1 illustrates schematically a context in which embodiments are implemented; [Figure 2] 1 shows a schematic example of the division that a picture of pixels of an original video may undergo; [Figure 3] 1 illustrates schematically a method for encoding a video stream. [Figure 4] 1 illustrates schematically a method for decoding an encoded video stream. [Figure 5A] 1 illustrates schematically an example of a hardware architecture of a processing module that may implement an encoding module or a decoding module in which various aspects and embodiments are implemented; [Figure 5B] 1 illustrates a block diagram of a first example system in which various aspects and embodiments may be implemented. [Figure 5C] 1 illustrates a block diagram of a second example system in which various aspects and embodiments may be implemented. [Figure 6] 1 illustrates an embodiment that allows for reducing the complexity of the film grain synthesis process. [Figure 7A] 1 illustrates a schematic representation of a film grain modeling framework. [Figure 7B] 1 illustrates a schematic of the film grain synthesis and renoising process. [Figure 8A] 3A schematically represents neighboring samples of a current block that can be used to estimate the mean value of the samples of the current block. [Figure 8B] 3A and 3B schematically represent DC coefficients that can be used to estimate the DC coefficients of the current block. DETAILED DESCRIPTION OF THE INVENTION

[0026] The following example embodiments are described in the context of video formats similar to Versatile Video Coding (VVC), developed by a joint team of ITU-T and ISO / IEC experts known as the Joint Video Experts Team (JVET). However, these embodiments are not limited to video encoding / decoding methods that correspond to VVC. These embodiments are particularly applicable to various video formats, including, for example, HEVC (ISO / IEC 23008-2 - MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265), AVC (ISO / CEI 14496-10), EVC (Essential Video Coding / MPEG-5), AV1, AV2, and VP9.

[0027] FIG. 1 illustrates schematically an example of a context in which embodiments may be implemented.

[0028] 1, system 11, which may be a camera, a storage device, a computer, a server, or any device capable of delivering a video stream, transmits a video stream to system 13 using communication channel 12. The video stream is encoded and transmitted by system 11, or is received and / or stored by system 11 and then transmitted. Communication channel 12 may be a wired (e.g., Internet or Ethernet) or wireless (e.g., Wi-Fi, 3G, 4G, or 5G) network link.

[0029] A system 13, which may be, for example, a set-top box, receives and decodes the video stream to produce a sequence of reconstructed pictures. Post-processing, such as a film grain compositing process, is applied to the reconstructed pictures.

[0030] The sequence of acquired post-processed reconstructed pictures is then transmitted using a communication channel 14, which may be a wired or wireless network, to a display system 15, which then displays the pictures.

[0031] In an embodiment, system 13 is included in a display system 15. In that case, system 13 and display 15 are included in a TV, computer, tablet, smartphone, head-mounted display, etc.

[0032] Figures 2, 3 and 4 show examples of video formats.

[0033] 2 shows an example of the division that a picture of a pixel 21 of an original video sequence 20 may undergo. Here, the pixel is considered to consist of three components: a luma component and two chroma components. However, other types of pixels may contain fewer or more components, such as only a luma component or an additional depth or transparency component.

[0034] A picture is divided into multiple coding entities. First, as represented by reference numeral 23 in FIG. 2, a picture is divided into a grid of blocks called coding tree units (CTUs). A CTU consists of an N×N block of luma samples and two corresponding blocks of chroma samples. N is generally a power of 2, with a maximum value of, for example, 128. Second, a picture is divided into one or more groups of CTUs. For example, a picture can be divided into one or more tile rows and tile columns, where a tile is a sequence of CTUs that covers a rectangular area of the picture. In some cases, a tile can be divided into one or more bricks, each consisting of at least one CTU row within the tile. Above the concept of tiles and bricks, there is another coding entity called a slice, which can include at least one tile of a picture or at least one brick of a tile.

[0035] In the example of Figure 2, as represented by reference numeral 22, picture 21 is divided into three slices S1, S2 and S3 in raster scan slice mode, each containing multiple tiles (not shown), with each tile containing only one brick.

[0036] As represented by reference numeral 24 in Figure 2, a CTU may be divided into a hierarchical tree of one or more sub-blocks called coding units (CUs). The CTU is the root (i.e., parent node) of the hierarchical tree and may be divided into multiple CUs (i.e., child nodes). Each CU is a leaf of the hierarchical tree if it has not been further divided into smaller CUs, or a parent node of smaller CUs (i.e., child nodes) if it has been further divided.

[0037] In the example of FIG. 2, CTU 24 is first divided into "4" rectangular CUs using a quadtree-type division. The top-left CU is not further divided and is therefore a leaf of the hierarchical tree, i.e., it is not a parent node of other CUs. The top-right CU is further divided into "4" smaller rectangular CUs, also using a quadtree-type division. The bottom-right CU is divided vertically into "2" rectangular CUs using a binary tree-type division. The bottom-left CU is divided vertically into "3" rectangular CUs using a ternary tree-type division.

[0038] During picture encoding, the partitioning is adaptive, and each CTU is partitioned to optimize the compression efficiency of the CTU criteria.

[0039] In HEVC, the concepts of prediction unit (PU) and transform unit (TU) are introduced. In fact, in HEVC, the coding entities used for prediction (i.e., PU) and transformation (i.e., TU) can be subdivisions of a CU. For example, as shown in Figure 2, a CU of size 2Nx2N can be divided into a PU 2411 of size Nx2N or a PU 2411 of size 2NxN. Furthermore, the CU can be divided into a TU 2412 of size NxN "4" or a TU 2413 of size NxN.

[0040]

number

[0041] Note that in VVC, except for a few specific cases, the frontiers of TUs and PUs are aligned with the frontier of a CU, so a CU generally contains one TU and one PU.

[0042] In this application, the term "block" or "picture block" can be used to refer to any one of a CTU, a CU, a PU, and a TU. Furthermore, the term "block" or "picture block" can be used to refer to a macroblock, a partition, and a sub-block as specified in H.264 / AVC or other video coding standards, or more generally to refer to an array of samples of multiple sizes.

[0043] In this application, the terms "reconstructed" and "decoded" can be used interchangeably, the terms "pixel" and "sample" can be used interchangeably, and the terms "image," "picture," "subpicture," "slice," and "frame" can be used interchangeably. Typically, although not necessarily, the term "reconstructed" is used on the encoder side, and the term "decoded" is used on the decoder side.

[0044]

[0033] Figure 3 illustrates a schematic diagram of a method for encoding a video stream, which is performed by an encoding module. For example, the encoding method of Figure 3 is performed by a processing module of system 11. The processing module corresponds to processing module 500, which is described in detail below in connection with Figure 5A. Variations of this method for encoding are contemplated, but for the sake of clarity, the method for encoding of Figure 3 will be described below without describing all possible variations.

[0045] Before being encoded, the current original picture of the original video sequence may undergo pre-processing, for example, in a pre-processing step 301, a film grain analysis is applied to the original picture.

[0046] FIG. 7A shows a schematic representation of the film grain modeling framework.

[0047] The process of FIG. 7A is performed, for example, during step 301.

[0048] In step 3011, processing module 500 obtains the original picture and removes film grain from the original picture using a denoising process. Many approaches for film grain noise removal have been proposed in the literature. For example, during step 3011, processing module 500 applies the denoising process described in J.C. Kit Yan and Hatzinakos, "Signal-dependent film grain noise removal and generation based on higher-order statistics," Proceedings of the IEEE Signal Processing Workshop on Higher-Order Statistics, July 1997, Banff, Canada.

[0049] In step 3012, the processing module 500 analyzes the denoised picture to determine smooth regions. Indeed, it is important to ensure that only smooth regions of the picture are used to estimate the film grain model, since edges and textures can affect the estimation of film grain intensity and pattern. To determine smooth regions of the input picture, the processing module may, for example, apply a Canny edge detector to the denoised image at different scales, followed by a dilation operation.

[0050] In step 3013, the processing module 500 subtracts the denoised picture from the original picture to obtain a noisy picture.

[0051] In step 3014, the processing module 500 estimates the film grain intensity and pattern from the noisy picture using the determined smooth regions. In these smooth regions, the film grain pattern is modeled using an autoregressive model (AR). Let G(x,y) be the zero-average film grain sample at the current position (x,y) in the picture. For a lag parameter L=2, the grain sample G(x,y) is calculated as follows: G(x,y)=a0.G(x-2,y-2)+a1.G(x-1,y-2)+a2.G(x,y-2)+...+z (Formula 1) In the formula, a0,...,a n is the AR coefficient, G(x+k,y+m) is the film grain sample value in the causal neighborhood of the current position (x,y), and z is the unit variance Gaussian noise obtained from a predefined set stored at the decoder and encoder side. i is determined by the lag parameter L, which is equal to 2L(L+1) for the luma component and 2L(L+1)+1 for the chroma component. For the chroma component, one additional coefficient a iis to capture the correlation with the luminance grain sample at the same spatial location. The lag L can take values from 0 to 3. L=0 corresponds to modeling Gaussian noise, while higher values of L may correspond to film grain with larger grain sizes. The AR coefficients a0...a n For example, it is estimated by a method based on the Yule-Walker AR formula.

[0052] It should be noted that instead of the AR model, other types of film grain models may be used, such as the frequency filtering model defined in the document SMPTE: Film Grain Technology - Specifications for H.264|MPEG-4 AVC Bitstreams / RDD 5-2006.

[0053] The film grain intensity can vary with the signal intensity. When adding film grain to the luminance component, the following model is used: Y'(x,y)=Y(x,y)+f(Y(x,y)).G(x,y) (Equation 2). where Y'(x,y) is the resulting luma sample at location (x,y) renoised with film grain, Y(x,y) is the reconstructed luma sample at location (x,y), and G(x,y) is the film grain sample at location (x,y). f() is a piecewise linear function that scales the film grain according to the luma component value, which is fitted by measuring the noise intensity over smooth regions. This piecewise linear function can be implemented as a pre-computed look-up table (LUT) that is initialized before performing film grain synthesis. Fitting the scaling function to the data can be done in various ways. For example, the scaling function is determined by using a least-squares fit to the local standard deviation of the smooth regions relative to their local mean intensity value. Some additional criteria can be used, such as the scaling function being equal to 0 for luma values of 0. A similar approach is applied to determine the scaling function for the chroma component.

[0054] Note that in equation (Equation 2), the scaling function f() gives a different value for each sample Y(x,y). In other implementations, a single scaling value may be calculated for a block of samples. In that case, the average of the sample values of the block

[0055]

number

[0056]

number

[0057]

number

[0058]

number

[0059]

number

[0060]

number

[0061] The picture output by the preprocessing step 301 (such as the denoised picture produced during step 3011) is referred to below as the preprocessed picture.

[0062] Encoding a preprocessed picture begins with dividing the preprocessed picture during step 302, as described in connection with Figure 2. Thus, the preprocessed picture is partitioned into CTUs, CUs, PUs, TUs, etc.

[0063] Then, for each block, the encoding module determines the encoding mode between intra-prediction and inter-prediction.

[0064] Intra prediction consists in predicting the pixels of the current block from a prediction block derived from pixels of reconstructed blocks located in the causal neighborhood of the current block to be coded, according to an intra prediction method, during step 303. The results of intra prediction are a prediction direction indicating which pixels of the neighboring blocks to use, and a residual block resulting from the calculation of the difference between the current block and the prediction block.

[0065] Inter prediction consists of predicting pixels of a current block from blocks of pixels (called reference blocks) of pictures preceding or following the current picture (this picture is called the reference picture). During the encoding of a current block by an inter prediction method, a block of the reference picture that is closest to the current block according to a similarity criterion is determined in a motion estimation step 304. During step 304, a motion vector is determined that indicates the location of the reference block in the reference picture. This motion vector is used during a motion compensation step 305, during which a residual block is calculated in the form of the difference between the current block and the reference block. In the first video compression standard, the unidirectional inter prediction mode described above was the only available inter mode. As video compression standards have evolved, the family of inter modes has grown significantly and now includes many different inter modes.

[0066] During a selection step 306, the encoding module selects from the tested prediction modes (intra prediction modes, inter prediction modes) the prediction mode that optimizes the compression performance according to a rate / distortion optimization criterion (i.e., RDO criterion).

[0067] Once a prediction mode is selected, the residual block is transformed during step 307. Then, during step 309, the transformed block is quantized.

[0068] It should be noted that the encoding module may skip the transform and apply quantization directly to the untransformed residual signal.

[0069] If the current block is coded according to an intra prediction mode, the prediction direction and the transformed and quantized residual block are coded by the entropy encoder during step 310. If the current block is coded according to inter prediction, the motion vector of the block is predicted, if appropriate, from a prediction vector selected from a set of motion vector predictors derived from reconstructed blocks located spatially and temporally close to the block being coded. Then, motion information is coded by the entropy encoder during step 310 in the form of a motion residual and an index for identifying the prediction vector. The transformed and quantized residual block is coded by the entropy encoder during step 310.

[0070] Note that the encoding module can bypass both transform and quantization, i.e., entropy encoding is applied to the residual without applying the transform or quantization processes. The result of the entropy encoding is inserted into the coded video stream 311.

[0071] Furthermore, some CUs (or TUs) may be coded without residual, i.e., with all transform coefficients equal to 0. This information is signaled by coded block flags (CBFs). For example, VVC uses three CBFs to indicate whether a CU is coded with residual or not. tu_y_coded_flag equal to '1' specifies that the luma component of the CU contains one or more transform coefficient levels not equal to 0. tu_y_coded_flag equal to '0' specifies that all transform coefficients of the CU are equal to 0. tu_cb_coded_flag equal to '1' specifies that the Cb component of the CU contains one or more transform coefficient levels not equal to 0. tu_cb_coded_flag equal to '0' specifies that all transform coefficients of the CU are equal to 0. tu_cr_coded_flag equal to '1' specifies that the Cr component of the CU contains one or more transform coefficient levels not equal to 0. tu_cr_coded_flag equal to '0' specifies that all transform coefficients of the CU are equal to 0.

[0072] The encoded video stream 311 may be accompanied by metadata such as a supplemental enhancement information (SEI) message. For example, the SEI message, defined in standards such as AVC, HEVC, or VVC (or the Versatile Supplemental Enhancement Information (VSEI) message for coded video bitstreams—H.274), is a data container or syntax structure associated with a video stream and is composed of metadata that provides information related to the video stream. For example, the SEI message is defined to carry film grain information in the document C Gomila, A. Kobilansky, “SEI message for film grain encoding,” ISO / IEC JTC1 / SC29 / WG11, ITU-T SG16 Q.6 document JVT-H022, Geneva, Switzerland, May 2003. This SEI message can transfer information that enables a decoder to apply a film grain synthesis process, which includes, for example, the parameters a of the AR model described by equation (Equation 1): i and a set of points of the piecewise linear scaling function f() for each color component.

[0073] After the quantization step 309, the current block is reconstructed so that its corresponding pixels can be used for future prediction. This reconstruction phase is also called a prediction loop. Thus, inverse quantization is applied to the transformed and quantized residual block during step 312, and an inverse transform is applied during step 313. Depending on the prediction mode used for the block obtained during step 314, a predictive block of the block is reconstructed. If the current block is coded according to an inter prediction mode, the coding module applies motion compensation using the motion vector of the current block, if appropriate, to identify a reference block for the current block during step 316. If the current block is coded according to an intra prediction mode, the prediction direction corresponding to the current block is used to reconstruct the predictive block of the current block during step 315. The predictive block and the reconstructed residual block (if any) are added to obtain the reconstructed current block.

[0074] After reconstruction, in-loop filtering intended to reduce coding artifacts is applied to the reconstructed blocks during step 317. This filtering is called in-loop filtering because it is performed in the prediction loop in order to obtain the same reference pictures at the decoder as the encoder, thus avoiding drift between the encoding and decoding processes. In-loop filtering tools include deblocking filtering, SAO (Sample Adaptive Offset) and ALF (Adaptive Loop Filtering).

[0075] Once a block is reconstructed, it is inserted into a reconstructed picture stored in a reconstructed picture memory 319, commonly referred to as a Decoded Picture Buffer (DPB), during step 318. The reconstructed picture so stored can then serve as a reference picture for other pictures to be coded.

[0076] Figure 4 illustrates schematically a method performed by a decoding module for decoding an encoded video stream 311 that was encoded according to the method described in relation to Figure 3. For example, the decoding method of Figure 4 may be performed by a processing module 500 of system 13. Variations of this decoding method are contemplated, but for the sake of clarity, the decoding method of Figure 4 will be described below without describing all possible variations.

[0077] The decoding is performed block by block. For the current block, the process starts with entropy decoding of the CTU that contains the current block (to determine the division of the CTU), in step 410, followed by entropy decoding of information representing the current block. The entropy decoding makes it possible to obtain at least the prediction mode of the block.

[0078] If the block is coded according to an inter prediction mode, entropy decoding makes it possible to obtain, where appropriate, the prediction vector index, the motion residual, and the residual block (if any). During step 408, a motion vector is reconstructed for the current block using the prediction vector index and the motion residual.

[0079] If the block is coded according to an intra prediction mode, entropy decoding makes it possible to obtain the prediction direction and the residual block, if any. Steps 412, 413, 414, 415, 416 and 417 implemented by the decoding module are identical in all respects to steps 312, 313, 314, 315, 316 and 317, respectively, implemented by the coding module.

[0080] In step 418, the decoded blocks are saved in a decoded picture, and the decoded picture is stored in DPB 419. When the decoding module decodes a given picture, the picture stored in DPB 419 is identical to the picture stored in DPB 319 by the encoding module during the encoding of said given picture. The decoded picture may also be output by the decoding module, for example for display.

[0081] Following the in-loop filtering (i.e., following the generation of the decoded picture), a post-processing step 421 can be applied. In particular, film grain may be added during the post-processing step 421. In that case, the post-processing step 421 includes a film grain synthesis and re-noising process.

[0082] FIG. 7B shows a schematic representation of the film grain synthesis and renoising process.

[0083] In step 4211, the processing module 500 obtains the film grain mode parameters. For example, the processing module 500 obtains the parameters a of the AR model described in equation (Equation 1). i and a set of points of the piecewise linear scaling function f() for each color component.

[0084] In step 4212, the processing module 500 calculates the parameter a of the AR model described in equation (Equation 1). i to generate film grain samples. Generally, the film grain samples are generated in the form of blocks of film grain samples. The size of the blocks of film grain samples is generally predefined, e.g., equal to 32x32 for luma blocks and equal to 16x16 for chroma blocks.

[0085] In step 4213, the processing module 500 adds a block of film grain samples to a block of reconstructed samples of the same size using Equation 2. In other implementations, Equation 3 can be used instead of Equation 2. In that case, the sample values of the reconstructed block

[0086]

number

[0087]

number

[0088] As can be seen, the shape and location of the blocks of film grain samples used in the film grain synthesis process do not take into account the division of the picture (as described in Figure 2). This can be problematic because the blocks resulting from this division are considered to be uniform enough on a rate / distortion basis to be coded together. Therefore, there is no reason to divide the picture differently. Furthermore, the film grain synthesis process involves the extraction of features of the picture samples in order to obtain film grain samples adapted to the picture content. The extraction of these features increases the computational cost on the decoder side. The extraction process does not take into account that some of these features may have been already available during decoding or could easily be derived from data obtained during the decoding process.

[0089] 5A, 5B, and 5C illustrate examples of devices, apparatuses, and / or systems that may implement various embodiments.

[0090] 5A shows a schematic diagram of an example of a hardware architecture of a processing module 500 capable of implementing an encoding module or a decoding module capable of implementing the encoding method of FIG. 3 and the decoding method of FIG. 4, respectively, modified according to different aspects and embodiments. The encoding module is included in system 11, for example, if this system is responsible for encoding a video stream. The decoding module is included in system 13, for example.

[0091] The processing module 500 includes a processor or central processing unit (CPU) 5000, including, by way of non-limiting example, one or more microprocessors, general purpose computers, special purpose computers, and processors based on multi-core architectures, connected by a communication bus 5005; a random access memory (RAM) 5001; a read only memory (ROM) 5002; and a memory device such as an electrically erasable programmable read-only memory (EEPROM), a read only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), flash, a magnetic disk drive, and / or an optical disk drive, or a secure digital (SD) card reader and / or a hard disk drive. The system includes a storage unit 5003, which may include non-volatile and / or volatile memory, including, but not limited to, a storage media reader and / or a network-accessible storage device such as a hard disk drive (HDD), and at least one communication interface 5004 for exchanging data with other modules, devices, or systems. The communication interface 5004 may include, but is not limited to, a transceiver configured to send and receive data over a communication channel. The communication interface 5004 may include, but is not limited to, a modem or a network card.

[0092] If the processing module 500 implements a decoding module, the communications interface 5004 may, for example, enable the processing module 500 to receive an encoded video stream and provide a sequence of decoded pictures. If the processing module 500 implements an encoding module, the communications interface 5004 may, for example, enable the processing module 500 to receive and encode a sequence of original picture data and provide an encoded video stream.

[0093] The processor 5000 can execute instructions loaded into the RAM 5001 from the ROM 5002, an external memory (not shown), a storage medium, or a communication network. When the processing module 5000 is powered on, the processor 5000 can read instructions from the RAM 5001 and execute them. These instructions form a computer program that causes the processor 5000 to implement, for example, the decoding method as described in relation to Figure 4 and / or the encoding method described in relation to Figure 3 and the method described in relation to Figure 6, including various aspects and embodiments described later in this document.

[0094] All or part of the algorithms and steps of the methods of Figures 3, 4 and 6 may be implemented in software form by execution of an instruction set by a programmable machine such as a digital signal processor (DSP) or a microcontroller, or may be implemented in hardware form by a machine or dedicated component such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

[0095] As can be seen, microprocessors, general purpose computers, special purpose computers, processors based or not based on multi-core architectures, DSPs, microcontrollers, FPGAs, and ASICs are electronic circuits adapted or configured to at least partially implement the methods of Figures 3, 4, and 6.

[0096] FIG. 5C illustrates a block diagram of an example system 13 in which various aspects and embodiments can be implemented. System 13 can be embodied as a device including various components, described below, configured to perform one or more of the aspects and embodiments described herein. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected consumer electronics, and head-mounted displays. Elements of system 13, singly or in combination, can be embodied in a single integrated circuit (IC), multiple ICs, and / or separate components. For example, in at least one embodiment, system 13 includes a processing module 500 that implements a decoding module. In various embodiments, system 13 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports. In various embodiments, system 13 is configured to implement one or more of the aspects described herein.

[0097] Input to processing module 500 can be provided via various input modules, as shown in block 531. Such input modules include, but are not limited to, (i) a radio frequency (RF) module, for example, receiving RF signals transmitted over the air from a broadcast station, (ii) a component (COMP) input module (or set of COMP input modules), (iii) a Universal Serial Bus (USB) input module, and / or (iv) a High Definition Multimedia Interface (HDMI) input module. Another example, not shown in FIG. 5C, is composite video.

[0098] In various embodiments, the input modules of block 531 have associated respective input processing elements, as known in the art. For example, the RF module may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a frequency band), (ii) downconverting the selected signal, (iii) band-limiting again to a narrower frequency band to select a signal frequency band, which in particular embodiments may be referred to (for example) as a channel, (iv) demodulating the downconverted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF module of various embodiments includes one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a downconverter, a demodulator, an error corrector, and a demultiplexer. The RF section may include, for example, a tuner that performs various of these functions, including downconverting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency near baseband) or to baseband. In one set-top box embodiment, the RF module and its associated input processing elements receive RF signals transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, downconverting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the above (and other) elements, omit some of these elements, and / or add other elements that perform similar or different functions. Adding elements can include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF module includes an antenna.

[0099] Additionally, the USB module and / or HDMI module may include respective interface processors for connecting system 13 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of the input processing, e.g., Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or, if desired, within processing module 500. Similarly, aspects of the USB or HDMI interface processing may be implemented, if desired, in a separate interface IC or within processing module 500. The demodulated, error corrected, and demultiplexed stream is provided to processing module 500.

[0100] The various elements of system 13 may be provided within a unitary housing, where the various elements may be interconnected and transmit data between them using any suitable connection arrangement, e.g., an internal bus known in the art, including an inter-IC (I2C) bus, wiring, and printed circuit boards. For example, in system 13, processing module 500 is interconnected to the other elements of system 13 by bus 5005.

[0101] The communication interface 5004 of the processing module 500 enables the system 13 to communicate over the communication channel 12. As already mentioned above, the communication channel 12 may be implemented, for example, in a wired and / or wireless medium.

[0102] In various embodiments, data is streamed or otherwise provided to system 13 using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in such embodiments is received via communication channel 12 and communication interface 5004 adapted for Wi-Fi communication. Typically, communication channel 12 in such embodiments is connected to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. In other embodiments, the RF connection of input block 531 is used to provide streaming data to system 13. As indicated above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as a cellular network or a Bluetooth network.

[0103] System 13 can provide output signals to various output devices, including display system 15, speakers 535, and other peripheral devices 536. Display system 15 in various embodiments includes, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. Display system 15 can be for a television, a tablet, a laptop, a mobile phone, a head-mounted display, or other device. Display system 15 can also be integrated with other components (e.g., like a smartphone) or separate (e.g., an external monitor for a laptop). In various example embodiments, other peripheral devices 536 include one or more of a standalone digital video disc (or digital versatile disc) (both terms referred to as DVR), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 536 to provide functionality based on the output of system 13. For example, a disc player performs the function of playing the output of system 13.

[0104] In various embodiments, control signals are communicated between system 13 and display system 15, speakers 535, or other peripheral devices 536 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable control between devices with or without user intervention. Output devices can be communicatively coupled to system 13 via dedicated connections through respective interfaces 532, 533, and 534. Alternatively, output devices can connect to system 13 using communication channel 12 via communication interface 5004 or using a dedicated communication channel corresponding to communication channel 12 of FIG. 5C via communication interface 5004. Display system 15 and speakers 535 can be integrated into a single unit with other components of system 13 in an electronic device such as, for example, a television. In various embodiments, display interface 532 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0105] Display system 15 and speakers 535 may alternatively be separate from one or more of the other components. In various embodiments in which display system 15 and speakers 535 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0106] FIG. 5B illustrates a block diagram of an example system 11 in which various aspects and embodiments can be implemented. System 11 is very similar to system 13. System 11 can be embodied as a device including various components, described below, configured to perform one or more of the aspects and embodiments described herein. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smartphones, tablet computers, cameras, and servers. The elements of system 11, singly or in combination, can be embodied in a single integrated circuit (IC), multiple ICs, and / or separate components. For example, in at least one embodiment, system 11 includes a processing module 500 that implements an encoding module. In various embodiments, system 11 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports. In various embodiments, system 11 is configured to implement one or more of the aspects described herein.

[0107] Input to processing module 500 may be provided via various input modules as shown in block 531, already described with respect to FIG. 5C.

[0108] The various elements of system 11 may be provided within a unitary housing, where the various elements may be interconnected and transmit data between them using any suitable connection arrangement, e.g., internal buses known in the art, including an inter-IC (I2C) bus, wiring, and printed circuit boards. For example, in system 11, processing module 500 is interconnected to the other elements of system 11 by bus 5005.

[0109] The communication interface 5004 of the processing module 500 enables the system 11 to communicate over the communication channel 12 .

[0110] In various embodiments, data is streamed or otherwise provided to system 11 using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in such embodiments is received via communication channel 12 and communication interface 5004 adapted for Wi-Fi communication. Typically, communication channel 12 in such embodiments is connected to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. In other embodiments, the RF connection of input block 531 is used to provide streaming data to system 11.

[0111] As indicated above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0112] The data provided to system 11 can be provided in different formats. In various embodiments, these data are encoded and conform to known video compression formats such as AV1, VP9, VVC, HEVC, AVC, EVC, AV2, etc. In various embodiments, these data are raw data provided, for example, by picture and / or audio capture modules connected to or included in system 11. In that case, processing module 500 is responsible for encoding these data.

[0113] System 11 can provide output signals to various output devices, such as system 13, which can store and / or decode the output signals.

[0114] Various implementations involve decoding. As used herein, "decoding" may encompass all or part of the processes performed on a received encoded video stream, e.g., to generate a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and prediction. In various embodiments, such processes also or alternatively include processes performed by decoders of various implementations described herein, e.g., processes for applying film grain synthesis processes in post-processing steps.

[0115] Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or to the broader decoding process as a whole will be clear based on the context of the specific description and will be well understood by one of ordinary skill in the art.

[0116] Various implementations involve encoding. Similar to the above discussion regarding "decoding," as used herein, "encoding" may encompass all or part of the processes performed on an input video sequence, for example, to generate an encoded video stream. In various embodiments, such processes include one or more of the processes typically performed by an encoder, such as, for example, segmentation, prediction, transformation, quantization, and entropy coding. In various embodiments, such processes also or alternatively include processes performed by the encoder of the various implementations described herein, for example, to apply a film grain modeling process and / or to encode metadata representing a film grain model, for example, in the form of an SEI message.

[0117] Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or to the broader encoding process as a whole will be clear based on the context of the specific description and will be well understood by one of ordinary skill in the art.

[0118] It should be noted that the syntax element names used herein are descriptive terms and therefore do not preclude the use of other syntax element names.

[0119] Where a figure is presented as a flow diagram, it should be understood that the figure also provides a block diagram of the corresponding apparatus. Similarly, where a figure is presented as a block diagram, it should be understood that the figure also provides a flow diagram of the corresponding method / process.

[0120] Various embodiments refer to rate-distortion optimization. In particular, a balance or trade-off between rate and distortion is usually considered during the encoding process. Rate-distortion optimization is usually formulated to minimize a rate-distortion function, which is a weighted sum of rate and distortion. There are various approaches to solving the rate-distortion optimization problem. For example, these approaches may be based on extensive testing of all encoding options, including all considered modes or coding parameter values, with a thorough evaluation of their encoding costs and the associated distortion of the reconstructed signal after encoding and decoding. Also, to reduce encoding complexity, faster approaches may be used, particularly calculation of approximate distortion based on a prediction or prediction residual signal rather than a reconstructed signal. A mixture of these two approaches may also be used, such as by using approximate distortion for only some of the possible encoding options and full distortion for others. Other approaches evaluate only a subset of the possible encoding options. More generally, many approaches employ any of a variety of techniques to perform the optimization, but the optimization does not necessarily involve a thorough evaluation of both the encoding cost and the associated distortion.

[0121] Implementations and aspects described herein may be implemented as, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the discussed feature may also be implemented in other forms (e.g., an apparatus or a program). For example, an apparatus may be implemented in appropriate hardware, software, and firmware. A method may be implemented, for example, in a processor, where a processor refers to a general processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include, for example, communication devices such as computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

[0122] References to "one embodiment" or "an embodiment" or "one implementation" or "an implementation," as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with that embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment," or "in one implementation" or "in an implementation" in various places throughout this application, as well as other variations thereof, are not necessarily all referring to the same embodiment.

[0123] Additionally, the application may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, retrieving information from memory, or retrieving information from, for example, another device, module, or user.

[0124] Additionally, the application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0125] Additionally, the application may refer to "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" typically involves in some manner, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0126] Use of any of " / ", "and / or", "at least one of", "one or more of", e.g., "A / B", "A and / or B", "at least one of A and B", "one or more of A and B" should be understood to be intended to encompass selection of only the first listed alternative (A), or selection of only the second listed alternative (B), or selection of both alternatives (A and B). As a further example, "A, B, and / or C" and "at least one of A, B, and C", "one or more of A, B, and C" are intended to encompass selection of only the first listed alternative (A), or selection of only the second listed alternative (B), or selection of only the third listed alternative (C), or selection of only the first and second listed alternatives (A and B), or selection of only the first and third listed alternatives (A and C), or selection of only the second and third listed alternatives (B and C), or selection of all three alternatives (A, B, and C). This can be expanded to include as many items as are listed, as would be apparent to one skilled in this and related arts.

[0127] Also, as used herein, the term "signaling" specifically refers to indicating something to a corresponding decoder. For example, in certain embodiments, an encoder signals the use of certain encoding tools. In this way, embodiments may use the same parameters on both the encoder and decoder sides. Thus, for example, an encoder may transmit certain parameters to a decoder (explicit signaling) so that the decoder can use the same certain parameters. In contrast, if the decoder already has the certain parameters as well as other parameters, signaling without transmission (implicit signaling) may be used to simply allow the decoder to know and select the certain parameters. By avoiding transmitting any actual capabilities, bit savings are realized in various embodiments. It will be appreciated that signaling can be achieved in various ways. For example, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder in various embodiments. While the above relates to the verb form of the term "signal," the term "signal" may also be used herein as a noun.

[0128] As will be apparent to those skilled in the art, implementations can generate a variety of signals formatted to carry information that can be stored or transmitted, for example. The information can include, for example, instructions for performing a method or data generated by one of the described implementations. For example, the signal can include a signal indicating how to apply a CC encoding tool. For example, such a signal can be formatted as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding an encoded video stream and modulating a carrier wave with the encoded video stream. The information carried by the signal can be, for example, analog or digital information. As is known, the signal can be transmitted over a variety of different wired or wireless links. The signal can be stored on a processor-readable medium.

[0129] FIG. 6 illustrates an embodiment that allows reducing the complexity of the film grain synthesis process.

[0130] The process of Figure 6 is performed during the decoding process described in connection with Figure 4. The process of Figure 6 is performed by processing module 500 of system 13.

[0131] In step 601, the processing module 500 obtains at least one characteristic of a current sample of a picture before reconstructing the current sample. Since each sample of a picture belongs to a block (e.g., a CU), the at least one characteristic of the current sample is generally at least one characteristic of the block that contains the sample, referred to as the current block.

[0132] In a first embodiment, the at least one characteristic of the current sample includes the position and shape (width and height) of the current block containing the current sample. This information is obtained during entropy decoding of the CTU containing the current block (step 410). Obtaining the position and shape of the current block does not require reconstruction of the block and can be obtained before inverse quantization of the block (step 412), inverse transform (step 413), INTRA or INTER prediction (steps 414, 408, 416 and 415) and in-loop filtering (step 417).

[0133] In step 602, the processing module 500 determines, from the at least one characteristic, parameters of a film grain synthesis process to be applied to the current sample.

[0134] In a first embodiment, when the at least one characteristic includes the position and shape of the current block, the processing module 500 determines the position and shape of at least one block of film grain samples to be applied to the current block that includes the current sample. In the first embodiment, the block of film grain samples has the same position and the same shape as the current block. In a variation of the first embodiment, for particularly large blocks, multiple blocks of film grain samples corresponding to subdivisions of the current block are determined.

[0135] In step 603, the processing module 500 stores the determined at least one characteristic. In a first embodiment, the processing module 500 stores the position and shape of the current block.

[0136] In step 604, the processing module 500 reconstructs the samples, which consists of reconstructing the current block by completing the process of FIG.

[0137] In step 605, the processing module applies a film grain synthesis process to the current sample using the stored shape and position of the current block with the determined parameters. In a first embodiment, step 605 is to determine a block of film grain samples having the same position and shape as the current block using equation (Equation 1), and add the block of film grain samples to the reconstructed current block using equation (Equation 2). The film grain synthesis process can be applied to only the luma component of the current block, or to the luma and chroma components of the current block.

[0138] In a first variation of the first embodiment, the film grain synthesis process involves applying equation (Equation 3), where a single scaling value

[0139]

number

[0140]

number

[0141]

number

[0142]

number

[0143]

number

[0144]

number

[0145]

number

[0146]

number

[0147] Some solutions are sample values of the current block

[0148]

number

[0149]

number

[0150] Figure 8A shows the value

[0151]

number

[0152] In the second solution, if the current block is coded in INTRA mode, the direction of INTRA prediction is used to find the value

[0153]

number

[0154]

number

[0155]

number

[0156] In the third solution, if the current block is coded in INTER mode, the value

[0157]

number

[0158] Figure 8B shows the value

[0159]

number

[0160]

number

[0161] In the fifth solution, an approximation of the DC value 801 of the block 800 is obtained by reconstructing the current block up to the inverse quantization step 412 and adding the DC value of the inverse quantized residual to the average value calculated in the first, second or third solution, or to the DC value calculated in the fourth solution.

[0162] In the sixth solution, we use the average of the block samples (i.e., block intensity) and the film grain intensity (i.e., a single scaling value

[0163]

number

[0164] In a second embodiment, the bits per pixel value bpp of the current block is calculated and the scaling function f() of equation (Equation 2) or the single scaling value of equation (Equation 3) is used.

[0165]

number

[0166]

number

[0167]

number

[0168] The calculation of the first additional scaling factor Y1 as a function of the bit per pixel value bpp can be implemented in the form of a LUT, a piecewise function, or a linear function. Table TAB1 shows the calculation of the first additional scaling factor Y1 as a function of the range of the bit per pixel value bpp.

[0169] [Table 1]

[0170] In a third embodiment, the surface bounded by the current block is used to calculate the scaling factor f(x) obtained for the current sample using either the scaling function f() in equation (Eq. 2) or a single scaling value in equation (Eq. 3).

[0171]

number

[0172]

number

[0173]

number

[0174] The calculation of the second additional scaling factor Y2 as a function of the block size may be implemented in the form of a LUT, a piecewise function, or an affine function. For example, an example of an affine function allowing the calculation Y2 may be: Y2=a.X+b where X is the current block size, and a and b are parameters of the affine model obtained, for example, using a training sequence or by defining the points. For example, the two points may be Y2=2 for block size=16, and Y2=0.5 for block size=1024. In that case, a=0.015 and b=1.9762.

[0175] In a fourth embodiment, the CBF of the current block is used to calculate the scaling function f() in equation (2) or the single scaling value f() in equation (3).

[0176]

number

[0177]

number

[0178]

number

[0179] In that case, Y3=1 if the CBF of the current block indicates that the prediction residual of the current block contains only zero transform coefficients, and Y3=2 if the CBF of the current block indicates that the prediction residual of the current block contains at least one non-zero transform coefficient.

[0180] In the particular case of a block coded in intra mode using DC mode or planar mode, and the CBF indicates that the current block is associated with a prediction residual containing only zero transform coefficients, the probability that the current block is uniform is very high. Thus, in a variation of the fourth embodiment, when the current block is coded in DC mode or planar mode, and the CBF indicates that the current block is associated with a prediction residual containing only zero transform coefficients, the intensity of the film grain sample is reduced by Y3=0.5.

[0181] In a fifth embodiment, an estimate of the texture complexity of the current block is used to generate a scaling function f() in equation (2) or a single scaling value in equation (3).

[0182]

number

[0183]

number

[0184]

number

[0185] In this case, if the variance V of the samples adjacent to the current block is high (e.g., V>5), Y4=3, and if the variance V of the samples adjacent to the current block is low (e.g., V<1), Y4=0.1. Furthermore, if the variance V of the samples adjacent to the current block is between "1" and "5", Y4=1.

[0186] Any combination of the first, second, third, fourth and fifth embodiments is possible.

[0187] example: The first embodiment, in which the edges of the block of film grain samples are aligned with the edges of the current block, can be combined with inserting additional scaling factors into equations (equation 2) or (equation 3), as described in the second, third, and fourth embodiments. The second, third, fourth, and fifth embodiments do not require that the edges of the film grain sample block be aligned with the edges of the current block. In fact, only the bits per pixel value (bpp), block size, or CBF of the current block containing the current sample is sufficient to implement the second, third, and fourth embodiments, respectively. In that case, the size of the film grain sample block is predefined, e.g., equal to 32x32 for a luma block and equal to 16x16 for a chroma block. If the size of the current block is smaller than the predetermined size, an area of the size of the current block can be randomly selected among the film grain sample blocks. If the size of the current block is larger than the predetermined size, a combination of several blocks of film grain samples can be used, and this combination is cropped if the block size is not a multiple of the predetermined size. The second, third, fourth and fifth embodiments can be combined. For example, in this case, the formula (Formula 2) can be changed to: Y'(x,y)=Y(x,y)+Y1.Y2.Y3.Y4.f(Y(x,y)).G(x,y) (Equation 12).

[0188] Several embodiments have been described above. The features of these embodiments may be provided alone or in any combination. Furthermore, the embodiments may include one or more of the following features, devices, or aspects, alone or in any combination, across various claim categories and types: A television, set-top box, mobile phone, tablet, or other electronic device that executes at least one of the described embodiments. A television, set-top box, mobile phone, tablet, or other electronic device that performs at least one of the described embodiments and displays the resulting picture (e.g., using a monitor, screen, or other type of display). A television, set-top box, mobile phone, tablet, or other electronic device that tunes to a channel (e.g., using a tuner) to receive a signal including an encoded video stream and that performs at least one of the described embodiments. A television, set-top box, mobile phone, tablet, or other electronic device that receives a signal containing an encoded video stream wirelessly (e.g., using an antenna) and that performs at least one of the described embodiments.

Claims

1. 1. A method comprising: Obtaining (601) at least one characteristic of a current sample of a picture before reconstructing said current sample; determining (602) from the at least one characteristic parameters of a film grain synthesis process to be applied to the current sample; storing (603) the determined parameters together with information representing the position of the current sample; Reconstructing the current sample (604); applying (605) the film grain synthesis process to the current sample using the determined parameters.

2. The method of claim 1 , wherein the at least one characteristic of the current sample is at least one characteristic of a current block that includes the current sample.

3. 3. The method of claim 2, wherein the at least one characteristic of the current block includes at least one of: a shape of the current block; a position of the current block; information representing a sum of bits used to code all components of the current block; information representing a surface bounded by the current block; and information indicating that a prediction residual of the current block includes at least one non-zero transform coefficient.

4. 4. The method of claim 3, wherein in response to the at least one characteristic of the current block including information representing a shape and a position of the current block, the parameters of a film grain synthesis process are a shape and a position of a block of film grain samples including a film grain sample to be applied to the current sample.

5. 5. The method of claim 1, wherein applying the film grain synthesis process to the current sample comprises adding a film grain value to a value of the current sample, the film grain value being obtained by weighting a value obtained using a film grain model by a first weighting factor that depends on the value of the current sample or the value of an average value of the samples of the current block.

6. 6. The method of claim 5 when dependent on claim 3 or 4, wherein the value obtained using a film grain model in response to the at least one characteristic of the current block including the sum of bits used to code all components of the current block is further weighted by a weighting factor that depends on the sum of bits used to code all components of the current block.

7. 7. A method according to claim 5 or claim 6 when dependent on claim 3 or claim 4, wherein the value obtained using a film grain model in response to the at least one characteristic of the current block including information representative of a surface bounded by the current block is further weighted by a weighting factor that depends on the surface bounded by the current block.

8. 8. The method of claim 5, 6 or 7 when dependent on claim 3 or 4, wherein in response to the at least one characteristic of the current block including information indicating that the prediction residual of the current block includes at least one non-zero transform coefficient, the value obtained using a film grain model is further weighted by a weighting factor that depends on the information indicating that the prediction residual of the current block includes at least one non-zero transform coefficient.

9. 1. A device including an electronic circuit, the electronic circuit comprising: Obtaining (601) at least one characteristic of a current sample of a picture before reconstructing said current sample; determining (602) from the at least one characteristic parameters of a film grain synthesis process to be applied to the current sample; storing (603) the determined parameters together with information representing the position of the current sample; Reconstructing the current sample (604); and applying (605) the film grain synthesis process to the current sample using the determined parameters.

10. The device of claim 9 , wherein the at least one characteristic of the current sample is at least one characteristic of a current block that includes the current sample.

11. 11. The device of claim 10, wherein the at least one characteristic of the current block includes at least one of: a shape of the current block; a position of the current block; information representing a sum of bits used to code all components of the current block; information representing a surface bounded by the current block; and information indicating that a prediction residual of the current block includes at least one non-zero transform coefficient.

12. 12. The device of claim 11 , wherein in response to the at least one characteristic of the current block including information representing a shape and a position of the current block, the parameters of a film grain synthesis process are a shape and a position of a block of film grain samples including a film grain sample to be applied to the current sample.

13. 13. The device of claim 9, wherein applying the film grain synthesis process to the current sample comprises adding a film grain value to a value of the current sample, the film grain value being obtained by weighting a value obtained using a film grain model by a first weighting factor that depends on the value of the current sample or the value of an average value of the samples of the current block.

14. 14. The device of claim 13 when dependent on claim 11 or 12, wherein the value obtained using a film grain model in response to the at least one characteristic of the current block including the sum of bits used to code all components of the current block is further weighted by a weighting factor that depends on the sum of bits used to code all components of the current block.

15. 13. A device as claimed in claim 13 or claim 6 when dependent on claim 3 or claim 4, wherein the value obtained using a film grain model in response to the at least one characteristic of the current block including information representative of a surface bounded by the current block is further weighted by a weighting factor that depends on the surface bounded by the current block.

16. 13. The device of claim 13 when dependent on claim 3 or 4 or claim 6 or 7, wherein, in response to the at least one characteristic of the current block including information indicating that the prediction residual of the current block includes at least one non-zero transform coefficient, the value obtained using a film grain model is further weighted by a weighting factor that depends on the information indicating that the prediction residual of the current block includes at least one non-zero transform coefficient.

17. A computer program comprising program code instructions for implementing the method according to any one of claims 1 to 8.

18. A non-transitory information storage medium storing program code instructions for implementing the method of any one of claims 1 to 8.

Images