Film grain processing method and apparatus, and storage medium

Abstract

The present disclosure relates to the technical field of picture processing. Provided are a film grain processing method and apparatus, and a storage medium, which can more flexibly simulate film grain visual effects. The method comprises: acquiring a decoded picture and film grain model parameters of the decoded picture; on the basis of the film grain model parameters, synthesizing film grains of the decoded picture; acquiring film grain fusion indication information, and on the basis of the film grain fusion indication information, performing fusion processing on overlapping film grains to obtain fused film grains; and adding the fused film grains to the decoded picture.

Description

Film grain processing method, device and storage medium

[0001] This disclosure claims priority to Chinese patent application No. 202410430838.8, filed on April 10, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of image processing technology, and in particular to a film grain processing method, device, and storage medium. Background Art

[0003] Film grain is generally defined as the random optical texture in processed photographic film, caused by the presence of small particles of metallic silver or dye clouds that develop from silver halide that has received sufficient photons. Film grain is a key characteristic of film photography and a crucial visual element in photographic art. Digital sensors operate differently from film, producing noise-free digital video without film grain. However, many content creators, pursuing the texture and atmosphere of film photography, use post-processing techniques to add simulated film grain effects to digital images.

[0004] Currently, film grain can be modeled on the encoder to obtain film grain parameters. These parameters are then used on the decoder to generate film grain and add it to the decoded video image. However, when generating overlapping film grain for a video image, how to handle this overlapping film grain remains a pressing technical challenge for those skilled in the art. Summary of the Invention

[0005] The embodiments of the present disclosure provide a film grain processing method, device, and storage medium, which can more flexibly simulate film grain visual effects.

[0006] In one aspect, a film grain processing method is provided, which is applied to a decoding end and includes:

[0007] Obtaining a decoded image and film grain model parameters of the decoded image;

[0008] Synthesize film grain of the decoded image based on film grain model parameters;

[0009] Acquiring film grain fusion indication information, and fusing overlapping film grains based on the film grain fusion indication information to obtain fused film grains;

[0010] Adds fused film grain to the decoded image.

[0011] In another aspect, a film grain processing method is provided, which is applied to an encoding end and includes:

[0012] Acquire a source image, and perform denoising on the source image to obtain a denoised source image;

[0013] Based on the source image and the denoised source image, film grain model parameters of the source image and film grain fusion indication information are determined. The film grain fusion indication information is used to indicate a fusion method for overlapping film grains.

[0014] In another aspect, a film grain processing device is provided, which is applied to a decoding end and includes:

[0015] The acquisition module is used to obtain the decoded image and the film grain model parameters of the decoded image.

[0016] The synthesis module is used to synthesize the film grain of the decoded image based on the film grain model parameters.

[0017] The acquisition module is further configured to acquire film grain fusion indication information, and fuse the overlapping film grains based on the film grain fusion indication information to obtain fused film grains.

[0018] Adds module for adding fused film grain to decoded images.

[0019] In another aspect, a film grain processing device is provided, which is applied to an encoding end and includes:

[0020] An acquisition module is used to acquire a source image and perform denoising on the source image to obtain a denoised source image;

[0021] The determination module is used to determine the film grain model parameters of the source image and film grain fusion indication information based on the source image and the denoised source image, wherein the film grain fusion indication information is used to indicate the fusion method of overlapping film grains.

[0022] On the other hand, a communication device is provided, comprising: a memory and a processor; the memory and the processor are coupled; the memory is used to store a computer program; and the processor implements the film grain processing method of any of the above embodiments when executing the computer program.

[0023] In another aspect, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the film grain processing method of any of the above embodiments is implemented.

[0024] In another aspect, a computer program product is provided. The computer program product includes computer program instructions. When the computer program instructions are executed by a processor, the film grain processing method described in any one of the above embodiments is implemented.

[0025] Based on the technical solutions provided in the present disclosure, for the overlapping film grain, film grain fusion indication information can be acquired, and the overlapping film grain is fused based on the film grain fusion indication information to obtain fused film grain, and the fused film grain is added to the decoded image. In this way, compared with the related art in which only one fixed processing method for the overlapping film grain is defined in different standards or private protocols of various manufacturers, the present disclosure can flexibly fuse the overlapping film grain in multiple possible fusion manners, so as to achieve the purpose of more accurately and flexibly simulating the visual effect of the film grain, thereby bringing a better viewing experience to the viewer. BRIEF DESCRIPTION OF DRAWINGS

[0026] The accompanying drawings are used to provide a further understanding of the technical solutions of the present disclosure, and constitute a part of the specification, and are used to explain the technical solutions of the present disclosure together with the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions of the present disclosure.

[0027] FIG. 1 is a schematic diagram of an architecture of a coding and decoding system provided by an embodiment of the present disclosure;

[0028] FIG. 2 is a schematic diagram of a decoded image provided by an embodiment of the present disclosure;

[0029] FIG. 3 is a schematic diagram of an architecture of another coding and decoding system provided by an embodiment of the present disclosure;

[0030] FIG. 4 is a flowchart of a film grain processing method provided by an embodiment of the present disclosure;

[0031] FIG. 5 is a flowchart of another film grain processing method provided by an embodiment of the present disclosure;

[0032] FIG. 6 is a flowchart of another film grain processing method provided by an embodiment of the present disclosure;

[0033] FIG. 7 is a flowchart of another film grain processing method provided by an embodiment of the present disclosure;

[0034] FIG. 8 is a schematic diagram of a structure of a film grain processing apparatus provided by an embodiment of the present disclosure;

[0035] FIG. 9 is a schematic diagram of a structure of another film grain processing apparatus provided by an embodiment of the present disclosure;

[0036] FIG. 10 is a schematic diagram of a structure of a communication apparatus provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present disclosure.

[0038] Unless the context requires otherwise, throughout the specification and claims, the term "comprise" and its other forms, such as the third person singular form "comprises" and the present participle form "comprising", are to be interpreted as open and inclusive, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "example", "specific example" or "some examples" are intended to indicate that the particular features, structures, materials or characteristics associated with the embodiment or example are included in at least one embodiment or example of the present disclosure. The schematic representation of the above terms does not necessarily refer to the same embodiment or example. In addition, the particular features, structures, materials or characteristics may be included in any one or more embodiments or examples in any appropriate manner.

[0039] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of such features. Throughout this disclosure, unless otherwise specified, "plurality" means two or more.

[0040] In the embodiments of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present disclosure should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0041] Additionally, the use of “based on” is meant to be open and inclusive, as a process, step, calculation, or other action “based on” one or more stated conditions or values ​​may, in practice, be based on additional conditions or values ​​beyond those stated.

[0042] Video image compression relies heavily on prediction in temporal, spatial or cross-component domain, while film grain is random in nature and thus difficult to be coded by typical video image coding tools. To preserve film grain, one solution is to use a smaller quantization parameter in video compression to better preserve details. But this solution requires a relatively high bit rate (code rate) for video applications such as adaptive streaming and broadcasting. Another solution is to model film grain at the encoding end to obtain film grain parameters, which are then used at the decoding end to generate film grain and add it to the decoded video image. This can fully exploit the potential of video image compression technology.

[0043] Film grain technology is one of the technologies currently being discussed and researched in the international video coding standard organization. The film grain modeling and synthesis methods defined in different standards are not exactly the same, and the support for film grain is also different.

[0044] For example, the High Performance Video Coding (H.265 / High Efficiency Video Coding, HEVC) standard and the Versatile Video Coding (H.266 / Versatile Video Coding, VVC) series standard formulated by the Joint Video Experts Team (JVET) under the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) and the International Telecommunication Union (ITU) use film grain characteristics (FGC) and supplemental enhancement information (SEI) to indicate film grain parameters used to synthesize film grain when decoding and rendering video or image data, thereby achieving the purpose of preserving film grain.

[0045] Referring to FIG. 1, it is a schematic diagram of the framework of a coding system used in the video coding standard formulated by the JVET. As shown in FIG. 1, the system includes an encoding end and a decoding end.

[0046] The encoding end includes an encoder, a noise removal module and a film grain modeling module.

[0047] The encoder is configured to compress and encode the input source video to obtain an encoded video bitstream.

[0048] The denoising module is used to pre-process and denoise the input source video to obtain a denoised video, wherein the denoised video includes a video image sequence after film grain has been removed or attenuated.

[0049] The film grain modeling module is used to parameterize the film grain model based on the difference between the denoised video and the source video to obtain film grain parameters (i.e., parameter values ​​of the film grain model).

[0050] The film grain parameters may be transmitted to the decoding end along with the encoded video stream; or may be transmitted to the decoding end through an external transmission method.

[0051] The decoding end includes a decoder and a film grain model.

[0052] The decoder is used to decode the input encoded video stream to obtain a decoded video stream.

[0053] Film grain model, used to synthesize simulated film grain based on input film grain parameters.

[0054] Furthermore, by adding simulated film grain parameters to the decoded video stream, a decoded reconstructed video with simulated film grain can be obtained.

[0055] In some embodiments, in addition to the codec system framework shown in FIG1 , there are other implementations of the codec system framework, the main difference being in the encoding end. For example, instead of encoding the source video image sequence to obtain an encoded video stream, the encoding end may encode the denoised video image sequence to obtain an encoded video. Alternatively, when the source video does not contain film grains, the encoding end may transmit predetermined film grain parameters to the decoding end for film grain synthesis. In this case, even if film grains do not exist in the source video, film grains may be added to the video at the decoder end to achieve a certain effect.

[0056] In the 33rd meeting of JVET, a new region-adaptive frame grouping supplemental enhancement information (FGC SEI) message is proposed in proposal JVET-AG0328, which allows applying film grain to local regions in a complete video frame. As shown in FIG. 2, the complete video frame includes image region 1, image region 2 and image region 3, and film grain model parameters can be independently set for each image region. In the proposal JVET-AG0328, different regions applying film grain are allowed to overlap, but it is stipulated that the film grain model parameters corresponding to the region with a larger region indication index value can only be used in the overlapping region. For example, there is an overlapping region A between image region 1 and image region 2 in FIG. 2, so the film grain model parameters corresponding to image region 2 with a larger region indication index value can be used in the overlapping region A. In this way, the application of film grain in the overlapping region is greatly limited. In addition, in the same image region, in order to more realistically simulate film grain, multiple simulation models may be used for superposition in the film grain modeling and synthesis process, that is, overlapping film grains are generated in the image region. At this time, based on the related art, the overlapping film grains cannot be flexibly processed.

[0057] Therefore, the embodiments of the present disclosure provide a film grain processing method, which comprises: first acquiring a decoded image and film grain model parameters of the decoded image, and synthesizing film grain of the decoded image based on the film grain model parameters, then acquiring film grain fusion indication information, and performing fusion processing on the overlapping film grains based on the film grain fusion indication information to obtain fused film grain, and adding the fused film grain to the decoded image.

[0058] In this way, for the overlapping film grains, film grain fusion indication information can be acquired, and the overlapping film grains can be fused based on the film grain fusion indication information to obtain fused film grain, and the fused film grain is added to the decoded image. In this way, compared with the related art in which only a fixed processing method for overlapping film grains is defined in different standards or private protocols of various manufacturers, the present disclosure can flexibly fuse the overlapping film grains in multiple possible fusion modes, so as to more accurately and flexibly simulate the visual effect of film grain, thereby bringing a better viewing experience to the viewer.

[0059] The coding system provided by the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0060] Referring to FIG. 3, an architecture diagram of a coding system is provided according to an embodiment of the present disclosure. As shown in FIG. 3, the coding system 100 includes an encoding end 110 and a decoding end 120.

[0061] The encoding end 110 generates an encoded video bitstream. Thus, the encoding end 110 can be referred to as a video encoding device. The decoding end 120 can decode the encoded video bitstream generated by the encoding end. Thus, the decoding end 120 can be referred to as a video decoding device. Various embodiments of the encoding end 110, the decoding end 120, or both can include one or more processors and a memory coupled to the one or more processors. The memory can include, but is not limited to, RAM, ROM, EEPROM, flash memory or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0062] In some embodiments, the encoding end 110 and the decoding end 120 can comprise various devices, including a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a telephone handset such as a so-called “smart” phone, a television, a camera, a display device, a digital media player, a video gaming console, an automobile computer, or similar device.

[0063] In some embodiments, the decoding end 120 can receive the encoded video bitstream from the encoding end 110 via a link. The link can comprise one or more media or devices capable of moving the encoded video data from the encoding end to the decoding end 120. In one example, the link can comprise one or more communication media that enable the encoded video bitstream to be transmitted from the encoding end 110 to the decoding end in real-time. In this example, the encoding end 110 can modulate the encoded video bitstream according to a communication standard, such as a wireless communication protocol, and can transmit the modulated video stream to the decoding end 120. The one or more communication media can include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media can form part of a packet-based network, such as a local area network, a wide-area network, or a global network such as the Internet. The one or more communication media can include routers, switches, base stations, or other equipment that facilitates communication from the encoding end 110 to the decoding end 120.

[0064] In some embodiments, the encoding end 110 can include an encoder, a denoising module, and a film grain modeling module as in FIG. 1.

[0065] The encoder is configured to compress and encode an input source video to obtain an encoded video bitstream.

[0066] The denoising module is configured to pre-process and denoise the input source video to obtain a denoised video. The denoised video includes a sequence of video images in which film grain is removed or attenuated.

[0067] The film grain modeling module is configured to parameterize a film grain model to obtain film grain parameters (i.e., parameter values of the film grain model). The film grain parameters can be transmitted to the decoding end 120 together with the encoded video bitstream, or transmitted to the decoding end 120 through an external transmission manner.

[0068] In some embodiments, the number of film grain modeling modules can also be multiple. Thus, multiple sets of film grain parameters can be obtained.

[0069] In some embodiments, the decoding end 120 can include a decoder and a film grain model as shown in FIG. 1.

[0070] The decoder is configured to decode the input encoded video bitstream to obtain a decoded video stream.

[0071] The film grain model is configured to synthesize simulated film grain according to the input film grain parameters.

[0072] In some embodiments, multiple simulated film grains can be sequentially added to the decoded video stream to obtain a decoded reconstructed video with simulated film grain.

[0073] In some embodiments, the number of film grain models can be multiple, so that multiple simulated film grains can be synthesized based on the multiple film grain models, respectively.

[0074] In some embodiments, for overlapping film grains, film grain fusion indication information can be obtained based on the film grain processing method provided in the present disclosure, and the overlapping film grains can be fused based on the film grain fusion indication information to obtain fused film grains, which are added to the decoded video stream to obtain a decoded reconstructed video with simulated film grain.

[0075] It can be understood that the application scenarios of the embodiments of the present disclosure are not limited. The system architecture and business scenarios described in the embodiments of the present disclosure are for more clearly illustrating the technical solutions of the present disclosure, and do not constitute a limitation on the technical solutions provided by the present disclosure. It can be known by those skilled in the art that, with the evolution of network architecture and the appearance of new business scenarios, the technical solutions provided by the present disclosure are also applicable to similar technical problems.

[0076] The embodiments provided by the present disclosure will be described in detail below with reference to the accompanying drawings.

[0077] As shown in FIG. 4, the disclosure provides a film grain processing method applied to a decoding end, the method comprising:

[0078] S101, obtaining a decoded image and film grain model parameters of the decoded image.

[0079] In some embodiments, the decoded image can include at least one of the following: an image sequence composed of continuous multiple frames of images, a video image sequence, a single image, a frame of image in the image sequence, and a partial image region in the frame of image.

[0080] It can be understood that, considering the randomness and diversity of the film grain effect in the video image, the film grain processing method provided by the embodiments of the disclosure can be adaptively applied to an image sequence, a single image, or a partial image region in a frame of image, etc.

[0081] In some embodiments, the decoding end can obtain a code stream, and obtain the decoded image by parsing the code stream.

[0082] The code stream (Data Rate) refers to the data flow used by a video file in a unit of time, also known as the code rate. For example, the code stream can be an image code stream or a transport stream or a media file including the image code stream.

[0083] For example, after obtaining the code stream, the decoding end can parse the frame information, encoding parameters, motion vectors, residual data, etc. in the code stream, and then use these information to perform inverse quantization, inverse transformation, motion compensation, etc. to reconstruct the image of each frame, i.e. to obtain the decoded image. The decoding method of the decoding end can use the method specified in the video decoding standard, which is not limited in the embodiments of the disclosure.

[0084] In some embodiments, the film grain model parameters are included in the code stream. Thus, the decoding end can obtain the film grain model parameters of the decoded image by parsing the code stream based on the obtained code stream.

[0085] In some embodiments, the film grain model parameters can include at least one of the following: the number of film grain models, the identification of the film grain model, the application range of the film grain model, the film grain synthesis parameters corresponding to the film grain model, and the way of adding the film grain to the decoded image.

[0086] The number of film grain models of the decoded image is used to indicate the number of film grain models corresponding to the decoded image. For example, if all image regions in the decoded image use the same film grain model, the number of film grain models of the decoded image is 1. For another example, the decoded image includes image region 1, image region 2, and image region 3, image region 1 uses film grain model 1, image region 2 uses film grain model 2, image region 3 uses film grain model 3 and film grain model 4, and the number of film grain models of the decoded image is 4.

[0087] The film grain model identifier is used to uniquely identify a film grain model corresponding to the decoded image. Based on the film grain model identifier, the decoding end can obtain the corresponding film grain model to synthesize film grain based on the film grain model. In an example, all film grain models corresponding to the decoded image can use one film grain model identifier, that is, all image regions of the decoded image use the same film grain model. In another example, the film grain model identifiers corresponding to different regions of the decoded image are different, that is, different image regions in the decoded image use different film grain models.

[0088] The film grain model application range refers to the image range to which the film grain model is applied. For example, the film grain model application range can indicate that the film grain model is applied to the entire decoded image, or that the film grain model is applied to part of the image region of the decoded image, etc.

[0089] The film grain synthesis parameter corresponding to the film grain model can also be referred to as the film grain processing parameter or the film grain modeling parameter corresponding to the film grain model. For example, based on the film grain model identifier, it is determined that the corresponding film grain model is a frequency domain filtering model, that is, the film grain of the image in the film grain model application range can be generated by using a synthesis method based on frequency domain filtering. Moreover, the film grain synthesis parameter corresponding to the film grain model can include the film grain luminance bit depth parameter fg_bit_depth_luma_minus8, the film grain chrominance bit depth parameter fg_bit_depth_chroma_minus8, the film grain corresponding image color gamut fg_colour_primaries, the color space fg_matric_coeffs, the transfer function fg_transfer_characteristics, the film grain scaling parameter fg_log2_scale_factor, the film grain intensity interval model number parameter fg_num_model_values_minus1[c] for different color components, and the film grain model value fg_comp_model_value[c][i][j] corresponding to each intensity interval for different color components, etc.

[0090] It should be understood that the above-mentioned film grain synthesis parameters are only one possible example, and the film grain synthesis parameters corresponding to the film grain model can also include other possible parameters, which are not limited in the present disclosure.

[0091] The manner in which the film grain is added to the decoded image is used to indicate how the synthesized film grain is added to the decoded image. That is, it is used to indicate how the synthesized film grain values are mixed with the pixel values in the decoded image. In one example, the manner in which the film grain is added to the decoded image corresponds to the entire decoded image, that is, the entire decoded image adopts the same manner in which the film grain is added to the decoded image. In another example, the manner in which the film grain is added to the decoded image corresponds to an image region of the decoded image, that is, different image regions of the entire decoded image adopt different manners in which the film grain is added to the decoded image. In yet another example, the manner in which the film grain is added to the decoded image corresponds to a sequence of video images.

[0092] In some embodiments, the obtained film grain model parameters include model parameters for constructing a plurality of different film grains. That is, the number of film grain models corresponding to the decoded image is a plurality, and each film grain model is applied to the entire decoded image or a partial image region of the decoded image.

[0093] For example, the obtained film grain model parameters can include at least one set of film grain model parameters, wherein each set of film grain model parameters corresponds to one film grain model. It should be understood that a set of film grain model parameters at least includes a film grain model identifier, a film grain model application range, film grain synthesis parameters corresponding to the film grain model, and a manner in which the film grain is added to the decoded image.

[0094] S102, based on the film grain model parameters, synthesizing film grains of the decoded image.

[0095] For example, the decoding end can synthesize each film grain corresponding to each region of the decoded image based on the film grain model identifier in the film grain model parameters of the decoded image, and each set of film grain synthesis parameters corresponding to the film grain synthesis method indicated by the film grain model identifier. For example, generate film grains based on the synthesis method based on frequency domain filtering indicated by the film grain model parameters. The method of film grain synthesis can adopt any method defined in a public standard or a private protocol, which is not limited in the embodiments of the present disclosure.

[0096] In some embodiments, the plurality of film grain models can be independent of each other, or can have a dependency relationship.

[0097] In some embodiments, the processing procedure of the plurality of film grain models can be parallel or serial. In the parallel processing procedure, the plurality of film grain models can be applied to synthesize corresponding film grains simultaneously without the order restriction. In the serial processing procedure, the plurality of film grain models have a predetermined processing order, and the expected film grain synthesis effect can be achieved by applying the plurality of specified film grain models in sequence.

[0098] In some embodiments, the synthesized film grains can overlap on the decoded image. Then, the following step S103 can be performed to perform fusion processing on the overlapping film grains to obtain fused film grains.

[0099] In implementation 1, the number of film grain models corresponding to the decoded image is a plurality, and the plurality of film grain models are applied to the entire decoded image.

[0100] At this time, the plurality of film grain models corresponding to the decoded image can be synthesized based on the film grain parameters, and the film grains corresponding to the plurality of film grain models can be applied to the entire decoded image. In this way, the film grains corresponding to different film grain models in the entire decoded image overlap, that is, there are overlapping film grains on the entire decoded image. Then, the following step S103 can be performed to perform fusion processing on the overlapping film grains in the entire decoded image to obtain fused film grains.

[0101] In implementation 2, the number of film grain models corresponding to the decoded image is a plurality, and each film grain model is applied to an image region of the decoded image. In this case, one image region can correspond to a plurality of film grain models.

[0102] At this time, the film grains corresponding to each image region of the decoded image can be synthesized based on the film grain parameters. For the image region corresponding to a plurality of film grain models, the film grains corresponding to the plurality of film grain models overlap in the image region. That is, there are overlapping film grains in the image region corresponding to the plurality of film grain models. Then, the following step S103 can be performed to perform fusion processing on the overlapping film grains in the image region to obtain fused film grains.

[0103] In implementation 3, the number of film grain models corresponding to the decoded image is a plurality, and each film grain model is applied to an image region of the decoded image. In this case, there is an overlapping region between different image regions. For example, the image region 1 and the image region 2 in FIG. 2 have an overlapping region A.

[0104] At this time, the film grain corresponding to each of the plurality of film grain models can be synthesized based on the film grain parameters. In this way, for example, the film grain corresponding to the film grain model applied to one image region of the decoded image includes the film grain generated based on the film grain model applied to the entire decoded image and the film grain generated based on the film grain model applied only to the image region. That is, the image region can correspond to a plurality of film grain models, and there are overlapping film grains in the image region. Further, the following step S103 can be performed to fuse the overlapping film grains in the image region to obtain fused film grains.

[0105] In some embodiments, each of the above-mentioned implementation modes 1-4 can be combined without contradiction. For example, the number of film grain models corresponding to the decoded image is a plurality, and there are film grain models corresponding to the entire decoded image and film grain models applied to one image region of the decoded image in the plurality of film grain models. In addition, one image region can correspond to a plurality of film grain models, or there are overlapping regions between different image regions, and the like possible implementation modes should be covered within the protection scope of the present disclosure, which will not be listed one by one here.

[0106] At this time, the film grain corresponding to each of the plurality of film grain models can be synthesized based on the film grain parameters. In this way, for example, the film grain corresponding to the film grain model applied to one image region of the decoded image includes the film grain generated based on the film grain model applied to the entire decoded image and the film grain generated based on the film grain model applied only to the image region. That is, the image region can correspond to a plurality of film grain models, and there are overlapping film grains in the image region. Further, the following step S103 can be performed to fuse the overlapping film grains in the image region to obtain fused film grains.

[0107] In some embodiments, each of the above-mentioned implementation modes 1-4 can be combined without contradiction. For example, the number of film grain models corresponding to the decoded image is a plurality, and there are film grain models corresponding to the entire decoded image and film grain models applied to one image region of the decoded image in the plurality of film grain models. In addition, one image region can correspond to a plurality of film grain models, or there are overlapping regions between different image regions, and the like possible implementation modes should be covered within the protection scope of the present disclosure, which will not be listed one by one here.

[0108] S103, obtain film grain fusion indication information, fuse the overlapping film grains based on the film grain fusion indication information to obtain fused film grains.

[0109] In some embodiments, the film grain fusion indication information is contained in a video parameter set (VPS), and the film grain fusion indication information is used to indicate a fusion manner of overlapping film grains in an entire video.

[0110] Alternatively, the film grain fusion indication information is contained in a sequence parameter set (SPS), and the film grain fusion indication information is used to indicate a fusion manner of overlapping film grains in a corresponding sequence of video images.

[0111] Alternatively, the film grain fusion indication information is contained in a picture parameter set (PPS) or an adaptive parameter set (APS), and the film grain fusion indication information is used to indicate a fusion manner of overlapping film grains in a frame of image corresponding to the PPS or the APS.

[0112] Alternatively, the film grain fusion indication information is contained in a supplemental enhancement information (SEI) message, and the film grain fusion indication information is used to indicate a fusion manner of overlapping film grains in a frame of image corresponding to the SEI message.

[0113] In some embodiments, the film grain fusion indication information is contained in a bitstream. Thus, a decoding end can obtain the film grain fusion indication information by parsing the bitstream based on the obtained bitstream.

[0114] It should be noted that the execution sequence shown in FIG. 4 is only an example, and does not limit the film grain processing method provided by the embodiments of the present disclosure. There is no certain order between obtaining the film grain fusion indication information in step S103 and obtaining the decoded image in step S101 and the film grain model parameters of the decoded image in step S101, and they can be performed simultaneously. Alternatively, the film grain model parameters of the decoded image are obtained after the bitstream is parsed to obtain the decoded image, and then the film grain fusion indication information is obtained. Alternatively, the film grain fusion indication information is obtained first, then the bitstream is parsed to obtain the decoded image, and then the film grain model parameters of the decoded image are obtained.

[0115] In some embodiments, the film grain fusion indication information is used to indicate a fusion manner of overlapping film grains in a fusion processing. In some embodiments, the film grain fusion indication information is used to indicate at least one of the following fusion manners:

[0116] only using the uppermost film grain;

[0117] only using the lowermost film grain;

[0118] taking a cumulative value of the overlapping film grains;

[0119] averaging overlapping film grains;

[0120] using only the highest priority film grain;

[0121] using only the most important film grain;

[0122] using only the film grain indicated by the preset flag.

[0123] In some embodiments, the film grain fusion indication information can be applied to an image region where there are overlapping film grains. In turn, the image region with overlapping film grains can be fused based on the fusion processing manner indicated by the film grain fusion indication information to obtain fused film grains.

[0124] For example, taking the above-mentioned implementation manner 1 as an example, there are overlapping film grains in the entire decoded image, so the film grain fusion indication information can be applied to the entire decoded image. In turn, the overlapping film grains in the entire decoded image can be fused based on the fusion processing manner indicated by the film grain fusion indication information, such as the fusion processing manner of using only the uppermost film grain in the overlapping film grains in the entire decoded image, so as to obtain the fused film grains of the entire decoded image.

[0125] Or, taking the above-mentioned implementation manner 2 as an example, there are overlapping film grains in the image region corresponding to the multiple film grain models, so the film grain fusion indication information can be applied to the image region corresponding to the multiple film grain models. In turn, the overlapping film grains in the image region can be fused based on the fusion processing manner indicated by the film grain fusion indication information, such as the fusion processing manner of using only the bottommost film grain in the overlapping film grains in the image region, so as to obtain the fused film grains of the image region.

[0126] Or, taking the above-mentioned implementation manner 3 as an example, there are overlapping film grains in the overlapping region between different image regions, so the film grain fusion indication information can be applied to the overlapping region between different image regions. In turn, the overlapping film grains in the overlapping region can be fused based on the fusion processing manner indicated by the film grain fusion indication information, such as the fusion processing manner of taking the cumulative value of the overlapping film grains in the overlapping region, so as to obtain the fused film grains of the overlapping region.

[0127] Or, taking the above-mentioned implementation manner 4 as an example, there are image regions corresponding to multiple film grain models, and there are overlapping film grains in the image regions, so that the film grain fusion indication information can be applied to the image regions. Further, the overlapping film grains in the image regions can be fused based on the fusion manner indicated by the film grain fusion indication information, for example, the overlapping film grains in the image regions are subjected to a fusion manner such as mean value taking, so as to obtain fused film grains of the image regions.

[0128] S104, add the fused film grains to the decoded image.

[0129] For example, the decoding end can add the final film grain values of each image region of the decoded image to the decoded image, and the fused film grains can be added to the image regions corresponding to the overlapping film grains.

[0130] In addition, the method of adding film grains to the decoded image can adopt an addition mode, a multiplication mode, or any other calculation mode. Alternatively, the method of adding film grains to the decoded image can adopt a hybrid mode mixed from multiple calculation modes. In addition, the method of adding film grains to the decoded image can correspond to each image region, or a method of adding film grains to the decoded image can be uniformly used in a complete video frame or a complete video sequence. The method can also be determined by the decoding end according to a preconfigured method.

[0131] Based on the technical solutions provided in the disclosure, for overlapping film grains, film grain fusion indication information can be obtained, and the overlapping film grains can be fused based on the film grain fusion indication information to obtain fused film grains, and the fused film grains can be added to the decoded image. In this way, compared with the related art in which only a fixed processing method for overlapping film grains is defined in different standards or private protocols of various manufacturers, the disclosure can flexibly fuse the overlapping film grains in multiple possible fusion manners, so as to achieve the purpose of more accurately and flexibly simulating the visual effect of film grains, thereby bringing a better viewing experience to the viewer.

[0132] In an example, as shown in FIG. 5, the above-mentioned embodiments can also be described as the following steps S11-S16:

[0133] S11, obtain and parse the code stream to obtain a decoded image.

[0134] S12, obtain film grain model parameters corresponding to the decoded image.

[0135] The film grain model parameters can include one or more sets of film grain model parameters corresponding to each image region of the decoded image.

[0136] S13, synthesizing film grain of the decoded image according to the film grain model parameter.

[0137] S14, obtaining film grain fusion indication information.

[0138] The film grain fusion indication information can be used to indicate indication information of a fusion manner when the film grains overlap.

[0139] S15, performing fusion processing on the overlapping film grains based on the film grain fusion indication information to obtain fused film grains.

[0140] For example, a numerical value of the final fused film grain is obtained.

[0141] S16, adding the final determined film grain of each region to the decoded image.

[0142] The fused film grain can be added to an image region of the corresponding decoded image where the film grains overlap.

[0143] In some embodiments, the disclosure also provides another film grain processing method, which can be applied to the encoding end, as shown in FIG. 6, the method comprises:

[0144] S201, obtaining a source image, and performing denoising processing on the source image to obtain a denoised source image.

[0145] For example, the source image can be an image sequence composed of continuous multiple frames of images, a video image sequence, a single image, a frame of image in the image sequence, or a partial image region in a frame of image.

[0146] Further, the original image can be preprocessed (or denoising processing) to achieve denoising to obtain a denoised source image.

[0147] The denoised source image is an image in which the film grains are removed or attenuated.

[0148] For example, the denoising processing can be a spatial domain denoising method, a transform domain denoising method, or a spatial domain and transform domain collaborative filtering method, etc., which is not limited in the embodiments of the disclosure.

[0149] S202, determining film grain model parameters of the source image and film grain fusion indication information based on the source image and the denoised source image, the film grain fusion indication information being used to indicate a fusion manner of overlapping film grains.

[0150] In some embodiments, the encoding end can first obtain the original film grains in the corresponding image, i.e., the film grains of the source image, according to the difference between the source image and the denoised source image.

[0151] For example, by analyzing the difference between the source image and the denoised source image, the encoder can obtain the original film grain of the corresponding image by calculating the difference value of the corresponding pixel points in the image and other algorithms.

[0152] In some embodiments, the encoder can determine the film grain model parameters of the source image based on the film grain of the source image obtained above. And one or more film grain models can be constructed based on the film grain of the source image and the preset film grain modeling method.

[0153] For example, the encoder can realize the film grain parameterization process through the film grain modeling module of the encoder by analyzing the characteristics of the film grain of the source image, estimate the film grain parameter values, and construct one or more film grain models.

[0154] The characteristics of the film grain can be the characteristics of the film grain in the entire image or the characteristics of the film grain in different regions of the image. The characteristics of the film grain include but are not limited to the shape of the film grain, the intensity of the film grain, the scaling parameter of the film grain, etc.

[0155] In addition, the film grain modeling method can be a frequency domain filtering-based film grain modeling method, an autoregressive coefficient-based film grain modeling method, etc., which is not specifically limited in the present disclosure.

[0156] In some embodiments, the film grain model parameters can include at least one of the following: the number of film grain models, the identification of the film grain model, the application range of the film grain model, the film grain synthesis parameter corresponding to the film grain model, and the way of adding the film grain to the decoded image.

[0157] For example, the film grain model parameters include model parameters for constructing multiple different film grains. And each film grain model is applied to the entire source image or part of the source image.

[0158] In some embodiments, the film grain model parameters of the source image can also be written into the code stream.

[0159] For example, when the one or more film grain models constructed can accurately simulate the original film grain in the source image, the encoder can write the single set or multiple sets of film grain model parameters corresponding to the image into the code stream.

[0160] It should be understood that the method of encoding the image to generate the code stream can adopt possible methods such as the H.26x series standard, which is not specifically limited in the present disclosure.

[0161] In some embodiments, the encoder can determine whether there is overlap in the application of the constructed film grain model. If there is, the encoder determines film grain fusion indication information.

[0162] For example, the encoder can determine whether there is overlap in the application range of the constructed film grain model.

[0163] In constructing the one or more film grain models, the encoder can determine whether the constructed film grain model can accurately simulate the film grain in the region by comparing the number of film grains estimated by the constructed film grain model with the number of original film grains in the region, and thus determine whether to perform the overlap application of the film grain model in the corresponding region.

[0164] When the constructed film grain model cannot accurately simulate the given film grain in the image, the encoder can determine that the constructed film grain model cannot simulate the given film grain in the image, i.e., multiple film grain models need to be constructed to jointly simulate the original film grain in the image. At this time, the encoder determines that the overlap application of the film grain model needs to be performed in the corresponding region of the image.

[0165] In some embodiments, the method for determining the synthesis accuracy of the film grain model can be to calculate the difference between the number of original film grains in the image and the number of film grains estimated by the constructed film grain model, and then determine whether the difference or an intermediate variable derived therefrom is greater than a pre-set threshold. For example, if the determined difference is greater than or equal to the pre-set threshold, it indicates that the constructed film grain model cannot accurately simulate the given film grain in the image. Conversely, if the difference is less than the pre-set threshold, it indicates that the constructed film grain model can accurately simulate the given film grain in the image.

[0166] Further, the film grain fusion indication information can be determined. The film grain fusion indication information is applied to the region of the image where the film grains overlap.

[0167] In some embodiments, the film grain fusion indication information is used to indicate at least one of the following fusion methods:

[0168] only using the topmost film grain;

[0169] only using the bottommost film grain;

[0170] taking the cumulative value of the overlapping film grains;

[0171] taking the mean value of the overlapping film grains;

[0172] only using the film grain with the highest priority;

[0173] only using the most important film grain particles;

[0174] only using the film grain particles indicated by the preset identifier.

[0175] In some embodiments, the film grain fusion indication information can be written into a bitstream.

[0176] For example, when it is determined that film grain model overlap application needs to be performed in an image region, the encoding end can analyze how the two or more film grain models that overlap should be fused to obtain a synthesized film grain that is closest to the original film grain effect, and determine the film grain fusion indication information corresponding to the fusion method. The fusion method of the overlapping film grain can depend on the accuracy of the final film grain synthesis. Then, the encoding end can write the determined film grain fusion indication information corresponding to the fusion method into a bitstream. For example, the bitstream can be an image bitstream or a transport stream or a media file containing the image bitstream.

[0177] In some embodiments, the film grain fusion indication information is contained in a VPS, indicating that the film grain fusion indication information is used to indicate the fusion method of the overlapping film grain in the entire video; or,

[0178] The film grain fusion indication information is contained in an SPS, indicating that the film grain fusion indication information is used to indicate the fusion method of the overlapping film grain in the video image sequence corresponding to the SPS; or,

[0179] The film grain fusion indication information is contained in a PPS or an APS, indicating that the film grain fusion indication information is used to indicate the fusion method of the overlapping film grain in a frame of image corresponding to the PPS or the APS; or,

[0180] The film grain fusion indication information is contained in an SEI message, indicating that the film grain fusion indication information is used to indicate the fusion method of the overlapping film grain in a frame of image corresponding to the SEI message.

[0181] In addition, the detailed description of steps S201-S202 can refer to the related description of steps S101-S104 above, which will not be repeated here.

[0182] Based on the above embodiments, in the case of multiple film grain model overlap application in the encoding process, how the overlapping film grain should be fused can be determined, and the corresponding film grain fusion indication information can be determined to indicate the fusion method of the overlapping film grain. In this way, the appropriate fusion method can be flexibly selected from a variety of possible fusion methods to fuse the overlapping film grain, so as to achieve the purpose of more accurate and flexible simulation of the visual effect of the film grain, thereby bringing a better viewing experience to the viewer.

[0183] An example, as shown in FIG. 7, the above embodiment can also be described as the following steps S21-S28:

[0184] S21, obtaining a source image.

[0185] S22, performing denoising processing on the source image to obtain a denoised source image.

[0186] S23, determining the original film grain of the source image according to the difference between the source image and the denoised source image.

[0187] S24, constructing a single or multiple film grain model according to the characteristics of the original film grain of the source image.

[0188] S25, writing the film grain model parameters of the single or multiple film grain model into a bitstream.

[0189] S26, determining whether there is an overlapping application scenario of the constructed film grain model.

[0190] If there is, the following step S27 is performed.

[0191] If there is not, the following step S28 is performed.

[0192] S27, writing the identification of the fusion mode of the film grain in the image when the film grain is overlapped, i.e., the above film grain fusion indication information into the bitstream.

[0193] In some embodiments, as shown in Table 1 below, the present disclosure provides a syntax of the above SEI message.

[0194] Table 1

[0195] In some embodiments, the film grain fusion indication information is contained in the SEI message, indicating that the film grain fusion indication information is used to indicate the fusion mode of the overlapped film grain in the image corresponding to the SEI message.

[0196] Wherein, fgr_cancel_flag: when the value is equal to 1, it is used to indicate that the film grain region feature SEI message cancels the persistence of any previous film grain region feature SEI message and does not use (or turn off) the related SEI function. Or, when the value of the fgr_cancel_flag field is equal to 0, it indicates that the film grain region feature information immediately follows.

[0197] fgr_single_model_per_picture_flag: when equal to 1, indicates that there is only one film grain model identifier in the decoded picture associated with this SEI message. When equal to 0, indicates that there are two or more film grain model identifiers in the decoded picture associated with this SEI message, i.e. different film grain models are used in different regions of the decoded picture and are indicated by different film grain model identifiers.

[0198] fgr_model_id: indicates the type of film grain synthesis model used. For example, when the value is different, it can be a frequency filtering model or an auto-regression model. The present disclosure does not make specific limitations.

[0199] fgr_model_parameters(): indicates the film grain synthesis parameters used in the decoded picture associated with this film grain characteristic SEI message. In some embodiments, the film grain synthesis parameters can be derived from the film grain parameters defined in the Film Grain characteristic SEI message in the H.274 standard or from the film grain parameters defined in the AV1 standard.

[0200] fgr_single_blendng_mode_per_picture_flag: when equal to 1, indicates that only one blending mode is supported when the synthesized film grain is added to the decoded picture associated with this SEI message. When equal to 0, indicates that two or more blending modes are supported when the synthesized film grain is added to the decoded picture associated with this SEI message, i.e. different blending modes are used in different regions of the decoded picture.

[0201] fgr_blending_mode_id: indicates the blending mode used to add the synthesized film grain to the decoded picture after film grain synthesis processing. It should be understood that the method of adding the synthesized film grain to the decoded picture can use an additive mode or a multiplicative mode or any other existing calculation method, and the present disclosure does not make specific limitations.

[0202] fgr_overlapping_mix_mode_id: when film grain model overlap application needs to be performed in the corresponding region of the picture, indicates how the two or more overlapping film grain models should be fused to obtain the final film grain value in the film grain model overlap region.

[0203] The fusion method for fusing the overlapping film grain includes but is not limited to at least one of the following methods:

[0204] only the topmost film grain particle is used;

[0205] only the bottommost film grain particle is used;

[0206] taking the cumulative value of overlapping film grain particles;

[0207] taking the mean value of overlapping film grain particles;

[0208] only the highest priority film grain particle is used;

[0209] only the most important film grain particle is used;

[0210] only the film grain particle indicated by the preset identifier is used.

[0211] fgr_region_information_present_flag: indicates whether the film grain synthesis is applied to a local region in the picture to be decoded associated with the SEI message.

[0212] fgr_active_regions_number_minus1: this field, after being incremented by 1, indicates the number of regions in the picture to be decoded associated with the SEI message to which the region-adaptive film grain synthesis is applied.

[0213] fgr_active_models_number_minus1: this field, after being incremented by 1, indicates the number of film grain models applied to a specific region in the picture to be decoded associated with the SEI message.

[0214] fgr_persistence_flag: used to indicate the persistence of the current film grain region characteristics SEI message. When fgr_persistence_flag is equal to 1, it indicates that the film grain region characteristics SEI message is not only applicable to the current decoded picture, but also applicable to the subsequent decoded pictures within the persistence scope (peisistence_scope). When fgr_persistence_flag is equal to 0, it indicates that the current film grain region characteristics SEI message is only applicable to the current decoded picture. For example, the persistence scope (peisistence_scope) of the film grain region characteristics SEI message can be an access unit (AU), a coded video sequence (CVS), or unspecified, which is not specifically limited in the disclosure.

[0215] In some embodiments, the above field information can be freely combined or appropriately adjusted in different application scenarios, which will not be listed one by one here.

[0216] The above describes the scheme of the embodiments of the present disclosure mainly from the perspective of the method. It can be understood that the film grain processing apparatus includes at least one of a hardware structure and a software module corresponding to each function to implement the above functions. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present disclosure.

[0217] It can be understood that the film grain processing apparatus includes a hardware structure and / or a software module corresponding to each function to implement the above functions. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present disclosure.

[0218] The embodiments of the present disclosure can divide the film grain processing apparatus into functional modules according to the above method embodiments. For example, each functional module can correspond to each function, or two or more functions can be integrated into one functional module. The above integrated module can be implemented in the form of hardware or software. It should be noted that the division of the modules in the embodiments of the present disclosure is illustrative, and is only a logical functional division. When actually implemented, there can be another division manner. The following takes the example of dividing each functional module corresponding to each function.

[0219] FIG. 8 is a structural schematic diagram of a film grain processing apparatus provided by the embodiments of the present disclosure, which is applied to a decoding end and can execute the film grain processing method provided by the above method embodiments. As shown in FIG. 8, the film grain processing apparatus 800 includes an acquisition module 801 and a synthesis module 802. In other embodiments, the film grain processing apparatus 800 further includes an adding module 803.

[0220] The acquisition module 801 is configured to acquire a decoded image and a film grain model parameter of the decoded image.

[0221] The synthesis module 802 is configured to synthesize a film grain of the decoded image based on the film grain model parameter.

[0222] The acquisition module 801 is further configured to acquire film grain fusion indication information, and perform fusion processing on the overlapped film grains based on the film grain fusion indication information to obtain fused film grains.

[0223] The adding module 803 is configured to add the fused film grains into the decoded image.

[0224] In some embodiments, the film grain model parameters include model parameters for constructing a plurality of different film grains, each film grain model is applied to the entire decoded image or a partial image region of the decoded image, and the film grain fusion indication information is applied to an image region where the film grains overlap.

[0225] In some embodiments, the acquisition module 801 is further configured to acquire a bitstream, and acquire the decoded image by parsing the bitstream.

[0226] In some embodiments, the film grain model parameters are included in the bitstream.

[0227] In some embodiments, the film grain fusion indication information is included in the bitstream.

[0228] In some embodiments, the film grain model parameters include at least one of the following: a number of film grain models, a film grain model identifier, an application range of a film grain model, a film grain synthesis parameter corresponding to a film grain model, and a manner in which a film grain is added to a decoded image.

[0229] In some embodiments, the film grain fusion indication information is used to indicate at least one of the following fusion manners:

[0230] only using the uppermost film grain;

[0231] only using the lowermost film grain;

[0232] taking a cumulative value of the overlapped film grains;

[0233] taking a mean value of the overlapped film grains;

[0234] only using the film grain with the highest priority;

[0235] only using the film grain with the highest importance;

[0236] only using the film grain indicated by a preset identifier.

[0237] In some embodiments, the film grain fusion indication information is included in a video parameter set (VPS), and indicates that the film grain fusion indication information is used to indicate a fusion manner of overlapped film grains in an entire video; or,

[0238] The film grain fusion indication information is contained in a sequence parameter set (SPS), and indicates that the film grain fusion indication information is used to indicate a fusion manner of overlapped film grains in a video image sequence corresponding to the SPS; or

[0239] The film grain fusion indication information is contained in a picture parameter set (PPS) or an adaptive parameter set (APS), and indicates that the film grain fusion indication information is used to indicate a fusion manner of overlapped film grains in a frame of image corresponding to the PPS or the APS; or

[0240] The film grain fusion indication information is contained in a supplemental enhancement information (SEI) message, and indicates that the film grain fusion indication information is used to indicate a fusion manner of overlapped film grains in a frame of image corresponding to the SEI message.

[0241] For more details of the above-mentioned obtaining module 801, synthesizing module 802, and adding module 803, and more details of each technical feature and beneficial effects, please refer to the above-mentioned corresponding method embodiment part, which will not be repeated here.

[0242] FIG. 9 is a structural schematic diagram of another film grain processing device provided by the embodiment of the present disclosure, which is applied to an encoding end and can execute the film grain processing method provided by the above-mentioned method embodiment. As shown in FIG. 9, the film grain processing device 900 includes an obtaining module 901 and a determining module 902. In another embodiment, the film grain processing device 900 further includes a processing module 903.

[0243] The obtaining module 901 is configured to obtain a source image, and perform denoising processing on the source image to obtain a denoised source image.

[0244] The determining module 902 is configured to determine film grain model parameters of the source image and film grain fusion indication information based on the source image and the denoised source image, and the film grain fusion indication information is used to indicate a fusion manner of overlapped film grains.

[0245] In some embodiments, the film grain model parameters include model parameters used to construct a plurality of different film grains, each film grain model is applied to the entire decoded image or a partial region of the decoded image, and the film grain fusion indication information is applied to an image region where there are overlapped film grains.

[0246] In some embodiments, the processing module 903 is configured to write the film grain model parameters of the source image into a bitstream.

[0247] In some embodiments, the processing module 903 is further configured to write the film grain fusion indication information into the bitstream.

[0248] In some embodiments, the processing module 903, the film grain model parameter comprises at least one of the following: the number of film grain models, the identification of the film grain model, the application range of the film grain model, the film grain synthesis parameter corresponding to the film grain model, and the way of adding the film grain into the decoded image.

[0249] In some embodiments, the processing module 903, the film grain fusion indication information is used to indicate at least one of the following fusion methods:

[0250] only using the uppermost film grain;

[0251] only using the bottommost film grain;

[0252] taking the cumulative value of the overlapping film grains;

[0253] taking the mean value of the overlapping film grains;

[0254] only using the film grain with the highest priority;

[0255] only using the film grain with the highest importance;

[0256] only using the film grain indicated by the preset identification.

[0257] In some embodiments, the processing module 903, the film grain fusion indication information is contained in the video parameter set VPS, indicating that the film grain fusion indication information is used to indicate the fusion method of the overlapping film grains in the entire video; or,

[0258] the film grain fusion indication information is contained in the sequence parameter set SPS, indicating that the film grain fusion indication information is used to indicate the fusion method of the overlapping film grains in the video image sequence corresponding to the SPS; or,

[0259] the film grain fusion indication information is contained in the picture parameter set PPS or the adaptive parameter set APS, indicating that the film grain fusion indication information is used to indicate the fusion method of the overlapping film grains in the corresponding one frame image of the PPS or the APS; or,

[0260] the film grain fusion indication information is contained in the supplemental enhancement information SEI message, indicating that the film grain fusion indication information is used to indicate the fusion method of the overlapping film grains in the corresponding one frame image of the SEI message.

[0261] For more detailed descriptions of the above-mentioned acquisition module 901, determination module 902 and processing module 903, as well as more detailed descriptions of various technical features and beneficial effects, please refer to the above-mentioned corresponding method embodiment part, which will not be repeated here.

[0262] It should be noted that the modules in FIG. 8 or FIG. 9 can also be referred to as units, for example, the sending module can be referred to as a sending unit. In addition, in the embodiments shown in FIG. 8 or FIG. 9, the names of the various modules can also be different from those shown in the figure, for example, the sending module can also be referred to as a communication module, and the receiving module can also be referred to as a communication module.

[0263] Each unit or module in FIG. 8 or FIG. 9, if implemented in the form of a software functional module and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present disclosure essentially or substantially, or all or part of the technical solutions, can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods in the various embodiments of the present disclosure. The storage medium storing the computer software product includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media capable of storing program codes.

[0264] In the case of implementing the functions of the above-mentioned integrated modules in the form of hardware, the embodiments of the present disclosure provide a structural diagram of a communication device. As shown in FIG. 10, the communication device 1000 includes a processor 1002, a communication interface 1003, and a bus 1004. Optionally, the communication device 1000 can also include a memory 1001.

[0265] The processor 1002 can be various exemplary logical blocks, modules and circuits described in combination with the content of the present disclosure. The processor 1002 can be a central processing unit, a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in combination with the content of the present disclosure. The processor 1002 can also be a combination of computing functions, such as one or more microprocessor combinations, DSP and microprocessor combinations, etc.

[0266] The communication interface 1003 is used to connect with other devices through a communication network. The communication network can be an Ethernet, a wireless access network, a wireless local area network (WLAN), etc.

[0267] The memory 1001 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this.

[0268] As a possible implementation, the memory 1001 can exist independently of the processor 1002, and the memory 1001 can be connected to the processor 1002 through the bus 1004, for storing instructions or program codes. When the processor 1002 invokes and executes the instructions or program codes stored in the memory 1001, the method provided by the embodiments of the present disclosure can be implemented.

[0269] In another possible implementation, the memory 1001 can also be integrated with the processor 1002.

[0270] The bus 1004 can be an extended industry standard architecture (EISA) bus or the like. The bus 1004 can be divided into an address bus, a data bus, a control bus, and the like. For the convenience of representation, only one thick line is shown in FIG. 10, but it does not mean that there is only one bus or only one type of bus.

[0271] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of functional modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device or apparatus is divided into different functional modules to complete all or part of the above described functions.

[0272] The embodiments of the present disclosure further provide a computer readable storage medium. All or part of the flow of the above-mentioned method embodiments can be directed by computer instructions to complete the relevant hardware, and the program can be stored in the above-mentioned computer readable storage medium. When the program is executed, it can include the flow of each method embodiment as described above. The computer readable storage medium can be the memory of any of the preceding embodiments. The above-mentioned computer readable storage medium can also be an external storage device of the above-mentioned device or apparatus, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the above-mentioned device or apparatus. Further, the above-mentioned computer readable storage medium can include both the internal storage unit of the above-mentioned device or apparatus and the external storage device. The above-mentioned computer readable storage medium is used to store the above-mentioned computer program and other programs and data required by the above-mentioned device or apparatus. The above-mentioned computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0273] The embodiments of the present disclosure further provide a computer program product, which contains a computer program, and when the computer program product runs on a computer, it makes the computer execute any method provided in the above embodiments.

[0274] Although the present disclosure is described herein in conjunction with various embodiments, it is understood that other variations of the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed disclosure, from an inspection of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and the indefinite articles "a" or "an" do not exclude a plurality. A single processor or other unit can fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to an advantage.

[0275] Although the present disclosure is described herein in conjunction with specific features and embodiments thereof, it is understood that modifications and combinations can be made thereto within the spirit and scope of the disclosure, which are not described explicitly herein. Accordingly, the description and drawings are to be regarded as illustrative in nature and are not to be considered as limiting the scope of the disclosure as defined in the appended claims. Obviously, various modifications and changes can be made to the present disclosure by those skilled in the art without departing from the spirit and scope of the present disclosure. Thus, it is intended that the present disclosure cover modifications and variations of this disclosure provided they come within the scope of the appended claims and their equivalents. It is therefore submitted that the disclosure be limited as by the appended claims.

[0276] The above merely provides a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and any change or replacement within the technical scope disclosed by the present disclosure should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A film grain processing method, characterized in that: The method comprises: Obtaining a decoded image and film grain model parameters of the decoded image; synthesizing film grain of the decoded image based on the film grain model parameters; Acquiring film grain fusion indication information, and fusing the overlapping film grains based on the film grain fusion indication information to obtain fused film grains; The fused film grain is added to the decoded image.

2. The method according to claim 1, characterized in that The film grain model parameters include model parameters for constructing a plurality of different film grains, each film grain model is applied to the entire decoded image or a partial image area of ​​the decoded image, and the film grain fusion indication information is applied to image areas with overlapping film grains.

3. The method according to claim 1, characterized in that The obtaining of the decoded image comprises: Obtain a code stream, and obtain the decoded image by parsing the code stream.

4. The method according to claim 3, characterized in that The film grain model parameters are included in the bitstream.

5. The method according to claim 3, characterized in that The film grain fusion indication information is included in the bitstream.

6. The method according to claim 1, characterized in that The film grain model parameters include at least one of the following: the number of film grain models, a film grain model identifier, an application range of the film grain model, film grain synthesis parameters corresponding to the film grain model, and a method of adding film grain to a decoded image.

7. The method according to claim 1, characterized in that The film grain fusion indication information is used to indicate at least one of the following fusion methods: Only the top layer of film grain is used; Only the bottom layer of film grain is used; taking a cumulative value of the overlapping film grains; averaging the overlapping film grains; Only the highest priority film grain is used; Only the most significant film grain is used; Use only the film grain indicated by the preset logo.

8. The method according to claim 1, characterized in that The film grain fusion indication information is included in the video parameter set VPS, indicating that the film grain fusion indication information is used to indicate a fusion method of overlapping film grains in the entire video; or, The film grain fusion indication information is included in a sequence parameter set SPS, indicating that the film grain fusion indication information is used to indicate a fusion method for overlapping film grains in a video image sequence corresponding to the SPS; or The film grain fusion indication information is included in the picture parameter set PPS or the adaptive parameter set APS, indicating that the film grain fusion indication information is used to indicate a fusion method for overlapping film grains in a frame of image corresponding to the PPS or the APS; or The film grain fusion indication information is included in the supplemental enhancement information SEI message, indicating that the film grain fusion indication information is used to indicate a fusion method for overlapping film grains in a frame of image corresponding to the SEI message.

9. A film grain processing method, characterized in that: The method comprises: Acquire a source image, and perform denoising on the source image to obtain a denoised source image; Based on the source image and the denoised source image, film grain model parameters of the source image and film grain fusion indication information are determined, where the film grain fusion indication information is used to indicate a fusion method for overlapping film grains.

10. The method according to claim 9, characterized in that The film grain model parameters include model parameters for constructing a plurality of different film grains, each film grain model is applied to the entire decoded image or a partial area of ​​the decoded image, and the film grain fusion indication information is applied to image areas with overlapping film grains.

11. The method according to claim 9, characterized in that The method further comprises: The film grain model parameters of the source image are written into a bitstream.

12. The method according to claim 9, characterized in that The method further comprises: The film grain fusion indication information is written into the bitstream.

13. The method according to claim 9, characterized in that The film grain model parameters include at least one of the following: the number of film grain models, a film grain model identifier, an application range of the film grain model, film grain synthesis parameters corresponding to the film grain model, and a method of adding film grain to a decoded image.

14. The method according to claim 9, characterized in that The film grain fusion indication information is used to indicate at least one of the following fusion methods: Only the top layer of film grain is used; Only the bottom layer of film grain is used; taking a cumulative value of the overlapping film grains; averaging the overlapping film grains; Only the highest priority film grain is used; Only the most significant film grain is used; Use only the film grain indicated by the preset logo.

15. The method according to claim 9, characterized in that The film grain fusion indication information is included in the video parameter set VPS, indicating that the film grain fusion indication information is used to indicate a fusion method of overlapping film grains in the entire video; or, The film grain fusion indication information is included in a sequence parameter set SPS, indicating that the film grain fusion indication information is used to indicate a fusion method for overlapping film grains in a video image sequence corresponding to the SPS; or The film grain fusion indication information is included in the picture parameter set PPS or the adaptive parameter set APS, indicating that the film grain fusion indication information is used to indicate a fusion method for overlapping film grains in a frame of image corresponding to the PPS or the APS; or The film grain fusion indication information is included in the supplemental enhancement information SEI message, indicating that the film grain fusion indication information is used to indicate a fusion method for overlapping film grains in a frame of image corresponding to the SEI message.

16. A film grain processing device, characterized in that: include: memory and processor; The memory is coupled to the processor; The memory is used to store instructions executable by the processor; When the processor executes the instructions, the method according to any one of claims 1 to 15 is performed.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a processor, the processor executes the method according to any one of claims 1 to 15.

18. A computer program product, characterized in that The computer program product comprises a computer program, which, when run on a computer, causes the computer to perform the method according to any one of claims 1 to 15 .

Images