Decoding method, coding method, electronic equipment and storage medium

Abstract

The invention provides a decoding method, a coding method, electronic equipment and a storage medium. The method comprises the following steps: decoding a code stream to obtain a decoded image; obtaining film particle adaptive indication information corresponding to the decoded image; determining film particle parameters corresponding to the decoded image according to the film particle adaptive indication information; synthesizing film particles corresponding to the decoded image according to the film particle parameters; and adding the film particles into the decoded image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, for example, to a decoding method, an encoding method, an electronic device and a storage medium. Background Art

[0002] Film grain technology is one of the technologies being discussed and studied in the current international video codec standard organization. The film grain modeling and synthesis methods defined in different standards are not exactly the same, and the support for film grain is also different. In the encoding and decoding process of images or videos, a single film grain modeling and synthesis method is usually used, which cannot be guaranteed to be applicable to different types of film grain modeling and synthesis, resulting in poor image quality. For example, the film grain model used has different effects on different types of film grains, and different film grain models have different effects on the same film grain; film grain synthesis works better for high-resolution images, but not for low-resolution images; different regions of the same frame of an image have different encoding methods and parameters due to different textures or brightness, and a film grain model produces different visual effects in different regions of the same frame of an image, etc. Summary of the invention

[0003] The present application provides a decoding method, an encoding method, an electronic device and a storage medium.

[0004] The embodiment of the present application provides a decoding method, which is applied to a decoding end, including:

[0005] Decoding the code stream to obtain a decoded image;

[0006] Obtaining film grain adaptive indication information corresponding to the decoded image;

[0007] Determining film grain parameters corresponding to the decoded image according to the film grain adaptation indication information;

[0008] synthesizing film grains corresponding to the decoded image according to the film grain parameters;

[0009] The film grain is added to the decoded image.

[0010] The embodiment of the present application provides an encoding method, which is applied to an encoding end, including:

[0011] Encode the image to generate a code stream;

[0012] modeling film grain in the image based on film grain characteristics in the image;

[0013] Film grain adaptation indication information is determined and the film grain adaptation indication information is written into the bitstream.

[0014] An embodiment of the present application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned decoding method or encoding method when executing the program.

[0015] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned decoding method or encoding method is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a film grain system framework provided by an embodiment;

[0017] Figure 2 A flowchart of a decoding method provided by an embodiment;

[0018] Figure 3 A flowchart of an encoding method provided by an embodiment;

[0019] Figure 4 A flowchart of decoding a video image code stream provided by an embodiment;

[0020] Figure 5 A processing flow chart of film grain adaptive indication information provided by an embodiment when the film grain model adaptive indication information is film grain model adaptive indication information;

[0021] Figure 6 A processing flow chart of film grain adaptive indication information provided by an embodiment when the film grain area adaptive indication information is film grain area adaptive indication information;

[0022] Figure 7 A processing flow chart of film grain adaptive indication information provided by an embodiment when the film grain layered adaptive indication information is film grain layered adaptive indication information;

[0023] Figure 8 A flowchart of encoding a video image code stream provided by an embodiment;

[0024] Fig. 9 A schematic diagram of the structure of a decoding device provided by an embodiment;

[0025] Fig.10 A schematic diagram of the structure of an encoding device provided by an embodiment;

[0026] Fig.11 A schematic diagram of the hardware structure of an electronic device provided by an embodiment. DETAILED DESCRIPTION

[0027] The present application is described below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily. It should also be noted that, for ease of description, only the parts related to the present application rather than all structures are shown in the accompanying drawings.

[0028] Film grain is produced during the physical process of photographic film exposure and development (exposure and development of silver halide crystals) and is widely present in film and television content. However, digital sensors do not go through the above process, so what is produced is noise-free digital video without film grain. Many content creators, especially in the film industry, generally believe that film grain is a pleasant noise that enhances the natural appearance of video content. Re-adding film noise to video can improve the visual experience and is generally considered to be part of the creative intent.

[0029] In the existing methods, film grain synthesis works better for high-resolution images, but not so well for low-resolution images; different regions of the same frame of image have different textures or brightness, so the encoding methods and parameters used are also different, which leads to different visual effects produced by a film grain model in different regions of the same frame of image, etc. The present application provides a decoding method and encoding method, which can perform adaptive film grain modeling and synthesis according to the characteristics of film grain, more accurately simulate the real film grain feeling, and thus bring better viewing experience to viewers.

[0030] Figure 1 A schematic diagram of a film particle system framework provided by an embodiment, such as Figure 1 As shown in FIG, in the video codec standard formulated by JVET, the encoder first compresses and encodes the input source video image to obtain an encoded video stream. On the other hand, the denoising module preprocesses the input source video image to achieve denoising (the encoder itself can also be used to perform the denoising process), and outputs a video image sequence after the film grain is removed or attenuated. The film grain modeling process is also the film grain parameterization process. By analyzing the difference between the denoised video and the source video, the film grain model parameter value is estimated. The film grain parameters can be transmitted to the decoding end together with the encoded video stream, or provided to the decoding end through some external transmission method.

[0031] At the decoding end, the film grain parameter values ​​are parsed and input into the film grain synthesis process to generate simulated film grains. Before the decoded video is output, the film grains are added to the decoded video to output a decoded reconstructed video with simulated film grains.

[0032] In addition to the film grain system framework shown in the figure above, there are other film grain system framework implementations. The main difference lies in the encoding end. For example, the encoding end does not encode the source video image sequence to obtain the encoded video stream, but can encode the denoised video image sequence to obtain the encoded video; or when the source video does not contain film grains, the encoding end can transmit the predetermined film grain parameters to the decoding end for film grain synthesis. In this case, even if there is no film grain in the source video, film grains can be added to the video at the decoder end to achieve a certain effect.

[0033] At the 29th JVET meeting, the proposal JVET-AC0199 provided film grain quality assessment results obtained by frequency-based methods and autoregression-based methods, which mentioned: 1. For videos with heavy film grain (heavy film grain), the synthesized grain is coarse and has obvious repetitive patterns. 2. For videos with light film grain (light / low film grain), the synthesized grain effect is very good. 3. The frequency-based FGS method and the autoregression (AR)-based FGS method show different results in different situations. It can be seen that a film grain synthesis method cannot produce relatively good results for different types of film grain at the same time.

[0034] It is also mentioned in the output document JVET-AC2020: The greater the image resolution, the greater the perception of film grain. Film grain is clearly visible in movies and high-definition images; it gradually loses its importance in standard-definition images and becomes imperceptible in smaller formats. Both the H.265 / HEVC standard and the H.266 / VVC standard already support the concept of "multi-layer video coding / scalable video coding". The video bitstream may include a base layer (Base Layer) sub-stream and at least one enhancement layer (Enhancement Layer) sub-stream, and the enhancement layer sub-stream may be an independent enhancement layer sub-stream or a dependent enhancement layer sub-stream. The base layer sub-stream can be decoded independently to provide low-quality video content, the independent enhancement layer sub-stream can be decoded independently to provide higher-quality video content, and the dependent enhancement layer sub-stream needs to be decoded together with the base layer sub-stream to provide higher-quality video content. Therefore, in the scalable video coding scenario, film grain is more valuable for high-quality video images output by higher enhancement layers, and is of little significance for low-quality video images output by the base layer.

[0035] In addition, different areas of the same frame of video image may have different textures (such as the background part and the key part of the picture) or different brightness (bright areas and dark areas), and because the encoding parameters used may be different, different film grain synthesis methods may produce different visual effects in different areas of the same frame of image.

[0036] Usually, only a single film grain processing method is defined in different standards or private protocols of various manufacturers, without considering the film grain characteristics in video images. Moreover, film grain processing is often adjusted for a video sequence or a frame of video images, and the visual effects in different areas of a frame of image or with different image texture characteristics may be different. In addition, in scalable coding scenarios, the image quality after film grain processing cannot be guaranteed for images of different qualities output at different layers.

[0037] Figure 2 A flowchart of a decoding method provided by an embodiment, such as Figure 2 As shown, the method provided in this embodiment includes steps 110 to 150.

[0038] In step 110, the code stream is decoded to obtain a decoded image.

[0039] In step 120, film grain adaptive indication information corresponding to the decoded image is obtained.

[0040] In step 130, film grain parameters corresponding to the decoded image are determined according to the film grain adaptation indication information.

[0041] In step 140, film grains corresponding to the decoded image are synthesized according to the film grain parameters.

[0042] In step 150, the film grain is added to the decoded image.

[0043] In this embodiment, the decoding end obtains a code stream to be decoded, parses the code stream, and obtains a decoded image corresponding to the code stream; obtains film grain adaptation indication information corresponding to the decoded image, where the film grain adaptation indication information may be included in the code stream or may be obtained in other preconfigured ways; then, determines film grain parameters corresponding to the decoded image according to the film grain adaptation indication information, which may be a set of film grain parameters or multiple sets of film grain parameters; synthesizes film grains corresponding to the decoded image according to a film grain synthesis method indicated by an identifier in the film grain parameters, and adds the film grains to the decoded image.

[0044] In one embodiment, the film grain adaptive indication information includes at least one of the following: film grain model adaptive indication information, film grain region adaptive indication information, and film grain layer adaptive indication information.

[0045] In one embodiment, the film grain adaptation indication information includes at least one of the following:

[0046] Film grain adaptive enable flag, used to indicate whether to enable film grain adaptive processing;

[0047] Film grain adaptive flag, used to indicate whether to enable film grain adaptive processing.

[0048] In one embodiment, the film grain parameters include film grain model parameters and film grain processing parameters, and the film grain processing parameters correspond to the film grain processing method corresponding to the film grain model parameters.

[0049] In one embodiment, the film grain adaptation indication information includes at least one of the following:

[0050] Film grain model adaptive enable flag, used to indicate whether to enable film grain model adaptive processing;

[0051] Film grain model adaptive flag, used to indicate whether to enable film grain model adaptive processing;

[0052] Film grain model adaptation mode, which indicates the range in which the film grain model should be applied.

[0053] In one embodiment, obtaining film grain adaptive indication information corresponding to a decoded image includes:

[0054] The film grain model adaptation indication information is determined according to the SEI message type.

[0055] In one embodiment, the film grain adaptation indication information includes film grain model adaptation indication information;

[0056] The determining, according to the film grain adaptation indication information, a film grain parameter corresponding to the decoded image comprises:

[0057] determining whether to switch the film grain model according to the film grain model adaptation indication information;

[0058] If yes, then obtain the film grain parameters corresponding to the switched film grain model;

[0059] Otherwise, obtain the film grain parameters corresponding to the film grain model before switching.

[0060] In one embodiment, the film grain adaptive indication information includes film grain region adaptive indication information;

[0061] The film grain area adaptive indication information includes at least one of the following:

[0062] Film grain area adaptation enable flag, used to indicate whether to enable film grain area adaptation processing;

[0063] Film grain area adaptation flag, used to indicate whether to enable film grain area adaptation processing;

[0064] The region information corresponding to the film grain parameter is used to indicate the region to which the film grain parameter is applied in the decoded image.

[0065] In one embodiment, the film grain adaptive indication information includes film grain region adaptive indication information;

[0066] The determining, according to the film grain adaptation indication information, a film grain parameter corresponding to the decoded image comprises:

[0067] Acquiring film grain parameters corresponding to each region in the decoded image according to the film grain region adaptive indication information;

[0068] The step of synthesizing film grains corresponding to the decoded image according to the film grain parameters includes:

[0069] The film grains corresponding to the corresponding areas are synthesized according to the film grain parameters corresponding to each area.

[0070] In one embodiment, the film grain adaptive indication information includes film grain layered adaptive indication information;

[0071] The film grain layering adaptive indication information includes at least one of the following:

[0072] Film grain layering adaptive enable flag, used to indicate whether to enable film grain layering adaptive processing;

[0073] Film grain layering adaptive flag, used to indicate whether to enable film grain layering adaptive processing;

[0074] Film grain adaptive layering flag, used to indicate whether the layer corresponding to the flag enables film grain adaptive layering processing;

[0075] The layer identification information corresponding to the film grain parameter is used to indicate the layer identification corresponding to the decoded image to which the film grain parameter is applied;

[0076] Image resolution information corresponding to the film grain parameter, used to indicate the image resolution corresponding to the decoded image to which the film grain parameter is applied;

[0077] Film grain layering adaptation mode, which indicates the extent to which the film grain layering adaptation processing is applied.

[0078] In one embodiment, the film grain adaptive indication information includes film grain layered adaptive indication information;

[0079] The determining, according to the film grain adaptation indication information, a film grain parameter corresponding to the decoded image comprises:

[0080] Acquire film grain parameters corresponding to each layer in the decoded image according to the film grain layered adaptive indication information;

[0081] The step of synthesizing film grains corresponding to the decoded image according to the film grain parameters includes:

[0082] The film grains corresponding to the corresponding layers are synthesized according to the film grain parameters corresponding to each layer.

[0083] In one embodiment, the film grain adaptation indication information is included in a SEI message; the SEI message includes at least one of the following:

[0084] Film grain adaptation enabling flag, used to indicate whether the code stream associated with the SEI message allows film grain adaptation processing to be enabled;

[0085] Film grain adaptation flag, used to indicate whether the code stream associated with the SEI message enables film grain model adaptation processing;

[0086] Film grain basic parameters, used to indicate the film grain basic parameters used by the code stream associated with the SEI message;

[0087] Film grain layered adaptive flag, used to indicate whether the code stream associated with the SEI message enables film grain layered adaptive processing;

[0088] Film grain model adaptation flag, used to indicate whether the code stream associated with the SEI message enables film grain model adaptation processing;

[0089] Film grain model adaptation mode, which indicates the application range of the film grain model;

[0090] Film grain layered adaptive mode: used to indicate whether the code stream associated with the SEI message enables the film grain layered adaptive processing mode;

[0091] Film grain area adaptation flag, used to indicate whether the code stream associated with the SEI message enables film grain area adaptation processing;

[0092] The quantity parameter is used to indicate the quantity of the rectangular area for film grain adaptive processing;

[0093] Position parameter, used to indicate the position of the film grain adaptive processing rectangular area;

[0094] Size parameter, used to indicate the size of the rectangular area for film grain adaptive processing;

[0095] Film grain area model parameters, used to indicate the film grain model parameters corresponding to the rectangular area of ​​film grain adaptive processing;

[0096] The film grain area processing parameters are used to indicate the film grain processing parameters corresponding to the rectangular area of ​​film grain adaptive processing.

[0097] Figure 3 A flowchart of an encoding method provided by an embodiment, such as Figure 3 As shown, the method provided in this embodiment includes steps 210 to 230.

[0098] In step 210, the image is encoded to generate a code stream.

[0099] In step 220, film grain in the image is modeled based on film grain characteristics in the image.

[0100] In step 230, film grain adaptation indication information is determined and written into the bitstream.

[0101] In this embodiment, an image is first acquired, and the image may be a single image or a frame of an image in a video sequence; the image is encoded to generate a bitstream, and the film grain characteristics may be based on the film grain characteristics in the entire image, or the film grain characteristics in different regions of the image; the film grain characteristics include but are not limited to the intensity of the film grain, the scaling parameters of the film grain, etc., and the film grain in the image is modeled according to the film grain characteristics in the image; film grain adaptive indication information corresponding to the image is determined according to the adopted adaptive film grain modeling process, and the film grain adaptive indication information is written into the bitstream.

[0102] In one embodiment, the film grain adaptive indication information includes at least one of the following: film grain model adaptive indication information, film grain region adaptive indication information, and film grain layer adaptive indication information.

[0103] In one embodiment, the film grain adaptation indication information includes at least one of the following:

[0104] Film grain adaptive enable flag, used to indicate whether to enable film grain adaptive processing;

[0105] Film grain adaptive flag, used to indicate whether to enable film grain adaptive processing.

[0106] In one embodiment, modeling the film grain in the image according to the characteristics of the film grain in the image includes at least one of the following:

[0107] performing model adaptive film grain modeling on film grains in the image according to film grain characteristics in the image;

[0108] performing regional adaptive film grain modeling on film grain in the image according to film grain characteristics in the image;

[0109] Hierarchical adaptive film grain modeling is performed on film grain in the image according to film grain characteristics in the image.

[0110] The data transmission method is exemplarily described below through specific embodiments.

[0111] The following embodiments will introduce methods of film grain model adaptive processing, film grain region adaptive processing, and film grain layer adaptive processing respectively. The film grain adaptive processing method may include any one or more of the grain model adaptive processing, film grain region adaptive processing, and film grain layer adaptive processing.

[0112] Embodiment 1

[0113] Film grain is difficult to encode using typical video image coding tools. This is because video image compression relies heavily on predictions in the temporal, spatial, or cross-component domains, while film grain is essentially random. To preserve film grain, one approach is to use lower quantization parameters during video compression to better preserve details, but for video applications such as adaptive streaming and broadcasting, a relatively high bit rate is required. Another solution is to model film grain at the encoding end to obtain film grain parameters, and then use the film grain parameters at the decoding end to synthesize the film grain and add it to the decoded video image, which can give full play to the potential of video image compression technology. For example, the High Efficiency Video Coding (H.265 / HEVC) standard and the Versatile Video Coding (H.266 / VVC) series of standards developed by the Joint Video Expert Teams (JVET) under ISO / IEC and ITU use FGC (Film Grain Characteristic) SEI (Supplemental Enhancement Information) messages to indicate the film grain parameters used for synthesizing film grains when decoding and rendering video or image data, thereby achieving the purpose of retaining film grains.

[0114] This embodiment provides a method for decoding a video image code stream, which is applied to a device for decoding a video code stream. The input of the device is a video image code stream or a media file, and the output is a decoded and reconstructed video image. Figure 4 The following is a flowchart of decoding a video image code stream provided by an embodiment. Figure 4 As shown, the method includes:

[0115] Step 1. Obtain a code stream, parse the code stream to obtain a decoded image (i.e., a decoded image);

[0116] The decoding end obtains the code stream to be decoded, parses the code stream to be decoded, and obtains a decoded image corresponding to the code stream to be decoded. Parsing the code stream to obtain the decoded image can be implemented according to the method specified in the video coding standard corresponding to the code stream type.

[0117] It should be noted that the video itself is composed of a series of continuous image sequences. The method in the embodiment of the present application can be applied to a single image or a frame image in a video sequence. Therefore, the concepts of video, video image or image mentioned in the subsequent description can be equivalently replaced.

[0118] Step 2. Obtain film grain adaptive indication information corresponding to the decoded image;

[0119] There is no necessary order between obtaining the film grain adaptive indication information corresponding to the decoded image and parsing the bitstream in step 1 to obtain the decoded image, and they can be performed simultaneously, or the bitstream can be parsed first to obtain the decoded image and then the film grain adaptive indication information is obtained, or the film grain adaptive indication information can be obtained first and then the bitstream can be parsed to obtain the decoded image.

[0120] The film grain adaptation indication information may be a film grain adaptation flag (fg_adaptive_flag) for indicating whether to enable film grain adaptation processing. The film grain adaptation indication information may also include a film grain adaptation enabling flag (fg_adaptive_enabled_flag) for indicating whether to allow the film grain adaptation processing to be enabled; when the value of fg_adaptive_enabled_flag is 1, it indicates that the film grain adaptation processing is allowed to be enabled, and at this time, whether to enable the film grain adaptation processing is further determined by the film grain adaptation flag.

[0121] The film grain adaptive flag should at least include one or more of the film grain model adaptive flag, the film grain region adaptive flag, or the film grain layer adaptive flag. The film grain adaptive enable flag should at least include one or more of the film grain model adaptive enable flag, the film grain region adaptive enable flag, or the film grain layer adaptive enable flag.

[0122] The film grain adaptation indication information may be included in the bitstream. For example: when the film grain model adaptation indication identifier fg_adaptive_flag is included in a video parameter set (VPS), it means that the film grain adaptation indication information is used to indicate whether the entire video enables film grain adaptive processing; when the film grain adaptation indication information is included in a sequence parameter set (SPS), it means that the film grain model adaptation indication information is used to indicate whether the video image sequence corresponding to the sequence parameter set enables film grain model adaptive processing; when the film grain model adaptation indication information is included in a picture parameter set (PPS) or an adaptive parameter set (APS), it means that the film grain model adaptation indication information is used to indicate whether a frame of image corresponding to the picture parameter set or the adaptive parameter set enables film grain model adaptive processing; when the film grain model adaptation indication information is included in an SEI message, it means that the film grain model adaptation indication information is used to indicate whether a frame of image corresponding to the SEI message enables film grain model adaptive processing.

[0123] It should be noted that if the video parameter set includes the film grain adaptation flag vps_fg_adaptive_flag, and the sequence parameter set also includes the film grain model adaptation flag pps_fg_adaptive_flag, if the value of vps_fg_adaptive_flag is inconsistent with the value of pps_fg_adaptive_flag, then whether the sequence corresponding to the sequence parameter set allows the enabling of film grain model adaptive processing may be based on the value of pps_fg_adaptive_flag.

[0124] Different film grain adaptation indication information may be included in one or more positions of a video parameter set VPS, a sequence parameter set SPS, a picture parameter set PPS or an adaptation parameter set APS or a specific SEI message in whole or in part.

[0125] The film grain adaptation indication information may be included in the bitstream or obtained through other preconfigured methods. When the film grain adaptation indication information includes multiple types of film grain adaptation indication information, the multiple types of film grain adaptation indication information may be partially obtained through the bitstream and partially obtained through preconfigured information or other methods.

[0126] Step 3: Obtain (ie, determine) film grain parameters corresponding to the decoded image according to the film grain adaptive indication information.

[0127] The film grain parameters include but are not limited to film grain model parameters and film grain processing parameters corresponding to the film grain processing method indicated by the film grain model parameters. The film grain model parameters may be film grain model identifiers, and the film grain processing parameters may also be film grain synthesis parameters or film grain modeling parameters. For example, when the film grain model identifier indicates that a frequency filtering-based film grain synthesis method is adopted, the corresponding film grain processing parameters may include a luminance bit depth parameter (fg_bit_depth_luma_minus8) of the film grains, a chrominance bit depth parameter (fg_bit_depth_chroma_minus8), an image color gamut (fg_colour_primaries), a color space fg_matric_coeffs, a transfer function (fg_transfer_characteristics) and other parameters corresponding to the film grains, a blending mode parameter (fg_blending_mode_id) of the film grains and the decoded image, a film grain scaling parameter (fg_log2_scale_factor), a model number parameter (fg_num_model_values_minus1[c]) of the film grains for intensity intervals of different color components, a model value (fg_comp_model_value[c][i][j]) of the film grains for each intensity interval of different color components, and the like.

[0128] The film grain parameters corresponding to the decoded image may be a set of film grain parameters or multiple sets of film grain parameters. For example, as described in step 2, when the film grain adaptation indication information includes film grain region adaptation indication information, the film grain parameters corresponding to the decoded image may include multiple sets of film grain parameters corresponding to different regions in the decoded image.

[0129] Step 4: Synthesize film grains corresponding to the decoded image according to the film grain parameters.

[0130] Film grains corresponding to the decoded image are synthesized according to the film grain model identifier included in the film grain parameters and the film grain parameters corresponding to the film grain synthesis method indicated by the film grain model identifier.

[0131] When the film grain adaptation indication information includes film grain region adaptation indication information, the film grains corresponding to the decoded image include film grains corresponding to each region in the decoded image.

[0132] Step 5. Add the film grain to the decoded image.

[0133] The method of adding the film grains to the decoded image may adopt an addition mode or a multiplication mode, etc. The method of adding the film grains to the decoded image may be specified by the mixing mode parameters of the film grains and the decoded image included in the film grain parameters in step 3, or may be determined by the decoding end according to pre-configured parameters.

[0134] Embodiment 2

[0135] Typically, different standards or private protocols of various manufacturers generally only define a specific film grain processing method, without considering providing a film grain model adaptive processing method for the film grain characteristics in the video image to achieve better visual effects.

[0136] This embodiment provides a method for decoding a video image code stream, which is applied to a device for decoding a video code stream. The input of the device is a video image code stream or a media file, and the output is a decoded and reconstructed video image. Figure 5 A processing flow chart of a film grain adaptive indication information provided by an embodiment when the film grain model adaptive indication information is film grain model adaptive indication information, such as Figure 5 As shown, the method includes:

[0137] Step 1. Obtain the code stream and parse the code stream to obtain the decoded image;

[0138] Step 2. Obtain film grain model adaptive indication information corresponding to the decoded image;

[0139] Referring to step 2 in Example 1, the film grain adaptive indication information in this embodiment is film grain model adaptive indication information, and the film grain model adaptive indication information corresponding to the decoded image is obtained. The film grain model adaptive indication information may include a film grain model adaptive flag fg_model_adaptive_flag, which is used to indicate whether film grain model adaptive processing is enabled, and may also include a film grain adaptive enabling flag fg_adaptive_enabled_flag, which is used to indicate whether film grain adaptive processing is allowed to be enabled.

[0140] Table 1 takes the film grain model adaptation enabling flag included in the video parameter set as an example to illustrate the relevant syntax elements and semantic information:

[0141] Table 1 Syntax table of film grain model adaptation enabling flag in video parameter set

[0142]

[0143] When the value of the film grain adaptive enable flag vps_fg_adaptive_enable_flag is 0, it indicates that the video sequence does not allow the film grain adaptive processing to be enabled; when the value of vps_fg_adaptive_enable_flag is 1, it indicates that the video sequence allows the film grain adaptive processing to be enabled.

[0144] Table 2 takes the film grain model adaptation enabling flag and the film grain adaptation flag included in the image parameter set as an example to illustrate the relevant syntax elements and semantic information:

[0145] Table 2 Syntax table of film grain model adaptation enable flag and film grain adaptation flag in image parameter set

[0146]

[0147] When the film grain adaptive enable flag pps_fg_adaptive_enable_flag value is 0, it indicates that the video image does not allow the film grain adaptive processing to be enabled; when the pps_fg_adaptive_enable_flag value is 1, it indicates that the video image allows the film grain adaptive processing to be enabled;

[0148] When the film grain model adaptive flag pps_fg_model_adaptive_flag value is 0, it indicates that the film grain model adaptive processing is enabled for the video image; when the pps_fg_model_adaptive_flag value is 1, it indicates that the film grain model adaptive processing is enabled for the video image.

[0149] The film grain model adaptation indication information may also be a film grain model adaptation mode, which is used to indicate the adaptation range (i.e., application range) of the switched film grain model. Table 3 takes the film grain model adaptation mode included in the image parameter set as an example to illustrate the relevant syntax elements and semantic information:

[0150] Table 3 Syntax table of film grain model adaptive identification in image parameter set

[0151]

[0152] pic_fg_model_adaptive_mode is the adaptive mode of the film grain model. When the value of pic_fg_model_adaptive_mode is 0, it means that the model switch is only for the image corresponding to the current parameter set, that is, the new film grain model and parameters of each switch are only applied to the image corresponding to the current parameter set; when the value of pic_fg_model_adaptive_mode is 1, it means that the model switch is for the image corresponding to the current parameter set and all subsequent images, that is, the film grain model and parameters received after the current parameter set will be applied to the image corresponding to the current parameter set and all subsequent images until other new film grain models and parameters are received;

[0153] The film grain model adaptation indication information may also be implicitly derived and determined through a plurality of SEI message types including different film grain models. For example, the film grain parameters corresponding to the previous decoded image are transmitted through the FGC SEI message, while the film grain parameters corresponding to the current decoded image are transmitted through SEI messages of types such as user_data_unregistered or user_data_registered_itu_t_t35. Therefore, these two different types of SEI messages carrying film grain parameters may be considered as film grain model adaptation indication information.

[0154] Step 3: Obtain film grain parameters corresponding to the decoded image according to the film grain model adaptive indication information.

[0155] Step 3 in this embodiment can be further divided into the following steps 3.1-3.4.

[0156] Step 3.1. Determine whether to switch the film grain model according to the film grain model adaptation indication information, if yes, execute step 3.3, otherwise execute step 3.2;

[0157] Step 3.2. Obtain the film grain parameter value corresponding to the original film grain model;

[0158] The original film grain model refers to the film grain model corresponding to the previously decoded image, and the previously decoded image refers to the decoded image obtained by parsing the code stream before the decoded image. Therefore, obtaining the film grain parameters corresponding to the original film grain model includes the following three situations: one is that the film grain parameter values ​​used in the current decoded image are exactly the same as the film grain parameter values ​​of the previously decoded image, and the film grain parameters corresponding to the previously decoded image are directly obtained; the second situation is that the film grain parameter values ​​used in the current decoded image are different from the film grain parameter values ​​of the previously decoded image, and the film grain parameters corresponding to the current decoded image are directly parsed from the code stream or other out-of-band transmission methods. The third situation is that the film grain parameter values ​​used in the current decoded image are partially different from the film grain parameter values ​​of the previously decoded image, and part of the film grain parameter values ​​corresponding to the current decoded image are obtained from the code stream parsing or other out-of-band transmission methods, and the remaining film grain parameter values ​​are obtained from the film grain parameters corresponding to the previously decoded image.

[0159] Step 3.3. Obtain the film grain parameter value corresponding to the switched film grain model;

[0160] The film grain parameter values ​​corresponding to the switched film grain model are obtained by parsing the bitstream or other out-of-band transmission methods.

[0161] Step 3.4. Determine whether the decoded image has film grain enabled, if so, execute step 4, otherwise return to execute step 1;

[0162] Step 3.4 is an optional step. For example, this step can be performed when it is allowed to add film grains to some images in a video sequence and not to add film grains to some images. Otherwise, step 4 is directly performed.

[0163] When film grains are allowed to be added to some images in a video sequence and not to be added to some images, the film grain parameter value corresponding to the film grain model obtained in step 3.2 or step 3.3 may be empty, which means that film grains are not enabled.

[0164] Step 4: Synthesize film grains corresponding to the decoded image according to the film grain parameter value.

[0165] Step 5. Add the film grain to the decoded image.

[0166] Embodiment 3

[0167] Film grain processing is often performed on a video sequence or a frame of video image, without considering providing a film grain region adaptive processing method for film grain characteristics and image texture characteristics in different regions of a frame of image.

[0168] This embodiment provides a method for decoding a video image code stream, which is applied to a device for decoding a video code stream. The input of the device is a video image code stream or a media file, and the output is a decoded and reconstructed video image. Figure 6 A processing flow chart of a film grain adaptive indication information when the film grain region adaptive indication information is provided in an embodiment, such as Figure 6 As shown, the method includes:

[0169] Step 1. Obtain the code stream and parse the code stream to obtain the decoded image;

[0170] Step 2. Obtain film grain region adaptive indication information corresponding to the decoded image;

[0171] Referring to step 2 in the first embodiment, the film grain adaptive indication information in this embodiment is film grain region adaptive indication information, and the film grain region adaptive indication information corresponding to the decoded image is obtained.

[0172] According to the film grain region adaptive indication information, it can be determined that different regions in the decoded image may correspond to different film grain parameters;

[0173] The film grain area adaptation indication information may be a film grain area adaptation enabling flag, which is used to indicate whether to enable film grain area adaptation processing.

[0174] The film grain region adaptation indication information may be a film grain region adaptation flag fg_region_adaptive_flag indicating whether to enable film grain region adaptation processing for a video image, and the flag may be included in a video parameter set VPS, a sequence parameter set SPS, a picture parameter set PPS or an adaptation parameter set APS, or may be included in a specific SEI message. For example: Table 4 takes the film grain region adaptation enabling flag and the film grain region adaptation flag included in the picture parameter set as an example to illustrate the relevant syntax elements and semantic information.

[0175] Table 4 Syntax table of film grain region adaptation enable flag and film grain adaptation flag in image parameter set

[0176]

[0177] When the film grain adaptive enable flag pps_fg_adaptive_enable_flag value is 0, it indicates that the video image does not allow the film grain adaptive processing to be enabled; when the pps_fg_adaptive_enable_flag value is 1, it indicates that the video image allows the film grain adaptive processing to be enabled;

[0178] When the film grain model adaptation flag pps_fg_region_adaptive_flag value is 0, it indicates that the video image enables film grain region adaptive processing; when the pps_fg_region_adaptive_flag value is 1, it indicates that the video image enables film grain region adaptive processing.

[0179] The film grain area adaptive indication information may also be area information corresponding to a film grain parameter, and the area information corresponding to the film grain parameter is used to indicate the location area where the film grain parameter is applied in the decoded image. The film grain area adaptive indication information may include multiple area information corresponding to different film grain parameters. The multiple area information corresponding to these different film grain parameters may be one or more area information corresponding to each / group of film grain parameters, or may include one / group of main film grain parameters and at least one / group of other film grain parameters corresponding to area information, wherein the one / group of main film grain parameters is applied to other areas in the decoded image except for the area information corresponding to the at least one / group of other film grain parameters. For example: Table 5 takes the area information of film grain adaptive processing included in the image parameter set as an example to illustrate the relevant syntax elements and semantic information.

[0180] Table 5 Syntax table of region information for film grain adaptive processing in image parameter set

[0181]

[0182]

[0183] When the film grain adaptive enable flag pps_fg_adaptive_enable_flag value is 0, it indicates that the video image does not allow the film grain adaptive processing to be enabled; when the pps_fg_adaptive_enable_flag value is 1, it indicates that the video image allows the film grain adaptive processing to be enabled;

[0184] pps_num_of_adaptive_regions: indicates the number of film grain adaptive processing regions;

[0185] fg_region_info[i]: indicates the information of the i-th film grain adaptive processing region.

[0186] The area information can be described by the area position and size, for example, the upper right corner vertex position and length and width can determine a rectangular area, or it can be described by the block size and serial number, such as the position of the 8th 8*8 adaptive processing block. The size of the adaptive processing block can be pre-set or determined by other means, such as determining the size and order of several optional adaptive processing blocks. The size of the adopted adaptive block can be known through a given serial number.

[0187] fg_region_parameters[i]() indicates the film grain parameters corresponding to the i-th film grain adaptive processing region.

[0188] Step 3: Obtain film grain parameters corresponding to each area in the decoded image according to the film grain area adaptive indication information.

[0189] According to the description of step 3 in embodiment 1 and embodiment 2, the film grain parameters corresponding to each region may include the following two situations: the film grain model parameters corresponding to different regions are the same but the film grain modeling parameters are different; or the film grain model parameters and the film grain modeling parameters corresponding to different regions are different.

[0190] Step 4: Synthesize the film grains corresponding to each area of ​​the decoded image according to the film grain parameters.

[0191] Film grains corresponding to each region of the decoded image are synthesized according to the film grain model identifier included in the film grain parameters and the film grain parameters corresponding to the film grain synthesis method indicated by the film grain model identifier.

[0192] When the film grain adaptation indication information includes film grain region adaptation indication information, the film grains corresponding to the decoded image may include different film grains corresponding to different regions in the decoded image.

[0193] Step 5. Add the film grain to the decoded image.

[0194] According to the description of step 3 and step 4, the film grains are added to the corresponding areas of the decoded image.

[0195] Embodiment 4

[0196] This embodiment provides a method for decoding a video image code stream, which is applied to a decoding device for a video code stream. The input of the device is a video image code stream or a media file, and the output is a decoded and reconstructed video image. The method takes into account providing film grain layered adaptive processing for images of different qualities of outputs of different layers in a scalable coding scenario. Figure 7A processing flow chart of a film grain adaptive indication information provided by an embodiment when the film grain layered adaptive indication information is film grain layered adaptive indication information is shown in FIG. Figure 7 As shown, the method includes:

[0197] Step 1. Obtain the code stream and parse the code stream to obtain the decoded image;

[0198] The bitstream in this embodiment is a multi-layer video coding bitstream, which can also be called a scalable video coding bitstream. The multi-layer video coding bitstream can include a base layer (Base Layer) sub-codestream and at least one enhancement layer (Enhancement Layer) sub-codestream, and the enhancement layer sub-codestream can be an independent enhancement layer sub-codestream or a dependent enhancement layer sub-codestream. Taking the H.266 / VVC codec standard as an example, when the value of the syntax element vps_max_layers_minus1 in the video parameter set VPS is not 0, it can be judged that the bitstream is a multi-layer video coding bitstream. Further, when the value of the syntax element vps_independent_layer_flag[i] (i>0) in the video parameter set VPS is 0, it indicates that the i-th enhancement layer is a dependent enhancement layer. When the value of the syntax element vps_independent_layer_flag[i] (i>0) in the video parameter set VPS is 1, it indicates that the i-th enhancement layer is an independent enhancement layer. The base layer sub-codestream can be independently decoded to provide low-quality video content. Generally, the base layer sub-code stream can be independently decoded to provide video image content of lower quality or lower resolution, the independent enhancement layer sub-code stream can be independently decoded to provide video image content of higher quality or higher resolution, and the dependent enhancement layer sub-code stream needs to be decoded together with the base layer sub-code stream to provide video image content of higher quality or higher resolution.

[0199] Therefore, the decoded image obtained by parsing in this step may be a decoded image of the same frame image with different qualities or different resolutions.

[0200] Step 2. Obtain film grain layered adaptive indication information corresponding to the decoded image;

[0201] Referring to step 2 in the first embodiment, the film grain adaptive indication information in this embodiment is film grain layered adaptive indication information, and the film grain layered adaptive indication information corresponding to the decoded image is obtained.

[0202] According to the film grain layered adaptive indication information, it can be determined that the decoded images corresponding to different layers may correspond to different film grain parameters; and the decoded images corresponding to the different layers may have different image qualities or different resolutions.

[0203] The film grain layering adaptive indication information may be a film grain layering adaptive enabling flag, which is used to indicate whether to enable film grain layering adaptive processing.

[0204] The film grain layering adaptation indication information may be a film grain layering adaptation flag fg_layers_adaptive_flag, which is used to indicate whether to enable film grain layering adaptive processing for a video image. When fg_layers_adaptive_flag is 1, it indicates that the code stream enables film grain layering adaptation, and when fg_layers_adaptive_flag is 0, it indicates that the code stream does not enable film grain layering adaptation. The flag may be included in a video parameter set VPS, a sequence parameter set SPS, a picture parameter set PPS, or an adaptive parameter set APS, or may be included in a specific SEI message.

[0205] The film grain layered adaptive indication information may also be layer identification information corresponding to the film grain parameter or image resolution information corresponding to the film grain parameter. The layer identification information corresponding to the film grain parameter is used to indicate the layer identification corresponding to the decoded image to which the film grain parameter is applied. The image resolution information corresponding to the film grain parameter is used to indicate the resolution corresponding to the decoded image to which the film grain parameter is applied. For example: Table 6 takes the layered information of the film grain adaptive processing included in the video parameter set as an example to illustrate the relevant syntax elements and semantic information.

[0206] Table 6 Syntax table of layered information for film grain adaptive processing in video parameter set

[0207]

[0208] vps_max_layers_minus1: plus 1 indicates the maximum number of coding layers contained in the bitstream;

[0209] vps_fg_adaptive_enable_flag: When the value is 0, it means that film grain adaptive processing is not allowed; when the value is 1, it means that film grain adaptive processing is allowed;

[0210] vps_fg_layer_adaptive_flag[i]: When the value is 0, it indicates that the coding layer with layer ID i does not enable film grain adaptive processing; when the value is 1, it indicates that the coding layer with layer ID i enables film grain adaptive processing;

[0211] fg_layer_parameters[i]() indicates the film grain parameters corresponding to the coding layer with layer identifier i.

[0212] The film grain layer adaptive indication information may be layer identification information corresponding to different film grain parameters. For example: when the layer identification is 0, it corresponds to a set of film grain parameters; when the layer identification is 1, it corresponds to another set of film grain parameters; when the layer identification is 2, it corresponds to yet another set of film grain parameters; or when the layer identification is greater than or equal to a given threshold a (0 < a < 6), it corresponds to a set of film grain parameters, and when the layer identification is less than a, it corresponds to another set of film grain parameters or no film grain parameters;

[0213] The film grain layer adaptive indication information may include multiple image resolution information corresponding to different film grain parameters. For example: when the image resolution reaches 720p (1280 * 720), it corresponds to a set of film grain parameters; when the image resolution reaches 1080p (1920 * 1080), it corresponds to another set of film grain parameters; when the image resolution reaches 4k (1920 * 1080), it corresponds to yet another set of film grain parameters;

[0214] The film grain layer adaptive indication information may also be the film grain layer adaptive mode information (i.e., the film grain adaptive layer identification) fg_layer_adaptive_mode. The film grain layer adaptive mode information is used to indicate which layer identification or resolution corresponding decoded images enable adaptive film grain processing. For example: when fg_layer_adaptive_mode is 0, it means that the bitstream applies film grain to all layer identifications or resolution corresponding decoded images; when fg_layer_adaptive_mode is 1, it means that the bitstream enables adaptive film grain processing for the decoded images corresponding to other than the base layer (layer identification is 0); when fg_layer_adaptive_mode is 2, it means that the bitstream only enables adaptive film grain processing for the decoded images corresponding to the highest enhancement layer (the value of the layer identification is the same as the value of vps_max_layers_minus1), etc.

[0215] Step 3. Determine whether to enable adaptive film grain processing for the decoded image. If so, execute Step 4; otherwise, execute Step 1;

[0216] As described in step 2, if the film grain layered adaptive indication information is the layer identification information corresponding to the film grain parameter or the image resolution information corresponding to the film grain parameter, and it can be determined according to the information that at least one decoded image corresponding to the layer identification has no corresponding film grain parameter, or when the film grain layered adaptive indication information is the film grain layered adaptive mode information, it can be determined according to the value of the film grain layered adaptive mode fg_layer_adaptive_mode that at least one decoded image corresponding to the layer identification does not enable film grain processing, it is necessary to determine whether the decoded image enables adaptive film grain processing, and the determination method is: whether the film grain processing is not enabled or there is no corresponding film grain parameter according to the layer identification or resolution corresponding to the decoded image, if the film grain processing is not enabled or there is no corresponding film grain parameter, return to step 1, and the acquired code stream can be the code stream corresponding to the next frame of the decoded image, or the sub-code stream of a different layer of the same frame of the decoded image. The second decoded image obtained by parsing the code stream can be the next frame of the decoded image, or the same frame of the image with different quality or resolution from the decoded image.

[0217] Step 4: Obtain film grain parameters corresponding to the decoded image according to the film grain layered adaptive indication information.

[0218] According to the description in step 2, the film grain layered adaptive indication information may be layer identification information corresponding to the film grain parameters or image resolution information corresponding to the film grain parameters, and the corresponding film grain parameters are obtained according to the layer identification or resolution of the decoded image.

[0219] Step 5. Synthesize film grains corresponding to the decoded image according to the film grain parameters.

[0220] Film grains corresponding to each region of the decoded image are synthesized according to the film grain model identifier included in the film grain parameters and the film grain parameters corresponding to the film grain synthesis method indicated by the film grain model identifier.

[0221] Step 6. Add the film grain to the decoded image.

[0222] Embodiment 5

[0223] This embodiment provides a method for encoding a video image code stream, which is applied to an encoding device for an input video image, wherein the input of the device is a source video image, and the output is a video image code stream or a media file. Figure 8 The following is a flow chart of encoding a video image code stream provided by an embodiment. Figure 8 shown.

[0224] Step 1. Acquire an image, which may be a single image or a frame of image in a video sequence;

[0225] Step 2: Encode the image and generate a bit stream;

[0226] There are many existing technologies for encoding images to generate code streams, such as H.26x series standards, etc. The use of any encoding standard to encode images is not limited here, so it will not be described in detail.

[0227] Step 3. Adaptively model the film grain in the image according to the characteristics of the film grain in the image;

[0228] The film grain characteristics may be based on the film grain characteristics of the entire image, or the film grain characteristics of different regions in the image; the film grain characteristics include but are not limited to the intensity of the film grain, the scaling parameters of the film grain, etc. Adaptive film grain modeling includes but is not limited to model-adaptive film grain modeling, regional-adaptive film grain modeling, layered-adaptive film grain modeling, etc. Model-adaptive film grain modeling refers to selecting the most appropriate film grain modeling method according to the film grain characteristics in the image, including but not limited to: a frequency-based film grain modeling method, an autoregressive coefficient-based film grain modeling method, etc. Regional-adaptive film grain modeling refers to performing film grain modeling according to the film grain characteristics of different regions in the image, and the film grain modeling methods used in different regions may be the same or different. When the same film grain modeling method is used in different regions, the film grain parameters corresponding to different regions may not be exactly the same; layered-adaptive film grain modeling refers to determining which output images need to enable film grain according to the film grain characteristics in the image.

[0229] The layered adaptive film grain modeling is only considered enabled when the image is scalable coded.

[0230] Step 4. Determine film grain adaptive indication information and write it into the bitstream;

[0231] Determining film grain adaptive indication information corresponding to the image according to the adaptive film grain modeling process adopted in step 3, wherein the film grain adaptive indication information may include at least one of model adaptive film grain indication information, region adaptive film grain indication information, and layer adaptive film grain indication information;

[0232] Step 5 outputs the code stream;

[0233] The code stream may be an image code stream or a transport stream or a media file containing an image code stream.

[0234] Embodiment 6

[0235] This embodiment provides an example of an SEI message for implementing film grain adaptive processing. Table 7 is a syntax table of an SEI message for film grain adaptive processing, in which the syntax structure and semantic information may correspond to the combination examples of various situations described in any of the above embodiments.

[0236] Table 7. SEI message syntax for film grain adaptive processing

[0237]

[0238]

[0239] Film grain adaptive SEI message semantic

[0240] fg_adaptive_enabled_flag (i.e., film grain adaptive enable flag): when fg_adaptive_enabled_flag=1, it indicates that the to-be-decoded bitstream associated with the SEI message allows the enabling of film grain adaptive processing; when fg_adaptive_enabled_flag=0, it indicates that the to-be-decoded bitstream associated with the SEI message does not allow the enabling of film grain adaptive processing. The film grain adaptive processing may include at least one of film grain model adaptive processing, film grain region adaptive processing, and film grain layer adaptive processing; correspondingly, the film grain adaptive enable flag may also be at least one of the film grain model adaptive processing flag, the film grain region adaptive processing flag, and the film grain layer adaptive processing flag;

[0241] The SEI message may also include: a film grain adaptation flag, which is used to indicate whether the code stream associated with the SEI message enables film grain adaptation processing;

[0242] fg_characteristic_base_parameters (i.e., film grain base parameters) indicates the basic film grain parameters used by the code stream associated with the SEI message, and the film grain parameters include film grain model parameters and film grain processing parameters. When the film grain model parameter indication can be the same as the film grain parameters defined in the Filmgraincharacteristic SEI message in the H.274 standard or the film grain parameters defined in the AV1 standard. It is not described here. If fg_characteristic_base_parameters() in the SEI message is empty, it means that the film grain parameters associated with the previous code stream are also applicable to the code stream associated with the current SEI message.

[0243] fg_layers_adaptive_flag (i.e., film grain layered adaptive flag): When fg_layers_adaptive_flag = 1, it indicates that the to-be-decoded bitstream associated with the SEI message enables film grain layered adaptive processing; when fg_layers_adaptive_flag = 0, it indicates that the to-be-decoded bitstream associated with the SEI message does not support or enable film grain layered adaptive processing.

[0244] fg_model_adaptive_flag (film grain model adaptive flag): When fg_model_adaptive_flag = 1, it indicates that the to-be-decoded bitstream associated with the SEI message enables film grain model adaptive processing; when fg_model_adaptive_flag = 0, it indicates that the to-be-decoded bitstream associated with the SEI message does not support or enable film grain model adaptive processing.

[0245] fg_model_adaptive_mode: Film grain model adaptive processing mode (film grain model adaptive mode), used to indicate the adaptive range of the film grain model after switching.

[0246] fg_layer_adaptive_mode: indicates the film grain layered adaptive processing mode (film grain layered adaptive mode); the value of fg_layer_adaptive_mode can be used to determine which output images of different quality corresponding to the bitstream need to add film grains. For specific values, please refer to the detailed description of step 3 in Example 4;

[0247] fg_region_adaptive_flag (i.e., film grain region adaptive flag): When fg_region_adaptive_flag=1, it indicates that the to-be-decoded bitstream associated with the SEI message supports enabling film grain region adaptive processing; when fg_region_adaptive_flag=0, it indicates that the to-be-decoded bitstream associated with the SEI message does not support disabling film grain region adaptive processing.

[0248] Num_of_adaptive_rect_regions (i.e., quantity parameter): indicates the number of rectangular regions for film grain adaptive processing;

[0249] adaptive_rect_region_top[i] (i.e. position parameter): indicates the top position of the i-th film grain adaptive processing rectangular region;

[0250] adaptive_rect_region_left[i] (i.e. position parameter): indicates the rightmost position of the i-th film grain adaptive processing rectangular region;

[0251] adaptive_rect_region_width[i] (i.e., size parameter): indicates the width of the i-th film grain adaptive processing rectangular region;

[0252] adaptive_rect_region_height[i] (i.e., size parameter): indicates the height of the i-th film grain adaptive processing rectangular region;

[0253] fg_region_model_parameters[i] (i.e., film grain region model parameters) indicates the film grain parameters corresponding to the i-th film grain adaptive processing rectangular region, and the film grain parameters include film grain model parameters and film grain processing parameters.

[0254] fg_region_parameters[i] (i.e., film grain region processing parameters) indicates the film grain processing parameters corresponding to the i-th film grain adaptive processing rectangular region. The film grain model parameters are consistent with the film model grain parameters contained in fg_characteristic_base_parameters().

[0255] This embodiment only provides an example of the syntax structure and semantic information of the film grain adaptive processing SEI message. In actual applications, only some of the syntax elements and their semantic information may be included, and any method in Embodiment 1 to Embodiment 5 may be implemented.

[0256] The embodiment of the present application also provides a decoding device. Fig. 9 FIG. 1 is a schematic diagram of a decoding device provided by an embodiment. Fig. 9 As shown, the decoding device comprises:

[0257] A decoding module 310, configured to decode the code stream to obtain a decoded image;

[0258] An acquisition module 320 is configured to acquire film grain adaptive indication information corresponding to the decoded image;

[0259] A determination module 330, configured to determine a film grain parameter corresponding to the decoded image according to the film grain adaptation indication information;

[0260] A synthesis module 340, configured to synthesize film grains corresponding to the decoded image according to the film grain parameters;

[0261] The adding module 350 is configured to add the film grains to the decoded image.

[0262] The decoding device of this embodiment performs adaptive film grain modeling and synthesis based on the characteristics of film grains, and more accurately simulates the real film graininess, thereby bringing a better viewing experience to the viewers.

[0263] In one embodiment, the film grain adaptive indication information includes at least one of the following: film grain model adaptive indication information, film grain region adaptive indication information, and film grain layer adaptive indication information.

[0264] In one embodiment, the film grain adaptation indication information includes at least one of the following:

[0265] Film grain adaptive enable flag, used to indicate whether to enable film grain adaptive processing;

[0266] Film grain adaptive flag, used to indicate whether to enable film grain adaptive processing.

[0267] In one embodiment, the film grain parameters include film grain model parameters and film grain processing parameters, and the film grain processing parameters correspond to the film grain processing method corresponding to the film grain model parameters.

[0268] In one embodiment, the film grain adaptation indication information includes at least one of the following:

[0269] Film grain model adaptive enable flag, used to indicate whether to enable film grain model adaptive processing;

[0270] Film grain model adaptive flag, used to indicate whether to enable film grain model adaptive processing;

[0271] Film grain model adaptation mode, which indicates the range in which the film grain model should be applied.

[0272] In one embodiment, the acquisition module 320 is configured to determine the film grain model adaptation indication information according to the SEI message type.

[0273] In one embodiment, the film grain adaptation indication information includes film grain model adaptation indication information;

[0274] The determining, according to the film grain adaptation indication information, a film grain parameter corresponding to the decoded image comprises:

[0275] determining whether to switch the film grain model according to the film grain model adaptation indication information;

[0276] If yes, then obtain the film grain parameters corresponding to the switched film grain model;

[0277] Otherwise, obtain the film grain parameters corresponding to the film grain model before switching.

[0278] In one embodiment, the film grain adaptive indication information includes film grain region adaptive indication information;

[0279] The film grain area adaptive indication information includes at least one of the following:

[0280] Film grain area adaptation enable flag, used to indicate whether to enable film grain area adaptation processing;

[0281] Film grain area adaptation flag, used to indicate whether to enable film grain area adaptation processing;

[0282] The region information corresponding to the film grain parameter is used to indicate the region to which the film grain parameter is applied in the decoded image.

[0283] In one embodiment, the film grain adaptive indication information includes film grain region adaptive indication information;

[0284] The determining, according to the film grain adaptation indication information, a film grain parameter corresponding to the decoded image comprises:

[0285] Acquiring film grain parameters corresponding to each region in the decoded image according to the film grain region adaptive indication information;

[0286] The step of synthesizing film grains corresponding to the decoded image according to the film grain parameters includes:

[0287] The film grains corresponding to the corresponding areas are synthesized according to the film grain parameters corresponding to each area.

[0288] In one embodiment, the film grain adaptive indication information includes film grain layered adaptive indication information;

[0289] The film grain layering adaptive indication information includes at least one of the following:

[0290] Film grain layering adaptive enable flag, used to indicate whether to enable film grain layering adaptive processing;

[0291] Film grain layering adaptive flag, used to indicate whether to enable film grain layering adaptive processing;

[0292] Film grain adaptive layering flag, used to indicate whether the layer corresponding to the flag enables film grain adaptive layering processing;

[0293] The layer identification information corresponding to the film grain parameter is used to indicate the layer identification corresponding to the decoded image to which the film grain parameter is applied;

[0294] Image resolution information corresponding to the film grain parameter, used to indicate the image resolution corresponding to the decoded image to which the film grain parameter is applied;

[0295] Film grain layering adaptation mode, which indicates the extent to which the film grain layering adaptation processing is applied.

[0296] In one embodiment, the film grain adaptive indication information includes film grain layered adaptive indication information;

[0297] The determining, according to the film grain adaptation indication information, a film grain parameter corresponding to the decoded image comprises:

[0298] Acquire film grain parameters corresponding to each layer in the decoded image according to the film grain layered adaptive indication information;

[0299] The step of synthesizing film grains corresponding to the decoded image according to the film grain parameters includes:

[0300] The film grains corresponding to the corresponding layers are synthesized according to the film grain parameters corresponding to each layer.

[0301] In one embodiment, the film grain adaptation indication information is included in a SEI message; the SEI message includes at least one of the following:

[0302] Film grain adaptation enabling flag, used to indicate whether the code stream associated with the SEI message allows film grain adaptation processing to be enabled;

[0303] Film grain adaptation flag, used to indicate whether the code stream associated with the SEI message enables film grain model adaptation processing;

[0304] Film grain basic parameters, used to indicate the film grain basic parameters used by the code stream associated with the SEI message;

[0305] Film grain layered adaptive flag, used to indicate whether the code stream associated with the SEI message enables film grain layered adaptive processing;

[0306] Film grain model adaptation flag, used to indicate whether the code stream associated with the SEI message enables film grain model adaptation processing;

[0307] Film grain model adaptation mode, which indicates the application range of the film grain model;

[0308] Film grain layered adaptive mode: used to indicate whether the code stream associated with the SEI message enables the film grain layered adaptive processing mode;

[0309] Film grain area adaptation flag, used to indicate whether the code stream associated with the SEI message enables film grain area adaptation processing;

[0310] The quantity parameter is used to indicate the quantity of the rectangular area for film grain adaptive processing;

[0311] Position parameter, used to indicate the position of the film grain adaptive processing rectangular area;

[0312] Size parameter, used to indicate the size of the rectangular area for film grain adaptive processing;

[0313] Film grain area model parameters, used to indicate the film grain model parameters corresponding to the rectangular area of ​​film grain adaptive processing;

[0314] The film grain area processing parameters are used to indicate the film grain processing parameters corresponding to the rectangular area of ​​film grain adaptive processing.

[0315] The decoding device proposed in this embodiment and the decoding method proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to any of the above embodiments, and this embodiment has the same beneficial effects as the decoding method.

[0316] The embodiment of the present application also provides a coding device. Fig.10 FIG. 1 is a schematic diagram of a coding device provided by an embodiment. Fig.10 As shown, the encoding device includes:

[0317] The encoding module 410 is configured to encode the image to generate a code stream;

[0318] A modeling module 420 configured to model the film grain in the image according to the characteristics of the film grain in the image;

[0319] The determination module 430 is configured to determine film grain adaptation indication information and write the film grain adaptation indication information into the bitstream.

[0320] The encoding device of this embodiment performs adaptive film grain modeling and synthesis based on the characteristics of film grains, and more accurately simulates the real film graininess, thereby bringing a better viewing experience to viewers.

[0321] In one embodiment, the film grain adaptive indication information includes at least one of the following: film grain model adaptive indication information, film grain region adaptive indication information, and film grain layer adaptive indication information.

[0322] In one embodiment, the film grain adaptation indication information includes at least one of the following:

[0323] Film grain adaptive enable flag, used to indicate whether to enable film grain adaptive processing;

[0324] Film grain adaptive flag, used to indicate whether to enable film grain adaptive processing.

[0325] In one embodiment, the determination module 430 is configured to include at least one of the following:

[0326] performing model adaptive film grain modeling on film grains in the image according to film grain characteristics in the image;

[0327] performing regional adaptive film grain modeling on film grain in the image according to film grain characteristics in the image;

[0328] Hierarchical adaptive film grain modeling is performed on film grain in the image according to film grain characteristics in the image.

[0329] The encoding device proposed in this embodiment and the encoding method proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to any of the above embodiments, and this embodiment has the same beneficial effects as the encoding method.

[0330] An embodiment of the present application also provides an electronic device, which can be used as a decoding end or as an encoding end. Fig.11 A schematic diagram of the hardware structure of an electronic device provided by an embodiment is shown in FIG. Fig.11 As shown, the electronic device provided by the present application includes a processor 510 and a memory 520; the processor 510 in the communication node may be one or more, Figure 5 A processor 510 is taken as an example; the memory 520 is configured to store one or more programs; the one or more programs are executed by the one or more processors 510, so that the one or more processors 510 implement the decoding method or encoding method as described in the embodiments of the present application.

[0331] The electronic device further includes: a communication device 530 , an input device 540 and an output device 550 .

[0332] The processor 510, memory 520, communication device 530, input device 540 and output device 550 in the electronic device may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0333] The input device 540 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 550 may include a display device such as a display screen.

[0334] The communication device 530 may include a receiver and a transmitter. The communication device 530 is configured to perform information transmission and reception communication according to the control of the processor 510.

[0335] The memory 520, as a computer-readable storage medium, can be configured to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the decoding method or encoding method described in the embodiment of the present application (for example, the decoding module 310, the acquisition module 320, the determination module 330, the synthesis module 340, the addition module 350 in the decoding device, or the encoding module 410, the modeling module 420, the determination module 430 in the encoding device). The memory 520 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to the use of the communication node, etc. In addition, the memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 520 may further include a memory remotely arranged relative to the processor 510, and these remote memories may be connected to the communication node via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0336] The present application also provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the decoding method or encoding method described in any one of the embodiments of the present application is implemented. The decoding method, applied to a decoding end, comprises: decoding a bit stream to obtain a decoded image;

[0337] Obtaining film grain adaptive indication information corresponding to the decoded image;

[0338] Determining film grain parameters corresponding to the decoded image according to the film grain adaptation indication information;

[0339] synthesizing film grains corresponding to the decoded image according to the film grain parameters;

[0340] The film grain is added to the decoded image.

[0341] Or, the encoding method, applied to the encoding end, includes:

[0342] Encode the image to generate a code stream;

[0343] modeling film grain in the image based on film grain characteristics in the image;

[0344] Film grain adaptation indication information is determined and the film grain adaptation indication information is written into the bitstream.

[0345] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (Random Access Memory, RAM), a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (ErasableProgrammable Read Only Memory, EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0346] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to: electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0347] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the above.

[0348] The computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and also conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0349] The above description is merely an exemplary embodiment of the present application and is not intended to limit the protection scope of the present application.

[0350] It will be appreciated by those skilled in the art that the term user terminal covers any suitable type of wireless user equipment, such as a mobile phone, a portable data processing device, a portable web browser or a vehicle-mounted mobile station.

[0351] In general, various embodiments of the present application can be implemented in hardware or dedicated circuits, software, logic or any combination thereof. For example, some aspects can be implemented in hardware, while other aspects can be implemented in firmware or software that can be executed by a controller, microprocessor or other computing device, although the present application is not limited thereto.

[0352] Embodiments of the present application may be implemented by executing computer program instructions by a data processor of a mobile device, for example in a processor entity, or by hardware, or by a combination of software and hardware. The computer program instructions may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages.

[0353] The block diagram of any logic flow in the drawings of the present application may represent program steps, or may represent interconnected logic circuits, modules and functions, or may represent a combination of program steps and logic circuits, modules and functions. A computer program may be stored in a memory. The memory may be of any type suitable for the local technical environment and may be implemented using any suitable data storage technology, such as but not limited to read-only memory (ROM), random access memory (RAM), optical memory devices and systems (digital versatile discs (DVD) or compact disks (CD), etc. Computer-readable media may include non-transitory storage media. The data processor may be of any type suitable for the local technical environment, such as but not limited to general-purpose computers, special-purpose computers, microprocessors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FGPAs) and processors based on multi-core processor architectures.

[0354] By way of exemplary and non-limiting examples, a detailed description of exemplary embodiments of the present application has been provided above. However, various modifications and adjustments to the above embodiments will be apparent to those skilled in the art when considered in conjunction with the accompanying drawings and claims, but will not depart from the scope of the present application. Therefore, the proper scope of the present application will be determined according to the claims.

Claims

1. A decoding method, applied to a decoding end, characterized in that: include: Decoding the code stream to obtain a decoded image; Obtaining film grain adaptive indication information corresponding to the decoded image; Determining film grain parameters corresponding to the decoded image according to the film grain adaptation indication information; synthesizing film grains corresponding to the decoded image according to the film grain parameters; The film grain is added to the decoded image.

2. The method according to claim 1, characterized in that The film grain adaptive indication information includes at least one of the following: film grain model adaptive indication information, film grain region adaptive indication information, and film grain layer adaptive indication information.

3. The method according to claim 1, characterized in that The film grain adaptation indication information includes at least one of the following: Film grain adaptive enable flag, used to indicate whether to enable film grain adaptive processing; Film grain adaptive flag, used to indicate whether to enable film grain adaptive processing.

4. The method according to claim 1, characterized in that: The film grain parameters include film grain model parameters and film grain processing parameters, and the film grain processing parameters correspond to the film grain processing methods corresponding to the film grain model parameters.

5. The method according to claim 1, characterized in that The film grain adaptation indication information includes at least one of the following: Film grain model adaptive enable flag, used to indicate whether to enable film grain model adaptive processing; Film grain model adaptive flag, used to indicate whether to enable film grain model adaptive processing; Film grain model adaptation mode, which indicates the range in which the film grain model should be applied.

6. The method according to claim 1, characterized in that Get the film grain adaptive indication information corresponding to the decoded image, including: The film grain model adaptation indication information is determined according to the SEI message type.

7. The method according to claim 1, characterized in that The film grain adaptation indication information includes film grain model adaptation indication information; The determining, according to the film grain adaptation indication information, a film grain parameter corresponding to the decoded image comprises: determining whether to switch the film grain model according to the film grain model adaptation indication information; If yes, then obtain the film grain parameters corresponding to the switched film grain model; Otherwise, obtain the film grain parameters corresponding to the film grain model before switching.

8. The method according to claim 1, characterized in that: The film grain adaptation indication information includes film grain area adaptation indication information; The film grain area adaptive indication information includes at least one of the following: Film grain area adaptation enable flag, used to indicate whether to enable film grain area adaptation processing; Film grain area adaptation flag, used to indicate whether to enable film grain area adaptation processing; The region information corresponding to the film grain parameter is used to indicate the region to which the film grain parameter is applied in the decoded image.

9. The method according to claim 1, characterized in that: The film grain adaptation indication information includes film grain area adaptation indication information; The determining, according to the film grain adaptation indication information, a film grain parameter corresponding to the decoded image comprises: Acquiring film grain parameters corresponding to each region in the decoded image according to the film grain region adaptive indication information; The step of synthesizing film grains corresponding to the decoded image according to the film grain parameters includes: The film grains corresponding to the corresponding areas are synthesized according to the film grain parameters corresponding to each area.

10. The method according to claim 1, characterized in that The film grain adaptation indication information includes film grain layered adaptation indication information; The film grain layering adaptive indication information includes at least one of the following: Film grain layering adaptive enable flag, used to indicate whether to enable film grain layering adaptive processing; Film grain layering adaptive flag, used to indicate whether to enable film grain layering adaptive processing; Film grain adaptive layering flag, used to indicate whether the layer corresponding to the flag enables film grain adaptive layering processing; The layer identification information corresponding to the film grain parameter is used to indicate the layer identification corresponding to the decoded image to which the film grain parameter is applied; Image resolution information corresponding to the film grain parameter, used to indicate the image resolution corresponding to the decoded image to which the film grain parameter is applied; Film grain layering adaptation mode, which indicates the extent to which the film grain layering adaptation processing is applied.

11. The method according to claim 1, characterized in that: The film grain adaptation indication information includes film grain layered adaptation indication information; The determining, according to the film grain adaptation indication information, a film grain parameter corresponding to the decoded image comprises: Acquire film grain parameters corresponding to each layer in the decoded image according to the film grain layered adaptive indication information; The step of synthesizing film grains corresponding to the decoded image according to the film grain parameters includes: The film grains corresponding to the corresponding layers are synthesized according to the film grain parameters corresponding to each layer.

12. The method according to claim 1, characterized in that The film grain adaptation indication information is included in an SEI message; the SEI message includes at least one of the following: Film grain adaptation enabling flag, used to indicate whether the code stream associated with the SEI message allows film grain adaptation processing to be enabled; Film grain adaptation flag, used to indicate whether the code stream associated with the SEI message enables film grain model adaptation processing; Film grain basic parameters, used to indicate the film grain basic parameters used by the code stream associated with the SEI message; Film grain layered adaptive flag, used to indicate whether the code stream associated with the SEI message enables film grain layered adaptive processing; Film grain model adaptation flag, used to indicate whether the code stream associated with the SEI message enables film grain model adaptation processing; Film grain model adaptation mode, which indicates the application range of the film grain model; Film grain layered adaptive mode: used to indicate whether the code stream associated with the SEI message enables the film grain layered adaptive processing mode; Film grain area adaptation flag, used to indicate whether the code stream associated with the SEI message enables film grain area adaptation processing; The quantity parameter is used to indicate the quantity of the rectangular area for film grain adaptive processing; Position parameter, used to indicate the position of the film grain adaptive processing rectangular area; Size parameter, used to indicate the size of the rectangular area for film grain adaptive processing; Film grain area model parameters, used to indicate the film grain model parameters corresponding to the rectangular area of ​​film grain adaptive processing; The film grain area processing parameters are used to indicate the film grain processing parameters corresponding to the rectangular area of ​​film grain adaptive processing.

13. A coding method, applied to a coding end, characterized in that: include: Encode the image to generate a code stream; modeling film grain in the image based on film grain characteristics in the image; Film grain adaptation indication information is determined and the film grain adaptation indication information is written into the bitstream.

14. The method according to claim 13, characterized in that The film grain adaptive indication information includes at least one of the following: film grain model adaptive indication information, film grain region adaptive indication information, and film grain layer adaptive indication information.

15. The method according to claim 13, characterized in that The film grain adaptation indication information includes at least one of the following: Film grain adaptive enable flag, used to indicate whether to enable film grain adaptive processing; Film grain adaptive flag, used to indicate whether to enable film grain adaptive processing.

16. The method according to claim 13, characterized in that Modeling film grain in the image according to film grain characteristics in the image includes at least one of the following: performing model adaptive film grain modeling on film grains in the image according to film grain characteristics in the image; performing regional adaptive film grain modeling on film grain in the image according to film grain characteristics in the image; Hierarchical adaptive film grain modeling is performed on film grain in the image according to film grain characteristics in the image.

17. An electronic device, characterized in that: include: memory, and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the decoding method according to any one of claims 1 to 12 or the encoding method according to any one of claims 13 to 16.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the decoding method according to any one of claims 1 to 12 or the encoding method according to any one of claims 13 to 16 is implemented.

Images