Decoding method, encoding method, electronic device, and storage medium

Abstract

A decoding method, an encoding method, an electronic device, and a storage medium, the decoding method including: decoding a codestream to obtain a decoded image; obtaining film grain adaptation instruction information corresponding to the decoded image; determining film grain parameters corresponding to the decoded image based on the film grain adaptation instruction information; synthesizing film grain corresponding to the decoded image based on the film grain parameters; and adding the film grain to the decoded image.

Description

[Technical Field]

[0001] The present application relates to the technical field of image processing, for example, to a decoding method, an encoding method, an electronic device, and a storage medium. [Background technology]

[0002] Film grain technology is one of the technologies being discussed and researched by the International Video Coding Standards Organization. The film grain modeling and synthesis methods defined in different standards are not completely identical, and their support for film grain varies. A single film grain modeling and synthesis method is typically used for image or video coding, failing to ensure its application to different types of film grain modeling and synthesis, resulting in poor image quality. For example, different film grain models achieve different effects for different types of film grain, and different film grain models achieve different effects for the same film grain. Film grain synthesis works better for high-resolution images but is significantly less effective for low-resolution images. Different regions of the same frame of an image may employ different coding methods and parameters due to differences in texture or brightness, resulting in different visual effects produced by a single film grain model in different regions of the same frame of an image. Summary of the Invention [Problem to be solved by the invention]

[0003] The present application provides a decoding method, an encoding method, an electronic device, and a storage medium. [Means for solving the problem]

[0004] The present embodiment is A decoding method applied to a decoding side, comprising: decoding the codestream to obtain a decoded image; obtaining film grain adaptation indication information corresponding to the decoded image; determining film grain parameters corresponding to the decoded image based on the film grain adaptation indication information; synthesizing a film grain corresponding to the decoded image based on the film grain parameters; adding the film grain to the decoded image. A decoding method is provided.

[0005] The present embodiment is A coding method applied to the code side, comprising: encoding the image to generate a codestream; modeling film grain in the image based on film grain characteristics in the image; determining film grain adaptation indication information and writing the film grain adaptation indication information into the codestream; An encoding method is provided.

[0006] The present embodiment is a memory, a processor, and a computer program stored in the memory and executable by the processor, the program implementing the decoding method or the encoding method when executed by the processor; An electronic device is also provided.

[0007] The present embodiment is a computer program is stored which, when executed by a processor, implements the decoding or encoding method; A computer-readable storage medium is also provided. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram of a film grain system frame according to one embodiment. [Figure 2] 1 is a flowchart of a decoding method according to an embodiment. [Figure 3]1 is a flowchart of an encoding method according to an embodiment. [Figure 4] 1 is a flowchart of a video image codestream decoding process according to one embodiment. [Figure 5] 10 is a flowchart illustrating a process performed when film grain adaptation instruction information is film grain model adaptation instruction information according to an embodiment. [Figure 6] 10 is a flowchart illustrating a process performed when film grain adaptation instruction information is film grain region adaptation instruction information according to an embodiment. [Figure 7] 10 is a flowchart illustrating a process performed when film grain adaptation instruction information is film grain layer adaptation instruction information according to an embodiment. [Figure 8] 1 is a flowchart of encoding a video image codestream according to one embodiment. [Figure 9] FIG. 2 is a structural schematic diagram of a decoding device according to an embodiment. [Figure 10] FIG. 1 is a structural schematic diagram of an encoding device according to an embodiment. [Figure 11] FIG. 1 is a schematic diagram illustrating the hardware structure of an electronic device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] The present application will be described below with reference to the drawings and examples. The specific examples described here are merely for the purpose of interpreting the present application. In the drawings, only parts relevant to the present application are shown.

[0010] Film grain is a result of the physical process of exposing and developing photographic film (exposing and developing silver halide crystals), and is prevalent in film and television content. However, digital sensors do not undergo this process, producing noise-free digital video, which is free of film grain. Many content producers, especially in the film industry, generally believe that film grain can enhance the natural appearance of video content with a pleasing noise. Adding film noise back into video can improve the visual experience and is usually considered part of the production intent.

[0011] In existing methods, film grain synthesis works better for high-resolution images but is not very effective for low-resolution images, and different regions of the same frame image adopt different encoding methods and parameters due to different textures or brightnesses, resulting in different visual effects generated by a single film grain model in different regions of the same frame image, etc. The decoding and encoding methods of the present application can perform adaptive film grain modeling and synthesis based on film grain characteristics, which can more accurately simulate the actual film grain and bring viewers a better viewing experience.

[0012] FIG. 1 is a schematic diagram of a film grain system frame according to one embodiment. As shown in FIG. 1, in the video coding standard developed by JVET, the encoding side first compresses and encodes input source video images using an encoder to obtain a coded video stream. A denoising module preprocesses the input source video images to remove noise (the encoder itself can also be used to perform the denoising process), and outputs a video image sequence after film grain has been removed or attenuated. The film grain modeling process, which is a film grain parameterization process, estimates the parameter values ​​of a film grain model by analyzing the differences between the denoised video and the source video. The film grain parameters may be transmitted to the decoding side along with the coded video stream or provided to the decoding side via an external transmission method.

[0013] On the decoding side, the film grain parameter values ​​are analyzed and input into a film grain synthesis process to generate simulated film grain, and before the decoded video is output, the film grain is added into the decoded video, and a decoded reconstructed video with the simulated film grain is output.

[0014] In addition to the film grain system frame shown in the above figures, there are other implementations of film grain system frames, the main difference being the encoding side. For example, instead of encoding a source video image sequence to obtain a coded video code stream, the encoding side may encode a video image sequence after noise has been removed to obtain a coded video. Alternatively, if the source video does not contain film grain, the encoding side can use pre-determined film grain parameters to transmit to the decoding side for film grain synthesis. In this case, even if there is no film grain in the source video, the decoder side can add film grain to the video to achieve a certain effect.

[0015] At the 29th JVET Conference, proposal JVET-AC0199 presented film grain quality assessment results obtained using frequency-based and autoregression-based methods, noting the following: 1. Videos with relatively heavy film grain have thicker synthesized grain and significant overlapping textures. 2. Videos with light / low film grain have a good synthesized grain effect. 3. Film grain synthesis (FGS) methods based on frequency filtering and autoregression (AR) produce different results in different situations. This means that a single film grain synthesis method cannot simultaneously achieve good results for different types of film grain.

[0016] The JVET-AC2020 output document noted that the greater the image resolution, the greater the perception of film grain. Film grain is prominent in movies and high-resolution images, gradually loses its significance in standard-resolution images, and becomes less visible in smaller formats. Meanwhile, the H.265 / HEVC and H.266 / VVC standards both support the concept of "multilayer video coding / scalable video coding." A video code stream may include one base layer subcode stream and at least one enhancement layer subcode stream, and the enhancement layer subcode stream may be an independent enhancement layer subcode stream or a dependent enhancement layer subcode stream. The base layer subcode stream can be independently decoded to provide low-quality video content, the independent enhancement layer subcode stream can be independently decoded to provide high-quality video content, and the dependent enhancement layer subcode stream needs to be decoded together with the base layer subcode stream to provide high-quality video content. Therefore, in the scene of scalable video coding, film grain is more significant for high quality video images output by a relatively high enhancement layer, and less significant for low quality video images output by a base layer.

[0017] Also, different areas of a video image in the same frame may have different textures (e.g., background and important screen areas) or different luminance (bright and dark areas), and different coding parameters may be employed, so different film grain synthesis methods may produce different visual effects in different areas of the image in the same frame.

[0018] Generally, different standards or private protocols of each manufacturer generally define only a single film grain processing method and do not consider the film grain characteristics in video images. Furthermore, film grain processing often adjusts one video sequence or one frame of video image, and the visual effect in different areas of one frame of image or for the texture characteristics of different images may be different. Furthermore, in scalable coding scenes, it is not possible to ensure the image quality after film grain processing for images of different qualities output by different layers.

[0019] FIG. 2 is a flowchart of a decoding method according to an embodiment. As shown in FIG. 2, the method according to this embodiment includes steps S110 to S150.

[0020] At S110, the codestream is decoded to obtain a decoded image.

[0021] At S120, film grain adaptation indication information corresponding to the decoded image is obtained.

[0022] In S130, a film grain parameter corresponding to the decoded image is determined based on the film grain adaptation indication information.

[0023] At S140, a film grain corresponding to the decoded image is synthesized based on the film grain parameters.

[0024] At S150, the film grain is added to the decoded image.

[0025] In this embodiment, the decoding side obtains a codestream to be decoded, analyzes the codestream, obtains a decoded image corresponding to the codestream, and obtains film grain adaptation instruction information corresponding to the decoded image. The film grain adaptation instruction information may be included in the codestream or obtained in other predetermined manners. Then, the decoding side determines film grain parameters corresponding to the decoded image based on the film grain adaptation instruction information, which may be one set of film grain parameters or multiple sets of film grain parameters. The decoding side synthesizes film grain corresponding to the decoded image based on the film grain synthesis method indicated by the identifier in the film grain parameters, and adds the film grain to the decoded image.

[0026] In one embodiment, the film grain adaptation indication information includes at least one of film grain model adaptation indication information, film grain region adaptation indication information, and film grain layer adaptation indication information.

[0027] In one embodiment, the film grain adaptation instruction information includes at least one of a film grain adaptation enable identifier for indicating whether film grain adaptation processing is enabled, and a film grain adaptation identifier for indicating whether film grain adaptation processing is enabled.

[0028] In one embodiment, the film grain parameters include film grain model parameters and film grain processing parameters, and the film grain processing parameters correspond to film grain processing methods corresponding to the film grain model parameters.

[0029] In one embodiment, the film grain adaptation instruction information includes at least one of a film grain model adaptation enable identifier for instructing whether to enable film grain model adaptation processing, a film grain model adaptation identifier for instructing whether to enable film grain model adaptation processing, and a film grain model adaptation mode for instructing the application range of the film grain model.

[0030] In one embodiment, obtaining film grain adaptation indication information corresponding to the decoded image includes determining film grain model adaptation indication information based on an SEI message type.

[0031] In one embodiment, the film grain adaptation instruction information includes film grain model adaptation instruction information, and determining film grain parameters corresponding to the decoded image based on the film grain adaptation instruction information includes: determining whether to switch film grain models based on the film grain model adaptation instruction information; if YES, obtaining film grain parameters corresponding to the film grain model after switching; and if NO, obtaining film grain parameters corresponding to the film grain model before switching.

[0032] In one embodiment, the film grain adaptation instruction information includes film grain region adaptation instruction information, which includes at least one of a film grain region adaptation enable identifier for instructing whether film grain region adaptation processing is enabled, a film grain region adaptation identifier for instructing whether film grain region adaptation processing is enabled, and region information corresponding to film grain parameters for indicating regions in the decoded image to which film grain parameters are applied.

[0033] In one embodiment, the film grain adaptation instruction information includes film grain region adaptation instruction information, determining film grain parameters corresponding to the decoded image based on the film grain adaptation instruction information includes obtaining film grain parameters corresponding to each region in the decoded image based on the film grain region adaptation instruction information, and synthesizing film grain corresponding to the decoded image based on the film grain parameters includes synthesizing film grain corresponding to the corresponding region based on the film grain parameters corresponding to each region, respectively.

[0034] In one embodiment, the film grain adaptation instruction information includes film grain layer adaptation instruction information, which includes at least one of a film grain layer adaptation enable identifier for indicating whether to enable film grain layer adaptation processing, a film grain layer adaptation identifier for indicating whether to enable film grain layer adaptation processing, a film grain adaptation layer identifier for indicating whether to enable film grain layer adaptation processing for a layer corresponding to the identifier, layer identifier information corresponding to the film grain parameter for indicating a layer identifier corresponding to a decoded image to which the film grain parameter has been applied, image resolution information corresponding to the film grain parameter for indicating an image resolution corresponding to the decoded image to which the film grain parameter has been applied, and a film grain layer adaptation mode for indicating an application range of the film grain layer adaptation processing.

[0035] In one embodiment, the film grain adaptation instruction information includes film grain layer adaptation instruction information, determining film grain parameters corresponding to the decoded image based on the film grain adaptation instruction information includes obtaining film grain parameters corresponding to each layer in the decoded image based on the film grain layer adaptation instruction information, and synthesizing film grain corresponding to the decoded image based on the film grain parameters includes synthesizing film grain corresponding to the corresponding layer based on the film grain parameters corresponding to each layer, respectively.

[0036] In one embodiment, the film grain adaptation indication information is included in an SEI message, and the SEI message includes: a film grain adaptation enable identifier for indicating whether a codestream associated with the SEI message enables film grain adaptation processing; a film grain adaptation identifier for indicating whether a codestream associated with the SEI message enables film grain model adaptation processing; a film grain base parameter for indicating a film grain base parameter adopted by the codestream associated with the SEI message; a film grain layer adaptation identifier for indicating whether a codestream associated with the SEI message enables film grain layer adaptation processing; a film grain model adaptation identifier for indicating whether a codestream associated with the SEI message enables film grain model adaptation processing; a film grain layer adaptation mode for indicating whether the codestream associated with the SEI message enables a film grain layer adaptation processing mode; a film grain region adaptation identifier for indicating whether the codestream associated with the SEI message enables film grain region adaptation processing; a number parameter for indicating the number of rectangular regions for film grain adaptation processing; a position parameter for indicating the position of the rectangular region for film grain adaptation processing; a size parameter for indicating the size of the rectangular region for film grain adaptation processing; a film grain region model parameter for indicating film grain model parameters corresponding to the rectangular region for film grain adaptation processing; and a film grain region processing parameter for indicating film grain processing parameters corresponding to the rectangular region for film grain adaptation processing.

[0037] FIG. 3 is a flowchart of an encoding method according to an embodiment. As shown in FIG. 3, the method according to this embodiment includes S210 to S230.

[0038] At S210, the image is encoded to generate a codestream.

[0039] At S220, film grain in the image is modeled based on film grain characteristics in the image.

[0040] At S230, film grain adaptation indication information is determined, and the film grain adaptation indication information is written into the codestream.

[0041] In this embodiment, first, an image is obtained, which may be a single image or an image of a frame in a video sequence, and the image is encoded to generate a codestream, and the film grain characteristics may be the film grain characteristics of the entire image or the film grain characteristics of different regions in the image, including film grain intensity, film grain scaling parameters, etc., and the film grain in the image is modeled based on the film grain characteristics in the image, and film grain adaptation indication information corresponding to the image is determined based on an adaptive film grain modeling process, and the film grain adaptation indication information is written into the codestream.

[0042] In one embodiment, the film grain adaptation indication information includes at least one of film grain model adaptation indication information, film grain region adaptation indication information, and film grain layer adaptation indication information.

[0043] In one embodiment, the film grain adaptation instruction information includes at least one of a film grain adaptation enable identifier for indicating whether film grain adaptation processing is enabled, and a film grain adaptation identifier for indicating whether film grain adaptation processing is enabled.

[0044] In one embodiment, modeling film grain in the image based on film grain characteristics in the image includes at least one of: performing model adaptive film grain modeling on film grain in the image based on film grain characteristics in the image; performing region adaptive film grain modeling on film grain in the image based on film grain characteristics in the image; and performing hierarchical adaptive film grain modeling on film grain in the image based on film grain characteristics in the image.

[0045] The data transmission method will be described below by way of example using specific embodiments.

[0046] In the following embodiments, methods of film grain model adaptation processing, film grain region adaptation processing, and film grain hierarchy adaptation processing will be described, respectively. The film grain adaptation processing method may include any one or more of the film grain model adaptation processing, film grain region adaptation processing, and film grain hierarchy adaptation processing methods. [Example]

[0047] Film grain is difficult to encode with video image coding tools because video image compression relies heavily on prediction in the temporal, spatial, or cross-component domains, while film grain is inherently random. To preserve film grain, one method is to use a low quantization parameter during video compression to better preserve details, but this requires a relatively high bitrate for video applications such as adaptive streaming and broadcasting. Another solution is to model film grain at the encoding side to obtain film grain parameters, and then synthesize film grain using the film grain parameters at the decoding side and add it to the decoded video image, thereby fully realizing the potential of video image compression technology. For example, the H.265 / High Efficiency Video Coding (HEVC) standard and the H.266 / Versatile Video Coding (VVC) series of standards, created by the Joint Video Expert Teams (JVET) of the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) and the International Telecommunication Union (ITU), use a Film Grain Characteristic (FGC) Supplemental Enhancement Information (SEI) message to indicate film grain parameters used for film grain synthesis when decoding and rendering video or image data, thereby achieving the purpose of retaining film grain.

[0048] This embodiment provides a video image codestream decoding method, which is applied to a video image codestream decoding device. The input of the device is a video image codestream or a media file, and the output is a decoded and reconstructed video image. Figure 4 is a flowchart of a video image codestream decoding method according to one embodiment. As shown in Figure 4, the method includes the following steps:

[0049] In S1, a codestream is obtained and analyzed to obtain a decoded image (ie, a decoded image).

[0050] The decoding side obtains a codestream to be decoded, analyzes the codestream to be decoded, and obtains a decoded image corresponding to the codestream to be decoded. The analysis of the codestream to obtain a decoded image can be achieved by a method specified in a video coding standard corresponding to the codestream type.

[0051] The video itself consists of a continuous sequence of images, and the methods in the embodiments of the present application may be applied to a single image or to an image of a frame within a video sequence, and therefore the concepts of video, video image or image referred to below can be used interchangeably.

[0052] At S2, film grain adaptation indication information corresponding to the decoded image is obtained.

[0053] There is no necessary order between obtaining the film grain adaptation instruction information corresponding to the decoded image and analyzing the code stream in S1 to obtain the decoded image; they may be performed simultaneously, or the code stream may be analyzed to obtain the decoded image and then the film grain adaptation instruction information may be obtained, or the film grain adaptation instruction information may be obtained and then the code stream may be analyzed to obtain the decoded image.

[0054] The film grain adaptation instruction information may be a film grain adaptation identifier (fg_adaptive_flag) for instructing whether film grain adaptation processing is enabled. The film grain adaptation instruction information may further include a film grain adaptation enable identifier (fg_adaptive_enabled_flag) for instructing whether film grain adaptation processing is enabled, and if the value of fg_adaptive_enabled_flag is 1, it indicates that enabling of film grain adaptation processing is allowed, and in this case, whether film grain adaptation processing is enabled is further determined by the film grain adaptation identifier.

[0055] The film grain adaptation identifier should at least include one or more of a film grain model adaptation identifier, a film grain region adaptation identifier, or a film grain layer adaptation identifier. The film grain adaptation enable identifier should at least include one or more of a film grain model adaptation enable identifier, a film grain region adaptation enable identifier, or a film grain layer adaptation enable identifier.

[0056] If the film grain adaptation indication information may be included in the codestream, for example, if the film grain model adaptation indication identifier fg_adaptive_flag is included in a Video Parameter Set (VPS), it indicates that the film grain adaptation indication information is used to indicate whether film grain adaptation processing is enabled for the entire video. If the film grain adaptation indication information is included in a Sequence Parameter Set (SPS), it indicates that the film grain model adaptation indication information is used to indicate whether film grain model adaptation processing is enabled for a video image sequence corresponding to the sequence parameter set. If the film grain model adaptation indication information is included in a Picture Parameter Set (PPS) or an Adaptive Parameter Set (ADAP), it indicates that the film grain model adaptation indication information is used to indicate whether film grain model adaptation processing is enabled for a video image sequence corresponding to the sequence parameter set. When the film grain model adaptation instruction information is included in an SEI message, it indicates that the film grain model adaptation instruction information is used to instruct whether to enable film grain model adaptation processing for one frame of image corresponding to the picture parameter set or adaptation parameter set. When the film grain model adaptation instruction information is included in an SEI message, it indicates that the film grain model adaptation instruction information is used to instruct whether to enable film grain model adaptation processing for one frame of image corresponding to the SEI message.

[0057] If a video parameter set includes the film grain adaptation identifier vps_fg_adaptive_flag and a sequence parameter set also includes the film grain model adaptation identifier pps_fg_adaptive_flag, and the value of vps_fg_adaptive_flag does not match the value of pps_fg_adaptive_flag, whether or not film grain model adaptation processing is enabled for the sequence corresponding to the sequence parameter set can be determined based on the value of pps_fg_adaptive_flag.

[0058] The different film grain adaptation indication information may be included in whole or in part in one or more locations of the video parameter set VPS, the sequence parameter set SPS, the picture parameter set PPS or the adaptation parameter set APS or in a specific SEI message.

[0059] The film grain adaptation indication information may be included in the codestream or may be obtained in other predetermined manners. If the film grain adaptation indication information includes multiple types of film grain adaptation indication information, some of the multiple types of film grain adaptation indication information may be obtained in the codestream and some of the film grain adaptation indication information may be obtained in provisioning information or other manners.

[0060] In S3, the film grain parameters corresponding to the decoded image are obtained (ie, determined) based on the film grain adaptation indication information.

[0061] The film grain parameters include a film grain model parameter and a film grain processing parameter corresponding to a film grain processing method indicated by the film grain model parameter, wherein the film grain model parameter may be a film grain model identifier, and the film grain processing parameter may be a film grain synthesis parameter or a film grain modeling parameter. For example, if the film grain model identifier indicates that a film grain synthesis method based on frequency filtering is to be adopted, the corresponding film grain processing parameters may include a film grain luminance bit depth parameter (fg_bit_depth_luma_minus8), a chroma bit depth parameter (fg_bit_depth_chroma_minus8), an image color gamut corresponding to the film grain (fg_colour_primaries), a color space fg_matric_coeffs, a transfer function (fg_transfer_characteristics), and other parameters, a film grain and decoded image blending mode parameter (fg_blending_mode_id), a film grain scaling parameter (fg_log2_scale_factor), a model number parameter of film grain intensity intervals for different color components (fg_num_model_values_minus1[c]), and model values ​​corresponding to each film grain intensity interval for different color components (fg_comp_model_value[c][i][j]).

[0062] The film grain parameters corresponding to the decoded image may be one set of film grain parameters or multiple sets of film grain parameters. For example, as described in S2, when the film grain adaptation instruction information includes film grain region adaptation instruction information, the film grain parameters corresponding to the decoded image may include multiple sets of film grain parameters corresponding to different regions in the decoded image.

[0063] In S4, a film grain corresponding to the decoded image is synthesized based on the film grain parameters.

[0064] Film grain corresponding to the decoded image is synthesized based on a film grain model identifier included in the film grain parameters and film grain parameters corresponding to a film grain synthesis method indicated by the film grain model identifier.

[0065] If the film grain adaptation instruction information includes film grain region adaptation instruction information, the film grain corresponding to the decoded image includes film grain corresponding to each region in the decoded image.

[0066] In S5, the film grain is added to the decoded image.

[0067] The method of adding the film grain to the decoded image can be an additive mode, a multiplication mode, etc. The method of adding the film grain to the decoded image can be specified by a mixing mode parameter of the film grain and the decoded image included in the film grain parameter in S3, or can be determined based on a parameter preset by the decoding side. [Example]

[0068] Usually, different standards or private protocols of each manufacturer generally only define one specific film grain processing method, and do not consider providing an adaptive processing method of the film grain model for the film grain characteristics in the video image to achieve a better visual effect.

[0069] This embodiment provides a video image codestream decoding method, which is applied to a video codestream decoding device. The input of the device is a video image codestream or a media file, and the output is a decoded and reconstructed video image. Figure 5 is a processing flowchart when the film grain adaptation indication information is film grain model adaptation indication information according to one embodiment. As shown in Figure 5, the method includes the following steps:

[0070] In S1, a codestream is obtained and the codestream is analyzed to obtain a decoded image.

[0071] In S2, film grain model adaptation instruction information corresponding to the decoded image is obtained.

[0072] Referring to S2 in the first embodiment, the film grain adaptation instruction information in the embodiment is film grain model adaptation instruction information, and film grain model adaptation instruction information corresponding to the decoded image is obtained, and the film grain model adaptation instruction information may include a film grain model adaptation identifier fg_model_adaptive_flag for indicating whether to enable film grain model adaptation processing, and may further include a film grain adaptation enable identifier fg_adaptive_enabled_flag for indicating whether to enable film grain adaptation processing.

[0073] Table 1 illustrates the relevant syntax elements and semantic information by taking the film grain model adaptation enable identifier as an example that is included in the video parameter set.

[0074] [Table 1]

[0075] If the value of the film grain adaptive enable identifier vps_fg_adaptive_enable_flag is 0, it indicates that the video sequence does not allow film grain adaptive processing to be enabled, and if the value of vps_fg_adaptive_enable_flag is 1, it indicates that the video sequence allows film grain adaptive processing to be enabled.

[0076] Table 2 illustrates the relevant syntax elements and semantics information by taking the film grain model adaptation enable identifier and film grain adaptation identifier as an example included in a picture parameter set.

[0077] [Table 2]

[0078] If the value of the film grain adaptive enable identifier pps_fg_adaptive_enable_flag is 0, it indicates that film grain adaptive processing is not allowed to be enabled for video images, if the value of pps_fg_adaptive_enable_flag is 1, it indicates that film grain adaptive processing is allowed to be enabled for video images, if the value of the film grain model adaptive identifier pps_fg_model_adaptive_flag is 0, it indicates that film grain model adaptive processing is enabled for video images, and if the value of pps_fg_model_adaptive_flag is 1, it indicates that film grain model adaptive processing is enabled for video images.

[0079] The film grain model adaptation instruction information may be a film grain model adaptation mode for instructing the adaptation range (i.e., application range) of the film grain model after switching. Table 3 describes related syntax elements and semantics information by taking the film grain model adaptation mode as an example, where the film grain model adaptation mode is included in a picture parameter set.

[0080] [Table 3]

[0081] pic_fg_model_adaptive_mode is a film grain model adaptive mode. When the value of pic_fg_model_adaptive_mode is 0, it indicates that the model switching is performed only on the image corresponding to the current parameter set, that is, the new film grain model and parameters are applied only to the image corresponding to the current parameter set each time the model is switched. When the value of pic_fg_model_adaptive_mode is 1, it indicates that the model switching is performed on the image corresponding to the current parameter set and all subsequent images, that is, the film grain model and parameters received after the current parameter set are applied to the image corresponding to the current parameter set and all subsequent images until other new film grain models and parameters are received. The film grain model adaptive instruction information can be implicitly inferred and determined by a plurality of SEI message types including different film grain models. For example, the film grain parameters corresponding to the decoded image of the previous frame are set to the FGC The film grain parameters transmitted by the SEI message and corresponding to the current decoded image are transmitted by SEI messages of types such as user_data_unregistered or user_data_registered_itu_t_t35, and therefore, SEI messages with these two different types of film grain parameters may be considered as film grain model adaptation instruction information.

[0082] In S3, film grain parameters corresponding to the decoded image are obtained based on the film grain model adaptation instruction information.

[0083] S3 in the embodiment can be further divided into the following S3.1 to S3.4.

[0084] In S3.1, it is determined whether to switch the film grain model according to the film grain model adaptation instruction information; if yes, execute S3.3; if no, execute S3.2.

[0085] In S3.2, the film grain parameter values ​​corresponding to the original film grain model are obtained.

[0086] The original film grain model refers to a film grain model corresponding to a previously decoded image, and the previously decoded image refers to a decoded image obtained by performing codestream analysis before the decoded image. Accordingly, there are three situations in obtaining film grain parameters corresponding to the original film grain model. First, the film grain parameter values ​​used for the current decoded image are exactly the same as those for the previously decoded image, so the film grain parameters corresponding to the previously decoded image are obtained directly. Second, the film grain parameter values ​​used for the current decoded image are different from those for the previously decoded image, so the film grain parameters corresponding to the current decoded image are obtained by directly analyzing the codestream or using other out-of-band transmission methods. Third, the film grain parameter values ​​used for the current decoded image are partially different from those for the previously decoded image, so some film grain parameter values ​​corresponding to the current decoded image are obtained by analyzing the codestream or using other out-of-band transmission methods, and the remaining film grain parameter values ​​are obtained from the film grain parameters corresponding to the previously decoded image.

[0087] In S3.3, the film grain parameter values ​​corresponding to the film grain model after switching are obtained.

[0088] The film grain parameter values ​​corresponding to the switched film grain model are parsed from the codestream or obtained by other out-of-band transmission methods.

[0089] In S3.4, it is determined whether film grain is enabled for the decoded image, if yes, execute S4, if no, execute back to S1.

[0090] S3.4 is an optional operation. For example, if it is acceptable to add film grain to some images in a video sequence and not to add film grain to some images, this operation can be performed. If the answer is NO, directly perform S4.

[0091] If it is permitted to add film grain to some images in a video sequence and not to add film grain to other images, the film grain parameter value corresponding to the film grain model obtained in S3.2 or S3.3 may be empty, which indicates that film grain is not enabled.

[0092] In S4, a film grain corresponding to the decoded image is synthesized based on the film grain parameter value.

[0093] In S5, the film grain is added to the decoded image. [Example]

[0094] Film grain processing often adjusts a single video sequence or a single frame of video image, and does not consider providing a method for adaptively processing film grain regions in response to film grain characteristics, image texture characteristics, etc. in different regions of a single frame of image.

[0095] This embodiment provides a video image codestream decoding method, which is applied to a video codestream decoding device. The input of the device is a video image codestream or a media file, and the output is a decoded and reconstructed video image. Figure 6 is a processing flowchart when the film grain adaptation indication information is film grain region adaptation indication information according to an embodiment. As shown in Figure 6, the method includes the following steps:

[0096] In S1, a codestream is obtained and the codestream is analyzed to obtain a decoded image.

[0097] In S2, film grain region adaptation indication information corresponding to the decoded image is obtained.

[0098] Referring to S2 in the first embodiment, the film grain adaptation instruction information in the embodiment is film grain region adaptation instruction information, and obtains the film grain region adaptation instruction information corresponding to the decoded image.

[0099] According to the film grain region adaptation indication information, it can be determined that different regions in the decoded image can correspond to different film grain parameters.

[0100] The film grain region adaptation instruction information may be a film grain region adaptation enable identifier for instructing whether or not to enable film grain region adaptation processing.

[0101] The film grain region adaptation indication information may be a film grain region adaptation identifier fg_region_adaptive_flag that indicates whether film grain region adaptation processing is enabled for a video image, and the identifier may be included in a video parameter set VPS, a sequence parameter set SPS, a picture parameter set PPS, or an adaptation parameter set APS, or may be included in a specific SEI message. For example, Table 4 describes related syntax elements and semantics information by taking the film grain region adaptation enable identifier and the film grain region adaptation identifier as an example that are included in a picture parameter set.

[0102] [Table 4]

[0103] If the value of the film grain adaptive enable identifier pps_fg_adaptive_enable_flag is 0, it indicates that film grain adaptive processing is not allowed to be enabled for video images, if the value of pps_fg_adaptive_enable_flag is 1, it indicates that film grain adaptive processing is allowed to be enabled for video images, if the value of the film grain model adaptive identifier pps_fg_region_adaptive_flag is 0, it indicates that film grain region adaptive processing is enabled for video images, and if the value of pps_fg_region_adaptive_flag is 1, it indicates that film grain region adaptive processing is enabled for video images.

[0104] The film grain region adaptation indication information may be region information corresponding to a film grain parameter, which is used to indicate a location region where the film grain parameter is applied in the decoded image. The film grain region adaptation indication information may include multiple region information corresponding to different film grain parameters. The multiple region information corresponding to different film grain parameters may include one or more region information pieces, each corresponding to one or more film grain parameters, or may include region information pieces corresponding to one or more main film grain parameters and at least one or more remaining film grain parameters, where the one or more main film grain parameters are applied to regions other than the region information pieces corresponding to the at least one or more remaining film grain parameters in the decoded image. For example, Table 5 describes related syntax elements and semantic information, taking the region information for film grain adaptation processing as an example included in a picture parameter set.

[0105] [Table 5]

[0106] If the value of the film grain adaptive enable identifier pps_fg_adaptive_enable_flag is 0, it indicates that film grain adaptive processing is not allowed to be enabled for video images, and if the value of pps_fg_adaptive_enable_flag is 1, it indicates that film grain adaptive processing is allowed to be enabled for video images, pps_num_of_adaptive_regions indicates the number of film grain adaptive processing regions, and fg_region_info[i] indicates information about the i-th film grain adaptive processing region.

[0107] The region information can be described by the region position and size. For example, a rectangular region can be determined by the position of the top right corner and its length and width. It can also be described by the block size and number. For example, the position of the eighth 8*8 adaptive processing block and the size of the adaptive processing block can be preset or determined in other ways. For example, the size and order of several optional adaptive processing blocks can be determined, and the size of the adaptive block to be used can be known by a predetermined number.

[0108] fg_region_parameters[i]() indicates the film grain parameters corresponding to the i-th film grain adaptive processing region.

[0109] In S3, film grain parameters corresponding to each region in the decoded image are obtained based on the film grain region adaptation instruction information.

[0110] According to S3 in Example 1 and the description in Example 2, the film grain parameters corresponding to each region may include two situations: the film grain model parameters corresponding to different regions are the same but the film grain modeling parameters are different, or the film grain model parameters and film grain modeling parameters corresponding to different regions are both different.

[0111] In S4, film grains corresponding to each region of the decoded image are synthesized based on the film grain parameters.

[0112] Film grain corresponding to each region of the decoded image is synthesized based on a film grain model identifier included in the film grain parameters and film grain parameters corresponding to a film grain synthesis method designated by the film grain model identifier.

[0113] When the film grain adaptation instruction information includes film grain region adaptation instruction information, the film grain corresponding to the decoded image may include different film grains corresponding to different regions in the decoded image.

[0114] In S5, the film grain is added to the decoded image.

[0115] According to the description of S3 and S4, the film grain is added to the corresponding area of ​​the decoded image. [Example]

[0116] This embodiment provides a video image codestream decoding method, which is applied to a video codestream decoding device. The input of the device is a video image codestream or a media file, and the output is a decoded and reconstructed video image. The method considers providing film grain layer adaptation processing for images of different qualities output by different layers in a scalable coding scene. Figure 7 is a processing flowchart when the film grain adaptation indication information according to one embodiment is film grain layer adaptation indication information. As shown in Figure 7, the method includes the following steps:

[0117] In S1, a codestream is obtained and the codestream is analyzed to obtain a decoded image.

[0118] In the embodiment, the codestream is a multi-layer video coding codestream, which may be called a scalable video coding codestream. The multi-layer video coding codestream may include one base layer subcodestream and at least one enhancement layer subcodestream, and the enhancement layer subcodestream may be an independent enhancement layer subcodestream or a dependent enhancement layer subcodestream. Taking the H.266 / VVC coding standard as an example, if the value of the syntax element vps_max_layers_minus1 in the video parameter set VPS is not 0, it can be determined that the codestream is a multi-layer video encoding codestream; furthermore, if the value of the syntax element vps_independent_layer_flag[i] (i>0) in the video parameter set VPS is 0, it indicates that the i-th enhancement layer is a dependent enhancement layer; if the value of the syntax element vps_independent_layer_flag[i] (i>0) in the video parameter set VPS is 1, it indicates that the i-th enhancement layer is an independent enhancement layer, and the base layer sub-codestream can be decoded independently to provide lower-quality video content. Typically, a base layer subcode stream can be decoded independently to provide low-quality or low-resolution video image content, an independent enhancement layer subcode stream can be decoded independently to provide high-quality or high-resolution video image content, and a dependent enhancement layer subcode stream must be decoded together with the base layer subcode stream to provide high-quality or high-resolution video image content.

[0119] Therefore, the decoded images analyzed in this operation may be decoded images of different quality or resolution of the same frame image.

[0120] At S2, film grain layer adaptation indication information corresponding to the decoded image is obtained.

[0121] Referring to S2 in the first embodiment, the film grain adaptation instruction information in the embodiment is film grain layer adaptation instruction information, and obtain the film grain layer adaptation instruction information corresponding to the decoded image.

[0122] Based on the film grain layer adaptation instruction information, it can be determined that the decoded images corresponding to different layers can correspond to different film grain parameters, and the decoded images corresponding to the different layers can have different image qualities or different resolutions.

[0123] The film grain layer adaptation instruction information may be a film grain layer adaptation enable identifier for instructing whether or not to enable film grain layer adaptation processing.

[0124] The film grain layer adaptation indication information may be a film grain layer adaptation identifier fg_layers_adaptive_flag for indicating whether to enable film grain layer adaptation processing for a video image. When fg_layers_adaptive_flag is 1, it indicates that the codestream enables film grain layer adaptation, and when fg_layers_adaptive_flag is 0, it indicates that the codestream does not enable film grain layer adaptation. The flag may be included in a video parameter set VPS, a sequence parameter set SPS, a picture parameter set PPS, or an adaptation parameter set APS, or may be included in a specific SEI message.

[0125] The film grain layer adaptation indication information may be layer identifier information corresponding to a film grain parameter or image resolution information corresponding to a film grain parameter. The layer identifier information corresponding to a film grain parameter is used to indicate a layer identifier corresponding to a decoded image in which the film grain parameter is used. The image resolution information corresponding to a film grain parameter is used to indicate a resolution corresponding to a decoded image in which the film grain parameter is used. For example, Table 6 describes related syntax elements and semantics information, taking as an example that layer information of film grain adaptation processing is included in a video parameter set.

[0126] [Table 6]

[0127] represents the maximum number of coding layers that will be included in the codestream after adding 1 to vps_max_layers_minus1; vps_fg_adaptive_enable_flag, if its value is 0, indicates that film grain adaptive processing is not allowed to be enabled, and if its value is 1, indicates that film grain adaptive processing is allowed to be enabled; vps_fg_layer_adaptive_flag[i], if its value is 0, indicates that film grain adaptive processing is not enabled for the coding layer with layer identifier i, and if its value is 1, indicates that film grain adaptive processing is enabled for the coding layer with layer identifier i; and fg_layer_parameters[i]() indicates the film grain parameters corresponding to the coding layer with layer identifier i.

[0128] The film grain layer adaptation instruction information may include layer identifier information corresponding to different film grain parameters. For example, when the layer identifier is 0, it corresponds to one set of film grain parameters; when the layer identifier is 1, it corresponds to another set of film grain parameters; when the layer identifier is 2, it corresponds to yet another set of film grain parameters; or when the layer identifier is greater than or equal to a predetermined threshold a (0 < a < 6), it corresponds to one set of film grain parameters, and when the layer identifier is less than a, it corresponds to another set of film grain parameters or there are no film grain parameters. The film grain layer adaptation instruction information may also include a plurality of image resolution information corresponding to different film grain parameters. For example, when the image resolution reaches 720p (1280 * 720), it corresponds to one set of film grain parameters; when the image resolution reaches 1080p (1920 * 1080), it corresponds to another set of film grain parameters; when the image resolution reaches 4k (1920 * 1080), it corresponds to yet another set of film grain parameters. The film grain layer adaptation instruction information may further be the film grain layer adaptation mode information (i.e., the film grain adaptation layer identifier) fg_layer_adaptive_mode. The film grain layer adaptation mode information is used to indicate which layer identifier or resolution corresponds to enabling adaptive film grain processing for the decoded image. For example, when fg_layer_adaptive_mode is 0, it means that the code stream applies film grain to the decoded images corresponding to all layer identifiers or resolutions; when fg_layer_adaptive_mode is 1, it means that adaptive film grain processing is enabled for the decoded images corresponding to other than the base layer (layer identifier is 0) in the code stream; when fg_layer_adaptive_mode is 2, it means that adaptive film grain processing is enabled only for the decoded image corresponding to the top enhancement layer (the value of the layer identifier is the same as the value of vps_max_layers_minus1) in the code stream, etc.

[0129] In S3, it is determined whether adaptive film grain processing is enabled for the decoded image; if yes, execute S4; if no, execute S1.

[0130] As described in S2, if the film grain layer adaptation instruction information is layer identifier information corresponding to film grain parameters or image resolution information corresponding to film grain parameters, and it can be ensured based on the information that a decoded image corresponding to at least one layer identifier does not have a corresponding film grain parameter, or if the film grain layer adaptation instruction information is film grain layer adaptive mode information and it can be determined based on the value of the film grain layer adaptive mode fg_layer_adaptive_mode that film grain processing is not enabled for the decoded image corresponding to at least one layer identifier, it is necessary to determine whether to enable adaptive film grain processing for the decoded image, and the determination method is as follows: determine whether film grain processing is enabled or whether there is no corresponding film grain parameter based on the layer identifier or resolution corresponding to the decoded image; if film grain processing is not enabled or there is no corresponding film grain parameter, return to S1, and the obtained codestream may be a codestream corresponding to an image of a frame next to the decoded image, or a sub-codestream of a different layer of an image of the same frame as the decoded image. The second decoded image obtained by analyzing the codestream may be an image of the frame next to the decoded image, or may be an image of the same frame as the decoded image but with a different quality or resolution.

[0131] In S4, a film grain parameter corresponding to the decoded image is obtained based on the film grain layer adaptation instruction information.

[0132] According to the description in S2, the film grain layer adaptation instruction information may be layer identifier information corresponding to a film grain parameter or image resolution information corresponding to a film grain parameter, and the corresponding film grain parameter is obtained based on the layer identifier or resolution of the decoded image.

[0133] In S5, a film grain corresponding to the decoded image is synthesized based on the film grain parameters.

[0134] Film grain corresponding to each region of the decoded image is synthesized based on a film grain model identifier included in the film grain parameters and film grain parameters corresponding to a film grain synthesis method designated by the film grain model identifier.

[0135] S6. Add the film grain to the decoded image. [Example]

[0136] This embodiment provides a video image codestream encoding method, which is applied to an input video image encoding device. The input of the device is a source video image, and the output is a video image codestream or a media file. Figure 8 is a flowchart of a video image codestream encoding method according to one embodiment, as shown in Figure 8.

[0137] At S1, an image is acquired, which may be a single image or an image of a frame in a video sequence.

[0138] At S2, the image is encoded to generate a codestream.

[0139] There are various related techniques for encoding an image to generate a codestream, such as the H.26x series standard, etc. Here, there is no restriction on which encoding standard is used to encode the image, so a description thereof will be omitted.

[0140] At S3, adaptive film grain modeling is performed on the film grain in the image based on the film grain characteristics in the image.

[0141] The film grain characteristics can be based on the film grain characteristics of the entire image or the film grain characteristics of different regions in the image, including film grain intensity, film grain scaling parameters, etc. Adaptive film grain modeling includes model-adaptive film grain modeling, region-adaptive film grain modeling, hierarchical adaptive film grain modeling, etc. Model-adaptive film grain modeling refers to selecting an optimal film grain modeling method based on the film grain characteristics in the image, including a frequency-based film grain modeling method and an autoregressive coefficient-based film grain modeling method, etc. Region-adaptive film grain modeling refers to performing film grain modeling based on the film grain characteristics of different regions in the image, and the film grain modeling methods used in different regions may be the same or different. When different regions adopt different film grain modeling methods, the film grain parameters corresponding to the different regions do not need to be completely identical, and hierarchical adaptive film grain modeling refers to determining which output image needs to have film grain enabled based on the film grain characteristics in the image.

[0142] The hierarchical adaptive film grain modeling is considered to be enabled only if the image employs scalable coding.

[0143] At S4, film grain adaptation indication information is determined and written into the codestream.

[0144] Determine film grain adaptation indication information corresponding to the image based on the adaptive film grain modeling process adopted in S3, where the film grain adaptation indication information includes at least one of model adaptive film grain indication information, region adaptive film grain indication information, and layer adaptive film grain indication information.

[0145] At S5, the codestream is output.

[0146] The codestream may be an image codestream, or a transport stream or media file containing an image codestream. [Example]

[0147] This embodiment shows an example of an SEI message for implementing film grain adaptation processing. Table 7 shows an SEI message syntax table for film grain adaptation processing, and the syntax structure and semantics information therein can correspond to the example combinations of various situations described in any of the above embodiments.

[0148] [Table 7]

[0149] Film grain adaptive SEI message semantic

[0150] The fg_adaptive_enabled_flag (i.e., film grain adaptive enable identifier) ​​indicates that, if fg_adaptive_enabled_flag=1, the codestream awaiting decoding associated with the SEI message allows film grain adaptive processing to be enabled, and, if fg_adaptive_enabled_flag=0, the codestream awaiting decoding associated with the SEI message does not allow film grain adaptive processing to be enabled. The film grain adaptation process may include at least one of film grain model adaptation, film grain region adaptation, and film grain layer adaptation. Correspondingly, the film grain adaptation enable identifier may be at least one of a film grain model adaptation enable identifier, a film grain region adaptation enable identifier, and a film grain layer adaptation enable identifier. The SEI message may include a film grain adaptation identifier for indicating whether film grain adaptation process is enabled for the codestream associated with the SEI message. fg_characteristic_base_parameters (i.e., film grain base parameters) indicate base film grain parameters adopted by the codestream associated with the SEI message, and the film grain parameters include film grain model parameters and film grain processing parameters. The film grain model parameter indication may be the same as the film grain parameters defined in the Film grain characteristic SEI message in the H.274 standard, or may be the film grain parameters defined in the AV1 standard. This will not be described in detail here. If fg_characteristic_base_parameters() in the SEI message is empty, it indicates that the film grain parameters associated with the previous codestream also apply to the codestream associated with the current SEI message.

[0151] The fg_layers_adaptive_flag (i.e., film grain layer adaptive identifier) ​​indicates that, if fg_layers_adaptive_flag=1, the codestream to be decoded associated with the SEI message has film grain layer adaptive processing enabled, and, if fg_layers_adaptive_flag=0, the codestream to be decoded associated with the SEI message does not have film grain layer adaptive processing enabled.

[0152] fg_model_adaptive_flag (film grain model adaptation identifier) ​​indicates that, if fg_model_adaptive_flag=1, the codestream to be decoded associated with the SEI message has film grain model adaptation processing enabled, and, if fg_model_adaptive_flag=0, the codestream to be decoded associated with the SEI message does not have film grain model adaptation processing enabled.

[0153] fg_model_adaptive_mode is a film grain model adaptive processing mode (film grain model adaptive mode) and is used to specify the adaptation range of the film grain model after switching.

[0154] fg_layer_adaptive_mode indicates the film grain layer adaptive processing mode (film grain layer adaptive mode), and the value of fg_layer_adaptive_mode determines which output image of different quality corresponding to the codestream needs to have film grain added to it. For specific values, refer to the detailed description of step 3 in Example 4. fg_region_adaptive_flag (i.e., film grain region adaptive identifier) ​​indicates that the codestream to be decoded associated with the SEI message supports enabling film grain region adaptive processing if fg_region_adaptive_flag=1, and indicates that the codestream to be decoded associated with the SEI message does not support enabling film grain region adaptive processing if fg_region_adaptive_flag=0.

[0155] num_of_adaptive_rect_regions (i.e., the number parameter) indicates the number of rectangular regions of the film grain adaptive processing, adaptive_rect_region_top[i] (i.e., the position parameter) indicates the top position of the i-th rectangular region of the film grain adaptive processing, adaptive_rect_region_left[i] (i.e., the position parameter) indicates the right-most position of the i-th rectangular region of the film grain adaptive processing, and adaptive_rect_region_width[i] (i.e., the size parameter) indicates the adaptive_rect_region_height[i] (i.e., size parameter) indicates the height of the rectangular region of the i-th film grain adaptive process; fg_region_model_parameters[i] (i.e., film grain region model parameters) indicates the film grain parameters corresponding to the rectangular region of the i-th film grain adaptive process, and the film grain parameters include film grain model parameters and film grain processing parameters.

[0156] fg_region_parameters[i] (i.e., film grain region processing parameters) indicates the film grain processing parameters corresponding to the i-th rectangular region of film grain adaptive processing, and the film grain model parameters and the film model grain parameters included in fg_characteristic_base_parameters() are kept consistent.

[0157] In this embodiment, only an example of the syntax structure and semantics information of the film grain adaptation processing SEI message is shown, and in actual application, only some of the syntax elements and other semantics information may be included, and any of the methods in embodiments 1 to 5 may be implemented.

[0158] An embodiment of the present application further provides a decoding device. Figure 9 is a structural diagram of a decoding device according to an embodiment. As shown in Figure 9, the decoding device comprises: a decoding module 310 configured to decode the codestream to obtain a decoded image; an acquisition module 320 configured to acquire film grain adaptation indication information corresponding to the decoded image; a determining module 330 configured to determine film grain parameters corresponding to the decoded image based on the film grain adaptation indication information; a synthesis module 340 configured to synthesize film grain corresponding to the decoded image based on the film grain parameters; an adding module 350 configured to add the film grain to the decoded image.

[0159] The decoding device of this embodiment performs adaptive film grain modeling and synthesis based on film grain characteristics, more accurately simulating the actual film grain, and providing a better viewing experience for viewers.

[0160] In one embodiment, the film grain adaptation indication information includes at least one of film grain model adaptation indication information, film grain region adaptation indication information, and film grain layer adaptation indication information.

[0161] In one embodiment, the film grain adaptation indication information comprises: a film grain adaptation enable identifier for indicating whether or not film grain adaptation processing is enabled; a film grain adaptation identifier for indicating whether film grain adaptation processing is enabled; and Contains at least one of the following:

[0162] In one embodiment, the film grain parameters include film grain model parameters and film grain processing parameters, and the film grain processing parameters correspond to film grain processing methods corresponding to the film grain model parameters.

[0163] In one embodiment, the film grain adaptation indication information comprises: The information includes at least one of a film grain model adaptation enable identifier for indicating whether or not to enable film grain model adaptation processing, a film grain model adaptation identifier for indicating whether or not to enable film grain model adaptation processing, and a film grain model adaptation mode for indicating the application range of the film grain model.

[0164] In one embodiment, the acquisition module 320 is configured to determine the film grain model adaptation indication information based on the SEI message type.

[0165] In one embodiment, the film grain adaptation instruction information includes film grain model adaptation instruction information, and determining film grain parameters corresponding to the decoded image based on the film grain adaptation instruction information includes: determining whether to switch film grain models based on the film grain model adaptation instruction information; if YES, obtaining film grain parameters corresponding to the film grain model after switching; and if NO, obtaining film grain parameters corresponding to the film grain model before switching.

[0166] In one embodiment, the film grain adaptation instruction information includes film grain region adaptation instruction information, which includes at least one of a film grain region adaptation enable identifier for instructing whether film grain region adaptation processing is enabled, a film grain region adaptation identifier for instructing whether film grain region adaptation processing is enabled, and region information corresponding to film grain parameters for indicating regions in the decoded image to which film grain parameters are applied.

[0167] In one embodiment, the film grain adaptation instruction information includes film grain region adaptation instruction information, determining film grain parameters corresponding to the decoded image based on the film grain adaptation instruction information includes obtaining film grain parameters corresponding to each region in the decoded image based on the film grain region adaptation instruction information, and synthesizing film grain corresponding to the decoded image based on the film grain parameters includes synthesizing film grain corresponding to the corresponding region based on the film grain parameters corresponding to each region, respectively.

[0168] In one embodiment, the film grain adaptation instruction information includes film grain layer adaptation instruction information, which includes at least one of a film grain layer adaptation enable identifier for indicating whether to enable film grain layer adaptation processing, a film grain layer adaptation identifier for indicating whether to enable film grain layer adaptation processing, a film grain adaptation layer identifier for indicating whether to enable film grain layer adaptation processing for a layer corresponding to the identifier, layer identifier information corresponding to the film grain parameter for indicating a layer identifier corresponding to a decoded image to which the film grain parameter has been applied, image resolution information corresponding to the film grain parameter for indicating an image resolution corresponding to the decoded image to which the film grain parameter has been applied, and a film grain layer adaptation mode for indicating an application range of the film grain layer adaptation processing.

[0169] In one embodiment, the film grain adaptation instruction information includes film grain layer adaptation instruction information, determining film grain parameters corresponding to the decoded image based on the film grain adaptation instruction information includes obtaining film grain parameters corresponding to each layer in the decoded image based on the film grain layer adaptation instruction information, and synthesizing film grain corresponding to the decoded image based on the film grain parameters includes synthesizing film grain corresponding to the corresponding layer based on the film grain parameters corresponding to each layer, respectively.

[0170] In one embodiment, the film grain adaptation indication information is included in an SEI message, and the SEI message includes: a film grain adaptation enable identifier for indicating whether a codestream associated with the SEI message enables film grain adaptation processing; a film grain adaptation identifier for indicating whether a codestream associated with the SEI message enables film grain model adaptation processing; a film grain base parameter for indicating a film grain base parameter adopted by the codestream associated with the SEI message; a film grain layer adaptation identifier for indicating whether a codestream associated with the SEI message enables film grain layer adaptation processing; a film grain model adaptation identifier for indicating whether a codestream associated with the SEI message enables film grain model adaptation processing; a film grain layer adaptation mode for indicating whether the codestream associated with the SEI message enables a film grain layer adaptation processing mode; a film grain region adaptation identifier for indicating whether the codestream associated with the SEI message enables film grain region adaptation processing; a number parameter for indicating the number of rectangular regions for film grain adaptation processing; a position parameter for indicating the position of the rectangular region for film grain adaptation processing; a size parameter for indicating the size of the rectangular region for film grain adaptation processing; a film grain region model parameter for indicating film grain model parameters corresponding to the rectangular region for film grain adaptation processing; and a film grain region processing parameter for indicating film grain processing parameters corresponding to the rectangular region for film grain adaptation processing.

[0171] The decoding device presented in this embodiment belongs to the same idea as the decoding method presented in the above embodiment, and the technical details not explained in detail in this embodiment can be referred to any of the above embodiments, and this embodiment has the same effect as the decoding method.

[0172] An embodiment of the present application further provides an encoding device. Figure 10 is a structural schematic diagram of an encoding device according to an embodiment. As shown in Figure 10, the encoding device includes: an encoding module 410 configured to encode the image to generate a codestream; a modeling module 420 configured to model film grain in the image based on film grain characteristics in the image; a determining module 430 configured to determine film grain adaptation indication information and write the film grain adaptation indication information into the codestream.

[0173] The encoding device of this embodiment performs adaptive film grain modeling and synthesis based on film grain characteristics, more accurately simulating the actual film grain, and providing a better viewing experience for viewers.

[0174] In one embodiment, the film grain adaptation indication information includes at least one of film grain model adaptation indication information, film grain region adaptation indication information, and film grain layer adaptation indication information.

[0175] In one embodiment, the film grain adaptation instruction information includes at least one of a film grain adaptation enable identifier for indicating whether film grain adaptation processing is enabled, and a film grain adaptation identifier for indicating whether film grain adaptation processing is enabled.

[0176] In one embodiment, the determination module 430 is configured to include at least one of: performing model adaptive film grain modeling for film grain in the image based on film grain characteristics in the image; performing region adaptive film grain modeling for film grain in the image based on film grain characteristics in the image; and performing hierarchical adaptive film grain modeling for film grain in the image based on film grain characteristics in the image.

[0177] The encoding device presented in this embodiment belongs to the same idea as the encoding method presented in the above embodiment, and the technical details not explained in detail in this embodiment can be referred to any of the above embodiments, and this embodiment has the same effect as the encoding method.

[0178] An embodiment of the present application further provides an electronic device, which may be a decoding side or an encoding side. Fig. 11 is a schematic diagram of the hardware structure of an electronic device according to an embodiment. As shown in Fig. 11, the electronic device according to the present application includes a processor 510 and a memory 520. The processor 510 in the communication node may be one or more. In Fig. 5, one processor 510 is taken as an example, and the memory 520 is configured to store one or more programs. When the one or more programs are executed by the one or more processors 510, the one or more processors 510 realize the decoding method or the encoding method according to an embodiment of the present application.

[0179] The electronic device further includes a communication device 530 , an input device 540 and an output device 550 .

[0180] The processor 510, memory 520, communication device 530, input device 540, and output device 550 in the electronic device can be connected via a bus or other methods, and FIG. 5 shows an example in which they are connected via a bus.

[0181] The input device 540 can be used to receive input numeric or textual information and generate key signal inputs related to user settings and function control of the electronic device. The output device 550 can include a display device such as a display.

[0182] The communication device 530 may include a receiver and a transmitter and is configured to transmit and receive information under the control of the processor 510.

[0183] The memory 520 can be used as a computer-readable storage medium to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the decoding or encoding method according to the embodiments of the present application (such as the decoding module 310, acquisition module 320, determination module 330, synthesis module 340, and addition module 350 in a decoding device, or the encoding module 410, modeling module 420, and determination module 430 in an encoding device). The memory 520 may include a program storage area and a data storage area, where the program storage area can store an operating system and / or application programs required for at least one function, and the data storage area can store data generated based on the use of the communication node. The memory 520 may also include high-speed random access memory and may further include non-volatile memory, such as at least one magnetic disk storage device, flash memory, or other non-volatile solid-state storage device. In some embodiments, the memory 520 may further include memory located remotely from the processor 510, and these remote memories may be connected to the device via a network. Examples of such networks include the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0184] An embodiment of the present application further provides a storage medium storing a computer program that, when executed by a processor, realizes any of the decoding methods or encoding methods of the embodiment of the present application, wherein the decoding method is applied to a decoding side, and includes: decoding a codestream to obtain a decoded image; obtaining film grain adaptation indication information corresponding to the decoded image; determining film grain parameters corresponding to the decoded image based on the film grain adaptation indication information; synthesizing film grain corresponding to the decoded image based on the film grain parameters; and adding the film grain to the decoded image.

[0185] Alternatively, the encoding method is applied to the code side, and includes: encoding an image to generate a codestream; modeling film grain in the image based on film grain characteristics in the image; determining film grain adaptation indication information; and writing the film grain adaptation indication information into the codestream.

[0186] The computer-readable medium of the embodiments of the present application may be any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. Examples (non-exhaustive list) of computer-readable storage media include an electrical connection having one or more leads, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium may be any tangible medium containing or storing a program that can be used in or in connection with an instruction execution system, apparatus, or device.

[0187] A computer-readable signal medium may include a propagated data signal, either in baseband or as part of a carrier wave, having computer-readable program code carried therein. Such a propagated data signal may take various forms, including an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which is capable of transmitting, propagating, or transporting a program for use in or in connection with an instruction execution system, apparatus, or device.

[0188] The program code contained in the computer readable medium may be transmitted over any suitable medium, including electrical wire, optical cable, radio frequency (RF), or the like, or any suitable combination of the above.

[0189] Computer program code for carrying out the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may run entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. When referring to a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., connected via the Internet using an Internet Service Provider).

[0190] The above are merely illustrative examples of the present application, and are not intended to limit the scope of protection of the present application.

[0191] Those skilled in the art will appreciate that the term user terminal includes any suitable type of wireless user equipment, including, for example, a mobile phone, a portable data processing device, a portable network browser, or a vehicle mounted mobile device.

[0192] In general, various embodiments of the present application may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware while other aspects may be implemented in firmware or software executable by a controller, microprocessor, or other computing device, and the present application is not limited thereto.

[0193] Embodiments of the present application may be implemented by execution of computer program instructions by a data processor of a mobile device, for example in a processor entity, by hardware, or by a combination of software and hardware. The computer program instructions may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or target code written in any combination of one or more programming languages.

[0194] Any logic flow block diagrams in the figures herein may represent program operations, interconnected logic circuits, modules, and functions, or a combination of program operations and logic circuits, modules, and functions. Computer programs may be stored in memory. The memory may be of any type suitable for the local technology environment and may be implemented with any suitable data storage technology, such as read-only memory (ROM), random access memory (RAM), optical storage devices and systems (Digital Video Disc (DVD) or Compact Disc (CD)), etc. Computer-readable media may also include non-transitory storage media. Data processors may be of any type suitable for the local technology environment, such as general-purpose computers, special-purpose computers, microprocessors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), and processors based on multi-core processor architectures.

Claims

1. A decoding method applied to a decoding side, comprising: decoding the codestream to obtain a decoded image; obtaining film grain adaptation indication information corresponding to the decoded image; determining film grain parameters corresponding to the decoded image based on the film grain adaptation indication information; synthesizing a film grain corresponding to the decoded image based on the film grain parameters; adding the film grain to the decoded image. Decryption method.

2. The film grain adaptation indication information comprises: at least one of film grain model adaptation instruction information, film grain region adaptation instruction information, and film grain layer adaptation instruction information; The method of claim 1.

3. The film grain adaptation indication information comprises: a film grain adaptation enable identifier for indicating whether or not film grain adaptation processing is enabled; a film grain adaptation identifier for indicating whether film grain adaptation processing is enabled; The method of claim 1.

4. the film grain parameters include film grain model parameters and film grain processing parameters; the film grain processing parameters correspond to a film grain processing method corresponding to the film grain model parameters; The method of claim 1.

5. The film grain adaptation indication information comprises: a film grain model adaptation enable identifier for indicating whether or not the film grain model adaptation process is enabled; a film grain model adaptation identifier for indicating whether film grain model adaptation processing is enabled; and a film grain model adaptation mode for indicating the application range of the film grain model; The method of claim 1.

6. Obtaining film grain adaptation indication information corresponding to the decoded image includes: determining the film grain model adaptation indication information based on a supplemental enhancement information (SEI) message type; The method of claim 1.

7. the film grain adaptation instruction information includes film grain model adaptation instruction information; determining a film grain parameter corresponding to the decoded image based on the film grain adaptation indication information, determining whether to switch film grain models according to the film grain model adaptation instruction information; In response to switching the film grain model, acquiring film grain parameters corresponding to the switched film grain model; In response to not switching the film grain model, obtaining film grain parameters corresponding to the film grain model before switching; The method of claim 1.

8. the film grain adaptation indication information includes film grain region adaptation indication information; The film grain region adaptation indication information comprises: a film grain region adaptation enable identifier for indicating whether or not the film grain region adaptation process is enabled; a film grain region adaptation identifier for indicating whether film grain region adaptation processing is enabled; and region information corresponding to the film grain parameters for indicating a region in the decoded image to which the film grain parameters are applied. The method of claim 1.

9. the film grain adaptation indication information includes film grain region adaptation indication information; determining a film grain parameter corresponding to the decoded image based on the film grain adaptation indication information, obtaining film grain parameters corresponding to different regions in the decoded image based on the film grain region adaptation indication information; synthesizing film grain corresponding to the decoded image based on the film grain parameters, synthesizing film grain corresponding to the corresponding regions based on film grain parameters corresponding to the different regions; The method of claim 1.

10. the film grain adaptation indication information includes film grain layer adaptation indication information; The film grain layer adaptation indication information comprises: a film grain layer adaptation enable identifier for indicating whether or not the film grain layer adaptation process is enabled; a film grain layer adaptation identifier for indicating whether film grain layer adaptation processing is enabled; and a film grain adaptation layer identifier for indicating whether film grain layer adaptation processing is enabled for the layer corresponding to the identifier; and layer identifier information corresponding to the film grain parameter for indicating a layer identifier corresponding to the decoded image to which the film grain parameter has been applied; image resolution information corresponding to the film grain parameters for indicating an image resolution corresponding to the decoded image to which the film grain parameters have been applied; a film grain layer adaptation mode for indicating the application range of the film grain layer adaptation process; The method of claim 1.

11. the film grain adaptation indication information includes film grain layer adaptation indication information; determining a film grain parameter corresponding to the decoded image based on the film grain adaptation indication information, obtaining film grain parameters corresponding to different layers in the decoded image based on the film grain layer adaptation instruction information; synthesizing film grain corresponding to the decoded image based on the film grain parameters, synthesizing film grain corresponding to a corresponding layer based on film grain parameters corresponding to the different layers; The method of claim 1.

12. the film grain adaptation indication information is included in an SEI message; The SEI message comprises: a film grain adaptation enable identifier for indicating whether a codestream associated with the SEI message is enabled for film grain adaptation processing; and a film grain adaptation identifier for indicating whether a codestream associated with the SEI message enables film grain model adaptation processing; and a film grain base parameter for indicating a film grain base parameter adopted by a codestream associated with the SEI message; a film grain layer adaptation identifier for indicating whether the codestream associated with the SEI message enables film grain layer adaptation processing; a film grain model adaptation identifier for indicating whether a codestream associated with the SEI message enables film grain model adaptation processing; and a film grain model adaptation mode for indicating the application range of the film grain model; a film grain layer adaptation mode for indicating whether the codestream associated with the SEI message enables a film grain layer adaptation processing mode; a film grain region adaptation identifier for indicating whether a codestream associated with the SEI message enables film grain region adaptation processing; and a number parameter for indicating the number of rectangular regions for film grain adaptation; a position parameter for indicating the position of a rectangular region for film grain adaptation processing; a size parameter for indicating the size of a rectangular region for film grain adaptation processing; a film grain region model parameter for indicating a film grain model parameter corresponding to a rectangular region of film grain adaptation processing; and a film grain region processing parameter for indicating film grain processing parameters corresponding to a rectangular region of film grain adaptive processing. The method of claim 1.

13. A coding method applied to the code side, comprising: encoding the image to generate a codestream; modeling film grain in the image based on film grain characteristics in the image; determining film grain adaptation indication information and writing the film grain adaptation indication information into the codestream; Encoding method.

14. the film grain adaptation instruction information includes at least one of film grain model adaptation instruction information, film grain region adaptation instruction information, and film grain layer adaptation instruction information; The method of claim 13.

15. The film grain adaptation indication information comprises: a film grain adaptation enable identifier for indicating whether or not film grain adaptation processing is enabled; a film grain adaptation identifier for indicating whether film grain adaptation processing is enabled; The method of claim 13.

16. Modeling film grain in the image based on film grain characteristics in the image includes: performing model adaptive film grain modeling for film grain in the image based on film grain characteristics in the image; performing region adaptive film grain modeling for film grain in the image based on film grain characteristics in the image; performing hierarchical adaptive film grain modeling for film grain in the image based on film grain characteristics in the image. The method of claim 13.

17. a memory and at least one processor; the memory is configured to store at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the decoding method according to any one of claims 1 to 12 or the encoding method according to any one of claims 13 to 16. electronic equipment.

18. A computer program is stored which, when executed by a processor, implements the decoding method according to any one of claims 1 to 12 or the encoding method according to any one of claims 13 to 16. A computer-readable storage medium.

Images