Gain map encoding method and apparatus, gain map decoding method and apparatus, device, and medium

Abstract

The present application relates to the technical field of communications, and discloses a gain map encoding method and apparatus, a gain map decoding method and apparatus, a device, and a medium. The method of the present application comprises: determining a first picture region and a second picture region in a gain map, the brightness of pixel points in the first picture region being greater than a first threshold, and the brightness of pixel points in the second picture region being less than or equal to the first threshold (S301); determining an encoding strategy of the gain map on the basis of the first picture region and the second picture region (S302); and encoding the gain map on the basis of the encoding strategy to obtain encoded information of the gain map (S303).

Description

Gain map encoding method, decoding method, device, equipment and medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on July 18, 2023, with application number 202310884272.1 and invention name “Gain map encoding method, decoding method, device, equipment and medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application belongs to the field of communication technology, and specifically relates to a gain map encoding method, decoding method, device, equipment and medium. Background Art

[0003] With advances in image acquisition and display technology, high dynamic range imaging and display can now be achieved on electronic devices (such as smartphones and tablets). Although an increasing number of devices support the display of high dynamic range still images, there are still a large number of electronic devices that only support standard dynamic range (SDR). High dynamic range images captured can be displayed well on both high dynamic range (HDR) and SDR display devices. HDR acquisition, storage, and display solutions based on gain maps are a commonly used method in the industry.

[0004] However, as additional information besides SDR and HDR images, the gain map usually increases the storage space requirements.

[0005] Summary of the Invention

[0006] The purpose of the embodiments of the present application is to provide a gain map encoding method, apparatus, device and medium, which can solve the problem of increased storage space requirements of existing gain maps.

[0007] In a first aspect, an embodiment of the present application provides a gain map encoding method, including:

[0008] Determine a first image area and a second image area in the gain map, wherein the brightness of pixels in the first image area is greater than a first threshold, and the brightness of pixels in the second image area is less than or equal to the first threshold;

[0009] determining a coding strategy for the gain map according to the first image region and the second image region;

[0010] The gain map is encoded according to the encoding strategy to obtain encoding information of the gain map.

[0011] In a second aspect, an embodiment of the present application provides a gain map decoding method, including:

[0012] Obtaining encoding information of the gain map;

[0013] decoding the encoded information to obtain the gain map;

[0014] The first image area and the second image area of ​​the gain map are encoded using different encoding strategies, the brightness of the pixels in the first image area is greater than a first threshold, and the brightness of the pixels in the second image area is less than or equal to the first threshold.

[0015] In a third aspect, an embodiment of the present application provides a gain map encoding device, including:

[0016] a first processing module, configured to determine a first image region and a second image region in a gain map, wherein the brightness of pixels in the first image region is greater than a first threshold, and the brightness of pixels in the second image region is less than or equal to the first threshold;

[0017] a second processing module, configured to determine a coding strategy for the gain map according to the first image area and the second image area;

[0018] The encoding module is configured to encode the gain map according to the encoding strategy to obtain encoding information of the gain map.

[0019] In a fourth aspect, an embodiment of the present application provides a gain map decoding device, including:

[0020] An acquisition module, used for acquiring encoding information of the gain map;

[0021] A decoding module, configured to decode the encoded information to obtain the gain map;

[0022] The first image area and the second image area of ​​the gain map are encoded using different encoding strategies, the brightness of the pixels in the first image area is greater than a first threshold, and the brightness of the pixels in the second image area is less than or equal to the first threshold.

[0023] In a fifth aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0024] In a sixth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0025] In the seventh aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method described in the first aspect or the second aspect.

[0026] In an eighth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the method described in the first aspect or the second aspect.

[0027] In an embodiment of the present application, after the gain map is divided into a first image region and a second image region using a first threshold, a coding strategy for the gain map can be determined based on the first image region and the second image region. The gain map is then encoded according to the determined coding strategy to obtain coding information for the gain map. In this way, image regions with different brightness are encoded using an appropriate coding strategy, which reduces the amount of data and saves storage space compared to directly storing the gain map. Because different coding strategies correspond to image regions with different brightness, compared to a single coding strategy, while ensuring high-quality encoding and reconstruction of image regions containing valid information, lower-quality encoding can be used for other image regions, reducing the storage space required for the coding information of the gain map. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a schematic diagram of one of the gain map applications;

[0029] Figure 2 is the second schematic diagram of gain map application;

[0030] FIG3 is a schematic diagram of the encoding method flow in an embodiment of the present application;

[0031] FIG4 is a schematic diagram of one of the application flow charts of the encoding method according to an embodiment of the present application;

[0032] FIG5 is a second schematic diagram of an application flow of the encoding method according to an embodiment of the present application;

[0033] FIG6 is a schematic flow chart of a decoding method according to an embodiment of the present application;

[0034] FIG7 is a schematic diagram of a decoding method application process according to an embodiment of the present application;

[0035] FIG8 is a second schematic diagram of the decoding method application process according to an embodiment of the present application;

[0036] FIG9 is a schematic diagram of a module structure of a device according to an embodiment of the present application;

[0037] FIG10 is a second schematic diagram of the module structure of the device according to an embodiment of the present application;

[0038] FIG11 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

[0039] FIG12 is a schematic structural diagram of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the accompanying drawings in the embodiments of this application to clearly describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0041] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0042] For ease of understanding, some of the contents involved in the embodiments of this application are described below:

[0043] Gain map (also known as gain map or enhancement map) is usually generated when capturing HDR images. As additional information, it can be stored in a specific file format, such as the format of the image or as additional information of a certain image format, and then encoded and transmitted. When displayed, the gain map is decoded. The gain map is generated based on the HDR image and its corresponding SDR image after tone mapping. The use of the gain map can be SDR and Gain map to generate the corresponding HDR image, or HDR and Gain map to generate the corresponding SDR image, as shown in Figures 1 and 2. In both formats, the Gain map needs to be stored.

[0044] Furthermore, with gain map-based HDR solutions, the areas within the HDR image that require brightening via the gain map are limited. This is primarily due to the following reasons: 1) If the brightened area is too large, it can cause visual discomfort to the human eye. Therefore, when generating the gain map, it is important to appropriately control the size of the brightened area. 2) If the brightened area is too large, the visual benefits of HDR are reduced. 3) In natural imagery, the areas of highlight brightness are typically limited.

[0045] The following describes in detail the gain map encoding method, decoding method, apparatus, device, and medium provided by the embodiments of the present application through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0046] As shown in FIG3 , a gain map encoding method according to an embodiment of the present application includes:

[0047] Step 301: Determine a first image area and a second image area in a gain map, wherein the brightness of pixels in the first image area is greater than a first threshold, and the brightness of pixels in the second image area is less than or equal to the first threshold.

[0048] The first threshold is a brightness threshold. In this step, for the gain map, the brightness of the pixels is compared with the first threshold to determine the first image area where the brightness of the pixels is greater than the first threshold, and the second image area where the brightness of the pixels is less than or equal to the first threshold.

[0049] Step 302: Determine a coding strategy for the gain map according to the first image region and the second image region.

[0050] Step 303: Encode the gain map according to the encoding strategy to obtain encoding information of the gain map.

[0051] Thus, according to steps 301-303 above, after the gain map is divided into a first image region and a second image region using the first threshold, a coding strategy for the gain map can be determined based on the first and second image regions. The gain map is then encoded according to the determined coding strategy to obtain coded information of the gain map. In this way, image regions with varying brightness are encoded using an appropriate coding strategy. This reduces the amount of data and saves storage space compared to directly storing the gain map. Because different coding strategies correspond to image regions with varying brightness, compared to a single coding strategy, while ensuring high-quality encoding and reconstruction of image regions containing valid information, lower-quality encoding can be used for other image regions, reducing the storage space required for the coded information of the gain map.

[0052] For example, as shown in FIG4 , a gain map is generated by combining an HDR image and an SDR image. Then, the gain map brightening area is identified, that is, the first image area (brightened area) in which the pixel value of the pixel point in the gain map is greater than the first threshold and the second image area (non-brightened area) in which the pixel value is less than or equal to the first threshold are determined. Next, based on the first image area and the second image area, the encoding strategies for different areas are determined respectively. Next, the first image area and the second image area are encoded according to different encoding strategies to obtain the encoding information of the gain map.

[0053] Optionally, determining the first image area and the second image area in the gain map includes:

[0054] Determining a segmentation size based on a target tone mapping method; wherein the target tone mapping method is a tone mapping method used when generating a standard dynamic range image from a high dynamic range image;

[0055] Based on the segmentation size, dividing the gain map into a plurality of regions;

[0056] Based on the first threshold, the first image region and the second image region among the plurality of regions are determined.

[0057] That is, the sizes of the first image area and the second image area are related to the tone mapping method used to generate the SDR image from the HDR image, and the method for generating the gain map based on the HDR image and the generated SDR image. When the encoding device (such as a user device) uses a tone mapping method, that is, a target tone mapping method, to generate the SDR image and the corresponding gain map, the segmentation size is related to the target tone mapping method, the method for generating the gain map based on HDR and SDR, and the encoding block size supported by the image encoder used. Generally, the segmentation size can be determined based on the minimum coding unit block size supported by the image encoder, such as when the segmentation size is greater than or equal to the minimum coding unit block size. After determining the segmentation size, the gain map is segmented based on the segmentation size to obtain multiple regions, and then combined with the first threshold, the multiple regions are divided into the first image region and the second image region. For example, after obtaining 20 regions based on the segmentation size, 8 first image regions and 12 second image regions are further obtained based on the first threshold.

[0058] Of course, the first image region and the second image region can also be identified based on an artificial intelligence learning method.

[0059] Among them, the first threshold is a pre-set brightness threshold. The setting of the first threshold can be adjusted according to different application scenarios and needs. For example, in a certain application scenario, due to its high requirements for HDR / SDR image quality, the threshold setting can set a larger area as the first image area (also called the brightened area, also called the ROI area of ​​interest), and a smaller area as the second image area (also called the non-brightened area, also called the non-ROI area); in another application scenario, due to its limitation on the storage space increased by the gain map, the threshold setting can set a smaller area as the first image area. The setting of the first threshold can also be related to the content of the image. For example, in a landscape picture during the day, there is sufficient light, and the first image area can be selected as a larger area by setting the threshold; in some night scenes, the area with lights is limited, and the first image area can be selected as a smaller area by setting the threshold.

[0060] Optionally, in this embodiment, after the gain map is generated and before encoding, some gain map preprocessing operations can be performed. For example, normalization is performed based on the pixel information of the gain map to convert the gain map pixels into floating-point numbers between 0 and 1; the normalized floating-point numbers are then digitally encoded according to a selected bit depth, such as 8-bit or 10-bit. Gain map segmentation and recognition can be performed before or after these preprocessing operations.

[0061] In this embodiment, the gain map can also be segmented based on the coding unit size selected by the encoder, such as the block size of a CTU. In this way, the size of the first image region and the second image region is the block size of one or more CTUs. For example, if the gain map is encoded using JPEG, the segmentation size of the gain map can be determined based on the 8x8 block size of the DCT transform in JPEG. Of course, the gain map can also be segmented based on other sizes, which will not be further described here.

[0062] In addition, in this embodiment, optionally, determining the encoding strategy of the gain map according to the first image area and the second image area includes:

[0063] In a case where the image unit to be encoded belongs to the first image area, determining the encoding strategy to be the first strategy;

[0064] In a case where the image unit to be encoded belongs to the second image area, determining the encoding strategy to be the second strategy;

[0065] The encoding quality of the first strategy is higher than the encoding quality of the second strategy.

[0066] That is to say, the first image region corresponds to the first strategy with higher coding quality, and the second image region corresponds to the second strategy with lower coding quality. For the image unit to be coded, the corresponding first strategy or second strategy will be adopted for coding based on the region to which the image unit to be coded belongs.

[0067] Among them, the high or low coding quality refers to whether the decoded image is closer to the original image before coding. For example, the higher the coding quality, the larger the Peak Signal to Noise Ratio (PSNR) and / or Structural SIMilarity (SSIM) in the image evaluation index of the decoded image.

[0068] As an implementation method, different coding strategies can be different coding rates. That is, the coding rate of the first image region is greater than the coding rate of the second image region.

[0069] In this embodiment, the image unit to be coded is the smallest coding unit supported by the image encoder. The segmentation block size of the gain map can be the same as the coding unit size supported by the image encoder. For example, in an image encoder based on HEVC, the image unit to be coded and the segmentation size are the same as the block size of the CTU. The image unit to be coded is combined with the first image region as the input information for coding strategy selection.

[0070] Optionally, in this embodiment, the first strategy includes multiple coding rules with different coding qualities;

[0071] When the image unit to be coded belongs to the first image region, determining that the coding strategy is the first strategy includes: [[ID=​​​​​​​​​​As an optional implementation, the first image region can be divided into a region with rich texture and a region with flat texture based on texture information such as edge information and high frequency information. The region with rich texture is encoded with the highest (first level) image quality, while the region with flat texture is encoded with the lowest (third level) image quality.

[0076] Similarly, for the second image area, a finer encoding quality level may be further set according to the brightness level or texture information, which will not be described in detail here.

[0077] Optionally, in this embodiment, determining the encoding strategy of the gain map according to the first image area and the second image area includes:

[0078] When the to-be-encoded image unit belongs to an edge portion of the first image area that is directly adjacent to the second image area, or an edge portion of the second image area that is directly adjacent to the first image area, determining that the encoding strategy is the third strategy;

[0079] The encoding quality of the third strategy is higher than that of the second strategy, and lower than that of the first strategy.

[0080] That is, the first strategy applies to all portions of the first image region except for the edge portion directly adjacent to the second image region; the second strategy applies to all portions of the second image region except for the edge portion directly adjacent to the first image region. A third strategy, whose encoding quality is higher than the second strategy but lower than the first strategy, is applied to the edge portion of the first image region directly adjacent to the second image region, and to the edge portion of the second image region directly adjacent to the first image region. This strategy ensures a smooth transition in quality between the adjacent regions of the first image region and the second image region, avoiding perceptible subjective distortion caused by large quality deviations between adjacent regions.

[0081] Optionally, in this embodiment, encoding the gain map according to the encoding strategy to obtain encoding information of the gain map includes:

[0082] adjusting encoding parameters based on the encoding strategy;

[0083] Encoding the gain map based on the encoding parameters to obtain the encoding information;

[0084] The encoding parameters include at least one of the following:

[0085] Quantization parameters;

[0086] transform coding coefficients;

[0087] Rate-distortion function.

[0088] That is, during the encoding process, the corresponding encoding quality or encoding bit rate will be achieved by adjusting at least one of the quantization parameter, transform coding coefficient, and rate-distortion function according to the determined encoding strategy. For example, for the image area corresponding to the first strategy, a smaller quantization parameter will be used for encoding to retain more information, and even lossless encoding can be performed when the highest encoding quality is selected; for the image area corresponding to the second strategy, a larger quantization parameter will be used for encoding, which will increase signal distortion and reduce the amount of encoded data. For another example, for the image area corresponding to the first strategy, most of the transform coding coefficients of the coding block will be retained; for the image area corresponding to the second strategy, some transform coding coefficients will be discarded, such as the high-frequency part of the transform coding coefficient can be directly set to zero or discarded to reduce the amount of data. For another example, for the image area corresponding to the first strategy, the rate-distortion function will be adjusted with a high bit rate as the target; for the image area corresponding to the second strategy, the rate-distortion function will be adjusted with a low bit rate as the target.

[0089] Optionally, in this embodiment, the quantization parameter includes at least one of the following:

[0090] Image-level quantization parameters;

[0091] Slice-level quantization parameters;

[0092] Coding tree unit (CTU) level quantization parameters;

[0093] Tile-level quantization parameters;

[0094] Coding unit (CU) level quantization parameter.

[0095] Taking HEVC still image coding (HEIC, HEIF) as an example, adjusting the quantization parameter to implement a coding strategy can be done at the picture level, slice level, CTU level, or CU level. In HEVC, CTU is the default minimum coding unit level where different quantization parameters can be set. Picture-level quantization parameters are set in the picture-level parameter set, specifying the initial value of the picture-level quantization parameter, such as the parameter init_qp_minus26 in the HEVC Picture Parameter Set (PPS). Slice-level quantization strategy can be implemented by setting slice_qp_delta in the slice header, which specifies the initial quantization parameter value for encoding and decoding in the current slice until the CU-level quantization offset value CuQpDeltaVal appears at the CU codec level. init_qp_minus26 plus 26 specifies the initial value of the encoding and decoding quantization parameter for each slice using the current PPS. For non-zero slice_qp_delta, the quantization parameter of the current slice is init_qp_minus26+26+slice_qp_delta. In order to achieve more refined rate control for different areas within the image, different quantization parameter settings or other rate-distortion rate control strategies can be supported by CU. In HEVC, this can be done by turning on the cu_qp_delta_enabled_flag flag in the PPS. cu_qp_delta_abs specifies the absolute value of the difference between the current CU luminance signal quantization parameter and the previous CU quantization parameter, and cu_qp_delta_sign_flag specifies the sign of the difference between the current CU luminance signal quantization parameter and the previous CU quantization parameter. CU-level rate control can be achieved through these CU-level quantization parameter settings.

[0096] Of course, other encoders, such as still image compression formats based on VVC or AV1 (AVIF) and other codecs, use a similar strategy to HEVC to adjust the quantization parameters at the image level, slice, CTU, or CU level to achieve coding quality control of the first and second image areas of the gain map. The gain map is divided into blocks based on the coding block size allowed by the standard. HEVC supports a maximum CTU block size of 64x64, VVC supports a maximum CTU block size of 256x256, and AV1 supports a maximum superblock block size of 128x128.

[0097] In this embodiment, different slices can be further defined for the distribution of the first image region and the second image region in the gain map, and different slice-level quantization parameters can be used to achieve different encoding qualities, such that the first image region has higher encoding quality than the second image region. Alternatively, different CU-level quantization parameters can be defined to achieve higher encoding quality for the first image region than for the second image region.

[0098] For example, HEVC supports a maximum CTU block size of 64x64. After determining the first image area and the second image area in the gain map, the gain map is divided into units to be coded according to the CTU block size. At this time, the divided units to be coded, the first image area, and the second image area serve as input information for the coding strategy, and the specific coding strategy is determined by judging the area to which the units to be coded belong. If the distribution of the first image area and the second image area is suitable for coding by slice, the coding quality control of the first image area and the second image area can be achieved by adjusting the coding strategy at the slice level, such as adjusting slice_qp_delta. If the CTU block distribution of the unit to be coded belongs to the first image area and cannot be applied to slice coding, the corresponding coding strategy can be implemented at the CU / CTU level, such as adjusting the CTU-level quantization parameter. At this time, the CTU block division, the first image area, and the second image area can be used as auxiliary information for selecting the coding strategy, and as input information for determining the coding parameters of the first image area and the second image area. Specifically, since the second image region does not contain any brightening information, the CTU encoding bitrate for the second image region can be set to a low bitrate. For example, the quantization parameter can be set to the maximum allowed value, minimizing the quantization coefficients to zero, thereby reducing the bitrate. For the first image region, a smaller quantization parameter or lossless compression can be used to ensure high-quality reconstructed images in this region.

[0099] Optionally, in this embodiment, encoding the gain map based on the encoding parameter to obtain the encoding information includes:

[0100] When the gain map is a multi-channel gain map and the encoding parameters include quantization parameters, different quantization parameters are used for encoding in different channels.

[0101] In this way, a multi-channel gain map can set different quantization strategies for different channels (such as luma and CB / CR chroma signals), that is, different channels will be encoded with different quantization parameters. For example, different quantization parameters for chroma and luma signals at different encoding and decoding levels can be specified through pps_cb_qp_offset, pps_cr_qp_offset, slice_cb_qp_offset, slice_cr_qp_offset, etc.

[0102] Optionally, in this embodiment, before determining the first image area and the second image area in the gain map, the method further includes:

[0103] The gain map is downsampled.

[0104] That is, the original gain map can be downsampled first, such as 1 / 4 or 1 / 16 downsampling, and then the first image area and the second image area are determined based on the downsampled gain map. Then, the encoding strategy of the two is determined based on the first image area and the second image area, and the downsampled gain map is encoded according to the encoding strategy, as shown in Figure 5, further reducing the storage space.

[0105] Among them, downsampling can use Bilinear filters, Lanczos filters, edge-preserving filters, artificial intelligence (AI)-based methods, etc.

[0106] To sum up, the gain map encoding method of the embodiment of the present application adopts an applicable encoding strategy for encoding image areas with different brightness. Without reducing the quality of HDR or SDR images reconstructed based on the gain map, it reduces the amount of data and saves storage space compared to directly storing the gain map.

[0107] The gain map encoding method provided in the embodiment of the present application can be executed by a gain map encoding device. In the embodiment of the present application, the gain map encoding method performed by the gain map encoding device is taken as an example to illustrate the gain map encoding device provided in the embodiment of the present application.

[0108] As shown in FIG6 , a gain map decoding method according to an embodiment of the present application includes:

[0109] Step 601, obtaining coding information of the gain map;

[0110] Step 602: Decode the coded information to obtain the gain map;

[0111] The first image area and the second image area of ​​the gain map are encoded using different encoding strategies, the brightness of the pixels in the first image area is greater than a first threshold, and the brightness of the pixels in the second image area is less than or equal to the first threshold.

[0112] In this way, after obtaining the coded information of the gain map coded by the method of the above embodiment, the gain map is obtained by decoding it.

[0113] Among them, if the decoding end is also the encoding end, obtaining the encoding information of the gain map can be extracting the encoding information of the gain map from the storage space; if the decoding end is different from the encoding end, obtaining the encoding information of the gain map can be the encoding information of the gain map received by the decoding end from the encoding end.

[0114] For example, as shown in Figure 7, the encoded gain map information is decoded to obtain the gain map. Next, an HDR image is generated based on the decoded gain map and the SDR image. If the HDR plus corresponding gain map storage format is used, the HDR and decoded gain map are combined to generate the SDR image.

[0115] It should be noted that, since the encoding method in the above embodiment complies with the encoding standard, such as HEVC / VVC / AV1 or other standards, the decoding end does not need special processing and reconstructs the gain map according to normal bitstream decoding.

[0116] Optionally, decoding the encoded information to obtain the gain map includes:

[0117] After decoding the encoded information, the decoded image is upsampled to obtain a gain map.

[0118] That is, if the gain map is downsampled at the encoding end, the decoding end needs to upsample the image after decoding so that the upsampled gain map can be combined with SDR / HDR to generate the corresponding HDR / SDR image, as shown in Figure 8. Among them, upsampling can use Bilinear filters, Lanczos filters, edge-preserving filters, AI-based methods, etc.

[0119] It's important to note that after the gain map is decoded, some post-processing may be performed before it is applied to the SDR / HDR image to generate the HDR / SDR image. For example, the decoded gain map may be converted to floating-point values ​​based on the encoding bit depth, and then the floating-point values ​​may be restored to the logarithmic domain using inverse normalization. Upsampling of the gain map can be performed before or after these post-processing steps. Finally, the gain map is applied to the SDR / HDR image.

[0120] It should be noted that this method is implemented in conjunction with the gain map encoding method of the above embodiment. The implementation of the above method embodiment is applicable to this method and can achieve the same technical effect.

[0121] As shown in FIG9 , a gain map encoding apparatus 900 according to an embodiment of the present application includes:

[0122] A first processing module 910 is configured to determine a first image region and a second image region in a gain map, wherein the brightness of pixels in the first image region is greater than a first threshold, and the brightness of pixels in the second image region is less than or equal to the first threshold;

[0123] A second processing module 920 is configured to determine a coding strategy for the gain map according to the first image region and the second image region;

[0124] The encoding module 930 is configured to encode the gain map according to the encoding strategy to obtain encoding information of the gain map.

[0125] Optionally, the first processing module is further configured to:

[0126] Determining a segmentation size based on a target tone mapping method; wherein the target tone mapping method is a tone mapping method used when generating a standard dynamic range image from a high dynamic range image;

[0127] Based on the segmentation size, dividing the gain map into a plurality of regions;

[0128] Based on the first threshold, the first image region and the second image region among the plurality of regions are determined.

[0129] Optionally, the second processing module is further configured to:

[0130] In a case where the image unit to be encoded belongs to the first image area, determining the encoding strategy to be the first strategy;

[0131] In a case where the image unit to be encoded belongs to the second image area, determining the encoding strategy to be the second strategy;

[0132] The encoding quality of the first strategy is higher than the encoding quality of the second strategy.

[0133] Optionally, the first strategy includes a plurality of encoding rules with different encoding qualities;

[0134] The second processing module is further configured to:

[0135] The encoding rule associated with the brightness level or texture information is determined according to the brightness level or texture information of the first image area to which the image unit to be encoded belongs.

[0136] Optionally, the second processing module is further configured to:

[0137] When the to-be-encoded image unit belongs to an edge portion of the first image area that is directly adjacent to the second image area, or an edge portion of the second image area that is directly adjacent to the first image area, determining that the encoding strategy is the third strategy;

[0138] The encoding quality of the third strategy is higher than that of the second strategy, and lower than that of the first strategy.

[0139] Optionally, the encoding module is further configured to:

[0140] adjusting encoding parameters based on the encoding strategy;

[0141] Encoding the gain map based on the encoding parameters to obtain the encoding information;

[0142] The encoding parameters include at least one of the following:

[0143] Quantization parameters;

[0144] transform coding coefficients;

[0145] Rate-distortion function.

[0146] Optionally, the quantization parameter includes at least one of the following:

[0147] Image-level quantization parameters;

[0148] Slice-level quantization parameters;

[0149] Coding tree unit level quantization parameter;

[0150] Block-level quantization parameters;

[0151] Coding unit level quantization parameter.

[0152] Optionally, the encoding module is further configured to:

[0153] When the gain map is a multi-channel gain map and the encoding parameters include quantization parameters, different quantization parameters are used for encoding in different channels.

[0154] Optionally, the device further comprises:

[0155] The third processing module is configured to downsample the gain map.

[0156] After dividing the gain map into a first image region and a second image region using a first threshold, the device can determine a coding strategy for the gain map based on the first and second image regions. The device then encodes the gain map according to the determined coding strategy to obtain coding information for the gain map. This allows image regions with varying brightness to be encoded using the appropriate coding strategy, reducing data volume and conserving storage space compared to storing the gain map.

[0157] The gain map encoding device in the embodiment of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other device other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., and the embodiment of the present application does not specifically limit it.

[0158] The gain map encoding device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0159] The gain map encoding device provided in the embodiment of the present application can implement each process implemented in the method embodiments of Figures 3 to 5. To avoid repetition, they are not described here.

[0160] As shown in FIG10 , an embodiment of the present application further provides a gain map decoding device, comprising:

[0161] An acquisition module 1010 is configured to acquire encoding information of a gain map;

[0162] A decoding module 1020, configured to decode the encoded information to obtain the gain map;

[0163] The first image area and the second image area of ​​the gain map are encoded using different encoding strategies, the brightness of the pixels in the first image area is greater than a first threshold, and the brightness of the pixels in the second image area is less than or equal to the first threshold.

[0164] Optionally, the decoding module is further configured to:

[0165] After decoding the encoded information, the decoded image is upsampled to obtain a gain map.

[0166] After acquiring the coded information of the gain map coded by the method of the above embodiment, the device obtains the gain map by decoding the coded information.

[0167] Optionally, as shown in Figure 11, an embodiment of the present application also provides an electronic device 1100, including a processor 1101 and a memory 1102, and the memory 1102 stores a program or instruction that can be run on the processor 1101. When the program or instruction is executed by the processor 1101, the various steps of the above-mentioned gain map encoding method or gain map decoding method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0168] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0169] FIG12 is a schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0170] The electronic device 1200 includes but is not limited to: a radio frequency unit 1201, a network module 1202, an audio output unit 1203, an input unit 1204, a sensor 1205, a display unit 1206, a user input unit 1207, an interface unit 1208, a memory 1209, and a processor 1210.

[0171] [Corrected 01.08.2024 according to Rule 91] Those skilled in the art will appreciate that electronic device 1200 may also include a power source (e.g., a battery) to power various components. The power source may be logically connected to processor 1210 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The electronic device structure shown in FIG12 does not limit the electronic device. The electronic device may include more or fewer components than shown, or may combine certain components, or have different component arrangements. These details are omitted here.

[0172] The processor 1210 is configured to determine a first image region and a second image region in the gain map, wherein the brightness of pixels in the first image region is greater than a first threshold, and the brightness of pixels in the second image region is less than or equal to the first threshold;

[0173] determining a coding strategy for the gain map according to the first image region and the second image region;

[0174] The gain map is encoded according to the encoding strategy to obtain encoding information of the gain map.

[0175] After dividing the gain map into a first image region and a second image region using a first threshold, the electronic device can determine a coding strategy for the gain map based on the first and second image regions, and then encode the gain map according to the determined coding strategy to obtain coding information for the gain map. In this way, image regions with different brightness are encoded using the appropriate coding strategy, reducing the amount of data and saving storage space compared to storing the gain map.

[0176] Optionally, the processor 1210 is further configured to: determine a segmentation size based on a target tone mapping method; wherein the target tone mapping method is a tone mapping method used when generating a standard dynamic range image from a high dynamic range image;

[0177] Based on the segmentation size, dividing the gain map into a plurality of regions;

[0178] Based on the first threshold, the first image region and the second image region among the plurality of regions are determined.

[0179] Optionally, the processor 1210 is further configured to:

[0180] In a case where the image unit to be encoded belongs to the first image area, determining the encoding strategy to be the first strategy;

[0181] In a case where the image unit to be encoded belongs to the second image area, determining the encoding strategy to be the second strategy;

[0182] The encoding quality of the first strategy is higher than the encoding quality of the second strategy.

[0183] Optionally, the first strategy includes a plurality of encoding rules with different encoding qualities;

[0184] The processor 1210 is further configured to:

[0185] The encoding rule associated with the brightness level or texture information is determined according to the brightness level or texture information of the first image area to which the image unit to be encoded belongs.

[0186] Optionally, the processor 1210 is further configured to:

[0187] When the to-be-encoded image unit belongs to an edge portion of the first image area that is directly adjacent to the second image area, or an edge portion of the second image area that is directly adjacent to the first image area, determining that the encoding strategy is the third strategy;

[0188] The encoding quality of the third strategy is higher than that of the second strategy, and lower than that of the first strategy.

[0189] Optionally, the processor 1210 is further configured to:

[0190] adjusting encoding parameters based on the encoding strategy;

[0191] Encoding the gain map based on the encoding parameters to obtain the encoding information;

[0192] The encoding parameters include at least one of the following:

[0193] Quantization parameters;

[0194] transform coding coefficients;

[0195] Rate-distortion function.

[0196] Optionally, the quantization parameter includes at least one of the following:

[0197] Image-level quantization parameters;

[0198] Slice-level quantization parameters;

[0199] Coding tree unit level quantization parameter;

[0200] Block-level quantization parameters;

[0201] Coding unit level quantization parameter.

[0202] Optionally, the processor 1210 is further configured to:

[0203] When the gain map is a multi-channel gain map and the encoding parameters include quantization parameters, different quantization parameters are used for encoding in different channels.

[0204] Optionally, the processor 1210 is further configured to:

[0205] The gain map is downsampled.

[0206] It should be understood that in an embodiment of the present application, the input unit 1204 may include a graphics processing unit (GPU) 12041 and a microphone 12042, and the graphics processor 12041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1206 may include a display panel 12061, and the display panel 12061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1207 includes a touch panel 12071 and at least one of other input devices 12072. The touch panel 12071 is also called a touch screen. The touch panel 12071 may include two parts: a touch detection device and a touch controller. Other input devices 12072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0207] The memory 1209 can be used to store software programs and various data. The memory 1209 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1209 may include a volatile memory or a non-volatile memory, or the memory 1209 may include both volatile and non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 1209 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0208] Processor 1210 may include one or more processing units. Optionally, processor 1210 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 1210.

[0209] Of course, the electronic device shown in FIG12 can also apply the gain map decoding method shown in the above embodiment, which will not be described in detail here.

[0210] An embodiment of the present application further provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned gain map encoding method or gain map decoding method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here.

[0211] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0212] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is configured to run programs or instructions to implement the various processes of the above-mentioned gain map encoding method or gain map decoding method embodiment, and can achieve the same technical effects. To avoid repetition, they are not further described here.

[0213] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0214] An embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-mentioned gain map encoding method or gain map decoding method embodiment, and can achieve the same technical effects. To avoid repetition, they are not described here.

[0215] The processors mentioned in the embodiments of the present application may include general-purpose processors, special-purpose processors, etc., such as a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), an artificial intelligence (AI) processor, a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a network processor (NP), a field programmable gate array (FPGA), or other programmable logic devices, gate circuits, transistors, discrete hardware components, etc.

[0216] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0217] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0218] [Corrected 01.08.2024 according to Rule 91] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A gain map encoding method, wherein: include: Determine a first image area and a second image area in the gain map, wherein the brightness of pixels in the first image area is greater than a first threshold, and the brightness of pixels in the second image area is less than or equal to the first threshold; Determining a coding strategy for the gain map according to the first image area and the second image area; The gain map is encoded according to the encoding strategy to obtain encoding information of the gain map.

2. The method according to claim 1, wherein: The determining of the first image area and the second image area in the gain map comprises: Determining a segmentation size based on a target tone mapping method; wherein the target tone mapping method is a tone mapping method used when generating a standard dynamic range image from a high dynamic range image; Based on the segmentation size, segmenting the gain map into a plurality of regions; Based on the first threshold, the first image region and the second image region among the plurality of regions are determined.

3. The method according to claim 1, wherein: The step of determining the encoding strategy of the gain map according to the first image area and the second image area includes: In a case where the image unit to be encoded belongs to the first image area, determining the encoding strategy to be a first strategy; In a case where the image unit to be encoded belongs to the second image area, determining that the encoding strategy is the second strategy; The encoding quality of the first strategy is higher than the encoding quality of the second strategy.

4. The method according to claim 3, wherein: The first strategy includes a plurality of encoding rules with different encoding qualities; The step of determining, when the to-be-encoded image unit belongs to the first image area, that the encoding strategy is the first strategy includes: The encoding rule associated with the brightness level or texture information is determined by the brightness level or texture information of the first image area to which the image unit to be encoded belongs.

5. The method according to claim 3, wherein: The step of determining the encoding strategy of the gain map according to the first image area and the second image area includes: In a case where the image unit to be encoded belongs to an edge portion in the first image area directly adjacent to the second image area, or an edge portion in the second image area directly adjacent to the first image area, determining that the encoding strategy is a third strategy; The encoding quality of the third strategy is higher than that of the second strategy, and lower than that of the first strategy.

6. The method according to claim 1, wherein: The step of encoding the gain map according to the encoding strategy to obtain encoding information of the gain map includes: adjusting encoding parameters based on the encoding strategy; Encoding the gain map based on the encoding parameters to obtain the encoding information; The encoding parameters include at least one of the following: Quantization parameters; Transform coding coefficients; Rate-distortion function.

7. The method according to claim 6, wherein: The quantization parameter includes at least one of the following: Image-level quantization parameters; Slice-level quantization parameters; Coding tree unit level quantization parameter; Block-level quantization parameters; Coding unit level quantization parameter.

8. The method according to claim 6, wherein: The step of encoding the gain map based on the encoding parameter to obtain the encoding information includes: When the gain map is a multi-channel gain map and the encoding parameters include quantization parameters, different quantization parameters are used for encoding in different channels.

9. The method according to claim 1, wherein: Before determining the first image area and the second image area in the gain map, the method further includes: The gain map is downsampled.

10. A gain map decoding method, wherein: include: Obtaining encoding information of the gain map; Decoding the encoded information to obtain the gain map; The first image area and the second image area of ​​the gain map are encoded using different encoding strategies, the brightness of the pixels in the first image area is greater than a first threshold, and the brightness of the pixels in the second image area is less than or equal to the first threshold.

11. The method according to claim 10, wherein decoding the coded information to obtain the gain map comprises: After decoding the encoded information, upsampling is performed on the decoded image to obtain the gain map.

12. A gain map encoding device, wherein: include: A first processing module, configured to determine a first image region and a second image region in a gain map, wherein the brightness of pixels in the first image region is greater than a first threshold, and the brightness of pixels in the second image region is less than or equal to the first threshold; A second processing module, configured to determine a coding strategy for the gain map according to the first image area and the second image area; The encoding module is used to encode the gain map according to the encoding strategy to obtain encoding information of the gain map.

13. The device according to claim 12, wherein: The determining of the first image area and the second image area in the gain map comprises: Determining a segmentation size based on a target tone mapping method; wherein the target tone mapping method is a tone mapping method used when generating a standard dynamic range image from a high dynamic range image; Based on the segmentation size, segmenting the gain map into a plurality of regions; Based on the first threshold, the first image region and the second image region among the plurality of regions are determined.

14. The device according to claim 12, wherein: The second processing module is also used for: In a case where the image unit to be encoded belongs to the first image area, determining the encoding strategy to be a first strategy; In a case where the image unit to be encoded belongs to the second image area, determining that the encoding strategy is the second strategy; The encoding quality of the first strategy is higher than the encoding quality of the second strategy.

15. The device according to claim 14, wherein: The first strategy includes a plurality of encoding rules with different encoding qualities; The second processing module is also used for: The encoding rule associated with the brightness level or texture information is determined by the brightness level or texture information of the first image area to which the image unit to be encoded belongs.

16. The device according to claim 14, wherein: The second processing module is also used for: In a case where the image unit to be encoded belongs to an edge portion in the first image area directly adjacent to the second image area, or an edge portion in the second image area directly adjacent to the first image area, determining that the encoding strategy is a third strategy; The encoding quality of the third strategy is higher than that of the second strategy, and lower than that of the first strategy.

17. The device according to claim 12, wherein: The encoding module is used for: adjusting encoding parameters based on the encoding strategy; Encoding the gain map based on the encoding parameters to obtain the encoding information; The encoding parameters include at least one of the following: Quantization parameters; Transform coding coefficients; Rate-distortion function.

18. The device according to claim 17, wherein: The quantization parameter includes at least one of the following: Image-level quantization parameters; Slice-level quantization parameters; Coding tree unit level quantization parameter; Block-level quantization parameters; Coding unit level quantization parameter.

19. The device according to claim 17, wherein: The encoding module is also used for: When the gain map is a multi-channel gain map and the encoding parameters include quantization parameters, different quantization parameters are used for encoding in different channels.

20. The device according to claim 12, wherein: The device also includes: The third processing module is used to downsample the gain map.

21. A gain map decoding device, wherein: include: An acquisition module, used for acquiring encoding information of a gain map; A decoding module, used for decoding the coded information to obtain the gain map; The first image area and the second image area of ​​the gain map are encoded using different encoding strategies, the brightness of the pixels in the first image area is greater than a first threshold, and the brightness of the pixels in the second image area is less than or equal to the first threshold.

22. The apparatus according to claim 21, wherein the decoding module is further configured to: After decoding the encoded information, the decoded image is upsampled to obtain a gain map.

23. An electronic device, wherein: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the method for encoding a gain map as described in any one of claims 1 to 9 or the steps of the method for decoding a gain map as described in any one of claims 10 to 11 are implemented.

24. A readable storage medium, wherein: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps of the gain map encoding method according to any one of claims 1 to 9 or the gain map decoding method according to any one of claims 10 to 11 are implemented.

25. A chip, wherein: The chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the gain map encoding method according to any one of claims 1 to 9, or the steps of the gain map decoding method according to any one of claims 10 to 11.

26. A computer program product, wherein: The computer program product is stored in a storage medium, and the computer program product is executed by at least one processor to implement the steps of the gain map encoding method according to any one of claims 1 to 9, or the gain map decoding method according to any one of claims 10 to 11.

Images