Tone mapping method and system and device

Abstract

The present application provides a tone mapping method and system, and a device, in order to provide a technical solution that can improve tone mapping effect. The method includes the steps of: obtaining an image frame to be displayed and metadata, the metadata including at least one metadata information unit; segmenting the image frame to be displayed into a plurality of segmentation regions in a target segmentation manner, and determining a correspondence relationship between the plurality of segmentation regions and at least one metadata information unit; for an i-th pixel, for a plurality of pixels included in a preset region, determining at least one associated segmentation region to which the plurality of pixels belong, and determining an associated metadata information unit corresponding to each associated segmentation region according to the correspondence relationship; obtaining a tone mapping sub-value corresponding to the associated segmentation region according to the associated metadata information unit, and allocating a corresponding weighting factor to the associated segmentation region; obtaining a tone mapping value according to the tone mapping sub-value and weighting factor corresponding to the associated segmentation region; and performing tone mapping according to the tone mapping value.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to Chinese Patent Application No. 202111080645.7, entitled "Tone Mapping Method and System, and Device," filed with the State Intellectual Property Office of China on September 15, 2021, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the field of display technology, and in particular to tone mapping methods and systems and devices. [Background technology]

[0003] With the development of capture end devices, currently, high dynamic range (HDR) videos or images can be acquired by capturing based on HDR video technology. In HDR videos or images, the brightness range of the displayable image is expanded, so that information in a wide brightness range can be recorded and brighter and darker details of the image can be displayed.

[0004] However, the display capabilities of existing display end devices are generally limited, and the brightness range information recorded in the HDR video or image captured by the capture end device cannot be displayed well. For example, the capture end device has a maximum brightness of 10,000 candela / m 2 (cd / m 2 , luminance units), and the display end device may capture pixels with a maximum luminance of 4000 cd / m 2 In this case, the capture end device may display only pixels whose luminance is 4000 cd / m in the HDR video or image captured by the capture end device. 2 ~10000cd / m 2Pixels where 0 cannot be displayed well. The prior art provides processing methods whereby tone mapping (TM) can be performed on an image to be displayed, so that high dynamic range image tones can be mapped to a display end device with low dynamic range display capabilities.

[0005] In the conventional tone mapping method, the local tone mapping method based on the tone mapping curve or the block-based method or other methods can be mainly used, but there exists a problem of insufficient tone mapping effect. Therefore, how to improve the tone mapping effect is worth studying. Summary of the Invention

[0006] The present application provides a tone mapping method and system, and a device, to provide a technical solution that can improve the tone mapping effect. [Means for solving the problem]

[0007] According to a first aspect, an embodiment of the present application provides a tone mapping method applied to a display end device, the method comprising the steps of: obtaining an image frame to be displayed and metadata, the metadata including at least one metadata information unit; segmenting the image frame to be displayed into a plurality of segmentation regions in a target segmentation method, and determining a correspondence between the plurality of segmentation regions and the at least one metadata information unit; and performing, for an ith pixel in the image frame to be displayed, the following operations: The method includes the steps of: determining, based on a plurality of pixels included in a preset region selected for the i-th pixel, at least one associated segmentation region to which the plurality of pixels belong; and determining, based on the correspondence relationship, an associated metadata information unit corresponding to each associated segmentation region; obtaining, based on the associated metadata information unit, tone mapping sub-values ​​corresponding to each associated segmentation region and allocating corresponding weighting factors to each associated segmentation region; and obtaining, based on the tone mapping sub-values ​​and weighting factors corresponding to each associated segmentation region, a tone mapping value for the i-th pixel; and performing tone mapping for the i-th pixel based on the tone mapping value, wherein i is any positive integer from 1 to N, and N is the number of pixels included in the image frame to be displayed.

[0008] The method provides a technical solution for performing fine local tone mapping on an image frame to be displayed, where local tone mapping is performed on pixels included in the image frame to be displayed based on a preset region, thereby improving the display effect of a display end device for the tone-mapped image frame to be displayed.

[0009] In a possible design, the operation of allocating a corresponding weighting factor to each associated segmented region may be performed in the following manner: Method 1: separately determining the number of pixels belonging to a preset region in each associated segmentation region and the total number of pixels corresponding to the preset region, and performing the following operation with respect to a first associated segmentation region: allocating a corresponding first weighting factor to the first associated segmentation region based on the number of pixels belonging to the preset region and the total number of pixels in the first associated segmentation region, where the first associated segmentation region is any one of the at least one associated segmentation region; and / or Method 2: separately determining preset feature values ​​corresponding to preset regions in each associated segmentation region, and performing the following operation with respect to a second associated segmentation region: assigning a corresponding second weighting factor to the second associated segmentation region based on a feature difference between the preset feature value corresponding to the preset region in the second associated segmentation region and the preset feature value of the i-th pixel, where the second associated segmentation region is any one of the at least one associated segmentation region. The present invention includes one or a combination of the following:

[0010] In this design, corresponding weighting factors may be allocated to different associated segmented regions based on local features of different associated segmented regions within the preset region corresponding to the pixel, for example, the number of pixels or preset feature values ​​of the pixels, whereby fine tone mapping of the pixels included in the image frame to be displayed can be performed, tone mapping can be performed based on the image features of the image frame to be displayed, and the display effect of the display end device for the tone-mapped image frame to be displayed can be improved.

[0011] In a possible design, the metadata further includes one or a combination of the following information: a first weighted intensity value or a second weighted intensity value, where the first weighted intensity value is used to adjust the first weighting factor based on a pixel count distribution status of the image frame to be displayed, and the second weighted intensity value is used to adjust the second weighting factor based on a preset feature value change status of the image frame to be displayed.

[0012] In this design, the weighting coefficients allocated to different segmented regions can be further adjusted using the weighted intensity value based on the overall characteristics of the image frames to be displayed. For example, when the image characteristics of the image frames to be displayed are similar, a large weighting coefficient can be allocated. When the image characteristics of the image frames to be displayed are different, a small weighting coefficient can be allocated. In this way, the boundary effect that may be generated by tone mapping can be reduced, and the display effect of the display end device for the tone-mapped image frames to be displayed can be improved.

[0013] In a possible design, the preset feature value is one of the following feature values: a luminance feature value or a color feature value. There is at least one preset feature value. In this design, the preset feature value may be a luminance feature value or a color feature value. In actual implementation, the preset feature value may be determined based on the image frame to be displayed. For example, if a luminance feature is important for the image frame to be displayed, a luminance feature value may be selected. If a color feature of the image frame to be displayed is prominent, a color feature value may be selected. In addition, the design does not limit the number of preset feature values, which improves the tone mapping effect for the image frame to be displayed.

[0014] In a possible design, the operation of obtaining a tone mapping sub-value corresponding to each associated segmented region based on an associated metadata information unit includes an operation of obtaining a tone mapping sub-value corresponding to an i-th pixel in the associated segmented region based on a tone mapping curve, wherein the associated metadata information unit includes parameters of the tone mapping curve.

[0015] In this design, each associated segmented region may correspond to a tone mapping curve, thereby allowing tone mapping of each associated segmented region to be performed to obtain multiple tone mapping sub-values, and the accuracy of the tone mapping sub-values ​​to be improved to perform accurate tone mapping effects.

[0016] In possible designs, the preset region selected for the i-th pixel is a region of a preset shape with a preset size selected using the i-th pixel as its center, a region determined based on preset shape and preset size information included in the metadata using the i-th pixel as its center, or a region determined based on preset shape and preset size information included in the metadata using the i-th pixel as its center.

[0017] In this design, a preset area is selected using the pixel to be processed (i-th pixel) as the center, and then fine local tone mapping is performed on the pixel to be processed based on multiple pixels included in the preset area, thereby improving the display effect of the display end device for the tone-mapped image frame to be displayed.

[0018] In a possible design, the step of determining the correspondence between the plurality of segmented areas and at least one metadata information unit includes, for any segmented area, selecting pixels at preset positions from the segmented area and obtaining coordinate information of the pixels at the preset positions, and determining a metadata information unit corresponding to the segmented area based on the coordinate information, and establishing a correspondence between the segmented area and the corresponding metadata information unit; or, for any metadata information unit, determining at least one segmented area corresponding to the one or more coordinate information based on the one or more coordinate information included in the metadata information unit, and establishing a correspondence between the metadata information unit and the corresponding at least one segmented area.

[0019] In this design, the coordinate information of the preset position is used as an index, so that the correspondence between the segmentation regions and the metadata information units can be accurately and easily determined, and the tone mapping sub-values ​​of each associated segmentation region can be calculated separately based on the associated metadata information units corresponding to the associated segmentation regions of the pixel to be processed.

[0020] In a possible design, determining the correspondence between the plurality of segmented regions and at least one metadata information unit includes traversing the plurality of segmented regions based on a target scan sequence and a segmented region identifier to determine a metadata information unit corresponding to each of the segmented regions, wherein the metadata information unit includes the corresponding segmented region identifier and the target scan sequence is a preset scan sequence or a scan sequence indicated in the metadata.

[0021] In this design, a target scanning sequence can be further used, so that the correspondence between the segmentation regions and the metadata information units can be accurately and easily determined, and the tone mapping sub-values ​​of each associated segmentation region can be calculated separately based on the associated metadata information units corresponding to the associated segmentation region of the pixel to be processed.

[0022] In a possible design, there is a one-to-one or many-to-one correspondence between the plurality of segmented regions and the at least one metadata information unit. In this design, there may be a one-to-one or many-to-one correspondence between the plurality of segmented regions and the at least one metadata information unit. The many-to-one correspondence can reduce the amount of metadata information that needs to be transmitted between the capture end device and the display end device, and can improve metadata transmission efficiency.

[0023] In a possible design, the target segmentation method is a preset segmentation method or a segmentation method indicated in metadata. In this design, the target segmentation method may be pre-stored separately by the capture end device and the display end device, or the segmentation method used by the display end device may be indicated by the capture end device, which can ensure the accuracy of tone mapping between the capture end device and the display end device.

[0024] According to a second aspect, an embodiment of the present application provides a metadata generation method applied to a capture end device, the method including the steps of: capturing image data to obtain a data frame to be displayed; segmenting the data frame to be displayed into a plurality of segmentation regions in a target segmentation method, and determining at least one corresponding metadata information unit for the plurality of segmentation regions; and transmitting the data frame to be displayed and the metadata to a display end device, where the metadata includes the at least one metadata information unit.

[0025] In a possible design, the method further includes determining a first weighted intensity value based on the number of pixels included in each of the plurality of segmented regions and / or determining a second weighted intensity value based on preset feature values ​​corresponding to each of the plurality of segmented regions, and associating the first weighted intensity value and / or the second weighted intensity value with metadata.

[0026] In a possible design, the preset feature value is one of the following feature values: a luminance feature value or a color feature value. There is at least one preset feature value.

[0027] In a possible design, the metadata information unit includes parameters of a tone mapping curve corresponding to the segmented region.

[0028] In a possible design, the metadata further includes preset size information, which indicates a preset size of a preset shape to be used when a preset region is selected for a pixel, or the metadata further includes a preset shape and preset size information.

[0029] In a possible design, the step of determining, for a plurality of segmented regions, at least one corresponding metadata information unit comprises: selecting a plurality of pixels at preset positions from the segmented region and obtaining coordinate information of the plurality of pixels at the preset positions; and associating the coordinate information with a metadata information unit; Includes:

[0030] In a possible design, the metadata information unit further includes a segmentation region identifier and a scanning sequence corresponding to the metadata information unit, the corresponding segmentation region identifier and scanning sequence being used by the display end device to determine a correspondence between the plurality of segmentation regions and the at least one metadata information unit.

[0031] In a possible design, there is a one-to-one or many-to-one correspondence between the plurality of segmented regions and the at least one metadata information unit.

[0032] In a possible design, the method further comprises associating the target segmentation method with the metadata to indicate to a display end device to segment the data frame to be displayed in the target segmentation method.

[0033] According to a third aspect, an embodiment of the present application provides a tone mapping processing device, the device including: a module configured to perform the method of the first aspect or any one of the possible implementations of the first aspect.

[0034] For example, the apparatus may include an acquisition module, a processing module, and a tone mapping module. The acquisition module is configured to acquire an image frame to be displayed and metadata, where the metadata includes at least one metadata information unit. The processing module is configured to segment the image frame to be displayed into a plurality of segmentation regions in a target segmentation manner, and determine a correspondence relationship between the plurality of segmentation regions and the at least one metadata information unit. The tone mapping module is configured to perform the following operations with respect to an i-th pixel in the image frame to be displayed: determine, based on a plurality of pixels included in the preset region selected for the i-th pixel, at least one associated segmented region to which the plurality of pixels belong, and determine, based on the correspondence, an associated metadata information unit corresponding to each associated segmented region; obtain, based on the associated metadata information unit, tone mapping sub-values ​​corresponding to each associated segmented region and assign corresponding weighting factors to each associated segmented region; obtain, based on the tone mapping sub-values ​​and weighting factors corresponding to each associated segmented region, a tone mapping value for the i-th pixel; and perform tone mapping for the i-th pixel based on the tone mapping value, where i is any positive integer from 1 to N, and N is the number of pixels included in the image frame to be displayed.

[0035] In a possible design, when configured to allocate a corresponding weighting factor to each associated segmented region, the tone mapping module specifically determines the number of pixels belonging to a preset region in each associated segmented region and the total number of pixels corresponding to the preset region separately, and performs the following operation with respect to a first associated segmented region: allocate a corresponding first weighting factor to the first associated segmented region based on the number of pixels belonging to the preset region and the total number of pixels in the first associated segmented region, where the first associated segmented region includes at least one associated segmented region. and / or separately determine preset feature values ​​corresponding to preset regions in each associated segmented region, and perform the following operation with respect to a second associated segmented region: assign a corresponding second weighting factor to the second associated segmented region based on a feature difference between the preset feature value corresponding to the preset region in the second associated segmented region and the preset feature value of the ith pixel, wherein the second associated segmented region is any one of the at least one associated segmented region.

[0036] In a possible design, the metadata further includes one or a combination of the following information: a first weighted intensity value or a second weighted intensity value: the first weighted intensity value is used to adjust the first weighting factor based on a pixel count distribution status of the image frame to be displayed; and the second weighted intensity value is used to adjust the second weighting factor based on a preset feature value change status of the image frame to be displayed.

[0037] In a possible design, the preset feature value is one of the following feature values: a luminance feature value or a color feature value. There is at least one preset feature value.

[0038] In a possible design, when configured to obtain a tone mapping sub-value corresponding to each associated segmented region based on an associated metadata information unit, the tone mapping module is specifically configured to obtain a tone mapping sub-value corresponding to the i-th pixel in the associated segmented region based on a tone mapping curve, and the associated metadata information unit includes parameters of the tone mapping curve.

[0039] In possible designs, the preset region selected for the i-th pixel is a region of a preset shape with a preset size selected using the i-th pixel as its center, a region determined based on preset shape and preset size information included in the metadata using the i-th pixel as its center, or a region determined based on preset shape and preset size information included in the metadata using the i-th pixel as its center.

[0040] In a possible design, when configured to determine a correspondence relationship between a plurality of segmented regions and at least one metadata information unit, the processing module is specifically configured to: for any segmented region, select a pixel at a preset position from the segmented region, obtain coordinate information of the pixel at the preset position, determine a metadata information unit corresponding to the segmented region based on the coordinate information, and establish a correspondence relationship between the segmented region and the corresponding metadata information unit; or for any metadata information unit, determine at least one segmented region corresponding to the one or more coordinate information based on one or more coordinate information included in the metadata information unit, and establish a correspondence relationship between the metadata information unit and the corresponding at least one segmented region.

[0041] In a possible design, when configured to determine a correspondence between a plurality of segmented regions and at least one metadata information unit, the processing module is specifically configured to: traverse the plurality of segmented regions based on a target scan sequence and the segmented region identifier to determine a metadata information unit corresponding to each of the segmented regions, wherein the metadata information unit includes the corresponding segmented region identifier, and the target scan sequence is a preset scan sequence or a scan sequence indicated in the metadata.

[0042] In a possible design, there is a one-to-one or many-to-one correspondence between the plurality of segmented regions and the at least one metadata information unit.

[0043] In a possible design, the target segmentation method is a pre-configured segmentation method or a segmentation method indicated in the metadata.

[0044] According to a fourth aspect, an embodiment of the present application provides a tone mapping apparatus, the apparatus including a module configured to perform a method according to the second aspect or any one of the possible implementations of the second aspect.

[0045] For example, the apparatus may include a capture module, a processing module, and a transceiver module. The capture module is configured to capture image data to obtain a data frame to be displayed. The processing module is configured to segment the data frame to be displayed into a plurality of segmented regions in a target segmentation method and determine at least one corresponding metadata information unit for the plurality of segmented regions. The transceiver module is configured to transmit the data frame to be displayed and the metadata to a display end device, the metadata including the at least one metadata information unit.

[0046] In a possible design, the processing module is further configured to determine a first weighted intensity value based on the number of pixels included in each of the plurality of segmented regions and / or determine a second weighted intensity value based on preset feature values ​​corresponding to each of the plurality of segmented regions, and associate the first weighted intensity value and / or the second weighted intensity value with metadata.

[0047] In a possible design, the preset feature value is one of the following feature values: a luminance feature value or a color feature value. There is at least one preset feature value.

[0048] In a possible design, the metadata information unit includes parameters of a tone mapping curve corresponding to the segmented region.

[0049] In a possible design, the metadata further includes preset size information, which indicates a preset size of a preset shape to be used when a preset region is selected for a pixel, or the metadata further includes a preset shape and preset size information.

[0050] In a possible design, when configured to determine at least one corresponding metadata information unit for a plurality of segmented regions, the processing module is specifically configured to select a plurality of pixels at preset positions from the segmented region, obtain coordinate information of the plurality of pixels at the preset positions, and associate the coordinate information with the metadata information unit.

[0051] In a possible design, the metadata information unit further includes a segmentation region identifier and a scanning sequence corresponding to the metadata information unit, the corresponding segmentation region identifier and scanning sequence being used by the display end device to determine a correspondence between the plurality of segmentation regions and the at least one metadata information unit.

[0052] In a possible design, there is a one-to-one or many-to-one correspondence between the plurality of segmented regions and the at least one metadata information unit.

[0053] In a possible design, the processing module is further configured to associate the target segmentation method with the metadata to indicate to a display end device to segment the data frame to be displayed in the target segmentation method.

[0054] According to a fifth aspect, an embodiment of the present application provides a display end device including a non-volatile memory and a processor coupled to each other, wherein the processor invokes program code stored in the memory to perform a method according to the first aspect or any one of the possible implementations of the first aspect.

[0055] According to a sixth aspect, an embodiment of the present application provides a capture end device including a non-volatile memory and a processor coupled to each other, wherein the processor invokes program code stored in the memory to perform a method according to the second aspect or any one of the possible implementations of the second aspect.

[0056] According to a seventh aspect, an embodiment of the present application provides a computer-readable storage medium comprising a program or instructions, which, when executed on a computer, performs a method according to the first aspect or any one of possible implementations of the first aspect, or a method according to the second aspect or any one of possible implementations of the second aspect.

[0057] It should be understood that for details of the technical effects that can be achieved by the second to seventh aspects, please refer to the description of the technical effects provided by the first aspect or any one of the possible implementations of the first aspect, and the details will not be described again here. [Brief explanation of the drawings]

[0058] [Figure 1] FIG. 1 is a schematic diagram of the relationship between PQ photoelectric transfer functions. [Figure 2] 1 is a schematic diagram of the relationship of the HLG photoelectric transfer function. [Figure 3] FIG. 1 is a schematic diagram of the SLF photoelectric transfer function relationship. [Figure 4] 1 is a schematic diagram of a possible system architecture to which an embodiment of the present application is applicable; [Figure 5] 1 is a schematic flowchart of a tone mapping method according to an embodiment of the present application; [Figure 6] FIG. 2 is a schematic diagram of segmentation in segmentation method A according to an embodiment of the present application; [Figure 7] FIG. 10 is a schematic diagram of segmentation in segmentation method B according to an embodiment of the present application. [Figure 8] FIG. 10 is a schematic diagram of segmentation in segmentation method C according to an embodiment of the present application. [Figure 9] FIG. 2 is a schematic diagram of a scanning sequence according to an embodiment of the present application; [Figure 10] FIG. 2 is a schematic diagram of determining relevant segmentation regions according to an embodiment of the present application; [Figure 11] FIG. 10 is another schematic diagram of determining relevant segmentation regions according to an embodiment of the present application; [Figure 12] FIG. 2 is a schematic diagram of brightness feature values ​​according to an embodiment of the present application; [Figure 13] 4 is another schematic flowchart of a tone mapping method according to an embodiment of the present application; [Figure 14] 1 is a schematic diagram of the structure of a tone mapping device according to an embodiment of the present application; [Figure 15] FIG. 2 is another schematic diagram of the structure of a tone mapping device according to an embodiment of the present application; [Figure 16] 1 is a schematic diagram of a structure of an electronic device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0059] In the embodiments of the present application, the term "at least one" refers to one or more, and "multiple" refers to two or more. And / or describes the association relationship between associated objects and indicates that three relationships may exist. For example, A and / or B may represent the following cases: only A is present, both A and B are present, and only B is present, where A and B may be singular or plural. The symbol " / " generally indicates an "or" relationship between associated objects. "At least one of" or similar expressions refers to any combination of these, including any combination of singular or plural items. For example, "at least one of" a, b, or c may refer to a, b, c, a and b, a and c, b and c, or a, b, and c. Any one of a, b, c, a and b, a and c, b and c, or a, b, and c may include a singular a, a singular b, and a singular c, or may include multiple a, multiple b, and multiple c.

[0060] In addition, unless otherwise specified, ordinal numbers such as "first" and "second" in the embodiments of the present application are used to distinguish between multiple objects, but are not intended to limit the order, chronological order, priority, or importance of the multiple objects. For example, a first priority criterion and a second priority criterion are merely used to distinguish between different criteria, and do not indicate different contents, priorities, or importance of the two criteria.

[0061] In addition, the terms "comprise" and "have" in the embodiments, claims, and accompanying drawings of this application are not exclusive. For example, a process, method, system, product, or device that includes a series of steps or modules / units is not limited to the enumerated steps or modules, and may further include unenumerated steps or modules / units.

[0062] In the field of digital image display, dynamic range can represent the ratio of the maximum grayscale value to the minimum grayscale value within the displayable image range. Currently, in most color digital images, each of the channels R, G, and B is stored using one byte, or 8 bits. Specifically, when the representation range of each channel is 0 to 255 grayscale levels, the dynamic range of the image is 0 to 255. However, the dynamic range of the same scene in the real world is generally 10 -3 ~10 6 and can be called high dynamic range (HDR). Compared to the high dynamic range, the dynamic range of a typical image is called low dynamic range (LDR), and the LDR is generally 1 to 100. Therefore, it can be understood that the imaging process of a digital camera is actually a mapping process from the high dynamic range of the real world to the low dynamic range of a photograph.

[0063] A greater dynamic range of an image indicates more scene details and richer brightness levels shown by the image, as well as a sharper visual effect.

[0064] The optical digital imaging process (e.g., the imaging process of a digital camera and the video playback process) converts the optical radiation of a real scene into an electrical signal by using an image sensor and storing it in the form of a digital image. Image display aims to reproduce the real scene described by the digital image using a display end device. The ultimate goal of the optical digital imaging process and image display is to enable users to obtain the same visual perception as they obtain when directly observing a real scene. The brightness levels that can actually be displayed by the optical radiation (optical signal) of a real scene are approximately linear. Therefore, optical signals are also called linear signals. However, in the process of converting optical signals to electrical signals in optical digital imaging, not all optical signals correspond to one electrical signal. In addition, the electrical signal obtained by the conversion is generally nonlinear. Therefore, electrical signals are also called nonlinear signals. The curve that converts optical signals to electrical signals is the optical electrotransfer function (OETF). The photoelectric transfer function in embodiments of the present application may include, but is not limited to, a perceptual quantizer (PQ) photoelectric transfer function, a hybrid log-gamma (HLG) photoelectric transfer function, and a scene luminance fidelity (SLF) photoelectric transfer function, etc.

[0065] The PQ photoelectric transfer function is a perceptual quantizer photoelectric transfer function based on the luminance perception model of the human eye. The PQ photoelectric transfer function represents the conversion relationship from the linear signal value of an image pixel to the nonlinear signal value in the PQ domain. Figure 1 is a schematic diagram of the PQ photoelectric transfer function relationship. The PQ photoelectric transfer function can be expressed as Equation (1-1).

number

[0066] The parameters in equation (1-1) are obtained by calculation using the following equations.

number

[0067] L denotes the linear signal value, which is normalized to [0,1].

[0068] L' denotes the nonlinear signal value, which ranges from [0, 1].

[0069]

number

[0070]

number

[0071]

number

[0072]

number

[0073]

number

[0074] The HLG photoelectric transfer function is obtained by improving the traditional Gamma curve. For the HLG photoelectric transfer function, the traditional Gamma curve is used in the lower half area, and a logarithmic curve is added to the upper half area. Figure 2 is a schematic diagram of the relationship of the HLG photoelectric transfer function. The HLG photoelectric transfer function shows the conversion relationship from the linear signal value of the image pixel to the nonlinear signal value in the HLG domain. The HLG photoelectric transfer function can be expressed as Equation (1-2).

number

[0075] L represents the linear signal value, which is in the range of [0,12]. L' represents the nonlinear signal value, which is in the range of [0,1]. a = 0.17883277, which represents the HLG photoelectric transfer coefficient. b = 0.28466892, which represents the HLG photoelectric transfer coefficient. c = 0.55991073, which represents the HLG photoelectric transfer coefficient.

[0076] The SLF photoelectric transfer function is the best-fit curve obtained based on the luminance distribution of the HDR scene when the optical characteristics of the human eye are met. Figure 3 shows a schematic diagram of the relationship of the SLF photoelectric transfer function.

[0077] The SLF photoelectric transfer curve shows the conversion relationship from the linear signal value of an image pixel to the nonlinear signal value of the SLF domain. The conversion relationship from the linear signal value of an image pixel to the nonlinear signal value of the SLF domain is expressed as Equation (1-3).

number

[0078] The SLF photoelectric transfer function can be expressed as equation (1-4).

number

[0079] L denotes the linear signal value, which is normalized to [0,1]. L' denotes the nonlinear signal value, which is in the range of [0,1]. p=2.3 denotes the SLF photoelectric transfer coefficient. m=0.14 denotes the SLF photoelectric transfer coefficient. a=1.12762 denotes the SLF photoelectric transfer coefficient. b=-0.12762 denotes the SLF photoelectric transfer coefficient.

[0080] Currently, capture end devices can capture HDR videos or images with a larger dynamic range (in the following embodiments, an image is used as an example for explanation, and it can be understood that a video can be represented as multiple frames of an image). However, the display capability of a display end device is limited and cannot fully match the dynamic range of the capture end device. In the prior art, a processing method is provided in which tone mapping (TM) can be performed on an image to be displayed, thereby enabling matching of images with different dynamic ranges. For example, tone mapping can be classified from a high dynamic range to a low dynamic range or from a low dynamic range to a high dynamic range. For example, an HDR image captured by a capture end device includes an illumination signal of 4000 nits (nits, illumination signal units), and the HDR display capability of the display end device (e.g., a television or mobile phone) is only 500 nits. In this case, the 4000 nit signal needs to be mapped to a 500 nit device. That is, the tone mapping process is from high to low. In another example, an HDR image includes an SDR signal of 100 nits, and the display capability of the target display end device is only 2000 nits. In this case, a 100 nit signal needs to be mapped to a 2000 nit device, i.e. the tone mapping process is from low to high.

[0081] Tone mapping is generally applied to matching a high-level HDR image captured by a capture end device with a low-level HDR image or an SDR image displayed by a display end device. Figure 4 is a schematic diagram of a possible system architecture to which an embodiment of the present application is applicable. The system architecture in this embodiment of the present application may include a capture end device 401, a display end device 402, and a transmission link 403. The display end device 402 may be further specifically classified into an HDR display device 402a and an SDR display device 402b.

[0082] The capture end device 401 is configured to capture or create HDR videos or images. For example, the capture end device 401 may be a video (or image) capture device or a video (or image) generation device. In a possible example, the capture end device 401 may further generate metadata based on the HDR videos or images. The metadata is used to record key information of the captured images of a scene or frame. In this embodiment of the present application, the metadata may include parameters of a tone mapping curve, dynamic metadata, static metadata, etc. Dynamic metadata may be understood as data associated with each frame of an image, where the data may change with different images, for example, the average, maximum, and minimum values ​​of pixel luminance within a scene. Static metadata may be understood as data associated with an image sequence, where the data remains unchanged within the image sequence.

[0083] The capture end device 401 may transmit data of HDR video or image, i.e., metadata, to the display end device 402 via the transmission link 403. Specifically, the HDR video or image and metadata may be transmitted in the form of one data packet, or may be transmitted separately in two data packets, which is not specifically limited in the embodiments of the present application.

[0084] The display end device 402 may be configured to receive the metadata and the HDR video or image, determine a tone mapping curve based on the parameters of the tone mapping curve included in the metadata, and perform tone mapping on the HDR image based on the tone mapping curve to convert the HDR image into display content applicable to the display end device 402. The display end device 402 may be classified into an HDR display device 402a and an SDR display device 402b. It should be noted that the HDR of the display end device 402 may be different from the HDR of the capture end device 401. Generally, the HDR of the display end device 402 is lower than the HDR of the capture end device 401. For example, the HDR of the capture end device 401 may be 10 -3 ~10 4 (This embodiment of the present application may also be referred to as "high-level HDR"). The HDR of the display end device 402 is 10 -1 ~10 3 (which may also be referred to as "low-level HDR" in this embodiment of the present application). It should be further understood that in different embodiments, the display end device 402 may further include a display end device having a display capability of a higher dynamic range than the HDR video or image generated by the capture end device 401. The display matching process of the display end device may also be applied to the system architecture, which is not limited in the present application.

[0085] Based on the content introduced in the background art, traditional tone mapping methods may mainly use tone mapping curve-based or block-based local tone mapping methods, but there exists a problem of insufficient tone mapping effect. For example, in tone mapping curve-based methods, tone mapping is generally performed based on a single local window. When a single shape is used to perform local processing, there may be boundary effects.

[0086] In view of this, an embodiment of the present application provides a tone mapping method to design a technical solution that can improve the tone mapping effect. The following describes the method provided in this embodiment of the present application with reference to the accompanying drawings.

[0087] 5 is a schematic flowchart of a tone mapping method applicable to the display end device 402 according to one embodiment of the present application. The process shown in FIG. 5 includes the following steps:

[0088] Step 501: The display end device 402 acquires an image frame to be displayed and metadata, where the metadata includes at least one metadata information unit. It should be understood that the image frame to be displayed may be an image or any image frame among multiple image frames included in a video. In addition, the image frame to be displayed may be data sent by the capture end device 401, or may be data obtained by another image processing procedure, such as global tone mapping. The metadata may be generated by the capture end device 401 based on the captured image frame to be displayed. A specific implementation of generating the metadata will be described in a subsequent processing procedure of the capture end device 401 and will not be described in detail here.

[0089] Step 502: The display end device 402 segments the image frame to be displayed into a plurality of segmentation regions in a target segmentation method, and determines a correspondence between the plurality of segmentation regions and at least one metadata information unit.

[0090] The target segmentation method may be a preset segmentation method or a segmentation method indicated in metadata. Optionally, the preset segmentation method may be preset by the display end device 402. It should be understood that the same preset segmentation method may be preset for the capture end device 401 and the display end device 402 in this scene to obtain accurate metadata. Alternatively, optionally, the target segmentation method of the display end device 402 may be indicated in metadata by the capture end device 401 to implement the same segmentation method for the capture end device 401 and the display end device 402.

[0091] For example, this embodiment of the present application provides the following several possible segmentation methods: It should be understood that the following segmentation methods are not limited in implementation.

[0092] In the segmentation method A, the display end device 402 uniformly segments the image frame to be displayed into a plurality of rectangular segmented regions based on first segmentation parameters corresponding to a first direction and a second direction, respectively. The first direction is perpendicular to the second direction. Optionally, the first segmentation parameter may be a segmentation number. In this case, the plurality of segmented regions obtained by segmentation are represented as rectangular regions of equal size.

[0093] FIG. 6 is a schematic diagram of segmentation in segmentation method A according to an embodiment of the present application. The first direction may be horizontal, and the second direction may be vertical. It is assumed that the number of segmentations in the first direction is represented by M1 (the value of M1 is generally ≦16), and the number of segmentations in the second direction is represented by M2 (the value of M2 is generally ≦16). In FIG. 6, in an example where the value of M1 is 4 and the value of M2 is also 4, the image frame to be displayed may be segmented into 16 segmentation regions, i.e., 16 rectangular regions of equal size. It should be understood that the values ​​of M1 and M2 may be the same or different. For example, the value of M1 may be 6, and the value of M2 may be 4.

[0094] In the segmentation method B, the display end device 402 segments the image frame to be displayed into a plurality of rectangular segmentation regions based on second segmentation parameters corresponding to the first direction and the second direction, respectively. Optionally, the second segmentation parameter may be a segmentation list. In this case, the plurality of segmentation regions obtained by segmentation are represented as rectangular regions of any size.

[0095] 7 is a schematic diagram of segmentation in segmentation method B according to an embodiment of the present application. It is assumed that the segmentation list in the first direction is represented as w[j1] (j1 may range from 1 to M1, and M1 indicates the number of segments in the first direction), and the segmentation list in the second direction is represented as h[j2] (j2 may range from 1 to M2, and M2 indicates the number of segments in the second direction). In FIG. 7, the value of M1 is 4, and the value of M2 is also 4. For example, every two values ​​of w[1], w[2], w[3], and w[4] included in FIG. 7 may be the same or different. Every two values ​​of h[1], h[2], h[3], and h[4] included in FIG. 7 may be the same or different.

[0096] In segmentation method C, the display end device 402 performs clustering on pixels included in the image frame to be displayed, and based on the clustering results, segments the image frame to be displayed into a plurality of irregular segmented regions, each of which includes at least one type of pixel.

[0097] FIG. 8 is a schematic diagram of segmentation in segmentation method C according to an embodiment of the present application. In the implementation of the present application, a target clustering method may be selected from multiple optional clustering methods to perform a clustering process on an image frame to be displayed. The specific clustering method is not limited in the present application. For example, the image frame to be displayed obtained by the clustering process may be segmented into 16 segmented regions shown in FIG. 8. For example, when performing clustering on an image frame to be displayed, the display end device 402 may first segment the image frame to be displayed into multiple rectangular regions based on the first segmentation parameter or the second segmentation parameter described above, then determine coordinate information and feature information (the feature information is, for example, color component values) of the center pixel of each rectangular region, and perform clustering analysis on each pixel and each center pixel included in the image frame to be displayed to obtain a clustering result. Finally, based on the clustering result, the schematic diagram of segmentation shown in FIG. 8 is obtained.

[0098] In addition, in order to facilitate the clustering analysis and reduce the amount of processing in the clustering process, when image segmentation is performed on the image frame to be displayed in the implementation of the present application, the image segmentation process may further be performed with reference to the image frame to be displayed as a thumbnail. For example, for the image frame to be displayed as a thumbnail, 1 / N1 of the resolution in the first direction of the captured image frame to be displayed and 1 / N2 of the resolution in the second direction may be selected. In this way, the clustering analysis is performed based on the image frame to be displayed as a thumbnail, thereby reducing the number of pixels in the clustering process.

[0099] It should be noted that, in the implementation of the present application, an appropriate segmentation method may be selected for an image having different characteristics. For example, segmentation method A may be applicable when the image frame to be displayed is an image frame whose local characteristics are not obvious, such as a pure-color image frame or a normal image, or may be applicable to any image frame. This is not limited in the present application. Segmentation method C may be applicable to an image frame having specific local characteristics, etc. It may be understood that the display end device 402 and / or the capture end device 401 may pre-store multiple segmentation methods and may further select a corresponding target segmentation method based on an image frame having different characteristics.

[0100] After obtaining the multiple segmented regions, the display end device 402 determines a metadata information unit corresponding to each segmented region based on at least one metadata information unit included in the metadata. Accordingly, parameters of a tone mapping curve, dynamic metadata, static metadata, etc. corresponding to the segmented regions may be obtained from the metadata information unit. Optionally, there may be a one-to-one correspondence between the multiple segmented regions and the at least one metadata information unit. That is, each segmented region may correspond to one metadata information unit. For example, in FIG. 8, 16 segmented regions may correspond to 16 metadata information units. Alternatively, there may be a many-to-one correspondence between the multiple segmented regions and the at least one metadata information unit. That is, one or more segmented regions may correspond to one metadata information unit. For example, in FIG. 8, segmented region 1 and segmented region 2 may correspond to one metadata information unit, segmented region 3 and segmented region 4 may correspond to one metadata information unit, etc. It can be understood that the many-to-one correspondence can reduce the amount of metadata data transmitted between the capture end device 401 and the display end device 402, and improve data transmission efficiency.

[0101] For example, this embodiment of the present application provides the following several possible examples of determining the correspondence between a plurality of segmented regions and at least one metadata information unit: It should be understood that the following examples are not limited in implementation.

[0102] Example 1: The capture end device 401 may indicate the corresponding segmentation region identifier in the metadata information unit.

[0103] Optionally, the segmentation region identifier may be the number of segmentation regions included in the metadata information unit. In implementation, the display end device 402 may traverse multiple segmentation regions included in the image frame to be displayed based on the target scanning sequence and the number of segmentation regions. The target scanning sequence includes, but is not limited to, left to right, then top to bottom, top to bottom, then left to right, etc. In addition, the target scanning sequence may be a preset scanning sequence or a scanning sequence indicated in metadata. It should be understood that the display end device 402 and the capture end device 401 use the same preset scanning sequence.

[0104] For example, Figure 9 is a schematic diagram of a scanning sequence according to an embodiment of the present application. The illustrated scanning sequence is from left to right, then from top to bottom. It is assumed that the 16 segmented areas corresponding to Figure 9 correspond to three metadata information units. The first metadata information unit corresponds to four segmented areas, the second metadata information unit corresponds to seven segmented areas, and the third metadata information unit corresponds to five segmented areas. In this case, the correspondence relationship between multiple segmented areas and at least one metadata information unit may be listed in Table 1-1 below.

[0105] [Table 1]

[0106] Based on the correspondence relationships described in the above Table 1-1, the display end device 402 may determine, based on the target scanning sequence, that the first metadata information unit corresponds to segmentation area 1 to segmentation area 4, the second metadata information unit corresponds to segmentation area 5 to segmentation area 11, and the third metadata information unit corresponds to segmentation area 12 to segmentation area 16.

[0107] Alternatively, optionally, the segmentation region identifier may be segmentation region sequence number information. In implementation, the display end device 402 may determine the segmentation region corresponding to the metadata information unit based on the segmentation region sequence number information included in the metadata information unit.

[0108] For example, the content shown in Figure 9 is used. In this example, the correspondence between the plurality of segmented regions and at least one metadata information unit may be listed in Table 1-2 below.

[0109] [Table 2]

[0110] Based on the correspondence relationship described in Table 1-2 above, the display end device 402 may directly determine the corresponding segmentation region based on the segmentation region sequence number information included in the metadata information unit.

[0111] Example 2: For any segmented region, the display end device 402 may further select one or more pixels at preset positions from the segmented region, obtain coordinate information of the one or more pixels at the preset positions, and determine a metadata information unit corresponding to the segmented region based on the coordinate information, thus establishing a correspondence between the segmented region and the corresponding metadata information unit. The preset positions may include one or more of the following positions: upper left corner, center, lower right corner, upper left corner, and lower right corner, etc. For example, for segmented region 1, the upper left corner, center corner, and lower right corner may be selected as the preset positions. The coordinate information of the upper left corner is (x1, y1), the coordinate information of the center is (x2, y2), and the coordinate information of the lower right corner is (x3, y3). Then, a metadata information unit including (x1, y1), (x2, y2), and (x3, y3) is determined, thereby obtaining a metadata information unit corresponding to segmented region 1.

[0112] It should be understood that for any metadata information unit, the display end device 402 may further determine at least one segmented area corresponding to the one or more coordinate information based on the one or more coordinate information included in the metadata information unit, and establish a correspondence between the metadata information unit and the corresponding at least one segmented area.

[0113] For the i-th pixel (where i is any positive integer from 1 to N, and N is the number of pixels included in the image frame to be displayed) in the image frame to be displayed, the display end device 402 performs the following steps 503 to 505 (it should be understood that in the following embodiments, an arbitrary pixel is used as an example, and the processing processes of other pixels included in the image frame to be displayed are similar and will not be described one by one in the following embodiments).

[0114] Step 503: Based on the plurality of pixels included in the preset area selected for the i-th pixel, determine at least one associated segmentation area to which the plurality of pixels belong, and based on the correspondence relationship, determine an associated metadata information unit corresponding to each associated segmentation area.

[0115] The preset region selected for the i-th pixel may be a region of a preset shape of a preset size selected using the i-th pixel as its center (for example, the preset shape may be a rectangle or another shape having a preset width and a preset length), a region determined based on preset shape and preset size information included in the metadata using the i-th pixel as its center (if the preset shape is a rectangle, the preset size information may be length and width information of the rectangle, or if the preset shape is a circle, the preset size information may be radius or diameter information of the circle), or a region determined based on preset shape and preset size information included in the metadata using the i-th pixel as its center. In other words, the preset shape and / or the preset size information of the preset shape used by the display end device 402 to select the preset region for the i-th pixel may be locally stored information or may be information indicated in the metadata.

[0116] For example, FIG. 10 is a schematic diagram of determining associated segmentation regions according to an embodiment of the present application. An image frame to be displayed is obtained using the segmentation method applied in FIG. 6. Referring to (a) and (b) of FIG. 10, when pixel 1 is processed, a rectangular preset region having pixel 1 as its center includes multiple pixels. The multiple pixels belong to segmentation region 1, segmentation region 2, segmentation region 5, and segmentation region 6. In this case, it can be determined that pixel 1 has four associated segmentation regions. Referring to (a) and (c) of FIG. 10, when pixel 2 is processed, the multiple pixels included in the rectangular preset region having pixel 2 as its center belong to segmentation regions 6, 7, 8, 10, 11, 12, 14, 15, and 16. In this case, it can be determined that pixel 2 has nine associated segmentation regions. It should be understood that the associated segmentation regions of a pixel are determined based on the pixel position and the preset region, and the number of associated segmentation regions of different pixels included in the image frame to be displayed may be the same or different. It should be noted that multiple pixels included in an image frame to be displayed may typically use preset regions with the same preset size information. However, this is not limited to this application, and preset regions with different preset size information may also be used in practice. Pixel 1 and pixel 2 shown in Figure 10 use different preset size information.

[0117] In another example, Figure 11 is another schematic diagram of determining associated segmentation regions according to an embodiment of the present application. An image frame to be displayed is obtained with the segmentation method applied in Figure 8. When pixel 3 is processed, multiple pixels included in a rectangular preset region having pixel 3 as its center belong to segmentation regions 9, 10, 13, 14, and 15. In this case, it can be determined that pixel 3 has five associated segmentation regions.

[0118] Based on the display end device 402 obtaining the correspondence between the plurality of segmentation areas and at least one metadata information unit in step 502, after the associated segmentation area of ​​the pixel is determined, key information contained in the associated segmentation area, such as parameters of the tone mapping curve, dynamic metadata, static metadata, and other metadata information, can be further obtained from the metadata information unit corresponding to the associated segmentation area to perform tone mapping of the pixel to be processed.

[0119] Step 504: The display end device 402 obtains tone mapping sub-values ​​corresponding to each associated segmented region based on the associated metadata information unit, and allocates corresponding weighting coefficients to each associated segmented region.

[0120] For example, the display end device 402 may obtain a tone mapping sub-value corresponding to the i-th pixel in the associated segmented region based on a tone mapping curve, and the associated metadata information unit includes parameters of the tone mapping curve. It may be understood that the number of tone mapping sub-values ​​for the i-th pixel depends on the number of associated segmented regions. For example, for pixel 1 shown in FIG. 10, if the number of associated segmented regions of pixel 1 is 4, four tone mapping sub-values ​​corresponding to pixel 1 may be obtained. For pixel 2 shown in FIG. 10, if the number of associated segmented regions of pixel 2 is 9, nine tone mapping sub-values ​​corresponding to pixel 2 may be obtained.

[0121] The parameters of the tone mapping curve included in the associated data information units described in the above examples are not limited in the implementation of the present application. For example, the parameters of the tone mapping curve may be determined based on the shape of the selected tone mapping curve. The shape of the tone mapping curve includes, but is not limited to, an S (sigmoidal) function curve, a cubic (or multi-order) spline function curve, a Bezier curve, and a gamma function curve.

[0122] In an embodiment in which the tone mapping curve is obtained based on the parameters of the tone mapping curve, the tone mapping curve is a cubic spline function. In the implementation of the present application, the display end device 402 determining the tone mapping curve of any relevant segmented region may include the following steps:

[0123] A1: Obtain the maximum luminance value (maxLum) from the relevant metadata information unit corresponding to the relevant segmented region.

[0124] A2: Determine the interval length (Len) of the cubic spline based on maxLum and the cubic spline interval information (Num3). Len can be expressed as in equation (2-1).

number

[0125] A3: Based on Len and the interval index (index, for identifying the first interval), the start position (TH[index-1]) and end position (TH[index]) of the first interval are obtained, where the first interval is any interval among the multiple intervals included in the cubic spline. TH[index] can be expressed as in Equation (2-2). TH[index]=index×Len(2-2)

[0126] A4: Obtain the cubic spline parameters P0, P1, P2, and P3 of the first interval based on the cubic spline interval information, the interval index, and the cubic spline interval value.

[0127] A4-1: Obtain the initial section gradient of the first section based on the cubic spline section value and the cubic spline section information. The initial section gradient can be expressed as in Equation (2-3).

number

[0128] gainOrg[index] indicates the initial interval gradient for the first interval, THValue[index] indicates the cubic spline interval value for the first interval, and THValue[index-1] indicates the cubic spline interval value for the interval before the first interval.

[0129] A4-2: Obtain the interval end point gradients of the first interval based on the initial interval gradient. The interval end point gradients include the start interval end point gradient, which can be expressed as in the following equation (2-4), and the end interval end point gradient, which can be expressed as in the following equation (2-5).

number

[0130] gainOrg[index-1] indicates the initial interval gradient of the interval before the first interval. gainStart[index] indicates the starting interval endpoint gradient.

number

[0131] gainOrg[index+1] indicates the initial interval gradient of the interval following the first interval. gainEnd[index] indicates the end interval endpoint gradient.

[0132] A4-3: The cubic spline parameters P0, P1, P2, and P3 of the first interval can be expressed as the following equations (2-6) to (2-9).

number

[0133] ValueA, ValueB, and ValueC in equations (2-8) and (2-9) are intermediate values, which can be obtained by calculation in the following manner. ValueA=THValue[index]-gainStart[index]×ValueC-THValue[index-1] ValueB=gainEnd[index]-gainStart[index] ValueC=TH[index]-TH[index-1]

[0134] A5: Based on the cubic spline parameters P0, P1, P2, and P3, the formula of the function corresponding to the tone mapping curve of the relevant segmented region can be obtained and expressed as the following formula (2-10): L'=H(L)=P0+P1×(L-TH[index-1])+P2×(L-TH[index-1]) 2 +P3×(L-TH[index-1]) 2 (2-10)

[0135] L denotes the linear signal value, which is normalized to [0,1]. L' denotes the nonlinear signal value, which is in the range [0,1]. H(L) denotes the function corresponding to the tone mapping curve.

[0136] In another embodiment of obtaining a tone mapping curve based on the parameters of the tone mapping curve, the formula of the function corresponding to the tone mapping curve may alternatively be expressed as the following formula (2-11):

number

[0137] L denotes a linear signal value, which is normalized to [0,1]. L' denotes a nonlinear signal value, which is in the range of [0,1]. F(L) denotes a function corresponding to a tone mapping curve. a, b, p, m, and n denote parameters of the tone mapping curve, which are used to adjust the shape of the tone mapping curve. It should be understood that different segmentation regions may correspond to different parameters of the tone mapping curve function described in (2-11). In implementation, the parameters of the tone mapping curve for each segmentation region may be indicated in the corresponding metadata information unit.

[0138] It should be noted that in implementation, different functions of the tone mapping curve may be used based on different forms of the tone mapping curve. The selected tone mapping curve is not limited in this application.

[0139] After obtaining the tone mapping curve corresponding to the associated segmented region, the display end device 402 performs tone mapping on the i-th pixel based on the tone mapping curve to obtain tone mapping sub-values ​​for the pixel in the associated segmented region. For example, for pixel 3 in Figure 11, a total of five tone mapping sub-values ​​may be obtained, including tone mapping sub-value 1 corresponding to segmented region 9, tone mapping sub-value 2 corresponding to segmented region 10, tone mapping sub-value 3 corresponding to segmented region 13, tone mapping sub-value 4 corresponding to segmented region 14, and tone mapping sub-value 5 corresponding to segmented region 15.

[0140] In one embodiment of tone mapping, the display end device 402 may perform a brightness adjustment process and / or a color adjustment process for the i-th pixel based on the tone mapping curve. Optionally, the color adjustment performed for the i-th pixel may be expressed as the following equation (3-1):

number

[0141] Rp, Gp, and Bp denote the three color components of the i-th pixel. f[id] denotes the function corresponding to the tone mapping curve, e.g., H(L) and F(L) described above. id denotes the segmentation region identifier.

[0142] The tone mapping sub-value of the i-th pixel in the associated segmented region can be expressed as the following equations (3-2) to (3-4). RpTM[i] = Rp × gain (3 - 2) GpTM[i] = Gp × gain (3-3) BpTM[i] = Bp × gain (3-4)

[0143] RpTM[i], GpTM[i], and BpTM[i] may denote the tone mapping sub-values ​​of the i-th pixel in the associated segmented region.

[0144] In another example, the display end device 402 may determine a tone mapping sub-value corresponding to each associated segmented region, and further determine a weighting factor corresponding to each associated segmented region. Thus, the influence of the tone mapping sub-value of each associated segmented region on the tone mapping value of the i-th pixel is determined based on the weighting factor.

[0145] Optionally, in the implementation of the present application, the display end device 402 may allocate a corresponding weighting factor to each associated segmented region. In addition, the display end device 402 may determine the weighting factor in a preset manner, or may determine the weighting factor according to instructions in metadata. The present application provides the following two possible methods for determining the weighting factor. In the implementation, one or a combination of the following methods may be used, and the following two methods are not limited in actual implementation.

[0146] Method 1: Separately determine the number of pixels belonging to a preset region in each associated segmentation region and the total number of pixels corresponding to the preset region, and perform the following operation with respect to a first associated segmentation region: assign a corresponding first weighting factor to the first associated segmentation region based on the number of pixels belonging to the preset region and the total number of pixels in the first associated segmentation region, where the first associated segmentation region is any one of the at least one associated segmentation region.

[0147] For example, referring to pixel 3 shown in FIG. 11, the number of pixels belonging to the preset region in each associated segmented region and the total number of pixels corresponding to the preset region may be listed in Table 2-1 below.

[0148] [Table 3]

[0149] Num[id] indicates the number of pixels in the segmented region included in the preset region, for example, the number of pixels included in each segmented region associated with pixel 3 in the preset region shown in (b) of Figure 11, where id indicates the segmented region identifier. Weight1[id] indicates the first weighting factor for the segmented region id.

[0150] Optionally, the first weighting factor may be positively correlated, for example linearly, with the number of pixels in the segmented region, in other words, a larger number of pixels in the segmented region indicates a larger first weighting factor corresponding to the segmented region, and the increase is linear.

[0151] Alternatively, optionally, the positive correlation may be an exponential correlation, and the exponential correlation may be determined based on a first weighted intensity value. The first weighted intensity value is used to adjust the first weighting factor based on the pixel number distribution status of the image frame to be displayed. For example, adjusting the first weighting factor using the first weighted intensity value may be expressed as the following equation (4-1):

number

[0152] NumStr[id] indicates a first weighted intensity value, which may be, for example, 2 or 0.5. The first weighted intensity value may be determined based on pixel similarity between multiple segmented regions in an image frame to be displayed. For example, if the pixels of multiple segmented regions are similar, a large first weighted intensity value may be set. Otherwise, a smaller first weighted intensity value may be set. It should be understood that the first weighted intensity value may be determined by the display end device 402 based on a preset rule or may be indicated by the capture end device 401 in metadata. This is not limited in the present application. In this way, the first weighted intensity value is set, and thereby, when the local features of the image frame to be displayed are clear, the boundary effect between segmented regions can be reduced based on a smaller first weighted intensity value, for example, 0.5.

[0153] Method 2: Separately determine a preset feature value corresponding to a preset region in each associated segmentation region and a preset feature value of the i-th pixel, and perform the following operation with respect to a second associated segmentation region: assign a corresponding second weighting factor to the second associated segmentation region based on a feature difference between the preset feature value corresponding to a subregion of the preset region in the second associated segmentation region and the preset feature value of the i-th pixel, where the second associated segmentation region is any one of the at least one associated segmentation region. The preset feature value may be a luminance feature value (AvgLum[id]) or a color feature value (AvgRGB[id][3], where [3] indicates three color components). It should be understood that the same preset feature value is selected as the preset feature value corresponding to the preset region in the associated segmentation region and the preset feature value of the i-th pixel.

[0154] There is at least one preset feature value. For example, there may be four luminance feature values (AvgLum[id](4)) in one segmented area, and there may also be four color feature values (AvgRGB[id](4)[3]) in one segmented area. FIG. 12 is a schematic diagram of luminance feature values according to an embodiment of the present application. A scenario with one luminance feature value is shown in (a) of FIG. 12, and a scenario with four luminance feature values is shown in (b) of FIG. 12. The number of luminance feature values can be obtained based on the luminance value having the maximum number or average maximum number within the segmented area.

[0155] In the foregoing example, when there are four preset feature values, at the time of implementing the present application, the target preset feature value selected from the four preset feature values can be further determined based on the correspondence between the center coordinates (x, y) of the sub-region of the preset region within the second related segmented area (or the coordinates (x, y) of the i-th pixel (current pixel)) and the center coordinates (xR, yR) of the second related segmented area. For example, referring to the content shown in (b) of FIG. 12, the upper left area has a luminance feature value of 1, the upper right area has a luminance feature value of 2, the lower left area has a luminance feature value of 3, and the lower right area has a luminance feature value of 4. When x ≤ xR and y ≥ yR, the luminance feature value 1 is used. When x > xR and y ≥ yR, the luminance feature value 2 is used. When x ≤ xR and y < yR, the luminance feature value 3 is used. When x > xR and y < yR, the luminance feature value 4 is used.

[0156] For example, referring to pixel 3 shown in FIG. 11, in the example where the preset region feature is a luminance feature value, the luminance feature value belonging to the preset region within each related segmented area and the luminance feature value of the i-th pixel can be described in Table 2-2 below.

[0157]

Table 4

[0158] AvgLum[id] indicates the luminance feature value of a segmented region included in the preset region, for example, the luminance feature value of a subregion belonging to the preset region within the segmented region associated with pixel 3 shown in FIG. 11(b), where id indicates the segmented region identifier. Weight2[id] indicates the second weighting factor of the segmented region id.

[0159] Optionally, the second weighting factor may be negatively correlated, for example linearly, with the difference value between the luminance feature value of the segmented area and the luminance feature value of the i-th pixel. In other words, a larger difference value between the luminance feature value of the segmented area and the luminance feature value of the i-th pixel indicates a smaller second weighting factor corresponding to the segmented area, and the decrease is linear.

[0160] Or, optionally, negative The correlation may be an exponential relationship, and the exponential relationship may be determined based on a second weighted intensity value. The second weighted intensity value is used to adjust the second weighting factor based on a preset feature value change status of the image frame to be displayed. For example, adjusting the second weighting factor using the second weighted intensity value may be expressed as the following equation (4-2):

number

[0161] g() indicates a normalization operation. For example, the signal is divided into a range of 0 to 1 by the maximum value of the signal. clip() indicates a clamping operation. min is the minimum value (e.g., 0). max is the maximum value (e.g., 1). Values ​​of the third operand included in clip() that are greater than max or less than min are truncated to max and min, respectively. For example, if the value of the third operand included in clip() is less than min, min is used. If the third operand included in clip() is greater than max, max is used. K is a preset value ranging from 0 to 1023. AvgLumStr[id] indicates a second weighted intensity value, which may be, for example, 2 or 0.5.

[0162] For example, the second weighted intensity value may be determined based on pixel luminance variations of multiple segmented regions in an image frame to be displayed. For example, if pixel luminance variations of multiple segmented regions are evident, a large second weighted intensity value may be set. Otherwise, a smaller second weighted intensity value may be set. It should be understood that the second weighted intensity value may be determined by the display end device 402 based on a preset rule or may be indicated by the capture end device 401 in metadata. This is not limited to this application. In this way, the second weighted intensity value is set, so that when local features of the image frame to be displayed are evident, the boundary effect between segmented regions can be reduced based on a smaller second weighted intensity value, for example, 0.5.

[0163] Optionally, the weighting factor corresponding to each relevant segmentation region may be a first weighting factor, which may be obtained by calculation according to equation (4-1), i.e., Weight[id] = Weight1[id], or may be a second weighting factor, which may be obtained by calculation according to equation (4-2), i.e., Weight[id] = Weight2[id], or may be determined based on both the first weighting factor and the second weighting factor, and may be expressed as the following equation (4-3): Weight[id]=W(Weight1[id],Weight2[id])(4-3)

[0164] W() denotes a preset function used to obtain a weighting factor corresponding to the relevant segmented region based on the first weighting factor and the second weighting factor. The preset function may be multiplication or addition, etc., and may be preset.

[0165] Step 505: The display end device 402 obtains a tone mapping value for the i-th pixel based on the tone mapping sub-value and weighting coefficient corresponding to each associated segmented region, and performs tone mapping on the i-th pixel based on the tone mapping value.

[0166] For example, in the present application, the tone mapping value of the i-th pixel can be obtained according to the following equations (5-1) to (5-5).

number

[0167] Z indicates the number of the associated segmented region of the ith pixel. Rtm, Gtm, and Btm indicate the tone mapping values. spline[id] indicates the function corresponding to the tone mapping curve. id indicates the segmented region identifier.

[0168] Additionally, in the implementation of this application, the tone mapping values ​​may alternatively be converted to color space from Rtm, Gtm, and Btm to obtain Ytm, Utm, and Vtm, with the corresponding tone mapping sub-values ​​being Yp, Up, and Vp.

[0169] Furthermore, in the implementation of the present application, after obtaining the tone mapping value of the i-th pixel, the display end device 402 may further perform a saturation adjustment on the image frame to be displayed. In other words, the display end device 402 performs a saturation adjustment on the i-th pixel based on the initial pixel value of the i-th pixel and the tone mapping value obtained in step 505.

[0170] It should be noted that in the above example, the information required to calculate the tone mapping value for each pixel may be processed by the display end device 402, or may be obtained by the capture end device 401 through calculation and indicated to the display end device 402 using metadata. In other words, the content that needs to be processed by each of the display end device 402 and the capture end device 401 may be allocated based on the computational capabilities, which is not limited in the present application. It can be understood that the display end device 402 may then perform tone mapping on the image frame to be displayed based on the information indicated in the metadata.

[0171] 13 is another schematic flowchart of a tone mapping method according to an embodiment of the present application, which is applicable to the capture end device 401 shown in FIG.

[0172] Step 1301: Capture image data to obtain a data frame to be displayed. For example, the image data may be captured by the capture end device 401 using an image capture device, or may be captured by the capture end device 401 using a pre-configured application program. 1 It may be created and generated by

[0173] Step 1302: Using a target segmentation method, segment the data frame to be displayed into multiple segmentation regions, and determine at least one corresponding metadata information unit for the multiple segmentation regions. For example, the target segmentation method may be multiple segmentation methods described above, such as segmentation method A shown in FIG. 6, segmentation method B shown in FIG. 7, and segmentation method C shown in FIG. 8, etc. In addition, the capture end device 401 and the display end device 402 use the same segmentation method. In addition, the capture end device 401 may further indicate the target segmentation method to the display end device 402 using metadata ... 1 may further use metadata to indicate to the display end device 402 first and second segmentation parameters, etc., that may be used in the target segmentation method.

[0174] For example, in the case of segmentation method A, the capture end device 401 may determine the number of vertical segmentations based on the histograms of different rows in the image data, and may determine the number of horizontal segmentations based on the histograms of different columns in the image data.

[0175] In another example, for segmentation method B, the capture end device 401 may perform clustering based on histograms of different rows in the image data to determine a vertical segmentation list, and may perform clustering based on histograms of different columns in the image data to determine a horizontal segmentation list.

[0176] Step 1303: Send a data frame to be displayed and metadata to the display end device 402, where the metadata includes at least one metadata information unit.

[0177] It should be understood that, in order to reduce the calculation amount of the display end device 402 and balance the contents that need to be processed separately by the capture end device 401 and the display end device 402, in the implementation of the present application, the capture end device 401 may obtain information such as the first weighted intensity value, the second weighted intensity value, and the preset feature value of the segmented region through calculation, and indicate it to the display end device 402 using metadata. It should be noted that the implementation process of the processing performed by the capture end device 401 is similar to the described content of the display end device 402. Therefore, the details will not be described again.

[0178] According to the same inventive concept as the aforementioned method, an embodiment of the present application provides a tone mapping processing device 1400. The tone mapping processing device 1400 may be applicable to the aforementioned display end device 402. As shown in FIG. 14 , the device may include: an acquisition module 1401, a processing module 1402, and a tone mapping module 1403.

[0179] The acquisition module 1401 is configured to acquire an image frame to be displayed and metadata, where the metadata includes at least one metadata information unit. The processing module 1402 is configured to segment the image frame to be displayed into a plurality of segmentation regions in a target segmentation manner, and determine a correspondence between the plurality of segmentation regions and the at least one metadata information unit. The tone mapping module 1403 is configured to perform the following operations with respect to an i-th pixel in an image frame to be displayed: based on a plurality of pixels included in a preset area selected for the i-th pixel, determine at least one associated segmentation area to which the plurality of pixels belong, and determine an associated metadata information unit corresponding to each associated segmentation area based on the correspondence relationship; based on the associated metadata information unit, obtain tone mapping sub-values ​​corresponding to each associated segmentation area and assign corresponding weighting factors to each associated segmentation area; and obtain a tone mapping value for the i-th pixel based on the tone mapping sub-values ​​and weighting factors corresponding to each associated segmentation area, and perform tone mapping for the i-th pixel based on the tone mapping value, where i is any positive integer from 1 to N, and N is the number of pixels included in the image frame to be displayed.

[0180] In a possible design, when configured to allocate a corresponding weighting factor to each associated segmentation region, the tone mapping module 1403 specifically determines the number of pixels belonging to a preset region in each associated segmentation region and the total number of pixels corresponding to the preset region separately, and performs the following operation with respect to a first associated segmentation region: allocate a corresponding first weighting factor to the first associated segmentation region based on the number of pixels belonging to the preset region and the total number of pixels in the first associated segmentation region, where the first associated segmentation region is a region including at least one associated segmentation region. and / or separately determine preset feature values ​​corresponding to preset regions in each associated segmented region, and perform the following operation with respect to a second associated segmented region: assign a corresponding second weighting factor to the second associated segmented region based on a feature difference between the preset feature value corresponding to the preset region in the second associated segmented region and the preset feature value of the ith pixel, wherein the second associated segmented region is any one of the at least one associated segmented region.

[0181] In a possible design, the metadata further includes one or a combination of the following information: a first weighted intensity value or a second weighted intensity value: the first weighted intensity value is used to adjust the first weighting factor based on a pixel count distribution status of the image frame to be displayed; and the second weighted intensity value is used to adjust the second weighting factor based on a preset feature value change status of the image frame to be displayed.

[0182] In a possible design, the preset feature value is one of the following feature values: a luminance feature value or a color feature value. There is at least one preset feature value.

[0183] In a possible design, when configured to obtain a tone mapping sub-value corresponding to each associated segmented region based on an associated metadata information unit, the tone mapping module 1403 is specifically configured to obtain a tone mapping sub-value corresponding to the i-th pixel in the associated segmented region based on a tone mapping curve, and the associated metadata information unit includes parameters of the tone mapping curve.

[0184] In possible designs, the preset region selected for the i-th pixel is a region of a preset shape with a preset size selected using the i-th pixel as its center, a region determined based on preset shape and preset size information included in the metadata using the i-th pixel as its center, or a region determined based on preset shape and preset size information included in the metadata using the i-th pixel as its center.

[0185] In a possible design, when configured to determine a correspondence between a plurality of segmented areas and at least one metadata information unit, the processing module 1402 is specifically configured to: for any segmented area, select a pixel at a preset position from the segmented area, obtain coordinate information of the pixel at the preset position, determine a metadata information unit corresponding to the segmented area based on the coordinate information, and establish a correspondence between the segmented area and the corresponding metadata information unit; or for any metadata information unit, determine at least one segmented area corresponding to the one or more coordinate information based on one or more coordinate information included in the metadata information unit, and establish a correspondence between the metadata information unit and the corresponding at least one segmented area.

[0186] In a possible design, when configured to determine a correspondence between a plurality of segmented regions and at least one metadata information unit, the processing module 1402 is specifically configured to traverse the plurality of segmented regions based on the target scan sequence and the segmented region identifier to determine a metadata information unit corresponding to each of the segmented regions, wherein the metadata information unit includes the corresponding segmented region identifier, and the target scan sequence is a preset scan sequence or a scan sequence indicated in the metadata.

[0187] In a possible design, there is a one-to-one or many-to-one correspondence between the plurality of segmented regions and the at least one metadata information unit.

[0188] In a possible design, the target segmentation method is a pre-configured segmentation method or a segmentation method indicated in the metadata.

[0189] According to the same inventive concept as the above-mentioned method, one embodiment of the present application comprises a tone mapping processing device 1 5 15, the tone mapping processing device 1500 may be applicable to the aforementioned capture end device 401. As shown in FIG. 15, the device 1500 may include a capture module 1501, a processing module 1502, and a transceiver module 1503.

[0190] The capture module 1501 is configured to capture image data to obtain a data frame to be displayed. The processing module 1502 is configured to segment the data frame to be displayed into a plurality of segmented regions in a target segmentation manner and determine at least one corresponding metadata information unit for the plurality of segmented regions. The transceiver module 1503 is configured to transmit the data frame to be displayed and the metadata to a display end device, where the metadata includes at least one metadata information unit.

[0191] In a possible design, the processing module 1502 is further configured to determine a first weighted intensity value based on the number of pixels included in each of the plurality of segmented regions and / or determine a second weighted intensity value based on preset feature values ​​corresponding to each of the plurality of segmented regions, and associate the first weighted intensity value and / or the second weighted intensity value with metadata.

[0192] In a possible design, the preset feature value is one of the following feature values: a luminance feature value or a color feature value. There is at least one preset feature value.

[0193] In a possible design, the metadata information unit includes parameters of a tone mapping curve corresponding to the segmented region.

[0194] In a possible design, the metadata further includes preset size information, which indicates a preset size of a preset shape to be used when a preset region is selected for a pixel, or the metadata further includes a preset shape and preset size information.

[0195] In a possible design, when configured to determine at least one corresponding metadata information unit for a plurality of segmented regions, the processing module 1502 is specifically configured to select a plurality of pixels at preset positions from the segmented region, obtain coordinate information of the plurality of pixels at the preset positions, and associate the coordinate information with the metadata information unit.

[0196] In a possible design, the metadata information unit further includes a segmentation region identifier and a scanning sequence corresponding to the metadata information unit, the corresponding segmentation region identifier and scanning sequence being used by the display end device to determine a correspondence between the plurality of segmentation regions and the at least one metadata information unit.

[0197] In a possible design, there is a one-to-one or many-to-one correspondence between the plurality of segmented regions and the at least one metadata information unit.

[0198] In a possible design, the processing module 1502 is further configured to associate the target segmentation method with the metadata to indicate to a display end device to segment the data frame to be displayed in the target segmentation method.

[0199] An embodiment of the present application further provides an electronic device 1600. As shown in FIG. 16 , the electronic device 1600 may be the display end device 402 or the capture end device 401 described in the previous embodiments. The electronic device 1600 may include a communication interface 1610 and a processor 1620. Optionally, the electronic device 1600 may further include a memory 1630. The memory 1630 may be located inside the electronic device or outside the electronic device. The acquisition module 1401, the processing module 1402, and the tone mapping module 1403 shown in FIG. 14 may all be implemented by the processor 1620. Alternatively, the capture module 1501, the processing module 1502, and the transceiver module 1503 shown in FIG. 15 may also be implemented by the processor 1620. Optionally, the communication interface 1610, the processor 1620, and the memory 1630 may be connected to each other via a communication line 1640. The communication lines 1640 may be a peripheral component interconnect (PCI) bus, an extended industry standard architecture (EISA) bus, etc. The communication lines 1640 may be categorized into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used to represent a bus in FIG. 16, but this does not mean that there is only one bus or only one type of bus.

[0200] In a possible implementation, the processor 1620 is configured to implement any of the methods performed by the capture end of FIG. 13 and to output, via the communication interface 1610, a bitstream encoded by the capture end, e.g., image frames to be displayed and metadata.

[0201] In implementation, the steps of the processing process may be implemented by integrated logic circuitry in hardware within the processor 1620 or by instructions in the form of software to perform the method performed by the capture end of Figure 13. For the sake of brevity, the details will not be described again here. Program code used by the processor 1620 to perform the aforementioned method may be stored in the memory 1630. The memory 1630 is coupled to the processor 1620.

[0202] Any communication interface in the embodiments of the present application may be a circuit, a bus, a transceiver, or another device that can be configured to exchange information, such as the communication interface 1610 in the electronic device 1600. For example, the other device may be a device connected to the electronic device 1600. For example, when the electronic device 1600 is a capture end device 401, the other device may be a display end device 402, etc.

[0203] The processor 1620 may cooperate with memory 1630. The memory 1630 may be non-volatile memory, such as a hard disk drive (HDD) or solid-state drive (SSD), or volatile memory, such as random access memory (RAM). The memory 1630 may be configured to hold or store desired program code in the form of instructions or data structures and may be any other medium that can be accessed by a computer, without being limited to such a medium.

[0204] The specific connection medium between the communication interface 1610, the processor 1620, and the memory 1630 is not limited in the embodiment of the present application. In this embodiment of the present application, the memory 1630, the processor 1620, and the communication interface 1610 are connected to each other via the bus in FIG. 16. In FIG. 16, the bus is represented using a thick line. The connection method between other components is merely an example for explanation and is not limited thereto. The bus may be classified into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is used to represent the bus in FIG. 16, but this does not mean that there is only one bus or only one type of bus.

[0205] In the embodiments of the present application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor, or any conventional processor, etc. The steps of the methods disclosed with reference to the embodiments of the present application may be performed directly by a hardware processor, or may be performed by a combination of hardware and software modules in the processor.

[0206] A coupling in the embodiments of the present application is an indirect coupling or communication connection between devices or modules for the exchange of information between the devices or modules, which may be electrical, mechanical, or in other forms.

[0207] Based on the above-mentioned embodiment, one embodiment of the present application further provides a computer storage medium. The storage medium stores a software program. When the software program is read and executed by one or more processors, the method provided in any one or more of the above-mentioned embodiments can be implemented. The computer storage medium may include any medium capable of storing program code, such as a USB flash drive, a removable hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0208] Based on the above-mentioned embodiments, one embodiment of the present application further provides a chip. The chip includes a processor configured to perform the functions of any one or more of the above-mentioned embodiments. Optionally, the chip further includes a memory, and the memory is configured to store necessary program instructions and data executed by the processor. The chip may include a chip, or may include a chip and another discrete device.

[0209] Those skilled in the art will understand that the embodiments of the present application may be provided as a method, a system, or a computer program product. Thus, the present application may take the form of a hardware-only embodiment, a software-only embodiment, or an embodiment having a combination of software and hardware. In addition, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0210] The present application has been described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that computer program instructions can be used to implement each process and / or each block in the flowcharts and / or block diagrams, and combinations of processes and / or blocks in the flowcharts and / or block diagrams. The computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or another programmable data processing device to generate a machine, whereby the instructions executed by the processor of the computer or another programmable data processing device generate an apparatus for performing the specific functions in one or more procedures of the flowcharts and / or one or more blocks of the block diagrams.

[0211] The computer program instructions may alternatively be stored in a computer-readable memory that can be configured to cause a computer or another programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable memory create an artifact that includes an instruction apparatus that implements a particular function in one or more steps of the flowcharts and / or one or more blocks of the block diagrams.

[0212] The computer program instructions may alternatively be loaded into a computer or other programmable data processing device such that a sequence of operations and steps are executed on the computer or other programmable device, thereby generating a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing a particular function in one or more procedures of the flowcharts and / or one or more blocks of the block diagrams.

[0213] It is apparent that those skilled in the art can make various modifications and variations to the embodiments of the present application without departing from the scope of the embodiments of the present application. The present application intends to cover these modifications and variations in the embodiments of the present application as long as they fall within the scope of the claims of the present application and their equivalent techniques. [Explanation of symbols]

[0214] 401 Capture End Device 402 Display End Device 402a HDR display device 402b SDR display device 403 Transmission Link 1400 Tone Mapping Processor 1401 Acquisition Module 1402 Processing Module 1403 Tone Mapping Module 1500 Tone Mapping Processor 1501 Capture Module 1502 Processing Module 1503 Transceiver Module 1600 Electronic Devices 1610 communication interface 1620 processor 1630 memory 1640 communication line

Claims

1. 1. A tone mapping method applied to a display end device, said method comprising: obtaining an image frame to be displayed and metadata, the metadata including at least one metadata information unit; segmenting the to-be-displayed image frame into a plurality of segmentation regions using a target segmentation method, and determining a correspondence between the plurality of segmentation regions and the at least one metadata information unit; For the i-th pixel in the image frame to be displayed, the following operations are performed: determining, based on a plurality of pixels included in a preset region selected for the i-th pixel, at least one associated segmentation region to which the plurality of pixels belong, and determining, based on the correspondence relationship, associated metadata information units corresponding to each of the associated segmentation regions; an operation of obtaining tone mapping sub-values ​​corresponding to each of the associated segmented regions based on the associated metadata information units, and allocating corresponding weighting factors to each of the associated segmented regions, wherein the weighting factors are adjusted using a first weighted intensity value based on a pixel number distribution status of the image frame to be displayed, or using a second weighted intensity value based on a preset feature value change status of a preset feature value, the preset feature value being one of a luminance feature value or a color feature value, of the image frame to be displayed; obtaining a tone mapping value for the i-th pixel based on the tone mapping sub-values ​​and the weighting coefficients corresponding to each of the associated segmented regions, and performing tone mapping on the i-th pixel based on the tone mapping value; and a step of executing Including, A method wherein i is any positive integer from 1 to N, and N is the number of pixels contained in the image frame to be displayed.

2. The operation of allocating a corresponding weighting factor to each of the associated segmented regions is carried out in the following manner: a method for performing the following operation with respect to a first associated segmentation region: separately determining the number of pixels belonging to the preset region and the total number of pixels corresponding to the preset region in each of the associated segmentation regions; and allocating a corresponding first weighting factor to the first associated segmentation region based on the number of pixels belonging to the preset region and the total number of pixels in the first associated segmentation region, wherein the first associated segmentation region is any one of the at least one associated segmentation region; and / or separately determining preset feature values ​​corresponding to the preset regions in each of the associated segmentation regions; and performing the following operation with respect to a second associated segmentation region: assigning a corresponding second weighting factor to the second associated segmentation region based on a feature difference between a preset feature value, the preset feature value being one of a luminance feature value or a color feature value, corresponding to the preset region in the second associated segmentation region and a preset feature value of the i-th pixel, wherein the second associated segmentation region is any one of the at least one associated segmentation regions.

10. The method of claim 1, comprising one or a combination of:

3. The method of claim 2 , wherein the metadata further includes one or a combination of the first weighted intensity value or the second weighted intensity value.

4. The operation of obtaining tone mapping sub-values ​​corresponding to each of the associated segmented regions based on the associated metadata information units comprises: an operation of obtaining a tone mapping sub-value corresponding to the i-th pixel in the associated segmented region based on a tone mapping curve, wherein the associated metadata information unit includes parameters of the tone mapping curve.

2. The method of claim 1, comprising:

5. 4. The method of claim 1, wherein the preset region selected for the ith pixel is a region of a preset shape and a preset size selected using the ith pixel as a center, a region determined based on preset shape and preset size information included in the metadata using the ith pixel as a center, or a region determined based on preset shape and preset size information included in the metadata using the ith pixel as a center.

6. The step of determining a correspondence between the plurality of segmented regions and the at least one metadata information unit comprises: For any of the segmented regions, selecting a pixel at a preset position from the segmented region and obtaining coordinate information of the pixel at the preset position, and determining a metadata information unit corresponding to the segmented region based on the coordinate information, and establishing a correspondence between the segmented region and the corresponding metadata information unit; or For any of the metadata information units, determining at least one segmented area corresponding to the one or more pieces of coordinate information based on the one or more pieces of coordinate information included in the metadata information unit, and establishing a correspondence between the metadata information unit and the corresponding at least one segmented area.

2. The method of claim 1, comprising:

7. The step of determining a correspondence between the plurality of segmented regions and the at least one metadata information unit comprises: traversing the plurality of segmented regions based on a target scan sequence and a segmented region identifier to determine a metadata information unit corresponding to each of the segmented regions, wherein the metadata information unit includes the corresponding segmented region identifier, and the target scan sequence is a preset scan sequence or a scan sequence indicated in the metadata.

2. The method of claim 1, comprising:

8. 8. The method of claim 1, wherein there is a one-to-one or many-to-one correspondence between the plurality of segmented regions and the at least one metadata information unit.

9. 8. The method of claim 1, wherein the target segmentation method is a preset segmentation method or a segmentation method indicated in the metadata.

10. 1. A tone mapping method applied to a capture end device, said method comprising: capturing image data to obtain a frame of data to be displayed; Segmenting the data frame to be displayed into a plurality of segmentation regions using a target segmentation method, and determining at least one corresponding metadata information unit for the plurality of segmentation regions; transmitting the data frame to be displayed and metadata to a display end device, the metadata including the at least one metadata information unit; Including, The i-th pixel of the data frame to be displayed is The metadata information unit is determined based on a plurality of pixels included in a preset region selected for the i-th pixel, and at least one associated segmentation region to which the plurality of pixels belong is determined, and the metadata information unit is determined to be correspondingly associated with each of the associated segmentation regions; The associated metadata information units corresponding to each of the associated segmented regions are used to obtain tone mapping sub-values ​​corresponding to each of the associated segmented regions, and allocate corresponding weighting factors to each of the associated segmented regions, wherein the weighting factors are adjusted using a first weighted intensity value based on a pixel number distribution status of the image frame to be displayed, or using a second weighted intensity value based on a preset feature value change status of a preset feature value, the preset feature value being one of a luminance feature value or a color feature value, of the image frame to be displayed; a tone mapping value for the i-th pixel is obtained based on the tone mapping sub-values ​​and the weighting coefficients corresponding to each of the associated segmented regions, and tone mapping is performed on the i-th pixel based on the tone mapping value, thereby configured to perform tone mapping; A method wherein i is any positive integer from 1 to N, and N is the number of pixels contained in the image frame to be displayed.

11. The method comprises: determining a first weighted intensity value based on the number of pixels included in each of the plurality of segmented regions; and / or determining second weighted intensity values ​​based on preset feature values, the preset feature values ​​being one of brightness feature values ​​or color feature values, corresponding to the plurality of segmented regions respectively; and Associating the first weighted intensity value and / or the second weighted intensity value with the metadata.

11. The method of claim 10, further comprising:

12. The method of claim 10 , wherein the metadata information unit includes parameters of a tone mapping curve corresponding to a segmented region.

13. the metadata further includes preset size information, the preset size information indicating a preset size of a preset shape to be used when a preset region is selected for a pixel; or The method of claim 10 , wherein the metadata further comprises preset shape and preset size information.

14. The step of determining at least one corresponding metadata information unit for the plurality of segmented regions comprises: selecting a plurality of pixels at preset positions from the segmented region and obtaining coordinate information of the plurality of pixels at the preset positions; associating said coordinate information with said metadata information unit; 11. The method of claim 10, comprising:

15. The metadata information unit further includes a segmentation region identifier and a scan sequence corresponding to the metadata information unit; The method of claim 10 , wherein the corresponding segmented region identifiers and the scanning sequence are used by the display end device to determine a correspondence between the plurality of segmented regions and the at least one metadata information unit.

16. 16. The method of claim 10, wherein there is a one-to-one or many-to-one correspondence between the plurality of segmented regions and the at least one metadata information unit.

17. The method comprises: associating the target segmentation method with the metadata to indicate to the display end device to segment the to-be-displayed data frame using the target segmentation method.

11. The method of claim 10, further comprising:

18. A display end device comprising a non-volatile memory and a processor coupled to each other, wherein the processor invokes program code stored in the memory to perform the method of any one of claims 1 to 3, 4, 6 or 7.

19. 16. A capture end device comprising a non-volatile memory and a processor coupled to each other, wherein the processor invokes program code stored in the memory to perform the method of any one of claims 10 to 12, 14, or 15.

20. 1. A tone mapping system comprising a display end device and a capture end device, The display end device is 1. A tone mapping method applied to a display end device, said method comprising: obtaining an image frame to be displayed and metadata, the metadata including at least one metadata information unit; segmenting the to-be-displayed image frame into a plurality of segmentation regions using a target segmentation method, and determining a correspondence between the plurality of segmentation regions and the at least one metadata information unit; For the i-th pixel in the image frame to be displayed, the following operations are performed: determining, based on a plurality of pixels included in a preset region selected for the i-th pixel, at least one associated segmentation region to which the plurality of pixels belong, and determining, based on the correspondence relationship, associated metadata information units corresponding to each of the associated segmentation regions; an operation of obtaining tone mapping sub-values ​​corresponding to each of the associated segmented regions based on the associated metadata information units, and allocating corresponding weighting factors to each of the associated segmented regions, wherein the weighting factors are adjusted using a first weighted intensity value based on a pixel number distribution status of the image frame to be displayed, or using a second weighted intensity value based on a preset feature value change status of a preset feature value, the preset feature value being one of a luminance feature value or a color feature value, of the image frame to be displayed; obtaining a tone mapping value for the i-th pixel based on the tone mapping sub-values ​​and the weighting coefficients corresponding to each of the associated segmented regions, and performing tone mapping on the i-th pixel based on the tone mapping value; and a step of executing Including, wherein i is any positive integer between 1 and N, and N is the number of pixels contained in the image frame to be displayed; The capture end device capturing image data to obtain a frame of data to be displayed; Segmenting the data frame to be displayed into a plurality of segmentation regions using a target segmentation method, and determining at least one corresponding metadata information unit for the plurality of segmentation regions; transmitting the data frame to be displayed and metadata to a display end device, the metadata including the at least one metadata information unit; Including, The i-th pixel of the data frame to be displayed is The metadata information unit is determined based on a plurality of pixels included in a preset region selected for the i-th pixel, and at least one associated segmentation region to which the plurality of pixels belong is determined, and the metadata information unit is determined to be correspondingly associated with each of the associated segmentation regions; The associated metadata information units corresponding to each of the associated segmented regions are used to obtain tone mapping sub-values ​​corresponding to each of the associated segmented regions, and allocate corresponding weighting factors to each of the associated segmented regions, wherein the weighting factors are adjusted using a first weighted intensity value based on a pixel number distribution status of the image frame to be displayed, or using a second weighted intensity value based on a preset feature value change status of a preset feature value, the preset feature value being one of a luminance feature value or a color feature value, of the image frame to be displayed; a tone mapping value for the i-th pixel is obtained based on the tone mapping sub-values ​​and the weighting coefficients corresponding to each of the associated segmented regions, and tone mapping is performed on the i-th pixel based on the tone mapping value, thereby configured to perform tone mapping; wherein i is any positive integer between 1 and N, and N is the number of pixels contained in the image frame to be displayed. Tone mapping system.

21. 10. A computer-readable storage medium storing a computer program that, when executed on a processor, performs the method of any one of claims 1 to 3, 4, 6, or 7.

22. 16. A computer-readable storage medium storing a computer program that, when executed on a processor, performs the method of any one of claims 10 to 12, 14, or 15.

Images