Color mapping systems and methods, display-side devices, data acquisition-side devices, and computer-readable storage media.

Abstract

The invention proposes a system and method for color shade mapping, a display-side device, an acquisition-side device, and a computer-readable storage medium to provide a technical solution that can improve the efficiency of color shade mapping. This method includes: acquiring the image frame to be displayed and its metadata, wherein the metadata includes at least one metadata information unit; segmenting the image frame to be displayed into segment regions in a targeted segmentation manner, and determining the correlation between the segment regions and at least one metadata information unit; and for pixel i: determining, for the pixels contained within the pre-established region, at least one related segment region to which the pixels belong, and determining, based on the correlation, the corresponding metadata information unit for each related segment region;Acquire, based on the relevant metadata unit, the color shade mapping subvalue corresponding to the relevant segment region, and allocate the corresponding weight coefficient to the relevant segment region; acquire the color shade mapping value based on the color shade mapping subvalue corresponding to the relevant segment region and the weight coefficient; and perform color shade mapping based on the color shade mapping value.

Description

A tone mapping method, device and system

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the Intellectual Property Office of the People's Republic of China on September 15, 2021, with application number 202111080645.7 and application name "A tone mapping method, device and system", the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of display technology, and in particular to a tone mapping method, device, and system. Background Art

[0004] With the development of capture devices, it is now possible to capture HDR videos or images based on high dynamic range (HDR) video technology. HDR videos or images expand the brightness range of the image that can be displayed, thus recording a wider range of brightness information, thereby showing more details in the bright and dark areas of the image.

[0005] However, the display capabilities of existing display devices are usually limited, and they cannot properly display the brightness range information recorded by the HDR video or image captured by the acquisition device. For example, the maximum brightness that the acquisition device can capture is 10,000 candelas / square meter (cd / m 2 , brightness unit) pixels, while the display device can only display a maximum brightness of 4000cd / m 2 If the pixel is 4000cd / m, the brightness of the HDR video or image collected by the acquisition end device is 4000cd / m 2 ~10000cd / m 2 The pixels may not be displayed well. The related art provides a method of performing tone mapping (TM) on the image to be displayed, so that the high dynamic range image can be tone mapped on a display device with low dynamic range display capability.

[0006] Traditional tone mapping methods mainly use tone mapping curves or block-based local tone mapping, but they suffer from poor tone mapping effects. Therefore, it is worth studying how to improve tone mapping effects.

[0007] Summary of the Invention

[0008] The present application provides a tone mapping method, device and system to provide a technical solution that can improve the tone mapping effect.

[0009] In a first aspect, an embodiment of the present application provides a tone mapping method, which is applied to a display terminal device. The method includes: obtaining an image frame to be displayed and metadata, wherein the metadata includes at least one metadata information unit; dividing the image frame to be displayed into multiple segmented areas using a target segmentation method, and determining the correspondence between the multiple segmented areas and the at least one metadata information unit. For the i-th pixel in the image frame to be displayed, the following steps are performed: determining at least one related segmented area to which the multiple pixels belong based on multiple pixels contained in a preset area selected by the i-th pixel, and determining the related metadata information unit corresponding to each of the related segmented areas based on the correspondence; obtaining a sub-tone mapping value corresponding to each of the related segmented areas based on the related metadata information unit, and assigning a corresponding weight factor to each of the related segmented areas; obtaining a tone mapping value of the i-th pixel based on the sub-tone mapping value and the weight factor corresponding to each of the related segmented areas; and tone mapping the i-th pixel based on the tone mapping value. Wherein, i is any positive integer from 1 to N, and N is the number of pixels contained in the image frame to be displayed.

[0010] This method provides a technical solution for performing localized refined tone mapping on image frames to be displayed. By implementing localized tone mapping based on preset regions for pixels contained in the image frames to be displayed, the display device can improve the display quality of the tone-mapped image frames to be displayed.

[0011] In a possible design, allocating a corresponding weight factor to each of the related segmented regions includes one or a combination of the following methods:

[0012] Method 1, respectively determining the number of pixels in each of the related segmented regions belonging to the preset region and the total number of pixels corresponding to the preset region; for a first related segmented region, performing: assigning a corresponding first weight factor to the first related segmented region based on the number of pixels in the first related segmented region belonging to the preset region and the total number of pixels, the first related segmented region being any one of the at least one related segmented region;

[0013] Method 2, respectively determining the preset characteristic value corresponding to the preset area in each of the related segmentation areas; for the second related segmentation area, executing: according to the characteristic difference between the preset characteristic value corresponding to the preset area in the second related segmentation area and the preset characteristic value of the i-th pixel, assigning a corresponding second weight factor to the second related segmentation area, where the second related segmentation area is any one of the at least one related segmentation area.

[0014] In this design, weight factors can be assigned to different related segmented regions based on their local characteristics, such as the number of pixels or preset feature values, within the preset region corresponding to the pixel. This assignment of weight factors enables refined tone mapping of the pixels contained in the image frame to be displayed, enabling tone mapping based on the image characteristics of the image frame to be displayed, thereby improving the display quality of the tone-mapped image frame on the display device.

[0015] In one possible design, the metadata further includes one or a combination of the following information: a first weighted intensity value and a second weighted intensity value. The first weighted intensity value is used to adjust the first weight factor based on the distribution of the number of pixels in the image frame to be displayed; and the second weighted intensity value is used to adjust the second weight factor based on changes in preset characteristic values ​​of the image frame to be displayed.

[0016] In this design, the weighting factors assigned to different segmented regions can be adjusted using weighted intensity values ​​based on the global features of the image frame to be displayed. For example, when the image features of the image frames to be displayed are relatively similar, a larger weighting factor can be assigned; when the image features of the image frames to be displayed are relatively different, a smaller weighting factor can be assigned. This can reduce the boundary effects that may occur during tone mapping and improve the display quality of the tone-mapped image frames on the display device.

[0017] In one possible design, the preset feature value is one of the following: a brightness feature value and a color feature value. The number of preset feature values ​​is at least one. In this design, the preset feature value can be a brightness feature value or a color feature value. In actual implementation, the preset feature value can be determined based on the image frame to be displayed. For example, if the brightness feature of the image frame to be displayed is more important, the brightness feature value can be selected; if the color feature of the image frame to be displayed is more prominent, the color feature value can be selected. Furthermore, this design does not limit the number of preset feature values, thereby improving the tone mapping effect of the image frame to be displayed.

[0018] In one possible design, the sub-tone mapping value corresponding to each of the related segmented areas is obtained based on the related metadata information unit, including: the related metadata information unit contains parameters of the tone mapping curve, and the sub-tone mapping value corresponding to the i-th pixel in the related segmented area is obtained based on the tone mapping curve.

[0019] In this design, each related segmented area can correspond to a tone mapping curve, so that tone mapping of each related segmented area can be achieved to obtain multiple sub-tone mapping values, which can improve the accuracy of the sub-tone mapping values ​​to obtain a more accurate tone mapping effect.

[0020] In one possible design, the preset area selected based on the i-th pixel is: an area of ​​a preset shape of a preset size selected with the i-th pixel as the center; or an area centered on the i-th pixel and determined based on a preset shape and preset size information contained in the metadata; or an area centered on the i-th pixel and determined based on the preset shape and preset size information contained in the metadata.

[0021] In this design, by selecting a preset area with the pixel to be processed (the i-th pixel) as the center, and then performing local fine-tuning tone mapping on the pixel to be processed based on multiple pixels contained in the preset area, the display effect of the image frame to be displayed after tone mapping on the display end device can be improved.

[0022] In one possible design, determining the correspondence between the multiple segmented regions and the at least one metadata information unit includes: for any of the segmented regions, selecting a pixel at a preset position from any of the segmented regions and obtaining coordinate information of the pixel at the preset position; determining the metadata information unit corresponding to any of the segmented regions based on the coordinate information; and establishing a correspondence between any of the segmented regions and the corresponding metadata information unit. Alternatively, for any of the metadata information units, determining, based on one or more coordinate information contained in the metadata information unit, at least one segmented region corresponding to the one or more coordinate information; and establishing a correspondence between any of the metadata information units and the corresponding at least one segmented region.

[0023] In this design, the coordinate information of the preset position is used as the index, which can more accurately and simply determine the correspondence between the segmented area and the metadata information unit, so that the sub-tone mapping value of each relevant segmented area can be calculated based on the relevant metadata information unit corresponding to the relevant segmented area of ​​the pixel to be processed.

[0024] In one possible design, determining the correspondence between the multiple segmented regions and the at least one metadata information unit includes: the metadata information unit contains a corresponding segmented region identifier, traversing the multiple segmented regions according to a target scanning order and the segmented region identifier, and determining the metadata information unit corresponding to each of the segmented regions; wherein the target scanning order is a preset scanning order, or a scanning order indicated in the metadata.

[0025] In this design, the target scanning order can also be used to more accurately and simply determine the correspondence between the segmented areas and the metadata information units, so that the sub-tone mapping values ​​of each relevant segmented area can be calculated separately based on the relevant metadata information units corresponding to the relevant segmented areas of the pixels to be processed.

[0026] In one possible design, the multiple segmented regions and the at least one metadata information unit have a one-to-one correspondence, or a many-to-one correspondence. In this design, the segmented regions and metadata information units can have not only a one-to-one correspondence, but also a many-to-one correspondence. This many-to-one correspondence can reduce the amount of metadata that needs to be transmitted between the acquisition and display devices, thereby improving metadata transmission efficiency.

[0027] In one possible design, the target segmentation method is a preset segmentation method or a segmentation method indicated in the metadata. In this design, the target segmentation method can be pre-stored by the acquisition end device and the display end device, respectively, or the segmentation method used by the display end device can be indicated to the acquisition end device, thereby ensuring the accuracy of tone mapping between the acquisition end device and the display end device.

[0028] In a second aspect, embodiments of the present application provide a method for generating metadata, which is applied to an acquisition terminal device. The method comprises: acquiring image data to obtain a data frame to be displayed; segmenting the data frame to be displayed into multiple segmented regions using a target segmentation method, and determining at least one metadata information unit corresponding to each of the multiple segmented regions; and transmitting the data frame to be displayed and the metadata to a display terminal device; the metadata including the at least one metadata information unit.

[0029] In one possible design, the method further includes: determining a first weighted intensity value based on the number of pixels respectively contained in the multiple segmented areas; and / or determining a second weighted intensity value based on preset feature values ​​respectively corresponding to the multiple segmented areas; and associating the first weighted intensity value and / or the second weighted intensity value with the metadata.

[0030] In a possible design, the preset characteristic value is one of the following characteristic values: a brightness characteristic value and a color characteristic value, wherein the number of the preset characteristic values ​​is at least one.

[0031] In one possible design, the metadata information unit includes parameters of a tone mapping curve corresponding to the segmented area.

[0032] In one possible design, the metadata also includes: preset size information; the preset size information is used to indicate the preset size of the preset shape used when selecting a preset area for pixels; or, the metadata also includes: preset shape and preset size information.

[0033] In one possible design, determining at least one metadata information unit corresponding to the multiple segmented areas includes: selecting pixels at multiple preset positions from the segmented areas, obtaining coordinate information of the pixels at the multiple preset positions; and associating the coordinate information with the metadata information unit.

[0034] In one possible design, the metadata information unit also includes: a segmented area identifier and a scanning order corresponding to the metadata information unit; wherein the corresponding segmented area identifier and the scanning order are used by the display end device to determine the correspondence between the multiple segmented areas and the at least one metadata information unit.

[0035] In a possible design, the multiple segmented regions and the at least one metadata information unit have a one-to-one correspondence or a many-to-one correspondence.

[0036] In one possible design, the method further includes associating the target segmentation method with the metadata to instruct the display terminal device to segment the data frame to be displayed using the target segmentation method.

[0037] In a third aspect, an embodiment of the present application provides a tone mapping processing device, which includes a module for executing the method described in the first aspect or any possible implementation of the first aspect.

[0038] Exemplarily, the apparatus may include: an acquisition module, a processing module, and a tone mapping module. The acquisition module is configured to acquire an image frame to be displayed and metadata, the metadata including at least one metadata information unit. The processing module is configured to segment the image frame to be displayed into multiple segmented regions using a target segmentation method, and determine a correspondence between the multiple segmented regions and the at least one metadata information unit. The tone mapping module is configured to, for an i-th pixel in the image frame to be displayed, perform the following operations: determining, based on multiple pixels contained in a preset region selected by the i-th pixel, at least one relevant segmented region to which the multiple pixels belong, and determining, based on the correspondence, relevant metadata information units corresponding to each relevant segmented region; obtaining, based on the relevant metadata information units, sub-tone mapping values ​​corresponding to each relevant segmented region, and assigning a corresponding weight factor to each relevant segmented region; obtaining a tone mapping value for the i-th pixel based on the sub-tone mapping values ​​and the weight factors corresponding to each relevant segmented region; and performing tone mapping on the i-th pixel based on the tone mapping value. Where i is any positive integer from 1 to N, and N is the number of pixels contained in the image frame to be displayed.

[0039] In one possible design, the tone mapping module, when used to assign a corresponding weight factor to each of the related segmented areas, is specifically used to: respectively determine the number of pixels in each of the related segmented areas belonging to the preset area, and the total number of pixels corresponding to the preset area; for a first related segmented area, execute: according to the number of pixels in the first related segmented area belonging to the preset area and the total number of pixels, assign a corresponding first weight factor to the first related segmented area, where the first related segmented area is any one of the at least one related segmented area; and / or, respectively determine the preset feature value corresponding to the preset area in each of the related segmented areas; for a second related segmented area, execute: according to the feature difference between the preset feature value corresponding to the preset area in the second related segmented area and the preset feature value of the i-th pixel, assign a corresponding second weight factor to the second related segmented area, where the second related segmented area is any one of the at least one related segmented area.

[0040] In one possible design, the metadata also includes one or a combination of the following information: a first weighted intensity value and a second weighted intensity value; wherein the first weighted intensity value is used to adjust the first weight factor according to the distribution of the number of pixels of the image frame to be displayed; and the second weighted intensity value is used to adjust the second weight factor according to the change of the preset characteristic value of the image frame to be displayed.

[0041] In a possible design, the preset characteristic value is one of the following characteristic values: a brightness characteristic value and a color characteristic value, wherein the number of the preset characteristic values ​​is at least one.

[0042] In one possible design, the tone mapping module is used to obtain the sub-tone mapping value corresponding to each of the related segmented areas based on the related metadata information unit, specifically, the related metadata information unit contains parameters of the tone mapping curve, and based on the tone mapping curve, the sub-tone mapping value corresponding to the i-th pixel in the related segmented area is obtained.

[0043] In one possible design, the preset area selected based on the i-th pixel is: an area of ​​a preset shape of a preset size selected with the i-th pixel as the center; or an area centered on the i-th pixel and determined based on a preset shape and preset size information contained in the metadata; or an area centered on the i-th pixel and determined based on the preset shape and preset size information contained in the metadata.

[0044] In one possible design, the processing module is used to determine the correspondence between the multiple segmented areas and the at least one metadata information unit, and is specifically used to: for any of the segmented areas, select pixels at a preset position from any of the segmented areas, and obtain coordinate information of the pixels at the preset position; determine the metadata information unit corresponding to any of the segmented areas based on the coordinate information; establish a correspondence between any of the segmented areas and the corresponding metadata information unit; or, for any of the metadata information units, determine at least one segmented area corresponding to the one or more coordinate information contained in the metadata information unit; and establish a correspondence between any of the metadata information units and the corresponding at least one segmented area.

[0045] In one possible design, the processing module is used to determine the correspondence between the multiple segmented regions and the at least one metadata information unit, including: the metadata information unit contains a corresponding segmented region identifier, and the multiple segmented regions are traversed according to the target scanning order and the segmented region identifier to determine the metadata information unit corresponding to each of the segmented regions; wherein the target scanning order is a preset scanning order, or a scanning order indicated in the metadata.

[0046] In a possible design, the multiple segmented regions and the at least one metadata information unit have a one-to-one correspondence or a many-to-one correspondence.

[0047] In one possible design, the target segmentation method is a preset segmentation method or a segmentation method indicated in the metadata.

[0048] In a fourth aspect, an embodiment of the present application provides a tone mapping device, which includes a module for executing the method described in the second aspect or any possible implementation of the second aspect.

[0049] Exemplarily, the apparatus may include: an acquisition module, a processing module, and a transceiver module. The acquisition module is configured to acquire image data to obtain a data frame to be displayed; the processing module is configured to segment the data frame to be displayed into multiple segmented regions using a target segmentation method and determine at least one metadata information unit corresponding to each of the multiple segmented regions; and the transceiver module is configured to transmit the data frame to be displayed and metadata to a display device; the metadata includes the at least one metadata information unit.

[0050] In one possible design, the processing module is further used to determine a first weighted intensity value based on the number of pixels respectively contained in the multiple segmented areas; and / or to determine a second weighted intensity value based on preset feature values ​​respectively corresponding to the multiple segmented areas; and to associate the first weighted intensity value and / or the second weighted intensity value with the metadata.

[0051] In a possible design, the preset characteristic value is one of the following characteristic values: a brightness characteristic value and a color characteristic value, wherein the number of the preset characteristic values ​​is at least one.

[0052] In one possible design, the metadata information unit includes parameters of a tone mapping curve corresponding to the segmented area.

[0053] In one possible design, the metadata also includes: preset size information; the preset size information is used to indicate the preset size of the preset shape used when selecting a preset area for pixels; or, the metadata also includes: preset shape and preset size information.

[0054] In one possible design, the processing module is used to determine at least one metadata information unit corresponding to the multiple segmented areas, specifically to select pixels at multiple preset positions from the segmented areas, obtain coordinate information of the pixels at the multiple preset positions; and associate the coordinate information with the metadata information unit.

[0055] In one possible design, the metadata information unit also includes: a segmented area identifier and a scanning order corresponding to the metadata information unit; wherein the corresponding segmented area identifier and the scanning order are used by the display end device to determine the correspondence between the multiple segmented areas and the at least one metadata information unit.

[0056] In a possible design, the multiple segmented regions and the at least one metadata information unit have a one-to-one correspondence or a many-to-one correspondence.

[0057] In a possible design, the processing module is further used to associate the target segmentation method with the metadata to instruct the display terminal device to segment the data frame to be displayed using the target segmentation method.

[0058] In a fifth aspect, an embodiment of the present application provides a display terminal device, comprising: a non-volatile memory and a processor coupled to each other, wherein the processor calls the program code stored in the memory to execute the method described in the first aspect or any possible implementation of the first aspect.

[0059] In a sixth aspect, an embodiment of the present application provides an acquisition terminal device, comprising: a non-volatile memory and a processor coupled to each other, wherein the processor calls the program code stored in the memory to execute the method described in the second aspect or any possible implementation of the second aspect.

[0060] In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium, including a program or instruction. When the program or instruction is run on a computer, the method in the first aspect or any possible implementation of the first aspect is executed, or the method in the second aspect or any possible implementation of the second aspect is executed.

[0061] It should be understood that the technical effects that can be achieved in the second to seventh aspects can be specifically described with reference to the technical effects brought about by the above-mentioned first aspect or any possible implementation method of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 is a schematic diagram of the PQ photoelectric transfer function relationship;

[0063] FIG2 is a schematic diagram of the HLG photoelectric transfer function relationship;

[0064] FIG3 is a schematic diagram of the SLF photoelectric transfer function relationship;

[0065] FIG4 is a schematic diagram of a possible system architecture applicable to an embodiment of the present application;

[0066] FIG5 is a flow chart of a tone mapping method provided in an embodiment of the present application;

[0067] FIG6 is a schematic diagram showing the segmentation method A provided for the implementation of this application;

[0068] FIG7 is a schematic diagram showing a segmentation method B provided for the implementation of this application;

[0069] FIG8 is a schematic diagram showing the segmentation method C provided in the present application;

[0070] FIG9 is a schematic diagram showing a scanning sequence provided in an embodiment of the present application;

[0071] FIG10 is a schematic diagram showing a method for determining relevant segmented regions according to an embodiment of the present application;

[0072] FIG11 shows another schematic diagram of determining related segmented areas according to an embodiment of the present application;

[0073] FIG12 is a schematic diagram showing a brightness characteristic value provided in an embodiment of the present application;

[0074] FIG13 is another schematic diagram of a flow chart of a tone mapping method provided in an embodiment of the present application;

[0075] FIG14 is a schematic structural diagram of a tone mapping device provided in an embodiment of the present application;

[0076] FIG15 is another structural diagram of a tone mapping device provided in an embodiment of the present application;

[0077] FIG16 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0078] The term "at least one" in the embodiments of the present application refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where any of a, b, c, ab, ac, bc, or abc can include a single a, a single b, a single c, or multiple a, multiple b, or multiple c.

[0079] Furthermore, unless otherwise indicated, ordinal numbers such as "first" and "second" in the embodiments of this application are used to distinguish between multiple objects and are not used to define the order, timing, priority, or importance of multiple objects. For example, the first priority criterion and the second priority criterion are only used to distinguish different criteria and do not indicate differences in the content, priority, or importance of the two criteria.

[0080] In addition, the terms "including" and "having" in the embodiments, claims, and drawings of this application are not exclusive. For example, a process, method, system, product, or device that includes a series of steps or modules / units is not limited to the listed steps or modules and may also include steps or modules / units that are not listed.

[0081] In the field of digital image display, dynamic range can be expressed as the ratio between the maximum grayscale value and the minimum grayscale value within the image display range. In most current color digital images, each channel of R, G, and B is stored in an 8-bit byte, that is, the range of each channel is 0 to 255 grayscale levels, where 0 to 255 is the dynamic range of the image. However, in the real world, the dynamic range of the same scene is usually around 10 -3 to 10 6 The range of dynamic range is called high dynamic range (HDR). Compared to high dynamic range, the dynamic range of ordinary images is called low dynamic range (LDR). The dynamic range that LDR can display is usually between 1 and 100. From this, we can understand that the imaging process of a digital camera is actually a mapping from the high dynamic range of the real world to the low dynamic range of the photograph.

[0082] The larger the dynamic range of an image, the more scene details the image displays, the richer the brightness levels, and the more realistic the visual effect.

[0083] The process of optical digital imaging (for example, the imaging process of a digital camera, the video playback process) is to convert the light radiation of a real scene into an electrical signal through an image sensor and save it in the form of a digital image. The purpose of image display is to reproduce the real scene described by a digital image through a display device. The ultimate goal of both is to enable the user to obtain the same visual perception as if he were directly observing the real scene. The brightness level in the real scene that can be displayed by light radiation (light signal) is almost linear, so the light signal is also called a linear signal. However, in the process of converting light signals into electrical signals in optical digital imaging, not every light signal corresponds to an electrical signal, and the converted electrical signal is usually nonlinear. Therefore, the electrical signal is also called a nonlinear signal. The curve that converts light signals into electrical signals is the optical electro transfer function (OETF). The OETF involved in the embodiments of the present application may include but is not limited to: perceptual quantizer (PQ) OETF, hybrid log-gamma (HLG) OETF, scene luminance fidelity (SLF) OETF, etc.

[0084] The PQ photoelectric transfer function (PQPTF) is a perceptually quantized photoelectric transfer function proposed based on the human eye's brightness perception model. The PQPTF represents the conversion relationship between the linear signal value of an image pixel and the nonlinear signal value in the PQ domain. Figure 1 shows a schematic diagram of the PQPTF relationship. The PQPTF can be expressed as formula (1-1):

[0085]

[0086] The parameters in formula (1-1) are calculated as follows:

[0087]

[0088] in,

[0089] L represents the linear signal value, and its value is normalized to [0, 1].

[0090] L' represents the nonlinear signal value, and its value range is [0, 1].

[0091] m1 is the PQ photoelectric transfer coefficient.

[0092] m2 is the PQ photoelectric transfer coefficient.

[0093] c1 is the PQ photoelectric transfer coefficient.

[0094] c2 is the PQ photoelectric transfer coefficient.

[0095] c3 is the PQ photoelectric transfer coefficient.

[0096] The HLG photoelectric transfer function is an improvement on the traditional Gamma curve. The HLG photoelectric transfer function applies the traditional Gamma curve in the low range and supplements the log curve in the high range. See Figure 2 for a schematic diagram of the HLG photoelectric transfer function relationship. The HLG photoelectric transfer function represents the conversion relationship between the linear signal value of the image pixel and the nonlinear signal value in the HLG domain. The HLG photoelectric transfer function can be expressed as formula (1-2):

[0097]

[0098] Where L represents the linear signal value, and its value range is [0, 12]. L' represents the nonlinear signal value, and its value range is [0, 1]. a = 0.17883277, represents the HLG photoelectric transfer coefficient. b = 0.28466892, represents the HLG photoelectric transfer coefficient. c = 0.55991073, represents the HLG photoelectric transfer coefficient.

[0099] The SLF photoelectric transfer function (SLF) is an optimal curve obtained based on the HDR scene brightness distribution, while satisfying the optical characteristics of the human eye. See Figure 3, which shows a schematic diagram of the SLF photoelectric transfer function.

[0100] The SLF photoelectric transfer curve represents the conversion relationship between the linear signal value of an image pixel and the nonlinear signal value in the SLF domain. The conversion relationship between the linear signal value of an image pixel and the nonlinear signal value in the SLF domain is shown in formula (1-3):

[0101]

[0102] The SLF photoelectric transfer function can be expressed as formula (1-4):

[0103]

[0104] in:

[0105] L represents the linear signal value, normalized to [0, 1]. L' represents the nonlinear signal value, ranging from [0, 1]. p = 2.3 represents the SLF photoelectric transfer coefficient. m = 0.14 represents the SLF photoelectric transfer coefficient. a = 1.12762 represents the SLF photoelectric transfer coefficient. b = -0.12762 represents the SLF photoelectric transfer coefficient.

[0106] At present, the acquisition end device can acquire HDR videos or images with a larger dynamic range (the following embodiments use images as examples for introduction, and it can be understood that videos can be represented as multiple frames of images), but the display capability of the display end device is limited and cannot be well matched with the dynamic range acquired by the acquisition end device. The related art provides a processing method that can be used to perform tone mapping (TM) on the image to be displayed, so that the matching of images under different dynamic ranges can be achieved. Exemplarily, tone mapping can be divided into mapping from high dynamic range to low dynamic range, or mapping from low dynamic range to high dynamic range. For example, the HDR image acquired by the acquisition end device includes a light signal of 4000 nits (nit, light signal unit), and the HDR display capability of the display end device (such as a TV, a mobile phone) is only 500 nits. In this case, the 4000 nit signal needs to be mapped to a 500 nit device, that is, a tone mapping process from high to low. For example, an HDR image includes a 100-nit SDR signal, but the target display device has a display capability of only 2000 nits. In this case, the 100-nit signal needs to be mapped to a 2000-nit device, which is a low-to-high tone mapping process.

[0107] Tone mapping is typically used to match high-level HDR images captured by an acquisition device with low-level HDR images or SDR images displayed by a display device. Referring to FIG4 , a schematic diagram of a possible system architecture applicable to an embodiment of the present application is shown. The system architecture of an embodiment of the present application may include an acquisition device 401, a display device 402, and a transmission link 403. The display device 402 may be further divided into an HDR display device 402a and an SDR display device 402b.

[0108] Among them, the acquisition end device 401 is used to acquire or produce HDR video or images. The exemplary acquisition end device 401 can be a video (or image) acquisition device, or it can be a video (or image) production device. In a possible example, the acquisition end device 401 can also generate metadata based on the HDR video or image. Metadata is used to record key information of the captured scene or image in the frame. In an embodiment of the present application, the metadata may include parameters of the tone mapping curve, dynamic metadata (dynamic metadata) and static metadata (static metadata), etc. Among them, dynamic metadata can be understood as data associated with each frame of the image, which can change with different pictures, such as the average, maximum, minimum, etc. of the pixel brightness in the scene. Static metadata can be understood as data associated with an image sequence, which remains unchanged within the image sequence.

[0109] The acquisition end device 401 can transmit the HDR video or image, that is, metadata, to the display end device 402 via the transmission link 403. Specifically, the HDR video or image and metadata can be transmitted in the form of a single data packet, or separately in two data packets, which is not specifically limited in this embodiment of the application.

[0110] The display device 402 can be used to receive metadata and HDR videos or images, and determine the tone mapping curve according to the parameters of the tone mapping curve contained in the metadata, and perform tone mapping on the HDR image according to the tone mapping curve to convert it into display content adapted for the display device 402. Among them, the display device 402 can be divided into an HDR display device 402a and an SDR display device 402b. It should be noted that the range of the HDR of the display device 402 and the HDR collected by the acquisition device 401 may be different. Usually, the HDR of the display device 402 is lower than the range of the HDR of the acquisition device 401. For example, the range of the HDR of the acquisition device 401 is 10 -3 to 10 4 (This embodiment of the present application may also be referred to as "high-level HDR"), and the HDR range of the display terminal device 402 is 10 -1 to 10 3Range (also referred to as "low-level HDR" in the embodiments of this application). It should also be understood that in different embodiments, the display end device 402 may also include a display end device with a display capability of a higher dynamic range than the HDR video or image generated by the acquisition end device 401, and its display adaptation process can also be applied to the present system architecture, which is not limited in this application.

[0111] As mentioned in the background, traditional tone mapping methods primarily employ tone mapping curves or block-based local tone mapping. However, these methods suffer from poor tone mapping performance. For example, tone mapping curves typically implement tone mapping within a single local window, which can lead to edge artifacts when using a relatively simple shape for local processing.

[0112] In view of this, an embodiment of the present application provides a tone mapping method for designing a technical solution that can improve the tone mapping effect. The method provided by the embodiment of the present application is described below with reference to the accompanying drawings.

[0113] FIG5 is a flow chart of a tone mapping method applicable to a display terminal device 402 according to an embodiment of the present application. The flow chart shown in FIG5 includes:

[0114] In step 501, the display end device 402 obtains the image frame to be displayed and metadata, wherein the metadata includes at least one metadata information unit. It should be understood that the image frame to be displayed can be an image, or any frame of multiple frames contained in a video. In addition, the image frame to be displayed can be data sent by the acquisition end device 401; or it can be data obtained after other image processing processes, such as global tone mapping. Among them, the metadata can be generated by the acquisition end device 401 based on the acquired image frame to be displayed. The specific implementation method of generating metadata will be introduced in the subsequent introduction of the processing flow of the acquisition end device 401, and will not be described in detail here.

[0115] Step 502: The display terminal device 402 divides the image frame to be displayed into a plurality of segmented areas by using a target segmentation method, and determines a correspondence between the plurality of segmented areas and the at least one metadata information unit.

[0116] The target segmentation method can be a preset segmentation method or a segmentation method indicated in the metadata. Optionally, the preset segmentation method can be pre-set by the display device 402. It should be understood that in order to obtain accurate metadata, in this scenario, the acquisition device 401 and the display device 402 can be pre-set to the same preset segmentation method. Alternatively, in order to ensure that the acquisition device 401 and the display device 402 use the same segmentation method, the target segmentation method of the display device 402 can also be indicated by the acquisition device 401 in the metadata.

[0117] For example, the present application provides the following possible segmentation methods. It should be understood that the following segmentation methods are not limited during implementation.

[0118] In segmentation mode A, the display device 402 divides the image frame to be displayed into a plurality of rectangular segmented regions according to first segmentation parameters corresponding to a first direction and a second direction, respectively. The first direction is perpendicular to the second direction. Optionally, the first segmentation parameter may be the number of segments, in which case the resulting multiple segmented regions are represented as rectangular regions of equal size.

[0119] As shown in Figure 6, a segmentation diagram of segmentation method A provided for the implementation of this application, the first direction can be a horizontal direction, and the second direction can be a vertical direction. Assume that the number of segmentations in the first direction is represented by M1 (the value of M1 is usually ≤16), and the number of segmentations in the second direction is represented by M2 (the value of M2 is usually ≤16). In Figure 6, M1 is taken as 4 and M2 is also taken as 4 as an example, then the image frame to be displayed can be divided into 16 divided areas, that is, 16 rectangular areas of equal size. It should be understood that the values ​​of M1 and M2 can be the same or different, for example, M1 can be 6, M2 can be 4, etc.

[0120] In segmentation mode B, the display device 402 divides the image frame to be displayed into a plurality of rectangular segmentation areas according to the second segmentation parameters corresponding to the first direction and the second direction, respectively. Optionally, the second segmentation parameter can be a segmentation list, in which case the plurality of segmentation areas obtained by the segmentation are represented as rectangular areas of any size.

[0121] As shown in FIG7 , a schematic diagram of the segmentation method B provided for the implementation of the present application is shown. Assume that the segmentation list in the first direction is represented by w[j1] (j1 can take values ​​from 1 to M1, and M1 represents the number of segmentations in the first direction), and the segmentation list in the second direction is represented by h[j2] (j2 can take values ​​from 1 to M2, and M2 represents the number of segmentations in the second direction). In FIG7 , M1 takes a value of 4 and M2 also takes a value of 4. For example, each two values ​​of w[1], w[2], w[3] and w[4] included in FIG7 can be the same or different, and each two values ​​of h[1], h[2], h[3] and h[4] included in FIG7 can be the same or different.

[0122] Segmentation method C: the display terminal device 402 clusters the pixels contained in the image frame to be displayed, and divides the image frame to be displayed into multiple irregular segmentation areas according to the clustering result, and each irregular segmentation area includes at least one type of pixels.

[0123] As shown in Figure 8, a segmentation diagram of segmentation method C provided for the implementation of this application. When implementing this application, a target clustering method can be selected from a variety of optional clustering methods to perform clustering processing on the image frame to be displayed, and the specific clustering method is not limited in this application. For example, the image frame to be displayed after clustering processing can be divided into 16 segmentation areas shown in Figure 8. Exemplarily, when the display terminal device 402 clusters the image frame to be displayed, it can first divide the image frame to be displayed into multiple rectangular areas according to the first segmentation parameter or the second segmentation parameter as described above; then determine the coordinate information and feature information (feature information such as color component value) of the center pixel point of each rectangular area; obtain a clustering result by performing cluster analysis on each pixel point contained in the image frame to be displayed and each of the central pixel points; finally, obtain a segmentation diagram as shown in Figure 8 based on the clustering result.

[0124] Furthermore, to facilitate cluster analysis and reduce the amount of cluster processing required, when implementing this application, when performing image segmentation on the image frames to be displayed, the image segmentation process can also be combined with thumbnailed image frames to be displayed. For example, the thumbnailed image frames to be displayed can be selected to have a resolution of 1 / N1 of the first direction and 1 / N2 of the second direction of the captured image frames to be displayed. In this way, cluster analysis based on the thumbnailed image frames to be displayed can reduce the number of pixels required during the clustering process.

[0125] It should be noted that when implementing this application, appropriate segmentation methods can be selected for images with different characteristics. Exemplarily, segmentation method A can be applied to image frames to be displayed whose local features are not obvious, such as solid color image frames or regular images; it can also be applied to any image frame, and this application does not limit this. Segmentation method C can be applied to image frames with certain local features, etc. It can be understood that the display end device 402 and / or the acquisition end device 401 can pre-store multiple segmentation methods, and the corresponding target segmentation method can be further selected based on image frames with different characteristics.

[0126] After obtaining the multiple segmented regions, the display device 402 determines the metadata information unit corresponding to each segmented region based on at least one metadata information unit included in the metadata. This allows the display device 402 to obtain the tone mapping curve parameters, dynamic metadata, and static metadata corresponding to the segmented regions from the metadata information unit. Optionally, the multiple segmented regions and the at least one metadata information unit can have a one-to-one correspondence; that is, each segmented region can correspond to one metadata information unit, such as the 16 segmented regions in Figure 8 can correspond to 16 metadata information units. Alternatively, the multiple segmented regions and the at least one metadata information unit can have a many-to-one correspondence; that is, one or more segmented regions can correspond to one metadata information unit, such as in Figure 8 where segmented regions 1 and 2 can correspond to one metadata information unit, segmented regions 3 and 4 can correspond to one metadata information unit, and so on. It will be appreciated that this many-to-one relationship can reduce the amount of metadata transmitted between the acquisition device 401 and the display device 402, thereby improving data transmission efficiency.

[0127] Illustratively, the embodiments of the present application provide the following possible examples for determining the correspondence between the multiple segmented regions and the at least one metadata information unit. It should be understood that the following examples are not limited in implementation.

[0128] Example 1: The collection end device 401 may indicate the corresponding segmented area identifier in the metadata information unit.

[0129] Optionally, the segmented region identifier may be the number of segmented regions included in the metadata information unit. During implementation, the display end device 402 may traverse the multiple segmented regions included in the image frame to be displayed based on the target scanning order and the number of segmented regions. The target scanning order includes, but is not limited to, from left to right and then from top to bottom, from top to bottom and then from left to right, etc.; and the target scanning order may be a preset scanning order, or may be a scanning order indicated in the metadata. It should be understood that the display end device 402 and the acquisition end device 401 use the same preset scanning order.

[0130] For example, FIG9 shows a schematic diagram of a scanning order provided by an embodiment of the present application, in which the scanning order is from left to right and then from top to bottom. Assume that the 16 segmented areas corresponding to FIG9 correspond to 3 metadata information units. Among them, the first metadata information unit corresponds to 4 segmented areas, the second metadata information unit corresponds to 7 segmented areas, and the third metadata information unit corresponds to 5 segmented areas. The correspondence between the multiple segmented areas and the at least one metadata information unit can be shown in the following Table 1-1:

[0131] Table 1-1

[0132] Segment area identifier metadata information unit identifier 4 1st metadata information unit 7 2nd metadata information unit 5 3rd metadata information unit

[0133] According to the corresponding relationship shown in Table 1-1 above, the display device 402 can determine, based on the target scanning order, that the segmented areas corresponding to the first metadata information unit are segmented areas 1 to 4, the segmented areas corresponding to the second metadata information unit are segmented areas 5 to 11, and the segmented areas corresponding to the third metadata information unit are segmented areas 12 to 16.

[0134] Alternatively, the segmented region identifier may be segmented region serial number information. During implementation, the display terminal device 402 may determine the segmented region corresponding to the metadata information unit based on the segmented region serial number information included in the metadata information unit.

[0135] For example, the content shown in Figure 9 is used. In this example, the correspondence between the multiple segmented areas and the at least one metadata information unit may be as shown in the following Table 1-2:

[0136] Table 1-2

[0137] Segment area identifier Metadata information unit identifier Segment area 1 to segment area 4 (first metadata information unit) Segment area 5 to segment area 11 (second metadata information unit) Segment area 12 to segment area 16 (third metadata information unit)

[0138] According to the corresponding relationship shown in Table 1-2 above, the display terminal device 402 can directly determine the corresponding segmented area according to the segmented area sequence number information included in the metadata information unit.

[0139] Example 2: The display end device 402 may also select one or more pixels at preset positions from any of the segmented areas, obtain coordinate information of the pixels at the one or more preset positions, and determine the metadata information unit corresponding to any of the segmented areas based on the coordinate information, thereby establishing a correspondence between any of the segmented areas and the corresponding metadata information unit. The preset positions may include one or more of the following positions: the upper left corner, the center, the lower right corner, the upper left corner, and the lower right corner. For example, for segmented area 1, the preset positions may be the upper left corner, the center, and the lower right corner, with the coordinate information of the upper left corner being (x1, y1), the coordinate information of the center being (x2, y2), and the coordinate information of the lower right corner being (x3, y3); then the metadata information unit containing (x1, y1), (x2, y2), and (x3, y3) is determined, thereby obtaining the metadata information unit corresponding to segmented area 1.

[0140] It should be understood that the display device 402 can also determine, for any of the metadata information units, one or more coordinate information contained in the metadata information unit, at least one segmented area corresponding to the one or more coordinate information; and establish a corresponding relationship between any of the metadata information units and the corresponding at least one segmented area.

[0141] For the i-th pixel in the image frame to be displayed (where i is any positive integer from 1 to N, and N is the number of pixels contained in the image frame to be displayed), the display terminal device 402 performs the following steps 503 to 505 (it should be understood that in the following embodiments, any pixel point is used as an example, and the processing process of other pixel points contained in the image frame to be displayed is similar, and will not be described one by one in the following embodiments), as follows:

[0142] Step 503: Based on the multiple pixels included in the preset area selected by the i-th pixel, determine at least one related segmented area to which the multiple pixels belong, and determine the related metadata information unit corresponding to each of the related segmented areas according to the corresponding relationship.

[0143] Among them, the preset area selected based on the i-th pixel is: an area of ​​a preset shape of a preset size selected with the i-th pixel as the center, for example, the preset shape can be a rectangle or other shape with a preset width and a preset length. Or it is: an area centered on the i-th pixel and determined according to the preset shape and the preset size information contained in the metadata, wherein if the preset shape is a rectangle, the preset size information can be the length and width information of the rectangle; if the preset shape is a circle, the preset size information can be the radius or diameter information of the circle. Or it is: an area centered on the i-th pixel and determined according to the preset shape and preset size information contained in the metadata. In other words, the preset shape and / or preset size information of the preset shape used by the display end device 402 to select the preset area for the i-th pixel can be information stored locally or information indicated in the metadata.

[0144] For example, FIG10 shows a schematic diagram of determining relevant segmentation areas provided by an embodiment of the present application, based on the image frame to be displayed obtained by the segmentation method adopted in FIG6. In combination with the contents shown in (a) and (b) of FIG10, when processing pixel 1, a plurality of pixels are included in the preset rectangular area centered on pixel 1, and the plurality of pixels belong to segmentation area 1, segmentation area 2, segmentation area 5, and segmentation area 6, respectively. Then, it can be determined that the relevant segmentation areas of pixel 1 include 4. In combination with the contents shown in (a) and (c) of FIG10, when processing pixel 2, a plurality of pixels included in the preset rectangular area centered on pixel 2 belong to segmentation areas 6, 7, 8, 10, 11, 12, 14, 15, and 16, respectively. Then, it can be determined that the relevant segmentation areas of pixel 2 include 9. It should be understood that the relevant segmentation areas of a pixel are determined based on the pixel position and the preset area, and the number of relevant segmentation areas of different pixels included in the image frame to be displayed can be the same or different. It should be noted that for the multiple pixel points contained in the displayed image frame, a preset area with the same preset size information can usually be used, but this application is not limited to this. Preset areas with different preset size information can also be used during implementation. For example, pixel point 1 and pixel point 2 shown in Figure 10 use different preset size information.

[0145] For another example, Figure 11 shows another schematic diagram of determining relevant segmented regions according to an embodiment of the present application, based on the image frame to be displayed obtained by the segmentation method used in Figure 8. When processing pixel 3, multiple pixels included in the preset rectangular area centered on pixel 3 are respectively divided into data segmentation regions 9, 10, 13, 14, and 15, so the number of relevant segmented regions for pixel 3 can be determined to be 5.

[0146] Based on the fact that the display end device 402 in step 502 can obtain the correspondence between multiple segmented areas and at least one metadata information unit, after determining the relevant segmented areas of the pixel points, it is possible to further obtain key information contained in the relevant segmented areas from the metadata information units corresponding to the relevant segmented areas, such as parameters of the tone mapping curve, dynamic metadata, static metadata and other metadata information, so as to realize the tone mapping of the pixel points to be processed.

[0147] Step 504: The display device 402 obtains the sub-tone mapping value corresponding to each of the related segmented regions according to the related metadata information unit, and assigns a corresponding weight factor to each of the related segmented regions.

[0148] Exemplarily, the relevant metadata information unit includes parameters of a tone mapping curve, and the display terminal device 402 can obtain the sub-tone mapping value corresponding to the i-th pixel in the relevant segmented area based on the tone mapping curve. It can be understood that the number of sub-tone mapping values ​​for the i-th pixel depends on the number of relevant segmented areas. For example, for pixel 1 shown in Figure 10, the number of relevant segmented areas for pixel 1 is 4, so 4 sub-tone mapping values ​​corresponding to pixel 1 can be obtained; for pixel 2 shown in Figure 10, the number of relevant segmented areas for pixel 2 is 9, so 9 sub-tone mapping values ​​corresponding to pixel 2 can be obtained.

[0149] The present application does not limit the parameters of the tone mapping curve contained in the relevant data information unit described in the above example. For example, the parameters of the tone mapping curve may be determined based on the selected tone mapping curve form. The tone mapping curve form includes, but is not limited to, a sigmoidal function curve, a cubic (or multiple) spline function curve, a Bezier curve, a gamma function curve, and the like.

[0150] An embodiment of obtaining a tone mapping curve based on the parameters of the tone mapping curve is that the tone mapping curve is a cubic spline function. When implementing this application, the display terminal device 402 determines the tone mapping curve of any relevant segmented area by including the following steps:

[0151] A1. Obtain a maximum brightness value (maxLum) from relevant metadata information units corresponding to the relevant segmented areas.

[0152] A2. Determine the cubic spline interval length (Len) based on the maxLum and cubic spline interval information (Num3). Wherein, Len can be expressed as formula (2-1):

[0153]

[0154] A3. Obtain the starting position (TH[index-1]) and ending position (TH[index]) of the first interval based on Len and the interval index (index, used to identify the first interval); the first interval is any interval among the multiple intervals included in the cubic spline. TH[index] can be expressed as formula (2-2):

[0155] TH[index]=index*Len (2-2)

[0156] A4. Obtain cubic spline parameters P0, P1, P2, and P3 of the first interval according to the cubic spline interval information, the interval index, and the cubic spline interval value.

[0157] A4-1, obtain the initial interval slope of the first interval based on the cubic spline interval value and the cubic spline interval information. The initial interval slope can be expressed as formula (2-3):

[0158]

[0159] Where gainOrg[index] represents the initial interval slope of the first interval, THValue[index] represents the cubic spline interval value of the first interval, and THValue[index-1] represents the cubic spline interval value of the interval before the first interval.

[0160] A4-2, obtain the interval endpoint slope of the first interval based on the initial interval slope. The interval endpoint slope includes the interval start endpoint slope, which can be expressed as the following formula (2-4); and the interval end endpoint slope, which can be expressed as the following formula (2-5):

[0161]

[0162] Where gainOrg[index-1] represents the initial interval slope of the interval before the first interval, and gainStart[index] represents the starting endpoint slope of the interval.

[0163]

[0164] Where gainOrg[index+1] represents the initial interval slope of the interval following the first interval, and gainEnd[index] represents the end endpoint slope of the interval.

[0165] A4-3, the cubic spline parameters P0, P1, P2, and P3 of the first interval can be expressed as the following formulas (2-6) to (2-9):

[0166] P0=THValue[index-1] (2-6)

[0167] P1=gainStart[index] (2-7)

[0168]

[0169]

[0170] Among them, ValueA, ValueB, and ValueC in formula (2-8) and formula (2-9) are intermediate values ​​and can be calculated as follows:

[0171] ValueA=THValue[index]-gainStart[index]*ValueC-THValue[index-1]

[0172] ValueB=gainEnd[index]-gainStart[index]

[0173] ValueC=TH[index]-TH[index-1]

[0174] A5. Based on the cubic spline parameters P0, P1, P2, and P3, the function expression corresponding to the tone mapping curve of the relevant segmented area can be obtained, which can be expressed as the following formula (2-10):

[0175] L′=H(L)=P0+P1*(L-TH[index-1])+P2*(L-TH[index-1]) 2 +P3*(L-TH[index-1]) 2 (2-10)

[0176] Wherein, L represents the linear signal value, which is normalized to [0, 1], L' represents the nonlinear signal value, which ranges from [0, 1], and H(L) represents the function corresponding to the tone mapping curve.

[0177] Another embodiment of obtaining a tone mapping curve based on the parameters of the tone mapping curve is that the function expression corresponding to the tone mapping curve can also be expressed as the following formula (2-11):

[0178]

[0179] Wherein, L represents the linear signal value, which is normalized to [0, 1], and L' represents the nonlinear signal value, which has a value range of [0, 1]. F(L) represents the function corresponding to the tone mapping curve. a, b, p, m, and n represent the parameters of the tone mapping curve, which are used to adjust the shape of the tone mapping curve. It should be understood that the parameters of the tone mapping curve function shown in (2-11) corresponding to different segmented regions may be different. During implementation, the parameters of the tone mapping curve for each segmented region may be indicated in the corresponding metadata information unit.

[0180] It should be noted that, during implementation, different tone mapping curve functions may be used based on different tone mapping curve shapes, and the selected tone mapping curve is not limited in this application.

[0181] After obtaining the tone mapping curve corresponding to the relevant segmented area, the display device 402 performs tone mapping on the i-th pixel based on the tone mapping curve to obtain the sub-tone mapping value of the pixel in the relevant segmented area. For example, for pixel 3 in Figure 11, the sub-tone mapping value 1 corresponding to segmented area 9, the sub-tone mapping value 2 corresponding to segmented area 10, the sub-tone mapping value 3 corresponding to segmented area 13, the sub-tone mapping value 4 corresponding to segmented area 14, and the sub-tone mapping value 5 corresponding to segmented area 15 can be obtained, for a total of 5 sub-tone mapping values.

[0182] In one embodiment of tone mapping, the display device 402 may perform a brightness adjustment process and / or a color adjustment process on the i-th pixel based on the tone mapping curve. Optionally, the color adjustment of the i-th pixel may be expressed as the following formula (3-1):

[0183]

[0184] Among them, Rp, Gp, Bp represent the three color components of the i-th pixel, f[id] represents the function corresponding to the tone mapping curve, such as H(L), F(L) mentioned above, etc.; id represents the segmentation area identifier.

[0185] The sub-tone mapping value of the i-th pixel in the relevant segmented area can be expressed as the following formulas (3-2) to (3-4):

[0186] RpTM[i]=Rp*gain (3-2)

[0187] GpTM[i]=Gp*gain (3-3)

[0188] BpTM[i]=Bp*gain (3-4)

[0189] Among them, RpTM[i], GpTM[i], BpTM[i] can be expressed as the sub-tone mapping value of the i-th pixel in the relevant segmented area.

[0190] In another example, in addition to determining the sub-tone mapping value corresponding to each relevant segmented area, the display end device 402 can also determine the weight factor corresponding to each relevant segmented area, thereby determining the degree of influence of the sub-tone mapping value of each relevant segmented area on the tone mapping value of the i-th pixel based on the weight factor.

[0191] Optionally, when implementing this application, the display device 402 may assign a corresponding weighting factor to each of the related segmented regions. Furthermore, the display device 402 may determine the weighting factor based on a preset method or based on metadata indications. This application provides the following two possible methods for determining the weighting factor. One or a combination of the following methods may be used during implementation, and actual implementation is not limited to the following two methods.

[0192] Method 1, respectively determining the number of pixels in each of the related segmented regions belonging to the preset region and the total number of pixels corresponding to the preset region. For a first related segmented region, performing: assigning a corresponding first weight factor to the first related segmented region based on the number of pixels in the first related segmented region belonging to the preset region and the total number of pixels. The first related segmented region is any one of the at least one related segmented region.

[0193] For example, with reference to pixel point 3 shown in FIG11 , the number of pixels in each of the related segmented areas belonging to the preset area, and the total number of pixels corresponding to the preset area, may be as shown in the following Table 2-1:

[0194] Table 2-1

[0195]

[0196] Where Num[id] represents the number of pixels in the segmented region within the preset region, and id represents the segmented region identifier. For example, for pixel 3 in Figure 11(b), the number of pixels in each relevant segmented region within the preset region. Weight1[id] represents the first weighting factor for the segmented region id.

[0197] Optionally, the first weight factor may be positively correlated with the number of pixels in the segmented area, such as a linear relationship; in other words, the more pixels in the segmented area, the higher the first weight factor corresponding to the segmented area, and the relationship increases linearly.

[0198] Alternatively, the positive correlation may be an indicative relationship, and the exponential relationship may be determined based on a first weighted intensity value. The first weighted intensity value is used to adjust the first weighting factor based on the distribution of the number of pixels in the image frame to be displayed. For example, adjusting the first weighting factor using the first weighted intensity value may be expressed as the following formula (4-1):

[0199]

[0200] Among them, NumStr[id] represents the first weighted intensity value, which can be 2 or 0.5, for example. The first weighted intensity value can be determined based on the pixel similarity of multiple segmented areas in the image frame to be displayed; for example, if the pixels of multiple segmented areas are relatively similar pixels, a larger first weighted intensity value can be set; otherwise, a smaller first weighted intensity value can be set. It should be understood that the first weighted intensity value can be determined by the display end device 402 based on preset rules, or it can be indicated in the metadata by the acquisition end device 401, and this application does not limit this. In this way, by setting the first weighted intensity value, when the local features of the image frame to be displayed are more obvious, a smaller first weighted intensity value, such as 0.5, can be used to reduce the boundary effect between the segmented areas.

[0201] Method 2, respectively determining the preset characteristic value corresponding to the preset area and the preset characteristic value of the i-th pixel in each of the related segmented areas. For the second related segmented area, executing: according to the characteristics of the preset characteristic value corresponding to the partial area of ​​the preset area and the preset characteristic value of the i-th pixel in the second related segmented area, assigning a corresponding second weight factor to the second related segmented area. The second related segmented area is any one of the at least one related segmented area. The preset characteristic value is a brightness characteristic value (AvgLum[id]), or a color characteristic value (AvgRGB[id][3], where [3] represents three color components), etc. It should be understood that the preset characteristic value corresponding to the preset area and the preset characteristic value of the i-th pixel in the related segmented area select the same preset characteristic value.

[0202] The number of the preset characteristic values ​​is at least one, for example, a segmented area may have 4 brightness characteristic values ​​(AvgLum[id](4)), and a segmented area may have 4 color characteristic values ​​(AvgRGB[id](4)[3]). FIG12 shows a schematic diagram of a brightness characteristic value provided in an embodiment of the present application, where FIG12(a) shows a scene with a brightness characteristic value of 1, and FIG12(b) shows a scene with a brightness characteristic value of 4. The number of brightness characteristic values ​​can be obtained based on the brightness value with the largest number or the highest average value in the segmented area.

[0203] In the above example, when the preset feature values are 4, during the implementation of this application, the target preset feature value selected from the 4 preset feature values can also be determined according to the relative relationship between the central coordinates (x, y) of the partial area belonging to the preset area in the second relevant segmentation area (which can also be the coordinates (x, y) of the i-th pixel (current pixel)) and the central coordinates (xR, yR) of the second relevant segmentation area. Exemplarily, in combination with the content shown in (b) of FIG. 12, the upper left area is the brightness feature value 1, the upper right area is the brightness feature value 2, the lower left area is the brightness feature value 3, and the lower right area is the brightness feature value 4. When x ≤ xR and y ≥ yR, the brightness feature value 1 is used; when x > xR and y ≥ yR, the brightness feature value 2 is used; when x ≤ xR and y < yR, the brightness feature value 3 is used; when x > xR and y < yR, the brightness feature value 4 is used.

[0204] For example, in combination with pixel point 3 shown in FIG. 11, and taking the preset area feature as the brightness feature value as an example, the brightness feature values of the partial areas belonging to the preset area in each of the relevant segmentation areas, and the brightness feature value of the i-th pixel, can be as shown in Table 2-2 below:

[0205] Table 2-2

[0206] Name Brightness Feature Value Second Weight Factor Segmentation Area 9 AvgLum[1] Weight2[1] Segmentation Area 10 AvgLum[2] Weight2[2] Segmentation Area 13 AvgLum[3] Weight2[3] Segmentation Area 14 AvgLum[4] Weight2[4] Segmentation Area 15 AvgLum[⑤] Weight2[5] The i-th Pixel AvgLum[i]

[0207] Among them, AvgLum[id] represents the brightness feature value of the segmentation area included under the preset area, and id represents the segmentation area identifier; it is the brightness feature value of the partial area of the relevant segmentation area of pixel point 3 shown in (b) of FIG. 11 that belongs to the preset area. Weight2[id] represents the second weight factor of segmentation area id.

[0208] Optionally, the second weight factor can be negatively correlated with the difference value between the brightness feature value of the segmentation area and the brightness feature value of the i-th pixel, such as a linear relationship; in other words, the greater the difference value between the brightness feature value of the segmentation area and the brightness feature value of the i-th pixel, the smaller the second weight factor corresponding to the segmentation area, and it decreases linearly.

[0209] Alternatively, the positive correlation may be an exponential relationship, and the exponential relationship may be determined based on a second weighted intensity value. The second weighted intensity value is used to adjust the second weighting factor based on changes in a preset characteristic value of the image frame to be displayed. For example, adjusting the second weighting factor using the second weighted intensity value may be expressed as the following formula (4-2):

[0210]

[0211] Here, g() represents a normalization operation, for example, dividing the signal by its maximum value to change its range to 0 to 1. clip() represents a clamping operation, min is the minimum value (for example, equal to 0), and max is the maximum value (for example, equal to 1). The value of the third operand in clip() that is greater than max or less than min is truncated to max and min respectively; for example, if the value of the third operand in clip() is less than min, min is used; if the third operand in clip() is greater than max, max is used. K is a preset value, ranging from 0 to 1023. AvgLumStr[id] represents the second weighted intensity value, for example, it can be 2 or 0.5.

[0212] Exemplarily, the second weighted intensity value can be determined based on the brightness change of pixels in multiple segmented areas in the image frame to be displayed. For example, if the brightness change of pixels in multiple segmented areas is more obvious, a larger second weighted intensity value can be set; otherwise, a smaller second weighted intensity value can be set. It should be understood that the second weighted intensity value can be determined by the display end device 402 based on preset rules, or it can be indicated in the metadata by the acquisition end device 401, and this application does not limit this. In this way, by setting the second weighted intensity value, when the local features of the image frame to be displayed are more obvious, a smaller second weighted intensity value, such as 0.5, can be used to reduce the boundary effect between the segmented areas.

[0213] Optionally, the weight factor corresponding to each relevant segmented region can be a first weight factor, which can be calculated according to the formula shown in formula (4-1), that is, Weight[id]=Weight1[id]; it can also be a second weight factor, which can be calculated according to the formula shown in formula (4-2), that is, Weight[id]=Weight2[id]; it can also be determined by the first weight factor and the second weight factor, which can be expressed as the following formula (4-3):

[0214] Weight[id]=W(Weight1[id], Weight2[id]) (4-3)

[0215] Wherein, W() represents a preset function for obtaining a weight factor corresponding to the relevant segmented region according to the first weight factor and the second weight factor. The preset function may be multiplication or addition, etc., and may be determined by presetting.

[0216] Step 505: The display device 402 obtains the tone mapping value of the i-th pixel based on the sub-tone mapping value and the weight factor corresponding to each of the related segmented areas; and performs tone mapping on the i-th pixel according to the tone mapping value.

[0217] Exemplarily, the present application may obtain the tone mapping value of the i-th pixel according to the following formulas (5-1) to (5-5):

[0218] MaxRGBP=max(Rp,Gp,Bp) (5-1)

[0219]

[0220]

[0221]

[0222]

[0223] Where Z represents the number of segmented regions associated with the i-th pixel. Rtm, Gtm, and Btm represent the tone mapping values. spline[id] represents the function corresponding to the tone mapping curve; and id represents the segmented region identifier.

[0224] In addition, when implementing this application, the tone mapping value can also be transformed from Rtm, Gtm, Btm to the color gamut space to obtain Ytm, Utm, Vtm; correspondingly, the sub-tone mapping value is Yp, Up, Vp.

[0225] Furthermore, in the implementation of the present application, after obtaining the tone mapping value of the i-th pixel, the display end device 402 may further perform saturation adjustment on the image frame to be displayed. In other words, the display end device 402 adjusts the saturation of the i-th pixel based on the initial pixel value of the i-th pixel and the tone mapping value obtained in step 505.

[0226] It should be noted that in the examples described above, the information required to calculate the tone mapping value for each pixel can be processed by display device 402; alternatively, the information can be calculated by acquisition device 401 and then indicated to display device 402 via metadata. In other words, the content to be processed by display device 402 and acquisition device 401 can be allocated based on their respective computing capabilities, and this application does not limit this. It is understood that the information indicated by the metadata can be used by display device 402 to subsequently implement tone mapping for the image frame to be displayed.

[0227] FIG13 is another flowchart of a tone mapping method provided in an embodiment of the present application, which is applicable to the acquisition terminal device 401 shown in FIG4 .

[0228] Step 1301: Capture image data to obtain a data frame to be displayed. For example, the image data may be captured by the acquisition terminal device 401 through a camera, or may be generated by the acquisition terminal device 402 through a preset application.

[0229] Step 1302: Use a target segmentation method to segment the data frame to be displayed into multiple segmented areas, and determine at least one metadata information unit corresponding to the multiple segmented areas. Exemplarily, the target segmentation method can be the multiple segmentation methods introduced in the above content, such as segmentation method A shown in Figure 6, segmentation method B shown in Figure 7, or segmentation method C shown in Figure 8, etc.; and the segmentation method adopted by the acquisition end device 401 and the display end device 402 is the same. In addition, the acquisition end device 401 can also indicate the target segmentation method to the display end device 402 through metadata. And, the acquisition end device 402 can also indicate the first segmentation parameter, the second segmentation parameter, etc. that may be involved in the target segmentation method to the display end device 402 through metadata.

[0230] For example, for segmentation method A, the acquisition end device may determine the number of segmentations in the vertical direction according to the histograms of different rows in the image data; and determine the number of segmentations in the horizontal direction according to the histograms of different columns in the image data.

[0231] In another example, for segmentation method B, the acquisition end device can cluster according to the histograms of different rows in the image data to determine the segmentation list in the vertical direction; and cluster according to the histograms of different columns in the image data to determine the segmentation list in the horizontal direction.

[0232] Step 1303: Send the data frame to be displayed and metadata to the display terminal device 402; the metadata includes the at least one metadata information unit.

[0233] It should be understood that in order to reduce the amount of computation required by the display device 402 and balance the content that needs to be processed by the acquisition device 401 and the display device 402, when implementing this application, the acquisition device 401 can calculate information such as the first weighted intensity value, the second weighted intensity value, and the preset feature value of the segmented region, and indicate this to the display device 402 through metadata. It should be noted that the implementation process of the processing performed by the acquisition device 401 is similar to that described for the display device 402, and therefore will not be repeated here.

[0234] According to the same inventive concept as the above method, the embodiment of the present application provides a tone mapping processing device 1400. The tone mapping processing device 1400 can be applied to the above display terminal device 402. Referring to Figure 14, the device may include: an acquisition module 1401, a processing module 1402 and a tone mapping module 1403.

[0235] An acquisition module 1401 is configured to acquire an image frame to be displayed and metadata, the metadata comprising at least one metadata information unit. A processing module 1402 is configured to segment the image frame to be displayed into multiple segmented regions using a target segmentation method and determine a correspondence between the multiple segmented regions and the at least one metadata information unit. A tone mapping module 1403 is configured to execute, for an i-th pixel in the image frame to be displayed, the following operations: determining, based on multiple pixels contained in a preset region selected by the i-th pixel, at least one associated segmented region to which the multiple pixels belong, and determining, based on the correspondence, associated metadata information units corresponding to each associated segmented region; obtaining, based on the associated metadata information units, a sub-tone mapping value corresponding to each associated segmented region and assigning a corresponding weight factor to each associated segmented region; obtaining a tone mapping value for the i-th pixel based on the sub-tone mapping value and the weight factor corresponding to each associated segmented region; and performing tone mapping on the i-th pixel based on the tone mapping value. Here, i is any positive integer from 1 to N, and N is the number of pixels in the image frame to be displayed.

[0236] In one possible design, the tone mapping module 1403, when used to assign a corresponding weight factor to each of the related segmented areas, is specifically used to: respectively determine the number of pixels in each of the related segmented areas belonging to the preset area, and the total number of pixels corresponding to the preset area; for a first related segmented area, execute: according to the number of pixels in the first related segmented area belonging to the preset area and the total number of pixels, assign a corresponding first weight factor to the first related segmented area, where the first related segmented area is any one of the at least one related segmented area; and / or, respectively determine the preset feature value corresponding to the preset area in each of the related segmented areas; for a second related segmented area, execute: according to the feature difference between the preset feature value corresponding to the preset area in the second related segmented area and the preset feature value of the i-th pixel, assign a corresponding second weight factor to the second related segmented area, where the second related segmented area is any one of the at least one related segmented area.

[0237] In one possible design, the metadata also includes one or a combination of the following information: a first weighted intensity value and a second weighted intensity value; wherein the first weighted intensity value is used to adjust the first weight factor according to the distribution of the number of pixels of the image frame to be displayed; and the second weighted intensity value is used to adjust the second weight factor according to the change of the preset characteristic value of the image frame to be displayed.

[0238] In a possible design, the preset characteristic value is one of the following characteristic values: a brightness characteristic value and a color characteristic value, wherein the number of the preset characteristic values ​​is at least one.

[0239] In one possible design, the tone mapping module 1403 is used to obtain the sub-tone mapping value corresponding to each of the related segmented areas based on the related metadata information unit, specifically, the related metadata information unit contains parameters of the tone mapping curve, and based on the tone mapping curve, the sub-tone mapping value corresponding to the i-th pixel in the related segmented area is obtained.

[0240] In one possible design, the preset area selected based on the i-th pixel is: an area of ​​a preset shape of a preset size selected with the i-th pixel as the center; or an area centered on the i-th pixel and determined based on a preset shape and preset size information contained in the metadata; or an area centered on the i-th pixel and determined based on the preset shape and preset size information contained in the metadata.

[0241] In one possible design, the processing module 1402 is used to determine the correspondence between the multiple segmented areas and the at least one metadata information unit, and is specifically used to: for any of the segmented areas, select pixels at a preset position from any of the segmented areas, and obtain coordinate information of the pixels at the preset position; determine the metadata information unit corresponding to any of the segmented areas based on the coordinate information; establish a correspondence between any of the segmented areas and the corresponding metadata information unit; or, for any of the metadata information units, determine at least one segmented area corresponding to the one or more coordinate information contained in the metadata information unit; and establish a correspondence between any of the metadata information units and the corresponding at least one segmented area.

[0242] In one possible design, the processing module 1402 is used to determine the correspondence between the multiple segmented regions and the at least one metadata information unit, including: the metadata information unit contains a corresponding segmented region identifier, and the multiple segmented regions are traversed according to the target scanning order and the segmented region identifier to determine the metadata information unit corresponding to each of the segmented regions; wherein the target scanning order is a preset scanning order, or a scanning order indicated in the metadata.

[0243] In a possible design, the multiple segmented regions and the at least one metadata information unit have a one-to-one correspondence or a many-to-one correspondence.

[0244] In one possible design, the target segmentation method is a preset segmentation method or a segmentation method indicated in the metadata.

[0245] According to the same inventive concept as the above method, the embodiment of the present application provides a tone mapping processing device 1400. The tone mapping processing device 1500 can be applied to the above acquisition terminal device 401. Referring to Figure 15, the device 1500 may include: an acquisition module 1501, a processing module 1502 and a transceiver module 1503.

[0246] The acquisition module 1501 is used to acquire image data and obtain a data frame to be displayed; the processing module 1502 is used to divide the data frame to be displayed into multiple segmented areas using a target segmentation method, and determine at least one corresponding metadata information unit for the multiple segmented areas; the transceiver module 1503 is used to send the data frame to be displayed and metadata to the display end device; the metadata includes the at least one metadata information unit.

[0247] In one possible design, the processing module 1502 is further used to determine a first weighted intensity value based on the number of pixels respectively contained in the multiple segmented areas; and / or to determine a second weighted intensity value based on preset feature values ​​respectively corresponding to the multiple segmented areas; and to associate the first weighted intensity value and / or the second weighted intensity value with the metadata.

[0248] In a possible design, the preset characteristic value is one of the following characteristic values: a brightness characteristic value and a color characteristic value, wherein the number of the preset characteristic values ​​is at least one.

[0249] In one possible design, the metadata information unit includes parameters of a tone mapping curve corresponding to the segmented area.

[0250] In one possible design, the metadata also includes: preset size information; the preset size information is used to indicate the preset size of the preset shape used when selecting a preset area for pixels; or, the metadata also includes: preset shape and preset size information.

[0251] In one possible design, the processing module 1502 is used to determine at least one metadata information unit corresponding to the multiple segmented areas, specifically to select pixels at multiple preset positions from the segmented areas, obtain coordinate information of the pixels at the multiple preset positions; and associate the coordinate information with the metadata information unit.

[0252] In one possible design, the metadata information unit also includes: a segmented area identifier and a scanning order corresponding to the metadata information unit; wherein the corresponding segmented area identifier and the scanning order are used by the display end device to determine the correspondence between the multiple segmented areas and the at least one metadata information unit.

[0253] In a possible design, the multiple segmented regions and the at least one metadata information unit have a one-to-one correspondence or a many-to-one correspondence.

[0254] In a possible design, the processing module 1502 is further used to associate the target segmentation method with the metadata to instruct the display terminal device to segment the data frame to be displayed using the target segmentation method.

[0255] An embodiment of the present application also provides an electronic device 1600, as shown in Figure 16. The electronic device 1600 can be the display end device 402 or the acquisition end device 401 introduced in the above embodiment. The electronic device 1600 may include a communication interface 1610 and a processor 1620. Optionally, the electronic device 1600 may also include a memory 1630. The memory 1630 may be arranged inside the electronic device or outside the electronic device. The acquisition module 1401, the processing module 1402 and the tone mapping module 1403 shown in Figure 14 above can all be implemented by the processor 1620. Alternatively, the acquisition module 1501, the processing module 1502 and the transceiver module 1503 shown in Figure 15 above can also be implemented by the processor 1620. The optional communication interface 1610, processor 1620, and memory 1630 can be interconnected via a communication line 1640; communication line 1640 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, for example. Communication line 1640 can be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, FIG16 shows only one thick line, but this does not imply that there is only one bus or only one type of bus.

[0256] In one possible implementation, the processor 1620 is configured to implement any of the methods performed by the acquisition end described in FIG. 13 , and output the encoded code stream by the acquisition end through the communication interface 1610 , such as image frames to be displayed, metadata, and the like.

[0257] During implementation, each step of the processing flow can be implemented by hardware integrated logic circuits or software instructions in processor 1620 to complete the method executed by the acquisition end in Figure 13. For the sake of brevity, this description is omitted here. The program code executed by processor 1620 to implement the above method can be stored in memory 1630. Memory 1630 is coupled to processor 1620.

[0258] Any communication interface involved in the embodiments of the present application may be a circuit, a bus, a transceiver, or any other device capable of exchanging information. For example, the communication interface 1610 in the electronic device 1600 may be, for example, a device connected to the electronic device 1600. For example, when the electronic device 1600 is the acquisition terminal device 401, the other device may be the display terminal device 402.

[0259] Processor 1620 may operate in conjunction with memory 1630. Memory 1630 may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or a volatile memory, such as random-access memory (RAM). Memory 1630 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0260] The specific connection medium between the communication interface 1610, processor 1620, and memory 1630 is not limited in the embodiments of the present application. In Figure 16, the embodiment of the present application shows that the memory 1630, processor 1620, and communication interface 1610 are connected via a bus. The bus is represented by a bold line in Figure 16. The connection method between other components is only for schematic illustration and is not limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one bold line is used in Figure 16, but this does not mean that there is only one bus or one type of bus.

[0261] The processors involved in the embodiments of the present application may be general-purpose processors, digital signal processors, application-specific integrated circuits, field programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0262] The coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, modules or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, modules or modules.

[0263] Based on the above embodiments, embodiments of the present application further provide a computer storage medium storing a software program that, when read and executed by one or more processors, can implement the methods provided by any one or more of the above embodiments. The computer storage medium may include any medium capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0264] Based on the above embodiments, embodiments of the present application further provide a chip, which includes a processor for implementing the functions involved in any one or more of the above embodiments. Optionally, the chip also includes a memory for storing program instructions and data necessary for execution by the processor. The chip can be composed of a single chip or can include a chip and other discrete devices.

[0265] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0266] The present application is described with reference to the flow chart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart flow or multiple flows and / or one box or multiple boxes of the block diagram.

[0267] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0268] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0269] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application also intends to include these modifications and variations.

Claims

1. A tone mapping method, It is characterized in that Applied to a display terminal device, the method comprises: Acquire an image frame to be displayed and metadata, wherein the metadata includes at least one metadata information unit; Using a target segmentation method to segment the image frame to be displayed into a plurality of segmented areas, and determining a corresponding relationship between the plurality of segmented areas and the at least one metadata information unit; For the i-th pixel in the image frame to be displayed, execute: Based on a plurality of pixels included in the preset area selected by the i-th pixel, determining at least one related segmented area to which the plurality of pixels belong, and determining a related metadata information unit corresponding to each of the related segmented areas according to the corresponding relationship; Obtaining a sub-tone mapping value corresponding to each of the related segmented regions according to the related metadata information unit, and assigning a corresponding weight factor to each of the related segmented regions; Based on the sub-tone mapping values ​​corresponding to each of the related segmented areas and the weight factor, a tone mapping value of the i-th pixel is obtained; and according to the tone mapping value, tone mapping is performed on the i-th pixel; Wherein, the i is any positive integer from 1 to N, and N is the number of pixels contained in the image frame to be displayed.

2. The method according to claim 1, It is characterized in that The assigning of corresponding weight factors to each of the related segmented regions includes one or a combination of the following methods: Determine the number of pixels in each of the related segmented areas belonging to the preset area and the total number of pixels corresponding to the preset area respectively; for a first related segmented area, perform: assign a corresponding first weight factor to the first related segmented area according to the number of pixels in the first related segmented area belonging to the preset area and the total number of pixels, the first related segmented area being any one of the at least one related segmented area; Determine respectively the preset feature value corresponding to the preset area in each of the related segmented areas; For the second related segmentation area, execute: according to the feature difference between the preset feature value corresponding to the preset area and the preset feature value of the i-th pixel in the second related segmentation area, assign a corresponding second weight factor to the second related segmentation area, and the second related segmentation area is any one of the at least one related segmentation area.

3. The method according to claim 2, It is characterized in that The metadata also includes one or a combination of the following information: a first weighted intensity value, a second weighted intensity value; Among them, the first weighted intensity value is used to adjust the first weight factor according to the distribution of the number of pixels of the image frame to be displayed; the second weighted intensity value is used to adjust the second weight factor according to the change of the preset characteristic value of the image frame to be displayed.

4. The method according to claim 2 or 3, It is characterized in that The preset characteristic value is one of the following characteristic values: a brightness characteristic value and a color characteristic value.

5. The method according to claim 4, It is characterized in that The number of the preset characteristic values ​​is at least one.

6. The method according to claim 1, It is characterized in that The obtaining, according to the relevant metadata information unit, a sub-tone mapping value corresponding to each of the relevant segmented areas comprises: The relevant metadata information unit includes parameters of a tone mapping curve, and a sub-tone mapping value corresponding to the i-th pixel in the relevant segmented area is obtained based on the tone mapping curve.

7. The method according to any one of claims 1 to 6, It is characterized in that The preset area selected based on the i-th pixel is: an area of ​​a preset shape of a preset size selected with the i-th pixel as the center; or: an area with the i-th pixel as the center and determined according to a preset shape and preset size information contained in the metadata; or: an area with the i-th pixel as the center and determined according to the preset shape and preset size information contained in the metadata.

8. The method according to claim 1, It is characterized in that The determining the correspondence between the plurality of segmented regions and the at least one metadata information unit includes: For any of the segmented areas, select a pixel at a preset position from any of the segmented areas, and obtain coordinate information of the pixel at the preset position; determine a metadata information unit corresponding to any of the segmented areas according to the coordinate information; establish a corresponding relationship between any of the segmented areas and the corresponding metadata information unit; or, For any of the metadata information units, determine at least one segmented area corresponding to the one or more coordinate information contained in the metadata information unit; and establish a corresponding relationship between any of the metadata information units and the corresponding at least one segmented area.

9. The method according to claim 1, It is characterized in that The determining the correspondence between the plurality of segmented regions and the at least one metadata information unit includes: The metadata information unit includes a corresponding segmented area identifier, and the multiple segmented areas are traversed according to a target scanning order and the segmented area identifier to determine the metadata information unit corresponding to each segmented area; wherein the target scanning order is a preset scanning order, or a scanning order indicated in the metadata.

10. The method according to any one of claims 1 to 9, It is characterized in that The plurality of segmented regions and the at least one metadata information unit are in a one-to-one correspondence relationship or a many-to-one correspondence relationship.

11. The method according to any one of claims 1 to 10, It is characterized in that The target segmentation method is a preset segmentation method or a segmentation method indicated in the metadata.

12. A tone mapping method, It is characterized in that Applied to a collection end device, the method includes: Collect image data to obtain data frames to be displayed; Using a target segmentation method to segment the data frame to be displayed into a plurality of segmented areas, and determining at least one metadata information unit corresponding to the plurality of segmented areas; The data frame to be displayed and metadata are sent to a display end device; the metadata includes the at least one metadata information unit.

13. The method according to claim 12, It is characterized in that The method further comprises: Determining a first weighted intensity value based on the number of pixels respectively included in the plurality of segmented areas; and / or, Determining a second weighted strength value based on preset feature values ​​respectively corresponding to the plurality of segmented regions; The first weighted intensity value and / or the second weighted intensity value are associated with the metadata.

14. The method according to claim 13, It is characterized in that The preset characteristic value is one of the following characteristic values: a brightness characteristic value and a color characteristic value.

15. The method according to claim 14, It is characterized in that The number of the preset characteristic values ​​is at least one.

16. The method according to claim 12, It is characterized in that The metadata information unit includes parameters of a tone mapping curve corresponding to the segmented area.

17. The method according to any one of claims 12 to 16, It is characterized in that The metadata further includes: preset size information; the preset size information is used to indicate the preset size of the preset shape used when selecting a preset area for pixels; or, The metadata also includes: preset shape and preset size information.

18. The method according to claim 12, It is characterized in that The step of determining at least one metadata information unit corresponding to the plurality of segmented regions comprises: Selecting pixels at a plurality of preset positions from the segmented area, and obtaining coordinate information of the pixels at the plurality of preset positions; The coordinate information is associated with the metadata information unit.

19. The method according to claim 12, It is characterized in that The metadata information unit also includes: a segmentation area identifier and a scanning order corresponding to the metadata information unit; The corresponding segmented area identifiers and the scanning order are used by the display terminal device to determine the corresponding relationship between the multiple segmented areas and the at least one metadata information unit.

20. The method according to any one of claims 12 to 19, It is characterized in that The plurality of segmented regions and the at least one metadata information unit are in a one-to-one correspondence relationship or a many-to-one correspondence relationship.

21. The method according to claim 12, It is characterized in that The method further comprises: The target segmentation method is associated with the metadata to instruct the display terminal device to segment the to-be-displayed data frame using the target segmentation method.

22. A display terminal device, It is characterized in that include: A non-volatile memory and a processor coupled to each other, wherein the processor calls a program code stored in the memory to execute the method according to any one of claims 1 to 11.

23. A collection terminal device, It is characterized in that include: A non-volatile memory and a processor coupled to each other, wherein the processor calls a program code stored in the memory to execute the method according to any one of claims 12 to 21.

24. A tone mapping system, It is characterized in that It comprises the display end device as described in claim 22 and the acquisition end device as described in claim 23.

25. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, which, when executed on a processor, implements the method of any one of claims 1-11 or claims 12-21.

Images