A tone mapping method, apparatus and system

Abstract

This application provides a tone mapping method, apparatus, and system to provide a technical solution that can improve tone mapping effect. The method involves acquiring an image frame to be displayed and metadata, the metadata including at least one metadata information unit; segmenting the image frame to be displayed into multiple segmented regions using a target segmentation method, and determining the correspondence between the multiple segmented regions and at least one metadata information unit. For the i-th pixel: for multiple pixels contained in a preset region, determining at least one related segmented region to which the multiple pixels belong, and determining the related metadata information unit corresponding to each related segmented region according to the correspondence; obtaining the sub-tone mapping value corresponding to each related segmented region based on the related metadata information unit, and assigning a corresponding weight factor to each related segmented region; obtaining the tone mapping value based on the sub-tone mapping value and weight factor corresponding to each related segmented region; and performing tone mapping based on the tone mapping value.

Description

Technical Field

[0001] This application relates to the field of display technology, and in particular to a tone mapping method, apparatus and system. Background Technology

[0002] With the development of acquisition equipment, it is now possible to acquire HDR video or images based on high dynamic range (HDR) video technology. HDR video or images expand the brightness range of displayable images, thus recording a larger range of brightness information, thereby revealing more details in both bright and dark areas of the image.

[0003] However, current display devices typically have limited display capabilities, and cannot adequately display the brightness range information recorded by the acquisition devices for HDR video or images. For example, the maximum brightness that an acquisition device can acquire is 10,000 candela per square meter (cd / m²). 2 The brightness unit is 4000 cd / m² (pixels), while the display device can only display a maximum brightness of 4000 cd / m². 2 If the pixel count is 4000 cd / m², then the brightness in the HDR video or image captured by the acquisition device is 4000 cd / m². 2 ~10000cd / m 2 Some pixels may not display well. Related technologies describe a method of performing tone mapping (TM) on the image to be displayed, which can tone map high dynamic range images onto display devices with low dynamic range display capabilities.

[0004] Traditional tone mapping methods mainly employ techniques based on tone mapping curves or block-based local tone mapping, but these methods suffer from unsatisfactory tone mapping results. Therefore, improving tone mapping performance is a worthwhile area of ​​research. Summary of the Invention

[0005] This application provides a tone mapping method, apparatus, and system to provide a technical solution that can improve tone mapping effects.

[0006] In a first aspect, embodiments of this application provide a tone mapping method applied to a display device. The method includes: acquiring an image frame to be displayed and metadata, the metadata including at least one metadata information unit; segmenting the image frame to be displayed into multiple segmented regions using a target segmentation method, and determining the correspondence between the multiple segmented regions and the at least one metadata information unit. For the i-th pixel in the image frame to be displayed, the following steps are performed: based on multiple pixels included in a preset region selected from the i-th pixel, determining at least one related segmented region to which the multiple pixels belong, and determining the related metadata information unit corresponding to each related segmented region according to the correspondence; obtaining a sub-tone mapping value corresponding to each related segmented region according to the related metadata information unit, and assigning a corresponding weight factor to each related segmented region; obtaining the tone mapping value of the i-th pixel based on the sub-tone mapping value and the weight factor corresponding to each related segmented region; and performing tone mapping on the i-th pixel according to the tone mapping value. Wherein, i takes any positive integer from 1 to N, and N is the number of pixels contained in the image frame to be displayed.

[0007] This method provides a technical solution for performing localized, refined tone mapping on an image frame to be displayed. By performing localized tone mapping on the pixels within the image frame based on a preset region, the display effect of the tone-mapped image frame can be improved on the display device.

[0008] In one possible design, assigning corresponding weight factors to each of the relevant segmented regions includes one or a combination of the following methods:

[0009] Method 1: Determine the number of pixels belonging to the preset region and the total number of pixels corresponding to the preset region in each of the relevant segmentation regions; for the first relevant segmentation region, perform the following: assign a corresponding first weight factor to the first relevant segmentation region based on the number of pixels belonging to the preset region and the total number of pixels in the first relevant segmentation region, wherein the first relevant segmentation region is any one of the at least one relevant segmentation region;

[0010] Method 2: Determine the preset feature value corresponding to the preset region in each of the relevant segmentation regions; for the second relevant segmentation region, perform the following: based on the feature difference between the preset feature value corresponding to the preset region in the second relevant segmentation region and the preset feature value of the i-th pixel, assign a corresponding second weight factor to the second relevant segmentation region, wherein the second relevant segmentation region is any one of the at least one relevant segmentation region.

[0011] In this design, based on the local features of different related segmented regions within a preset area corresponding to a pixel, such as the number of pixels or preset feature values, corresponding weight factors can be assigned to different related segmented regions. Through the allocation of weight factors, fine-grained tone mapping of pixels contained in the image frame to be displayed can be achieved. Thus, tone mapping can be realized based on the image features of the image frame to be displayed, thereby improving the display effect of the display device on the tone-mapped image frame.

[0012] In one possible design, the metadata further includes one or a combination of the following: a first weighted intensity value and a second weighted intensity value. The first weighted intensity value is used to adjust the first weighting factor based on the pixel distribution of the image frame to be displayed; the second weighted intensity value is used to adjust the second weighting factor based on changes in preset feature values ​​of the image frame to be displayed.

[0013] In this design, based on the global features of the image frames to be displayed, the weighting factors assigned to different segmented regions can be adjusted using weighted intensity values. For example, when the image features of the image frames to be displayed are similar, a larger weighting factor can be assigned; when the image features of the image frames to be displayed are quite different, a smaller weighting factor can be assigned. This reduces the boundary effects that tone mapping may produce and improves the display effect of the tone-mapped image frames on the display device.

[0014] In one possible design, the preset feature value is one of the following: a brightness feature value or a color feature value. The number of preset feature values ​​is at least one. In this design, the preset feature value can be either a brightness feature value or a color feature value, and in actual implementation, it can be determined based on the image frame to be displayed. For example, if the brightness feature is more important for the image frame to be displayed, a brightness feature value can be selected; if the color feature is more prominent, a color feature value can be selected. Furthermore, this design does not limit the number of preset feature values, thereby improving the tone mapping effect of the image frame to be displayed.

[0015] In one possible design, obtaining the sub-tone mapping value corresponding to each of the relevant segmented regions based on the relevant metadata information unit includes: the relevant metadata information unit contains parameters of the tone mapping curve, and the sub-tone mapping value corresponding to the i-th pixel in the relevant segmented region is obtained based on the tone mapping curve.

[0016] In this design, each relevant segmented region can correspond to a tone mapping curve, thereby enabling tone mapping for each relevant segmented region to obtain multiple sub-tone mapping values. This improves the accuracy of the sub-tone mapping values, resulting in a more accurate tone mapping effect.

[0017] In one possible design, the preset region selected based on the i-th pixel is: a region of a preset shape and a preset size selected with the i-th pixel as the center; or: a region centered on the i-th pixel and determined according to the preset shape and the preset size information contained in the metadata; or: a region centered on the i-th pixel and determined according to the preset shape and the preset size information contained in the metadata.

[0018] In this design, by selecting a preset area centered on the pixel to be processed (the i-th pixel), and then performing local fine-tuned tone mapping on the pixel to be processed based on the multiple pixels contained in the preset area, the display effect of the display device on the tone-mapped image frame to be displayed can be improved.

[0019] In one possible design, determining the correspondence between the plurality of segmented regions and the at least one metadata information unit includes: for any segmented region, selecting a pixel at a preset position within the segmented region and obtaining the coordinate information of the pixel at the preset position; determining the metadata information unit corresponding to any segmented region based on the coordinate information; and establishing a correspondence between any segmented region and the corresponding metadata information unit. Alternatively, for any metadata information unit, determining at least one segmented region corresponding to the one or more coordinate information pieces contained in the metadata information unit; and establishing a correspondence between any metadata information unit and the corresponding at least one segmented region.

[0020] In this design, the coordinate information of the preset position is used as an index, which can accurately and easily determine the correspondence between the segmented region and the metadata information unit. This allows for the calculation of the sub-tone mapping value of each relevant segmented region based on the relevant metadata information unit corresponding to the relevant segmented region of the pixel to be processed.

[0021] In one possible design, determining the correspondence between the plurality of segmented regions and the at least one metadata information unit includes: the metadata information unit contains a corresponding segmented region identifier; traversing the plurality of segmented regions according to the target scanning order and the segmented region identifier to determine the metadata information unit corresponding to each segmented region; wherein the target scanning order is a preset scanning order or a scanning order indicated in the metadata.

[0022] In this design, the target scanning order can also be used to determine the correspondence between the segmented regions and the metadata information units more accurately and easily. This allows for the calculation of the sub-tone mapping values ​​of each relevant segmented region based on the relevant metadata information units corresponding to the relevant segmented regions of the pixels to be processed.

[0023] In one possible design, the plurality of segmented regions and the at least one metadata information unit have a one-to-one correspondence or a many-to-one correspondence. In this design, the segmented regions and metadata information units can have not only a one-to-one correspondence but also a many-to-one correspondence. Through the many-to-one correspondence, the amount of metadata information that needs to be transmitted between the acquisition end device and the display end device can be reduced, thereby improving the transmission efficiency of metadata.

[0024] In one possible design, the target segmentation method is a preset segmentation method or a segmentation method indicated in the metadata. In this design, the target segmentation method can be pre-stored by both the acquisition device and the display device, or it can be a segmentation method used by the display device that is indicated by the acquisition device, thereby ensuring the accuracy of tone mapping between the acquisition device and the display device.

[0025] Secondly, embodiments of this application provide a method for generating metadata, applied to a data acquisition device. The method includes: acquiring image data to obtain a data frame to be displayed; segmenting the data frame to be displayed into multiple segmented regions using a target segmentation method, and determining at least one corresponding metadata information unit for each of the multiple segmented regions; sending the data frame to be displayed and the metadata to a display device; the metadata includes the at least one metadata information unit.

[0026] In one possible design, the method further includes: determining a first weighted intensity value based on the number of pixels contained in each of the plurality of segmented regions; and / or determining a second weighted intensity value based on preset feature values ​​corresponding to each of the plurality of segmented regions; and associating the first weighted intensity value and / or the second weighted intensity value with the metadata.

[0027] In one possible design, the preset feature value is one of the following feature values: brightness feature value, color feature value. The number of preset feature values ​​is at least one.

[0028] In one possible design, the metadata information unit includes parameters of the tone mapping curve corresponding to the segmented region.

[0029] In one possible design, the metadata may further include: preset size information; the preset size information is used to indicate the preset size of the preset shape used when selecting a preset area for a pixel; or, the metadata may further include: preset shape and preset size information.

[0030] In one possible design, determining at least one metadata information unit corresponding to the plurality of segmented regions includes: selecting pixels at multiple preset positions from the segmented regions, obtaining coordinate information of the pixels at the multiple preset positions, and associating the coordinate information with the metadata information unit.

[0031] In one possible design, the metadata information unit further includes: a segmentation region identifier and a scanning order corresponding to the metadata information unit; wherein the corresponding segmentation region identifier and the scanning order are used by the display device to determine the correspondence between the plurality of segmentation regions and the at least one metadata information unit.

[0032] In one possible design, the plurality of segmented regions and the at least one metadata information unit have a one-to-one correspondence or a many-to-one correspondence.

[0033] In one possible design, the method further includes: associating the target segmentation method with the metadata to instruct the display device to segment the data frame to be displayed using the target segmentation method.

[0034] Thirdly, embodiments of this application provide a tone mapping processing apparatus, which includes modules for performing the methods described in the first aspect or any possible implementation of the first aspect.

[0035] For example, the device may include: an acquisition module, a processing module, and a tone mapping module. The acquisition module is used to acquire an image frame to be displayed and metadata, the metadata including at least one metadata information unit. The processing module is used to segment the image frame to be displayed into multiple segmentation regions using a target segmentation method, and to determine the correspondence between the multiple segmentation regions and the at least one metadata information unit. The tone mapping module is used to perform the following for the i-th pixel in the image frame to be displayed: based on multiple pixels contained in a preset region selected from the i-th pixel, determine at least one related segmentation region to which the multiple pixels belong, and determine the related metadata information unit corresponding to each related segmentation region according to the correspondence; obtain a sub-tone mapping value corresponding to each related segmentation region according to the related metadata information unit, and assign a corresponding weight factor to each related segmentation region; obtain the tone mapping value of the i-th pixel based on the sub-tone mapping value and the weight factor corresponding to each related segmentation region; and perform tone mapping on the i-th pixel according to the tone mapping value; wherein i takes any positive integer from 1 to N, and N is the number of pixels contained in the image frame to be displayed.

[0036] In one possible design, the tone mapping module, when assigning corresponding weight factors to each of the relevant segmentation regions, specifically performs the following steps: determining the number of pixels belonging to the preset region and the total number of pixels corresponding to the preset region in each of the relevant segmentation regions; for a first relevant segmentation region, performing the following: assigning a corresponding first weight factor to the first relevant segmentation region based on the number of pixels belonging to the preset region and the total number of pixels in the first relevant segmentation region, wherein the first relevant segmentation region is any one of the at least one relevant segmentation region; and / or, determining a preset feature value corresponding to the preset region in each of the relevant segmentation regions; for a second relevant segmentation region, performing the following: assigning a corresponding second weight factor to the second relevant segmentation region based on the feature difference between the preset feature value corresponding to the preset region in the second relevant segmentation region and the preset feature value of the i-th pixel, wherein the second relevant segmentation region is any one of the at least one relevant segmentation region.

[0037] In one possible design, the metadata may further include one or a combination of the following: a first weighted intensity value and a second weighted intensity value; wherein the first weighted intensity value is used to adjust the first weighting factor according to the pixel distribution of the image frame to be displayed; and the second weighted intensity value is used to adjust the second weighting factor according to the changes in preset feature values ​​of the image frame to be displayed.

[0038] In one possible design, the preset feature value is one of the following feature values: brightness feature value, color feature value. The number of preset feature values ​​is at least one.

[0039] In one possible design, the tone mapping module is used to obtain the sub-tone mapping value corresponding to each of the relevant segmented regions based on the relevant metadata information unit. Specifically, the relevant metadata information unit includes parameters of the tone mapping curve, and the sub-tone mapping value corresponding to the i-th pixel in the relevant segmented region is obtained based on the tone mapping curve.

[0040] In one possible design, the preset region selected based on the i-th pixel is: a region of a preset shape and a preset size selected with the i-th pixel as the center; or: a region centered on the i-th pixel and determined according to the preset shape and the preset size information contained in the metadata; or: a region centered on the i-th pixel and determined according to the preset shape and the preset size information contained in the metadata.

[0041] In one possible design, the processing module is used to determine the correspondence between the plurality of segmented regions and the at least one metadata information unit, specifically by: for any segmented region, selecting a pixel at a preset position from any segmented region, and obtaining the coordinate information of the pixel at the preset position; determining the metadata information unit corresponding to any segmented region based on the coordinate information; establishing a correspondence between any segmented region and the corresponding metadata information unit; or, for any metadata information unit, determining at least one segmented region corresponding to the one or more coordinate information contained in the metadata information unit; establishing a correspondence between any metadata information unit and the corresponding at least one segmented region.

[0042] In one possible design, the processing module is used to determine the correspondence between the plurality of segmented regions and the at least one metadata information unit, including: the metadata information unit contains a corresponding segmented region identifier; the plurality of segmented regions are traversed according to the target scanning order and the segmented region identifier to determine the metadata information unit corresponding to each segmented region; wherein the target scanning order is a preset scanning order or a scanning order indicated in the metadata.

[0043] In one possible design, the plurality of segmented regions and the at least one metadata information unit have a one-to-one correspondence or a many-to-one correspondence.

[0044] In one possible design, the target segmentation method is a preset segmentation method or a segmentation method indicated in the metadata.

[0045] Fourthly, embodiments of this application provide a tone mapping apparatus, which includes modules for performing the methods described in the second aspect or any possible implementation of the second aspect.

[0046] For example, the device may include: an acquisition module, a processing module, and a transceiver module. The acquisition module is used to acquire image data to obtain a data frame to be displayed; the processing module is used to segment the data frame to be displayed into multiple segmented regions using a target segmentation method, and to determine at least one corresponding metadata information unit for each of the multiple segmented regions; the transceiver module is used to send the data frame to be displayed and the metadata to a display device; the metadata includes the at least one metadata information unit.

[0047] In one possible design, the processing module is further configured to determine a first weighted intensity value based on the number of pixels contained in each of the plurality of segmented regions; and / or determine a second weighted intensity value based on preset feature values ​​corresponding to each of the plurality of segmented regions; and associate the first weighted intensity value and / or the second weighted intensity value with the metadata.

[0048] In one possible design, the preset feature value is one of the following feature values: brightness feature value, color feature value. The number of preset feature values ​​is at least one.

[0049] In one possible design, the metadata information unit includes parameters of the tone mapping curve corresponding to the segmented region.

[0050] In one possible design, the metadata may further include: preset size information; the preset size information is used to indicate the preset size of the preset shape used when selecting a preset area for a pixel; or, the metadata may further include: preset shape and preset size information.

[0051] In one possible design, the processing module is used to determine at least one corresponding metadata information unit for the plurality of segmented regions, specifically for selecting pixels at multiple preset positions from the segmented regions, obtaining coordinate information of the pixels at the multiple preset positions, and associating the coordinate information with the metadata information unit.

[0052] In one possible design, the metadata information unit further includes: a segmentation region identifier and a scanning order corresponding to the metadata information unit; wherein the corresponding segmentation region identifier and the scanning order are used by the display device to determine the correspondence between the plurality of segmentation regions and the at least one metadata information unit.

[0053] In one possible design, the plurality of segmented regions and the at least one metadata information unit have a one-to-one correspondence or a many-to-one correspondence.

[0054] In one possible design, the processing module is further configured to associate the target segmentation method with the metadata, so as to instruct the display device to segment the data frame to be displayed using the target segmentation method.

[0055] Fifthly, embodiments of this application provide a display device, including: a non-volatile memory and a processor coupled to each other, wherein the processor calls program code stored in the memory to execute the method as described in the first aspect or any possible implementation thereof.

[0056] In a sixth aspect, embodiments of this application provide a data acquisition device, comprising: a non-volatile memory and a processor coupled to each other, wherein the processor calls program code stored in the memory to execute the method as described in the second aspect or any possible implementation thereof.

[0057] In a seventh aspect, embodiments of this application provide a computer-readable storage medium including a program or instructions that, when executed on a computer, cause a method as described in the first aspect or any possible implementation thereof to be performed, or cause a method as described in the second aspect or any possible implementation thereof to be performed.

[0058] It should be understood that the technical effects that can be achieved by aspects two through seven can be specifically described with reference to the technical effects brought about by aspect one or any possible implementation of aspect one above, and will not be repeated here. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the PQ photoelectric transfer function relationship;

[0060] Figure 2 This is a schematic diagram of the HLG photoelectric transfer function relationship;

[0061] Figure 3 This is a schematic diagram of the photoelectric transfer function relationship of SLF;

[0062] Figure 4 This is a schematic diagram of a possible system architecture applicable to the embodiments of this application;

[0063] Figure 5 A schematic flowchart illustrating a tone mapping method provided in an embodiment of this application;

[0064] Figure 6 The diagram shown illustrates the segmentation method A provided for implementation of this application.

[0065] Figure 7 The diagram shown illustrates the segmentation method B provided for implementation of this application.

[0066] Figure 8 The diagram shown illustrates the segmentation method C provided for implementation of this application.

[0067] Figure 9 The diagram shown illustrates a scanning sequence provided in an embodiment of this application.

[0068] Figure 10 The diagram shown is a schematic diagram of determining the relevant segmentation region provided in an embodiment of this application;

[0069] Figure 11The diagram shown is another schematic diagram illustrating the determination of relevant segmented regions according to an embodiment of this application;

[0070] Figure 12 The diagram shown is a schematic representation of a brightness characteristic value provided in an embodiment of this application.

[0071] Figure 13 This is another schematic flowchart of a tone mapping method provided in an embodiment of this application;

[0072] Figure 14 This is a schematic diagram of the structure of a tone mapping device provided in an embodiment of this application;

[0073] Figure 15 This is another schematic diagram of a tone mapping device provided in an embodiment of this application;

[0074] Figure 16 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0075] In the embodiments of this application, the term "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where any one of a, b, c, ab, ac, bc, or abc can include a single a, a single b, a single c, or multiple a, multiple b, multiple c.

[0076] Furthermore, unless otherwise stated, the ordinal numbers such as "first" and "second" mentioned in the embodiments of this application are used to distinguish multiple objects and are not used to limit the order, sequence, priority, or importance of multiple objects. For example, the first priority criterion and the second priority criterion are only used to distinguish different criteria, and do not indicate that the content, priority, or importance of these two criteria are different.

[0077] Furthermore, the terms "comprising" and "having" in the embodiments, claims, and drawings of this application are not exclusive. For example, a process, method, system, product, or device that includes a series of steps or modules / units is not limited to the listed steps or modules, but may also include steps or modules / units not listed.

[0078] In the field of digital image display, dynamic range represents the ratio between the maximum and minimum grayscale values ​​within the displayable range of an image. Currently, in most color digital images, each of the R, G, and B channels uses one byte (8 bits) for storage. This means that each channel represents a grayscale range of 0 to 255 levels; this 0 to 255 is the image's dynamic range. However, in the real world, the dynamic range of the same scene is typically around 10. -3 Up to 10 6 Within this range, it can be called high dynamic range (HDR). In contrast to high dynamic range, the dynamic range of a regular image is called low dynamic range (LDR), which typically displays a dynamic range between 1 and 100. Therefore, it can be understood that the imaging process of a digital camera is actually a mapping from the high dynamic range of the real world to the low dynamic range of the photograph.

[0079] The greater the dynamic range of an image, the more scene details it displays, the richer the brightness levels, and the more realistic the visual effect.

[0080] The process of optical digital imaging (e.g., the imaging process of a digital camera, the video playback process) involves converting the light radiation of a real scene into electrical signals through an image sensor and saving it as a digital image. The purpose of image display is to reproduce the real scene described by a digital image through a display device. The ultimate goal of both is to provide users with the same visual perception as if they were directly observing a real scene. The brightness levels in a real scene that light radiation (light signals) can display are almost linear; therefore, light signals are also called linear signals. However, in the process of converting light signals into electrical signals in optical digital imaging, not every light signal corresponds to an electrical signal; the converted electrical signals are usually non-linear. Therefore, electrical signals are also called non-linear signals. The curve that converts light signals into electrical signals is called the optical electro-transfer function (OETF). The OETF involved in the embodiments of this application may include, but is not limited to: perceptual quantizer (PQ) OETF, hybrid log-gamma (HLG) OETF, scene luminance fidelity (SLF) OETF, etc.

[0081] The PQ photoelectric transfer function is a perceptual quantization photoelectric transfer function proposed based on the human eye's brightness perception model. The PQ photoelectric transfer function represents the conversion relationship between linear signal values ​​of image pixels and nonlinear signal values ​​in the PQ domain. See also... Figure 1 The figure shows a schematic diagram of the PQ photoelectric transfer function relationship. The PQ photoelectric transfer function can be expressed as formula (1-1):

[0082]

[0083] The parameters in formula (1-1) are calculated as follows:

[0084]

[0085] in,

[0086] L represents a linear signal value, which is normalized to [0, 1].

[0087] L' represents the nonlinear signal value, and its value ranges from [0, 1].

[0088] m1 is the PQ photoelectric transfer coefficient.

[0089] m2 is the PQ photoelectric transfer coefficient.

[0090] c1 is the PQ photoelectric transfer coefficient.

[0091] c2 is the PQ photoelectric transfer coefficient.

[0092] c3 is the PQ photoelectric transfer coefficient.

[0093] The HLG photoelectric transfer function is an improvement upon the traditional Gamma curve. The HLG photoelectric transfer function uses the traditional Gamma curve in the lower range and supplements it with a log curve in the higher range. See also... Figure 2 This is a schematic diagram of the HLG photoelectric transfer function relationship. The HLG photoelectric transfer function represents the conversion relationship between the linear signal value of an image pixel and the nonlinear signal value in the HLG domain. The HLG photoelectric transfer function can be expressed as equation (1-2):

[0094]

[0095] Where L represents the linear signal value, with a range of [0, 12]. L' represents the nonlinear signal value, with a range of [0, 1]. a = 0.17883277 represents the HLG photoelectric transfer coefficient. b = 0.28466892 represents the HLG photoelectric transfer coefficient. c = 0.55991073 represents the HLG photoelectric transfer coefficient.

[0096] The SLF photoelectric transfer function is the optimal curve obtained based on the brightness distribution of an HDR scene, while satisfying the optical characteristics of the human eye. (See also...) Figure 3 , Figure 3 This is a schematic diagram of the photoelectric transfer function relationship of SLF.

[0097] The SLF photoelectric transfer curve represents the conversion relationship between the linear signal value of an image pixel and the nonlinear signal value in the SLF domain. The conversion relationship between the linear signal value of an image pixel and the nonlinear signal value in the SLF domain is shown in formula (1-3):

[0098]

[0099] The SLF photoelectric transfer function can be expressed as formula (1-4):

[0100]

[0101] in:

[0102] L represents the linear signal value, normalized to [0, 1], and L' represents the nonlinear signal value, ranging from [0, 1]. p = 2.3 represents the SLF photoelectric transfer coefficient. m = 0.14 represents the SLF photoelectric transfer coefficient. a = 1.12762 represents the SLF photoelectric transfer coefficient. b = -0.12762 represents the SLF photoelectric transfer coefficient.

[0103] Currently, acquisition devices can capture HDR videos or images with a wider dynamic range (the following examples use images as an example; video can be understood as multiple frames of images). However, the display capabilities of display devices are limited and cannot be well matched with the dynamic range captured by the acquisition device. Related technologies suggest using tone mapping (TM) on the image to be displayed, thereby achieving image matching across different dynamic ranges. For example, tone mapping can be divided into high dynamic range to low dynamic range mapping or low dynamic range to high dynamic range mapping. For instance, if the HDR image captured by the acquisition device includes a 4000 nit (nit, unit of light signal) illumination signal, while the HDR display capability of the display device (such as a television or mobile phone) is only 500 nits, then the 4000 nit signal needs to be mapped to the 500 nit device, i.e., a high-to-low tone mapping process. For example, an HDR image includes a 100-nit SDR signal, but the target display device only has a display capability of 2000 nits. Therefore, it is necessary to map the 100-nit signal to the 2000-nit device, which is a low-to-high tone mapping process.

[0104] Tone mapping is typically used to match high-resolution HDR images captured by acquisition devices with low-resolution HDR or SDR images displayed by display devices. See also... Figure 4 This is a schematic diagram of a possible system architecture applicable to an embodiment of this application. The system architecture of this embodiment may include a data acquisition device 401, a display device 402, and a transmission link 403. The display device 402 may further be divided into an HDR display device 402a and an SDR display device 402b.

[0105] The acquisition device 401 is used to acquire or create HDR video or images. An exemplary acquisition device 401 can be a video (or image) acquisition device or a video (or image) creation device. In one possible example, the acquisition device 401 can also generate metadata based on the HDR video or image. Metadata is used to record key information about the acquired scene or image in a frame. In this embodiment, the metadata may include parameters of the tone mapping curve, dynamic metadata, and static metadata. Dynamic metadata can be understood as data associated with each frame of the image, which can change depending on the image, such as the average, maximum, and minimum pixel brightness values ​​in the scene. Static metadata can be understood as data associated with an image sequence, which remains unchanged within the image sequence.

[0106] The acquisition device 401 can transmit HDR video or images, i.e., metadata, to the display device 402 via the transmission link 403. Specifically, the HDR video or images and metadata can be transmitted in the form of a single data packet or in two separate data packets; this embodiment does not impose any specific limitations.

[0107] Display device 402 can receive metadata and HDR video or images, determine the tone mapping curve based on the parameters of the tone mapping curve contained in the metadata, and perform tone mapping on the HDR image according to the tone mapping curve, converting it into display content adapted to display device 402. Display device 402 can be divided into HDR display device 402a and SDR display device 402b. It should be noted that the HDR range of display device 402 and the HDR range of acquisition device 401 can be different. Typically, the HDR range of display device 402 is lower than that of acquisition device 401; for example, the HDR range of acquisition device 401 is 10. -3 Up to 10 4 (This can also be referred to as "high-level HDR" in this embodiment), and the HDR range of the display device 402 is 10. -1 Up to 10 3 The range (also referred to as "low-level HDR" in this application embodiment). It should also be understood that, in different embodiments, the display device 402 may also include a display device with a higher dynamic range display capability than the HDR video or image generated by the acquisition device 401, and its display adaptation process can also be applied to this system architecture, which is not limited in this application.

[0108] Based on the background information, traditional tone mapping methods mainly employ tone mapping curves or block-based local tone mapping, but these methods suffer from poor tone mapping results. For example, tone mapping curve-based methods typically achieve tone mapping based on a single local window, which may lead to boundary effects when using a relatively simple shape for local processing.

[0109] In view of this, embodiments of this application provide a tone mapping method to design a technical solution that can improve tone mapping effects. The method provided by embodiments of this application will be described below with reference to the accompanying drawings.

[0110] Figure 5 This application provides a schematic flowchart of a tone mapping method applicable to a display device 402. Figure 5 The illustrated process includes:

[0111] Step 501: The display device 402 acquires the image frame to be displayed and metadata, wherein the metadata includes at least one metadata information unit. It should be understood that the image frame to be displayed can be an image, or any frame from multiple frames in a video. Furthermore, the image frame to be displayed can be data sent by the acquisition device 401; or it can be data obtained after other image processing procedures, such as global tone mapping. The metadata can be generated by the acquisition device 401 based on the acquired image frame to be displayed. The specific implementation method for generating metadata will be described later in the section introducing the processing flow of the acquisition device 401, and will not be detailed here.

[0112] Step 502: The display device 402 uses a target segmentation method to segment the image frame to be displayed into multiple segmentation regions, and determines the correspondence between the multiple segmentation regions and the at least one metadata information unit.

[0113] The target segmentation method can be a preset segmentation method or a segmentation method indicated in the metadata. Optionally, the preset segmentation method can be pre-set by the display device 402; it should be understood that, in order to obtain accurate metadata, the acquisition device 401 and the display device 402 can be pre-set with the same preset segmentation method in this scenario. Alternatively, to ensure that the acquisition device 401 and the display device 402 use the same segmentation method, the target segmentation method of the display device 402 can also be indicated by the acquisition device 401 in the metadata.

[0114] For example, the embodiments of this application provide the following possible segmentation methods. It should be understood that the following segmentation methods are not limited in implementation.

[0115] In segmentation method A, the display device 402 divides the image frame to be displayed into multiple rectangular segmentation regions according to the first segmentation parameters corresponding to the first and second directions, respectively. The first direction is perpendicular to the second direction. Optionally, the first segmentation parameter can be the number of segments, in which case the multiple segmentation regions are represented as rectangular regions of equal size.

[0116] like Figure 6 The diagram shown illustrates a segmentation method A provided for implementation of this application. The first direction can be horizontal, and the second direction can be vertical. Let M1 represent the number of segments in the first direction (M1 is typically ≤16), and M2 represent the number of segments in the second direction (M2 is typically ≤16). Figure 6Taking M1 and M2 as an example, the image frame to be displayed can be divided into 16 regions, that is, 16 rectangular regions of equal size. It should be understood that the values ​​of M1 and M2 can be the same or different. For example, M1 can be 6 and M2 can be 4.

[0117] In segmentation method B, the display device 402 divides the image frame to be displayed into multiple rectangular segmentation regions according to the second segmentation parameters corresponding to the first and second directions, respectively. Optionally, the second segmentation parameters can be a segmentation list, in which case the multiple segmentation regions obtained are represented as rectangular regions of arbitrary size.

[0118] like Figure 7 The diagram shown illustrates the segmentation method B provided for implementation of this application. Assume the segmentation list in the first direction is represented by w[j1] (j1 can take values ​​from 1 to M1, where M1 represents the number of segments in the first direction), and the segmentation list in the second direction is represented by h[j2] (j2 can take values ​​from 1 to M2, where M2 represents the number of segments in the second direction). Figure 7 In this example, M1 is represented by a value of 4, and M2 is also represented by a value of 4. Figure 7 Each pair of values ​​in w[1], w[2], w[3], and w[4] can be the same or different. Figure 7 Each pair of values ​​in h[1], h[2], h[3] and h[4] can be the same or different.

[0119] In segmentation method C, the display device 402 clusters the pixels contained in the image frame to be displayed, and divides the image frame to be displayed into multiple irregular segmentation regions based on the clustering results. Each irregular segmentation region includes at least one type of pixel.

[0120] like Figure 8 The diagram shown illustrates segmentation method C provided in this application. In implementing this application, a target clustering method can be selected from various optional clustering methods to perform clustering processing on the image frames to be displayed. The specific clustering method is not limited in this application. For example, the image frames to be displayed after clustering processing can be divided into... Figure 8 The image shows 16 segmented regions. For example, when clustering the image frame to be displayed, the display device 402 can first divide the image frame into multiple rectangular regions according to the first or second segmentation parameters described above; then determine the coordinate information and feature information (such as color component values) of the center pixel of each rectangular region; by performing clustering analysis on each pixel in the image frame to be displayed and each of the center pixels, a clustering result is obtained; finally, based on the clustering result, the following can be obtained: Figure 8 The diagram shows the segmentation.

[0121] Furthermore, to facilitate cluster analysis and reduce the processing workload, this application allows for image segmentation by incorporating a scaled-down version of the image frame to be displayed during the initial image segmentation process. The scaled-down image frame can be selected as 1 / N1 of the resolution in the first direction and 1 / N2 of the resolution in the second direction of the acquired image frame. This reduces the number of pixels involved in the clustering process by performing cluster analysis based on the scaled-down image frame.

[0122] It should be noted that, in implementing this application, appropriate segmentation methods can be selected for images with different characteristics. For example, segmentation method A can be applied to image frames whose local features are not obvious, such as solid color image frames or regular image frames; it can also be applied to any image frame, and this application does not limit this. Segmentation method C can be applied to image frames with certain local features. It is understood that the display device 402 and / or the acquisition device 401 can pre-store multiple segmentation methods, and the corresponding target segmentation method can be further selected based on image frames with different characteristics.

[0123] After obtaining multiple segmented regions, the display device 402 determines the metadata information unit corresponding to each segmented region based on the metadata including at least one metadata information unit. This allows it to obtain parameters of the tone mapping curve, dynamic metadata, and static metadata corresponding to the segmented region from the metadata information unit. Optionally, the multiple segmented regions and the at least one metadata information unit can have a one-to-one correspondence; that is, each segmented region can correspond to one metadata information unit. Figure 8 The 16 segmented regions can correspond to 16 metadata information units. Alternatively, the multiple segmented regions and the at least one metadata information unit can also have a many-to-one correspondence; that is, one or more segmented regions can correspond to one metadata information unit; such as... Figure 8 In the diagram, segmented region 1 and segmented region 2 can correspond to one metadata information unit, segmented region 3 and segmented region 4 can correspond to one metadata information unit, and so on. It can be understood that by establishing a many-to-one relationship, the amount of metadata transmitted between the acquisition device 401 and the display device 402 can be reduced, thereby improving data transmission efficiency.

[0124] For example, embodiments of this application provide the following possible examples of determining the correspondence between the plurality of segmented regions and the at least one metadata information unit. It should be understood that the following examples are not limited in implementation.

[0125] Example 1: The acquisition device 401 can indicate the corresponding segmentation region identifier in the metadata information unit.

[0126] Optionally, the segmentation region identifier can be the number of segmentation regions contained in the metadata information unit. In implementation, the display device 402 can traverse multiple segmentation regions contained in the image frame to be displayed based on the target scanning order and the number of segmentation regions. The target scanning order includes, but is not limited to, left-to-right followed by top-to-bottom, top-to-bottom followed by left-to-right, etc.; and the target scanning order can be a preset scanning order or a scanning order indicated in the metadata. It should be understood that the display device 402 and the acquisition device 401 use the same preset scanning order.

[0127] For example, Figure 9 The diagram shown illustrates a scanning sequence according to an embodiment of this application, where the scanning order is from left to right and then from top to bottom. Assuming... Figure 9 The 16 corresponding segmented regions correspond to 3 metadata information units. Specifically, the first metadata information unit corresponds to 4 segmented regions, the second metadata information unit corresponds to 7 segmented regions, and the third metadata information unit corresponds to 5 segmented regions. The correspondence between the multiple segmented regions and the at least one metadata information unit can be shown in Table 1-1 below:

[0128] Table 1-1

[0129] Segmentation Marker Metadata information unit identifier 4 The first metadata information unit 7 The second metadata information unit 5 The third metadata information unit

[0130] Based on the correspondence shown in Table 1-1 above, the display device 402 can determine the segmentation region corresponding to the first metadata information unit as segmentation region 1 to segmentation region 4, the segmentation region corresponding to the second metadata information unit as segmentation region 5 to segmentation region 11, and the segmentation region corresponding to the third metadata information unit as segmentation region 12 to segmentation region 16 based on the target scanning sequence.

[0131] Alternatively, the segmentation region identifier can also be segmentation region sequence information. In implementation, the display device 402 can determine the segmentation region corresponding to the metadata information unit based on the segmentation region sequence information contained in the metadata information unit.

[0132] For example, continue to use Figure 9 The content shown is illustrated below. The correspondence between the multiple segmented regions and the at least one metadata information unit in this example can be shown in Table 1-2 below:

[0133] Table 1-2

[0134] Segmentation Marker Metadata information unit identifier Segmentation region 1 to segmentation region 4 The first metadata information unit Segmentation region 5 to segmentation region 11 The second metadata information unit Segmentation region 12 to segmentation region 16 The third metadata information unit

[0135] Based on the correspondence shown in Table 1-2 above, the display device 402 can directly determine the corresponding segmented region based on the segmented region sequence information contained in the metadata information unit.

[0136] Example 2: The display device 402 can also select one or more pixels at preset positions within any of the segmented regions, and obtain the coordinate information of the pixels at the one or more preset positions; based on the coordinate information, determine the metadata information unit corresponding to any of the segmented regions; thereby establishing a correspondence between any of the segmented regions and the corresponding metadata information unit. The preset positions can include one or more of the following positions: top left corner, center, bottom right corner, top left corner, and bottom right corner. For example, for segmented region 1, the preset positions can be the top left corner, center, and bottom right corner, with the coordinates of the top left corner being (x1, y1), the center being (x2, y2), and the bottom right corner being (x3, y3); then, a metadata information unit containing (x1, y1), (x2, y2), and (x3, y3) can be determined, thereby obtaining the metadata information unit corresponding to segmented region 1.

[0137] It should be understood that the display device 402 may also, for any of the metadata information units, determine at least one segmented region corresponding to the one or more coordinate information contained in the metadata information unit; and establish a correspondence between any of the metadata information units and the corresponding at least one segmented region.

[0138] For the i-th pixel in the image frame to be displayed (where i takes any positive integer from 1 to N, and N is the number of pixels in the image frame to be displayed), the display device 402 performs the following steps 503 to 505 (it should be understood that in the following embodiments, any pixel is used as an example, and the processing of other pixels in the image frame to be displayed is similar, and will not be described in detail in the following embodiments), as follows:

[0139] Step 503: Based on the multiple pixels included in the preset area selected by the i-th pixel, determine at least one related segmentation region to which the multiple pixels belong, and determine the related metadata information unit corresponding to each of the related segmentation regions according to the correspondence relationship.

[0140] The preset region selected based on the i-th pixel is: a region of a preset size and preset shape centered on the i-th pixel. For example, the preset shape can be a rectangle or other shapes with a preset width and preset length. Alternatively, it can be: a region centered on the i-th pixel, determined according to the preset shape and preset size information contained in the metadata. If the preset shape is a rectangle, the preset size information can be the length and width of the rectangle; if the preset shape is a circle, the preset size information can be the radius or diameter of the circle. Or, it can be: a region centered on the i-th pixel, determined according to the preset shape and preset size information contained in the metadata. In other words, the preset shape and / or preset size information of the preset shape used by the display device 402 to select the preset region for the i-th pixel can be information stored locally or information indicated in the metadata.

[0141] For example, Figure 10 The diagram shown is a schematic diagram of determining the relevant segmentation region provided in an embodiment of this application, based on... Figure 6 The image frames to be displayed are obtained using the segmentation method employed. Combined with... Figure 10 As shown in (a) and (b), when processing pixel 1, a preset rectangular area centered on pixel 1 contains multiple pixels. These multiple pixels belong to segmentation region 1, segmentation region 2, segmentation region 5, and segmentation region 6, respectively. Therefore, it can be determined that pixel 1 has four related segmentation regions. Combined with... Figure 10 As shown in (a) and (c), when processing pixel 2, the multiple pixels within the preset rectangular area centered on pixel 2 belong to segmentation regions 6, 7, 8, 10, 11, 12, 14, 15, and 16, respectively. Therefore, it can be determined that pixel 2 has nine related segmentation regions. It should be understood that the related segmentation regions of a pixel are determined based on the pixel position and the preset regions. The number of related segmentation regions for different pixels contained in the image frame to be displayed can be the same or different. It should be noted that multiple pixels contained in the image frame to be displayed can usually use preset regions with the same preset size information, but this application does not limit this. In practice, preset regions with different preset size information can also be used, such as... Figure 10 The pixels shown, Pixel 1 and Pixel 2, use different preset size information.

[0142] For example, Figure 11 The diagram shown is another schematic diagram of determining the relevant segmented region provided in the embodiments of this application, based on Figure 8 The image frame to be displayed is obtained using the segmentation method employed. When processing pixel 3, multiple pixels within a preset rectangular area centered on pixel 3 are respectively divided into regions 9, 10, 13, 14, and 15, thus determining that the number of relevant segmented regions for pixel 3 is 5.

[0143] Based on the fact that the display device 402 can obtain the correspondence between multiple segmented regions and at least one metadata information unit in step 502, after determining the relevant segmented region of the pixel, the key information contained in the relevant segmented region, such as the parameters of the tone mapping curve, dynamic metadata and static metadata, can be further obtained from the metadata information unit corresponding to the relevant segmented region to realize tone mapping of the pixel to be processed.

[0144] Step 504: The display device 402 obtains the sub-tone mapping value corresponding to each of the relevant segmented regions based on the relevant metadata information unit, and assigns a corresponding weight factor to each of the relevant segmented regions.

[0145] For example, the relevant metadata information unit includes parameters of the tone mapping curve, and the display device 402 can obtain the sub-tone mapping value corresponding to the i-th pixel in the relevant segmented region based on the tone mapping curve. It can be understood that the number of sub-tone mapping values ​​for the i-th pixel depends on the number of relevant segmented regions; for example, for... Figure 10 The illustrated pixel 1 has four corresponding segmented regions, thus yielding four sub-tone mapping values ​​for pixel 1; for Figure 10 The pixel 2 shown has 9 related segmented regions, so the 9 sub-tone mapping values ​​corresponding to pixel 2 can be obtained.

[0146] In this application, the parameters of the tone mapping curves contained in the relevant data information units described in the above examples are not limited. For example, the parameters of the tone mapping curves can be determined based on the selected tone mapping curve shape. The tone mapping curve shape includes, but is not limited to, sigmoid function curves, cubic (or multi-dimensional) spline function curves, Bézier curves, gamma function curves, etc.

[0147] One embodiment of obtaining a tone mapping curve based on parameters of a tone mapping curve is that the tone mapping curve is a cubic spline function. In implementation of this application, the display device 402 determining the tone mapping curve of any relevant segmented region may include the following steps:

[0148] A1. Obtain the maximum brightness value (maxLum) from the relevant metadata information unit corresponding to the relevant segmented region.

[0149] A2. Determine the interval length (Len) of the cubic spline based on the maxLum and the cubic spline interval information (Num3).

[0150] Wherein, Len can be expressed as formula (2-1):

[0151]

[0152] A3. Based on the Len and the interval index (index, used to identify the first interval), obtain the start position (TH[index-1]) and end position (TH[index]) of the first interval; the first interval is any interval among the multiple intervals contained in the cubic spline. TH[index] can be expressed as formula (2-2):

[0153] TH[index] = index * Len (2-2)

[0154] A4. Based on the cubic spline interval information, interval index, and cubic spline interval value, obtain the cubic spline parameters P0, P1, P2, and P3 of the first interval.

[0155] A4-1, obtain the initial interval slope of the first interval based on the cubic spline interval values ​​and information. The initial interval slope can be expressed as formula (2-3):

[0156]

[0157] Where gainOrg[index] represents the initial interval slope of the first interval. THValue[index] represents the cubic spline interval value of the first interval, and THValue[index-1] represents the cubic spline interval value of the previous interval of the first interval.

[0158] A4-2, Obtain the interval endpoint slopes of the first interval based on the initial interval slope. The interval endpoint slopes include the interval start endpoint slope, which can be expressed as formula (2-4); and the interval end endpoint slope, which can be expressed as formula (2-5).

[0159]

[0160] Where gainOrg[index-1] represents the initial slope of the interval preceding the first interval. gainStart[index] represents the slope of the starting endpoint of the interval.

[0161]

[0162] Where gainOrg[index+1] represents the initial interval slope of the interval following the first interval. gainEnd[index] represents the slope of the interval's ending endpoint.

[0163] A4-3, the cubic spline parameters P0, P1, P2, and P3 of the first interval can be expressed by the following formulas (2-6) to (2-9):

[0164] P0 = THValue[index-1] (2-6)

[0165] P1 = gainStart[index] (2-7)

[0166]

[0167]

[0168] In formulas (2-8) and (2-9), ValueA, ValueB, and ValueC are intermediate values, which can be calculated in the following way:

[0169] ValueA=THValue[index]-gainStart[index]*ValueC-THValue[index-1]

[0170] ValueB=gainEnd[index]-gainStart[index]

[0171] ValueC=TH[index]-TH[index-1]

[0172] A5. Based on the cubic spline parameters P0, P1, P2, and P3, the function expression corresponding to the tone mapping curve of the relevant segmented region can be obtained, which can be expressed as the following formula (2-10):

[0173] L′=H(L)=P0+P1*(L-TH[index-1])+P2*(L-TH[index-1]) 2 +P3*(L-TH[index-1]) 2 (2-10)

[0174] Where L represents the linear signal value, which is normalized to [0, 1], and L' represents the nonlinear signal value, which takes values ​​in the range [0, 1]. H(L) represents the function corresponding to the tone mapping curve.

[0175] Another embodiment of obtaining the tone mapping curve based on the parameters of the tone mapping curve is as follows: the function expression corresponding to the tone mapping curve can also be expressed as the following formula (2-11).

[0176]

[0177] Where L represents a linear signal value, normalized to [0, 1], and L' represents a nonlinear signal value, ranging from [0, 1]. F(L) represents the function corresponding to the tone mapping curve. a, b, p, m, and n represent the parameters of the tone mapping curve, used to adjust its shape. It should be understood that the parameters of the tone mapping curve function shown in (2-11) for different segmented regions may be different. In implementation, the parameters of the tone mapping curve for each segmented region can be indicated in the corresponding metadata information unit.

[0178] It should be noted that, depending on the shape of the tone mapping curve, different tone mapping curve functions can be used during implementation, and this application does not limit the selected tone mapping curve.

[0179] After obtaining the tone mapping curve corresponding to the relevant segmented region, the display device 402 performs tone mapping on the i-th pixel based on the tone mapping curve to obtain the sub-tone mapping value of that pixel in the relevant segmented region. For example, for Figure 11 For pixel 3 in the segment, we can obtain sub-tone mapping values ​​1 corresponding to segment 9, 2 corresponding to segment 10, 3 corresponding to segment 13, 4 corresponding to segment 14, and 5 corresponding to segment 15, for a total of 5 sub-tone mapping values.

[0180] One embodiment of tone mapping involves the display device 402 adjusting the brightness and / or color of the i-th pixel based on a tone mapping curve. Optionally, adjusting the color of the i-th pixel can be expressed as the following formula (3-1):

[0181]

[0182] Where Rp, Gp, and Bp represent the three color components of the i-th pixel, f[id] represents the function corresponding to the tone mapping curve, such as H(L) and F(L) mentioned above; and id represents the segmentation region identifier.

[0183] The sub-tone mapping value of the i-th pixel in the relevant segmented region can be expressed by the following formulas (3-2) to (3-4):

[0184] RpTM[i]=Rp*gain (3-2)

[0185] GpTM[i] = Gp * gain (3-3)

[0186] BpTM[i] = Bp*gain (3-4)

[0187] Where RpTM[i], GpTM[i], and BpTM[i] can be represented as the sub-tone mapping values ​​of the i-th pixel in the relevant segmentation region.

[0188] In another example, in addition to determining the sub-tone mapping value corresponding to each relevant segmentation region, the display device 402 can also determine the weight factor corresponding to each relevant segmentation region, thereby determining the degree of influence of the sub-tone mapping value of each relevant segmentation region on the tone mapping value of the i-th pixel based on the weight factor.

[0189] Optionally, in implementation of this application, the display device 402 may assign corresponding weight factors to each of the relevant segmented regions. Furthermore, the display device 402 may determine the weight factors based on a preset method or based on metadata indications. This application provides the following two possible methods for determining weight factors; one or a combination of these methods may be used in implementation, and the actual implementation is not limited to these two methods.

[0190] Method 1 involves determining the number of pixels belonging to the preset region and the total number of pixels corresponding to the preset region in each of the relevant segmentation regions. For the first relevant segmentation region, the following is performed: Based on the number of pixels belonging to the preset region and the total number of pixels in the first relevant segmentation region, a corresponding first weighting factor is assigned to the first relevant segmentation region. The first relevant segmentation region is any one of the at least one relevant segmentation region.

[0191] For example, combining Figure 11 The number of pixels belonging to the preset region in each of the relevant segmented regions, and the total number of pixels corresponding to the preset region, shown in pixel 3, can be represented as shown in Table 2-1 below:

[0192] Table 2-1

[0193]

[0194] Where Num[id] represents the number of pixels in the segmented region contained in the preset region, and id represents the segmented region identifier; for example Figure 11 In Figure (b), pixel 3 is shown as the number of pixels contained in each relevant segmented region under the preset area. Weight1[id] represents the first weight factor of the segmented region id.

[0195] Optionally, the first weight factor can be positively correlated with the number of pixels in the segmented region, such as a linear relationship; in other words, the more pixels in the segmented region, the higher the first weight factor corresponding to the segmented region, and the growth is linear.

[0196] Alternatively, the positive correlation can also be an indicative relationship, where the exponential relationship can be determined based on a first weighted intensity value. This first weighted intensity value is used to adjust the first weighting factor according to the pixel distribution of the image frame to be displayed. For example, adjusting the first weighting factor using the first weighted intensity value can be expressed as the following formula (4-1):

[0197]

[0198] Wherein, NumStr[id] represents the first weighted intensity value, which can be, for example, 2 or 0.5. The first weighted intensity value can be determined based on the pixel similarity of multiple segmented regions in the image frame to be displayed; for example, if the pixels of multiple segmented regions are relatively similar pixels, a larger first weighted intensity value can be set; otherwise, a smaller first weighted intensity value can be set. It should be understood that the first weighted intensity value can be determined by the display device 402 based on preset rules, or it can be indicated by the acquisition device 401 in the metadata, and this application does not limit this. In this way, by setting the first weighted intensity value, when the local features of the image frame to be displayed are obvious, a smaller first weighted intensity value, such as 0.5, can reduce the boundary effect between segmented regions.

[0199] Method 2 involves determining the preset feature value corresponding to the preset region and the preset feature value of the i-th pixel in each of the relevant segmentation regions. For the second relevant segmentation region, the following is performed: based on the features of the preset feature value corresponding to the part of the preset region in the second relevant segmentation region and the preset feature value of the i-th pixel, a corresponding second weight factor is assigned to the second relevant segmentation region. The second relevant segmentation region is any one of the at least one relevant segmentation region. The preset feature value is a luminance feature value (AvgLum[id]) or a color feature value (AvgRGB[id][3], where [3] represents three color components), etc. It should be understood that the preset feature value corresponding to the preset region and the preset feature value of the i-th pixel in the relevant segmentation region are selected to be the same preset feature value.

[0200] Among them, the number of preset feature values ​​is at least one. For example, the brightness feature value of a segmented region can be 4 (AvgLum[id](4)) and the color feature value of a segmented region can also be 4 (AvgRGB[id](4)[3]). Figure 12 The diagram shown is a schematic representation of a brightness feature value provided in an embodiment of this application. Figure 12 Image (a) shows a scene with a brightness feature value of 1. Figure 12Subfigure (b) shows a scenario with four luminance feature values. Wherein, the number of luminance feature values can be obtained based on the maximum number or the maximum average number of luminance values in the divided regions.

[0201] In the above example, when the number of preset feature values is four, in the implementation of the present application, the target preset feature value selected from the four preset feature values can also be determined according to the relative relationship between the central coordinates (x, y) of the partial region, which belongs to the preset region and is within the second correlation divided region (it can also be the coordinates (x, y) of the i-th pixel (current pixel)), and the central coordinates (xR, yR) of the second correlation divided region. For example, with reference to Figure 12 the content shown in subfigure (b), the top-left region corresponds to luminance feature value 1, the top-right region corresponds to luminance feature value 2, the bottom-left region corresponds to luminance feature value 3, and the bottom-right region corresponds to luminance feature value 4. When x≤xR and y≥yR, luminance feature value 1 is used; when x>xR and y≥yR, luminance feature value 2 is used; when x≤xR and y<yR, luminance feature value 3 is used; when x>xR and y<yR, luminance feature value 4 is used.

[0202] For example, with reference to Figure 11 the pixel point 3 shown therein, and taking the preset region feature being a luminance feature value as an example, the luminance feature values belonging to the preset region in each of the correlation divided regions and the luminance feature value of the i-th pixel can be shown in the following Table 2-2:

[0203] Table 2-2

[0204] name Brightness characteristic value Second weighting factor Segmentation Region 9 AvgLum[1] Weight2[1] Segmentation region 10 AvgLum[2] Weight2[2] Segmentation Region 13 AvgLum[3] Weight2[3] Segmentation region 14 AvgLum[4] Weight2[4] Divide into 15 regions AvgLum[5] Weight2[5] The i-th pixel AvgLum[i]

[0205] Wherein, AvgLum[id] represents the luminance feature value of the divided region included under the preset region, and id represents the identifier of the divided region; for example, Figure 11 it is the luminance feature value of the partial region of the divided region correlated to pixel point 3 shown in subfigure (b) that belongs to the preset region. Weight2[id] represents the second weight factor of the divided region with the identifier id.

[0206] Optionally, the second weight factor can be in a negative correlation, such as a linear correlation, with the difference between the luminance feature value of the divided region and the luminance feature value of the i-th pixel; in other words, the larger the difference between the luminance feature value of the divided region and the luminance feature value of the i-th pixel is, the smaller the second weight factor corresponding to the divided region is, and the reduction is linear.

[0207] Alternatively, the positive correlation can be an exponential relationship, which can be determined based on a second weighted intensity value. The second weighted intensity value is used to adjust the second weighting factor according to the changes in preset feature values ​​of the image frame to be displayed. For example, adjusting the second weighting factor using the second weighted intensity value can be expressed as the following formula (4-2):

[0208]

[0209] Here, `g()` represents a normalization operation, for example, dividing the signal by its maximum value to change its range to 0 to 1. `clip()` represents a clamping operation, where `min` is the minimum value (e.g., equal to 0) and `max` is the maximum value (e.g., equal to 1). Values ​​in `clip()` containing a third operand that are greater than `max` or less than `min` are truncated to `max` and `min` respectively; for example, if the value in `clip()` containing a third operand is less than `min`, then `min` is used; if the value in `clip()` containing a third operand is greater than `max`, then `max` is used. `K` is a preset value, ranging from 0 to 1023. `AvgLumStr[id]` represents the second weighting strength value, for example, it can be 2 or 0.5.

[0210] For example, the second weighted intensity value can be determined based on the pixel brightness changes of multiple segmented regions in the image frame to be displayed. For instance, if the brightness changes of pixels in multiple segmented regions are significant, a larger second weighted intensity value can be set; otherwise, a smaller second weighted intensity value can be set. It should be understood that the second weighted intensity value can be determined by the display device 402 based on preset rules, or it can be indicated by the acquisition device 401 in the metadata; this application does not limit this. In this way, by setting the second weighted intensity value, when the local features of the image frame to be displayed are significant, a smaller second weighted intensity value, such as 0.5, can reduce the boundary effect between segmented regions.

[0211] Optionally, the weight factor corresponding to each relevant segmentation region can be a first weight factor, which can be calculated according to the formula shown in formula (4-1), i.e., Weight[id] = Weight1[id]; or it can be a second weight factor, which can be calculated according to the formula shown in formula (4-2), i.e., Weight[id] = Weight2[id]; or it can be determined by the first weight factor and the second weight factor together, as shown in the following formula (4-3):

[0212] Weight[id]=W(Weight1[id], Weight2[id]) (4-3)

[0213] Wherein, W() represents a preset function used to obtain the weight factors corresponding to the relevant segmentation regions based on the first weight factor and the second weight factor. This preset function can be multiplication or addition, etc., and can be determined by presetting.

[0214] Step 505: The display device 402 obtains the tone mapping value of the i-th pixel based on the sub-tone mapping value and the weight factor corresponding to each of the relevant segmented regions; and performs tone mapping on the i-th pixel according to the tone mapping value.

[0215] For example, the tone mapping value of the i-th pixel can be obtained according to the following formulas (5-1) to (5-5):

[0216] MaxRGBP=max(Rp,Gp,Bp) (5-1)

[0217]

[0218]

[0219]

[0220]

[0221] Where Z represents the number of relevant segmented regions for the i-th pixel. Rtm, Gtm, and Btm represent tone mapping values. spline[id] represents the function corresponding to the tone mapping curve; id represents the segmentation region identifier.

[0222] Furthermore, in the implementation of this application, the tone mapping values ​​can also be transformed from Rtm, Gtm, Btm to the color gamut space to obtain Ytm, Utm, Vtm; correspondingly, the sub-tone mapping values ​​are Yp, Up, Vp.

[0223] Furthermore, in this application, after obtaining the tone mapping value of the i-th pixel, the display device 402 can also adjust the saturation of the image frame to be displayed. In other words, the display device 402 adjusts the saturation of the i-th pixel based on the initial pixel value of the i-th pixel and the tone mapping value obtained in step 505.

[0224] It should be noted that in the examples described above, the information required to calculate the tone mapping value of each pixel can be processed by the display device 402; alternatively, it can be calculated by the acquisition device 401 and then indicated to the display device 402 via metadata. In other words, the processing capacity of the display device 402 and the acquisition device 401 can be allocated according to their computing capabilities, and this application does not impose any limitations on this. It is understood that the display device 402 can use the information indicated by the metadata to perform subsequent tone mapping of the image frame to be displayed.

[0225] Figure 13 This is another flowchart illustrating a tone mapping method provided in an embodiment of this application, applicable to... Figure 4 The data acquisition device 401 shown in the figure.

[0226] Step 1301: Acquire image data to obtain a data frame to be displayed. For example, the image data can be captured by the acquisition device 401 through a camera device, or it can be generated by the acquisition device 402 through a preset application.

[0227] Step 1302: Divide the data frame to be displayed into multiple segmented regions using a target segmentation method, and determine at least one corresponding metadata information unit for each of the multiple segmented regions. For example, the target segmentation method can be one of the various segmentation methods described above, such as... Figure 6 The segmentation method shown is A. Figure 7 The shown segmentation method B or Figure 8 The segmentation method shown is C, etc.; and the segmentation method used by the acquisition device 401 and the display device 402 is the same. Furthermore, the acquisition device 401 can also indicate the target segmentation method to the display device 402 via metadata. Additionally, the acquisition device 402 can also indicate the first segmentation parameter, second segmentation parameter, etc., that may be involved in the target segmentation method to the display device 402 via metadata.

[0228] For example, for segmentation method A, the acquisition device can determine the number of segments in the vertical direction based on the histograms of different rows in the image data; and determine the number of segments in the horizontal direction based on the histograms of different columns in the image data.

[0229] In another example, for segmentation method B, the acquisition device can cluster the histograms of different rows in the image data to determine the segmentation list in the vertical direction; and cluster the histograms of different columns in the image data to determine the segmentation list in the horizontal direction.

[0230] Step 1303: Send the data frame to be displayed and metadata to the display device 402; the metadata includes at least one metadata information unit.

[0231] It should be understood that, in order to reduce the computational load of the display device 402 and balance the content that the acquisition device 401 and the display device 402 need to process respectively, in this application, the acquisition device 401 can calculate information such as the first weighted intensity value, the second weighted intensity value, and the preset feature value of the segmented region, and instruct the display device 402 through metadata. It should be noted that the processing implementation process of the acquisition device 401 is similar to that described in the display device 402, and therefore will not be repeated.

[0232] Based on the same inventive concept as the method described above, this application provides a tone mapping processing apparatus 1400. The tone mapping processing apparatus 1400 can be applied to the aforementioned display device 402. See also... Figure 14 The device may include: an acquisition module 1401, a processing module 1402, and a tone mapping module 1403.

[0233] The acquisition module 1401 is used to acquire an image frame to be displayed and metadata, wherein the metadata includes at least one metadata information unit. The processing module 1402 is used to segment the image frame to be displayed into multiple segmentation regions using a target segmentation method, and to determine the correspondence between the multiple segmentation regions and the at least one metadata information unit. The tone mapping module 1403 is used to, for the i-th pixel in the image frame to be displayed, perform the following: based on multiple pixels contained in a preset region selected from the i-th pixel, determine at least one related segmentation region to which the multiple pixels belong, and determine the related metadata information unit corresponding to each related segmentation region according to the correspondence; obtain the sub-tone mapping value corresponding to each related segmentation region according to the related metadata information unit, and assign a corresponding weight factor to each related segmentation region; obtain the tone mapping value of the i-th pixel based on the sub-tone mapping value and the weight factor corresponding to each related segmentation region; and perform tone mapping on the i-th pixel according to the tone mapping value; wherein i takes any positive integer from 1 to N, and N is the number of pixels contained in the image frame to be displayed.

[0234] In one possible design, the tone mapping module 1403, when assigning corresponding weight factors to each of the relevant segmentation regions, specifically performs the following steps: determining the number of pixels belonging to the preset region and the total number of pixels corresponding to the preset region in each of the relevant segmentation regions; for a first relevant segmentation region, performing the following: assigning a corresponding first weight factor to the first relevant segmentation region based on the number of pixels belonging to the preset region and the total number of pixels in the first relevant segmentation region, wherein the first relevant segmentation region is any one of the at least one relevant segmentation region; and / or determining a preset feature value corresponding to the preset region in each of the relevant segmentation regions; for a second relevant segmentation region, performing the following: assigning a corresponding second weight factor to the second relevant segmentation region based on the feature difference between the preset feature value corresponding to the preset region in the second relevant segmentation region and the preset feature value of the i-th pixel, wherein the second relevant segmentation region is any one of the at least one relevant segmentation region.

[0235] In one possible design, the metadata may further include one or a combination of the following: a first weighted intensity value and a second weighted intensity value; wherein the first weighted intensity value is used to adjust the first weighting factor according to the pixel distribution of the image frame to be displayed; and the second weighted intensity value is used to adjust the second weighting factor according to the changes in preset feature values ​​of the image frame to be displayed.

[0236] In one possible design, the preset feature value is one of the following feature values: brightness feature value, color feature value. The number of preset feature values ​​is at least one.

[0237] In one possible design, the tone mapping module 1403 is used to obtain the sub-tone mapping value corresponding to each of the relevant segmented regions according to the relevant metadata information unit. Specifically, the relevant metadata information unit includes parameters of the tone mapping curve, and the sub-tone mapping value corresponding to the i-th pixel in the relevant segmented region is obtained based on the tone mapping curve.

[0238] In one possible design, the preset region selected based on the i-th pixel is: a region of a preset shape and a preset size selected with the i-th pixel as the center; or: a region centered on the i-th pixel and determined according to the preset shape and the preset size information contained in the metadata; or: a region centered on the i-th pixel and determined according to the preset shape and the preset size information contained in the metadata.

[0239] In one possible design, the processing module 1402 is used to determine the correspondence between the plurality of segmented regions and the at least one metadata information unit, specifically: for any segmented region, selecting a pixel at a preset position from any segmented region and obtaining the coordinate information of the pixel at the preset position; determining the metadata information unit corresponding to any segmented region based on the coordinate information; establishing a correspondence between any segmented region and the corresponding metadata information unit; or, for any metadata information unit, determining at least one segmented region corresponding to the one or more coordinate information contained in the metadata information unit; establishing a correspondence between any metadata information unit and the corresponding at least one segmented region.

[0240] In one possible design, the processing module 1402 is used to determine the correspondence between the plurality of segmented regions and the at least one metadata information unit, including: the metadata information unit contains a corresponding segmented region identifier, and the plurality of segmented regions are traversed according to the target scanning order and the segmented region identifier to determine the metadata information unit corresponding to each segmented region; wherein, the target scanning order is a preset scanning order or a scanning order indicated in the metadata.

[0241] In one possible design, the plurality of segmented regions and the at least one metadata information unit have a one-to-one correspondence or a many-to-one correspondence.

[0242] In one possible design, the target segmentation method is a preset segmentation method or a segmentation method indicated in the metadata.

[0243] Based on the same inventive concept as the method described above, this application provides a tone mapping processing apparatus 1400. The tone mapping processing apparatus 1500 can be applied to the aforementioned acquisition device 401. See also... Figure 15 The device 1500 may include: a data acquisition module 1501, a processing module 1502, and a transceiver module 1503. Among them,

[0244] The acquisition module 1501 is used to acquire image data to obtain a data frame to be displayed; the processing module 1502 is used to divide the data frame to be displayed into multiple segmented regions using a target segmentation method, and to determine at least one metadata information unit corresponding to the multiple segmented regions; the transceiver module 1503 is used to send the data frame to be displayed and the metadata to the display device; the metadata includes the at least one metadata information unit.

[0245] In one possible design, the processing module 1502 is further configured to determine a first weighted intensity value based on the number of pixels contained in each of the plurality of segmented regions; and / or determine a second weighted intensity value based on preset feature values ​​corresponding to each of the plurality of segmented regions; and associate the first weighted intensity value and / or the second weighted intensity value with the metadata.

[0246] In one possible design, the preset feature value is one of the following feature values: brightness feature value, color feature value. The number of preset feature values ​​is at least one.

[0247] In one possible design, the metadata information unit includes parameters of the tone mapping curve corresponding to the segmented region.

[0248] In one possible design, the metadata may further include: preset size information; the preset size information is used to indicate the preset size of the preset shape used when selecting a preset area for a pixel; or, the metadata may further include: preset shape and preset size information.

[0249] In one possible design, the processing module 1502 is used to determine at least one corresponding metadata information unit for the plurality of segmented regions, specifically for selecting pixels at multiple preset positions from the segmented regions, obtaining coordinate information of the pixels at the multiple preset positions, and associating the coordinate information with the metadata information unit.

[0250] In one possible design, the metadata information unit further includes: a segmentation region identifier and a scanning order corresponding to the metadata information unit; wherein the corresponding segmentation region identifier and the scanning order are used by the display device to determine the correspondence between the plurality of segmentation regions and the at least one metadata information unit.

[0251] In one possible design, the plurality of segmented regions and the at least one metadata information unit have a one-to-one correspondence or a many-to-one correspondence.

[0252] In one possible design, the processing module 1502 is further configured to associate the target segmentation method with the metadata, so as to instruct the display device to segment the data frame to be displayed using the target segmentation method.

[0253] This application also provides an electronic device 1600, such as... Figure 16As shown, the electronic device 1600 can be the display device 402 or the acquisition device 401 described in the above embodiments. The electronic device 1600 may include a communication interface 1610 and a processor 1620. Optionally, the electronic device 1600 may also include a memory 1630. The memory 1630 may be located inside the electronic device or outside the electronic device. Figure 14 The acquisition module 1401, processing module 1402, and tone mapping module 1403 shown can all be implemented by the processor 1620. Alternatively, the above... Figure 15 The acquisition module 1501, processing module 1502, and transceiver module 1503 shown can also be implemented by the processor 1620. The optional communication interface 1610, processor 1620, and memory 1630 can be interconnected via communication line 1640; communication line 1640 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication line 1640 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 16 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0254] In one possible implementation, processor 1620 is used to implement Figure 13 The acquisition terminal described herein executes any method and outputs the encoded bitstream, such as the image frame to be displayed, metadata, etc., through the communication interface 1610.

[0255] In the implementation process, each step of the processing flow can be completed through the integrated logic circuits in the processor 1620 or through software instructions. Figure 13 The method executed by the acquisition terminal is described below. For simplicity, it will not be elaborated further. The program code executed by the processor 1620 to implement the above method can be stored in the memory 1630. The memory 1630 and the processor 1620 are coupled.

[0256] Any communication interface involved in the embodiments of this application can be a circuit, a bus, a transceiver, or any other device that can be used for information interaction. For example, the communication interface 1610 in electronic device 1600. Exemplarily, the other device can be a device connected to electronic device 1600. For example, when electronic device 1600 is a data acquisition device 401, the other device can be a display device 402, etc.

[0257] Processor 1620 may operate in conjunction with memory 1630. Memory 1630 may be non-volatile memory, such as hard disk drive (HDD) or solid-state drive (SSD), or it may be volatile memory, such as random-access memory (RAM). Memory 1630 may be any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited to this.

[0258] This application embodiment does not limit the specific connection medium between the communication interface 1610, processor 1620, and memory 1630. This application embodiment... Figure 16 The memory 1630, processor 1620, and communication interface 1610 are connected via a bus, and the bus is in... Figure 16 The connections between other components are shown in bold and are for illustrative purposes only, not as limiting information. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 16 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0259] The processors involved in the embodiments of this application can be general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0260] The coupling in the embodiments of this application is an indirect coupling or communication connection between devices, modules, or modules, which can be electrical, mechanical, or other forms, and is used for information interaction between devices, modules, or modules.

[0261] Based on the above embodiments, this application also provides a computer storage medium storing software programs. When these software programs are read and executed by one or more processors, they can implement the methods provided in any one or more of the above embodiments. The computer storage medium may include various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory, random access memory, magnetic disk, or optical disk.

[0262] Based on the above embodiments, this application also provides a chip, which includes a processor for implementing the functions involved in any one or more of the above embodiments. Optionally, the chip further includes a memory for storing necessary program instructions and data executed by the processor. This chip can be composed of individual chips or can include chips and other discrete devices.

[0263] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0264] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0265] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0266] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0267] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A tone mapping method, characterized in that, Applied to a display device, the method includes: Obtain the image frame to be displayed and metadata, wherein the metadata includes at least one metadata information unit; The image frame to be displayed is divided into multiple segmentation regions using a target segmentation method, and the correspondence between the multiple segmentation regions and the at least one metadata information unit is determined. For the i-th pixel in the image frame to be displayed, perform the following: Based on the multiple pixels contained in the preset region selected by the i-th pixel, at least one related segmentation region to which the multiple pixels belong is determined, and the relevant metadata information unit corresponding to each of the related segmentation regions is determined according to the correspondence relationship. The relevant metadata information unit obtains the sub-tone mapping value corresponding to each relevant segmented region, and assigns a corresponding weight factor to each relevant segmented region based on the portion of each relevant segmented region belonging to the preset region; wherein, the weight factor is adjusted using a first weighted intensity value or a second weighted intensity value, the first weighted intensity value is determined based on the pixel number distribution of the image frame to be displayed, and the second weighted intensity value is determined based on the change of the preset feature value of the image frame to be displayed, the preset feature value being at least one of the following: brightness feature value, color feature value; Based on the sub-tone mapping value and the weight factor corresponding to each of the relevant segmented regions, the tone mapping value of the i-th pixel is obtained; and tone mapping is performed on the i-th pixel according to the tone mapping value. Where i takes any positive integer from 1 to N, and N is the number of pixels contained in the image frame to be displayed.

2. The method according to claim 1, characterized in that, The assignment of corresponding weight factors to each of the relevant segmented regions includes one or a combination of the following methods: The number of pixels belonging to the preset region and the total number of pixels corresponding to the preset region are determined in each of the relevant segmentation regions respectively; for the first relevant segmentation region, the following is performed: according to the number of pixels belonging to the preset region and the total number of pixels in the first relevant segmentation region, a corresponding first weight factor is assigned to the first relevant segmentation region, wherein the first relevant segmentation region is any one of the at least one relevant segmentation region; Determine the preset feature value corresponding to the preset region in each of the relevant segmented regions; For the second related segmentation region, the following is performed: based on the feature difference between the preset feature value corresponding to the preset region and the preset feature value of the i-th pixel in the second related segmentation region, a corresponding second weight factor is assigned to the second related segmentation region, wherein the second related segmentation region is any one of the at least one related segmentation region.

3. The method according to claim 2, characterized in that, The metadata also includes one or a combination of the following information: a first weighted strength value and a second weighted strength value.

4. The method according to claim 1, characterized in that, The step of obtaining the sub-tone mapping value corresponding to each of the relevant segmented regions based on the relevant metadata information unit includes: The relevant metadata information unit contains parameters of the tone mapping curve, and the sub-tone mapping value corresponding to the i-th pixel in the relevant segmentation region is obtained based on the tone mapping curve.

5. The method according to any one of claims 1 to 4, characterized in that, The preset region selected based on the i-th pixel is: a region of a preset size and preset shape selected with the i-th pixel as the center; or: a region determined with the i-th pixel as the center and based on the preset shape and the preset size information contained in the metadata; or: a region determined with the i-th pixel as the center and based on the preset shape and preset size information contained in the metadata.

6. The method according to claim 1, characterized in that, Determining the correspondence between the plurality of segmented regions and the at least one metadata information unit includes: For any of the segmented regions, a pixel at a preset position is selected from the segmented region, and the coordinate information of the pixel at the preset position is obtained; based on the coordinate information, the metadata information unit corresponding to any of the segmented regions is determined; a correspondence between any of the segmented regions and the corresponding metadata information unit is established; or, For any of the metadata information units, at least one segmented region corresponding to the one or more coordinate information contained in the metadata information unit is determined; and a correspondence between any of the metadata information units and the corresponding at least one segmented region is established.

7. The method according to claim 1, characterized in that, Determining the correspondence between the plurality of segmented regions and the at least one metadata information unit includes: The metadata information unit contains a corresponding segmentation region identifier. The multiple segmentation regions are traversed according to the target scanning order and the segmentation region identifier to determine the metadata information unit corresponding to each segmentation region. The target scanning order is a preset scanning order or a scanning order indicated in the metadata.

8. The method according to any one of claims 1 to 4, characterized in that, The plurality of segmented regions and the at least one metadata information unit have a one-to-one correspondence or a many-to-one correspondence.

9. The method according to any one of claims 1 to 4, characterized in that, The target segmentation method is a preset segmentation method or a segmentation method indicated in the metadata.

10. A tone mapping method, characterized in that, Applied to data acquisition devices, the method includes: Acquire image data to obtain the data frame to be displayed; The data frame to be displayed is divided into multiple segmented regions using a target segmentation method. A first weighted intensity value is determined based on the pixel number distribution of the multiple segmented regions, and / or a second weighted intensity value is determined based on the changes in preset feature values ​​of the multiple segmented regions, wherein the preset feature values ​​are at least one of the following: brightness feature value and color feature value; And for each of the multiple segmented regions, at least one corresponding metadata information unit is determined; The data frame to be displayed and metadata are sent to the display device; the metadata includes at least one metadata information unit, and the metadata also includes a first weighted intensity value and / or a second weighted intensity value, wherein the first weighted intensity value and / or the second weighted intensity value are used to adjust the weight factor corresponding to the relevant segmentation region of the pixel used when performing tone mapping on the pixel; wherein the weight factor is determined based on the portion of the relevant segmentation region that belongs to a preset region, and the preset region is selected for the pixel.

11. The method according to claim 10, characterized in that, The determination of a first weighted intensity value based on the pixel distribution of the multiple segmented regions, and / or the determination of a second weighted intensity value based on the changes in preset feature values ​​of the multiple segmented regions, includes: Based on the number of pixels contained in each of the plurality of segmented regions, the first weighted intensity value is determined; and / or, The second weighted intensity value is determined based on the preset feature values ​​corresponding to the multiple segmented regions; Associate the first weighted strength value and / or the second weighted strength value with the metadata.

12. The method according to claim 10, characterized in that, The metadata information unit includes parameters of the tone mapping curve corresponding to the segmented region.

13. The method according to any one of claims 10 to 12, characterized in that, The metadata also includes: preset size information; the preset size information is used to indicate the preset size of the preset shape used when selecting the preset region for the pixel; or, The metadata also includes: preset shape and preset size information.

14. The method according to claim 10, characterized in that, The step of determining at least one metadata information unit corresponding to the plurality of segmented regions includes: Select pixels at multiple preset positions from the segmented region and obtain the coordinate information of the pixels at the multiple preset positions; The coordinate information is associated with the metadata information unit.

15. The method according to claim 10, characterized in that, The metadata information unit further includes: the segmentation region identifier and scanning order corresponding to the metadata information unit; The corresponding segmentation region identifier and the scanning order are used by the display device to determine the correspondence between the plurality of segmentation regions and the at least one metadata information unit.

16. The method according to any one of claims 10 to 12, characterized in that, The plurality of segmented regions and the at least one metadata information unit have a one-to-one correspondence or a many-to-one correspondence.

17. The method according to claim 10, characterized in that, The method further includes: The target segmentation method is associated with the metadata to instruct the display device to segment the data frame to be displayed using the target segmentation method.

18. A display terminal device, characterized in that, include: A non-volatile memory and a processor are coupled together, the processor calling program code stored in the memory to perform the method as described in any one of claims 1-9.

19. A data acquisition terminal device, characterized in that, include: A non-volatile memory and a processor are coupled together, the processor calling program code stored in the memory to perform the method as described in any one of claims 10-17.

20. A tone mapping system, characterized in that, It includes the display device as described in claim 18 and the acquisition device as described in claim 19.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a processor, implements the method of any one of claims 1-9 or 10-17.

Images