Inverse tone mapping method and device, and storage medium

Abstract

The invention relates to the technical field of display, and discloses an inverse tone mapping method and device and a storage medium, and the method comprises the steps: converting original image data into linear RGB data, converting the linear RGB data into an LUV color space, separating a brightness component and a chrominance component, and carrying out the corresponding noise reduction processing of the brightness component and the chrominance component; determining an image brightness protection point according to the denoised brightness component, and segmenting to obtain a brightness mapping curve; carrying out weighted addition on the brightness mapping curve and a historical frame brightness mapping curve, carrying out infinite impulse response filtering, mapping a brightness channel by using the filtered mapping curve, and carrying out statistics on a brightness histogram; calculating a brightness adjustment curve according to the distribution data of the histogram; performing weighted addition on the brightness adjustment curve and the historical frame brightness adjustment curve, performing infinite impulse response filtering, and adjusting the brightness by using the filtered adjustment curve; and performing saturation adjustment on the chrominance component after noise reduction, and converting the adjusted image into a nonlinear HDR image with a set color gamut.

Description

Technical Field

[0001] The present invention relates to the field of display technology, and in particular to an inverse tone mapping method, device and storage medium. Background Art

[0002] In daily life, more and more screens support high brightness and high dynamic range (HDR) display, bringing users a more vivid visual experience. However, most of the video resources currently produced are still based on standard dynamic range (SDR) data with normal brightness, and the existing SDR video stock is huge. When these image materials are displayed on HDR screens, they cannot fully display the high dynamic range characteristics of the screen, limiting the improvement of visual effects.

[0003] In order to enhance the display effect of standard dynamic range video on HDR devices, a variety of HDR inverse tone mapping methods have been developed to convert SDR images into images close to HDR effects. However, these traditional methods often easily lead to overexposure, loss of bright details and dark shadows when processing highlight areas, as well as excessive noise in the picture, affecting the overall viewing experience. Summary of the invention

[0004] The purpose of the present invention is to provide an inverse tone mapping method, device and storage medium, which can adaptively adjust the brightness mapping of an image, reduce dead black, overexposure and brightness overflow, and effectively suppress or even eliminate noise that may exist in the picture during the processing.

[0005] In order to solve the above technical problems, the present invention provides an inverse tone mapping method, the method comprising:

[0006] Converting the original image data into linear RGB data, and converting the linear RGB data into a LUV color space to separate a brightness component and a chrominance component;

[0007] Performing corresponding noise reduction processing on the brightness component and the chrominance component respectively;

[0008] Determine the image brightness protection point according to the brightness component after noise reduction, and obtain the brightness mapping curve in segments with the image brightness protection point as the boundary;

[0009] The brightness mapping curve is weightedly added to the brightness mapping curve of the historical frame, and infinite impulse response filtering is performed, and the brightness channel is mapped using the filtered mapping curve, and a brightness histogram is calculated;

[0010] Calculating a brightness adjustment curve according to the distribution data of the brightness histogram;

[0011] The brightness adjustment curve is weightedly added to the brightness adjustment curve of the historical frame, and infinite impulse response filtering is performed, and the brightness is adjusted using the filtered adjustment curve;

[0012] The saturation of the chroma component after noise reduction is adjusted, and the image after brightness adjustment and chroma saturation adjustment is converted into a non-linear HDR image of a set color gamut.

[0013] In a first aspect, in the above-mentioned inverse tone mapping method provided by the present invention, the original image data is converted into linear RGB data, and the linear RGB data is converted into a LUV color space, including:

[0014] According to the color space of the original image data, the original image data is converted into linear RGB data using an inverse photoelectric conversion curve;

[0015] Performing a shift operation on the linear RGB data to increase the data bit depth;

[0016] Convert the shifted linear RGB data to LUV color space.

[0017] On the other hand, in the above-mentioned inverse tone mapping method provided by the present invention, determining the image brightness protection point according to the brightness component after noise reduction includes:

[0018] Calculating the average brightness of the image according to the brightness component after noise reduction;

[0019] Selecting a target brightness value between the average brightness of the image and the mid-grey brightness;

[0020] The target brightness value is increased by a set percentage, and the increased brightness value is used as the image brightness protection point.

[0021] On the other hand, in the above-mentioned inverse tone mapping method provided by the present invention, the brightness mapping curve is obtained segmentedly with the image brightness protection point as the boundary, including:

[0022] The brightness component after noise reduction is linearly scaled by a set multiple to form a first curve below the image brightness protection point;

[0023] Fitting a second curve passing through the image brightness protection point and saturation brightness;

[0024] The first curve and the second curve are spliced ​​according to cubic splines to obtain a brightness mapping curve; the brightness mapping curve has a smooth transition at the image brightness protection point.

[0025] On the other hand, in the above-mentioned inverse tone mapping method provided by the present invention, calculating the brightness adjustment curve according to the distribution data of the brightness histogram includes:

[0026] Dividing the distribution data of the brightness histogram according to dead black, dark part, middle gray, highlight and overexposure;

[0027] A brightness adjustment curve is obtained according to the proportions of the divided parts; the brightness adjustment curve passes through a mid-grey point.

[0028] On the other hand, in the above-mentioned inverse tone mapping method provided by the present invention, it also includes:

[0029] Performing flat area detection on the nonlinear HDR image, and using the detected flat area as a mask;

[0030] Scaling the nonlinear HDR image to create an image pyramid;

[0031] Performing stripe detection and smoothing processing on the image pyramid layer by layer, and weighted fusion of processed images of different layers to obtain a stripe-free image;

[0032] The nonlinear HDR image and the de-striped image are superimposed through the mask according to different weights to obtain a de-striped result.

[0033] On the other hand, in the above-mentioned inverse tone mapping method provided by the present invention, stripe detection and smoothing processing are performed layer by layer on the image pyramid, including:

[0034] The standard deviation of the flat area of ​​each layer of the image is calculated separately to obtain the block standard deviation, and the difference between the pixel and the surrounding pixels is calculated pixel by pixel;

[0035] According to the comparison result of the difference value and the block standard deviation, pixels meeting a set threshold are selected as pixels of the stripe boundary;

[0036] Smooths strip boundaries.

[0037] On the other hand, in the above-mentioned inverse tone mapping method provided by the present invention, it also includes:

[0038] Reduce the de-striping result to the target bit depth and introduce random jitter in the process of reducing the bit depth;

[0039] Add random brightness noise globally to the image reduced to the target bit depth and output HDR data.

[0040] In order to solve the above technical problems, the present invention further provides an inverse tone mapping device, the device comprising:

[0041] Memory for storing computer programs;

[0042] A processor is used to implement the steps of the above-mentioned inverse tone mapping method when executing the computer program.

[0043] In order to solve the above technical problem, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above inverse tone mapping method are implemented.

[0044] It can be seen from the above technical scheme that the present invention provides an inverse tone mapping method, which includes: converting original image data into linear RGB data, and converting the linear RGB data into LUV color space to separate brightness component and chromaticity component; performing corresponding noise reduction processing on brightness component and chromaticity component respectively; determining the image brightness protection point according to the brightness component after noise reduction, and obtaining the brightness mapping curve in segments with the image brightness protection point as the boundary; weightedly adding the brightness mapping curve and the brightness mapping curve of the historical frame, performing infinite impulse response filtering, mapping the brightness channel using the filtered mapping curve, and statistically calculating the brightness histogram; calculating the brightness adjustment curve according to the distribution data of the brightness histogram; weightedly adding the brightness adjustment curve and the brightness adjustment curve of the historical frame, performing infinite impulse response filtering, and adjusting the brightness using the filtered adjustment curve; performing saturation adjustment on the chromaticity component after noise reduction, and converting the image after brightness adjustment and chromaticity saturation adjustment to a nonlinear HDR image of a set color gamut.

[0045] The beneficial effect of the present invention is that the above-mentioned inverse tone mapping method provided by the present invention first converts the original image data into linear RGB data, and then converts it into LUV color space to separate the brightness component and the chroma component; respectively performs corresponding noise reduction processing on the brightness component and the chroma component to avoid that the noise becomes more obvious after the brightness is enhanced; then determines the image brightness protection point according to the brightness component after noise reduction, and obtains the brightness mapping curve in segments with the image brightness protection point as the boundary; weightedly adds the brightness mapping curve and the brightness mapping curve of the historical frame, performs infinite impulse response filtering, avoids sudden changes in brightness mapping between frames, uses the filtered mapping curve to map the brightness channel, and calculates the brightness histogram; then calculates the brightness adjustment curve according to the distribution data of the brightness histogram; weightedly adds the brightness adjustment curve and the brightness adjustment curve of the historical frame, performs infinite impulse response filtering, avoids sudden changes in brightness adjustment between frames, and uses the filtered adjustment curve to adjust the brightness, which can optimize the performance of highlights and dark parts; performs saturation adjustment on the chroma component after noise reduction; finally, converts the image after brightness adjustment and chroma saturation adjustment to a nonlinear HDR image of a set color gamut. This can adaptively adjust the brightness mapping of the image to obtain better brightness performance, reduce dead black, overexposure and brightness overflow, and effectively suppress or even eliminate the noise that may exist in the picture during the processing. This method can be widely used in a variety of HDR devices such as TVs, mobile phones, and monitors, allowing users to enjoy a more realistic visual experience when watching ordinary SDR images.

[0046] In addition, the present invention also provides a corresponding inverse tone mapping device and a computer-readable storage medium for the inverse tone mapping method, which have the same or corresponding technical features as the above-mentioned inverse tone mapping method and have the same effects as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A flowchart of an inverse tone mapping method provided by an embodiment of the present invention;

[0049] Figure 2 A specific flow chart of the inverse tone mapping method provided by an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of the structure of an inverse tone mapping device provided by an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of the structure of an inverse tone mapping device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 A flowchart of the inverse tone mapping method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes:

[0054] S101, converting the original image data into linear RGB data, and converting the linear RGB data into LUV color space to separate the brightness component and the chrominance component.

[0055] It should be noted that the original image data may be SDR data. Figure 2 A specific flow chart of the inverse tone mapping method provided by an embodiment of the present invention. Figure 2As shown, after inputting SDR data, the SDR data can be converted into linear RGB data, and the linear RGB data can be converted into CIE 1976 Yu'v' color space (i.e., a LUV color space) to separate the brightness component Y and the chromaticity component u'v'. That is, the brightness information is separated from the color information.

[0056] S102: Perform corresponding noise reduction processing on the brightness component and the chrominance component respectively.

[0057] In practice, the present invention can perform noise reduction processing of different intensities on the brightness and color channels respectively to avoid more obvious noise after brightness enhancement, and at the same time provide a purer input for the subsequent de-color banding module to avoid noise interfering with the judgment of the color bands.

[0058] S103, determining an image brightness protection point according to the brightness component after noise reduction, and obtaining a brightness mapping curve in segments with the image brightness protection point as a boundary.

[0059] It should be noted that the image brightness protection point is a specific control point or threshold set during the brightness processing of the image in order to avoid obvious abnormalities in the basic brightness of the image in the senses. The present invention determines the image brightness protection point based on the brightness component after noise reduction, and calculates the brightness mapping curve based on the image brightness protection point. The brightness mapping curve is used to describe the mapping relationship between the input brightness value and the output brightness value. It is a curve with the input brightness as the horizontal axis and the output brightness as the vertical axis, that is, it determines the output brightness value of each input brightness value after processing or conversion or corresponding.

[0060] S104 , weighted addition of the brightness mapping curve and the historical frame brightness mapping curve, infinite impulse response filtering, mapping the brightness channel using the filtered mapping curve, and calculating a brightness histogram.

[0061] In the implementation, the brightness mapping curve calculated by the current frame is weighted and added to the historical frame curve, and an infinite impulse response (IIR) filter is performed, and the filtered curve is used to map the brightness channel. Then, the brightness histogram can be statistically calculated by referring to the brightness channel.

[0062] S105: Calculate a brightness adjustment curve according to the distribution data of the brightness histogram.

[0063] It should be noted that the brightness adjustment curve is a curve that intuitively represents the brightness adjustment method. The original brightness value of the brightness histogram can be used as the horizontal axis, and the adjusted brightness value can be used as the vertical axis. Each point on the curve corresponds to an original brightness value and its adjusted brightness value. The image brightness can be adjusted by changing the shape of the curve. The present invention can appropriately brighten the dark part and darken the brightness of the highlight part to a certain extent, thereby obtaining better brightness performance.

[0064] S106 , weighted addition of the brightness adjustment curve and the historical frame brightness adjustment curve, infinite impulse response filtering, and use the filtered adjustment curve to adjust the brightness.

[0065] In implementation, the present invention can perform weighted addition of the brightness adjustment curve calculated for the current frame and the historical frame curve, perform infinite impulse response filtering, and use the filtered brightness adjustment curve to adjust the brightness.

[0066] S107, adjusting the saturation of the chromaticity component after noise reduction, and converting the image after brightness adjustment and chromaticity saturation adjustment to a nonlinear HDR image of a set color gamut.

[0067] In implementation, the present invention can convert the image after brightness adjustment and color saturation adjustment to the BT.2020 color gamut first, and then apply the inverse electro-optical transfer curve (EOTF) defined by the quantization perception (PQ) curve to the image in the BT.2020 color gamut to convert it into a nonlinear HDR image that complies with BT.2100.

[0068] In the above-mentioned inverse tone mapping method provided by the embodiment of the present invention, the original image data is first converted into linear RGB data, and then converted into LUV color space to separate the brightness component and the chromaticity component; the brightness component and the chromaticity component are respectively subjected to corresponding noise reduction processing to avoid more obvious noise after the brightness is enhanced; then the image brightness protection point is determined according to the brightness component after noise reduction, and the brightness mapping curve is obtained by segmentation with the image brightness protection point as the boundary; the brightness mapping curve is weightedly added to the brightness mapping curve of the historical frame, and infinite impulse response filtering is performed to avoid sudden changes in brightness mapping between frames, and the brightness channel is mapped using the filtered mapping curve to calculate the brightness histogram; then, according to the distribution data of the brightness histogram, the brightness adjustment curve is calculated; the brightness adjustment curve is weightedly added to the brightness adjustment curve of the historical frame, and infinite impulse response filtering is performed to avoid sudden changes in brightness adjustment between frames, and the brightness is adjusted using the filtered adjustment curve to optimize the performance of highlights and dark parts; the saturation of the chromaticity component after noise reduction is adjusted; finally, the image after brightness adjustment and chromaticity saturation adjustment is converted to a nonlinear HDR image of a set color gamut. This can adaptively adjust the brightness mapping of the image to obtain better brightness performance, reduce dead black, overexposure and brightness overflow, and effectively suppress or even eliminate the noise that may exist in the picture during the processing. This method can be widely used in a variety of HDR devices such as TVs, mobile phones, and monitors, allowing users to enjoy a more realistic visual experience when watching ordinary SDR images.

[0069] Furthermore, in a specific implementation, in the above-mentioned inverse tone mapping method provided in an embodiment of the present invention, step S101 converts the original image data into linear RGB data, and converts the linear RGB data into a LUV color space, which may specifically include: converting the original image data into linear RGB data using an inverse photoelectric conversion curve (OETF) according to the color space of the original image data; shifting the linear RGB data to increase the data bit depth; and converting the shifted linear RGB data into a LUV color space.

[0070] In implementation, the original image data is converted into linear RGB data using the OETF curve according to the color space of the original image data; the linear RGB data can then be left-shifted to increase the data bit depth and improve the calculation accuracy of subsequent operations; the shifted linear RGB data is then converted to the LUV color space to separate the color and brightness information.

[0071] Furthermore, in a specific implementation, in the above-mentioned inverse tone mapping method provided in an embodiment of the present invention, step S103 determines the image brightness protection point according to the brightness component after noise reduction, which may specifically include: calculating the average brightness of the image according to the brightness component after noise reduction; selecting a target brightness value between the average brightness of the image and the middle gray brightness; increasing the target brightness value by a set percentage, and using the increased brightness value as the image brightness protection point.

[0072] In implementation, the present invention can first calculate the average brightness of the image based on the brightness component after noise reduction; then select the larger value between the average brightness and the middle gray brightness (such as 18%) as the target brightness value, and finally increase the target brightness value by a set percentage (such as 10%) as the image brightness protection point.

[0073] Furthermore, in a specific implementation, in the above-mentioned inverse tone mapping method provided in an embodiment of the present invention, step S103 uses the image brightness protection point as a boundary to obtain a brightness mapping curve in segments, which may specifically include: linearly scaling the luminance component after noise reduction by a set multiple to form a first curve below the image brightness protection point; fitting a second curve passing through the image brightness protection point and the saturated brightness; splicing the first curve and the second curve according to cubic splines to obtain a brightness mapping curve; and the brightness mapping curve has a smooth transition at the image brightness protection point.

[0074] In practice, the present invention obtains a brightness mapping curve through segmented design. First, the portion of the curve below the brightness protection point can be linearly scaled to a set multiple of the original brightness, while the portion of the curve above the brightness protection point can be fitted with a curve passing through the brightness protection point and the saturated brightness. The two segments of the curve are spliced ​​by cubic splines to make the curve transition smooth at the brightness protection point.

[0075] Furthermore, in a specific implementation, in the above-mentioned inverse tone mapping method provided in an embodiment of the present invention, step S105 calculates a brightness adjustment curve according to the distribution data of the brightness histogram, which may specifically include: dividing the distribution data of the brightness histogram according to dead black, dark part, middle gray, highlight and overexposure; obtaining a brightness adjustment curve according to the proportion of each part after division; and the brightness adjustment curve passes through the middle gray point.

[0076] In implementation, the present invention can divide the histogram data according to dead black, dark part, middle gray, highlight, and overexposure, design an S-shaped or reverse S-shaped brightness adjustment curve according to the proportion of each part, and make the curve pass through the middle gray point.

[0077] Furthermore, in a specific implementation, the above-mentioned inverse tone mapping method provided in an embodiment of the present invention may also include: performing flat area detection on the nonlinear HDR image, and using the detected flat area as a mask; scaling the nonlinear HDR image to create an image pyramid; performing stripe detection and smoothing on the image pyramid layer by layer, and weightedly fusing the processed images of different layers to obtain a de-striped image; and superimposing the nonlinear HDR image and the de-striped image through a mask according to different weights to obtain a de-striped result.

[0078] In implementation, Figure 2 As shown, the present invention can first detect the flat area of ​​the nonlinear HDR image, and use the detected flat area as a mask. Then the nonlinear HDR image is scaled to create an image pyramid, and the image pyramid is striped and smoothed layer by layer at different scales. Subsequently, the images of different layers are weighted and added to obtain an overall strip-free image; the nonlinear HDR image and the strip-free image data are superimposed through a mask according to different weights to obtain a strip-free result. In this way, the stripes that may exist in the picture can be effectively suppressed or even eliminated.

[0079] Furthermore, in a specific implementation, the above steps may include: calculating the standard deviation of the flat area of ​​each layer of the image in blocks to obtain the block standard deviation, and calculating the difference between the pixel and the surrounding pixels pixel by pixel; selecting pixels that meet the set threshold as pixels of the strip boundary according to the comparison result of the difference and the block standard deviation; and smoothing the strip boundary.

[0080] In implementation, the present invention can calculate the standard deviation of the flat area of ​​each layer of the image in blocks, calculate the difference between the pixel and the surrounding pixels pixel by pixel, compare the relationship between the difference and the block standard deviation, and regard the pixels that meet the set threshold as strip boundary pixels. Use multiple pixels near the strip boundary to calculate the pixels at the boundary, and smooth the strip boundary.

[0081] Furthermore, in a specific implementation, the above-mentioned inverse tone mapping method provided in an embodiment of the present invention may also include: reducing the de-banding result to a target bit depth, and introducing random jitter in the process of reducing the bit depth; globally adding random luminance noise to the image reduced to the target bit depth, and outputting HDR data.

[0082] In implementation, Figure 2 As shown, the present invention can reduce the image bit depth to the target bit depth, introduce random jitter when reducing the bit depth, and avoid introducing stripes again. Then, a small amount of random brightness noise is added to the image globally, so that the image retains a certain film graininess, making the residual stripes less obvious, and also minimizing the stripes that may be introduced in the subsequent encoding process, while improving the image texture.

[0083] The HDR inverse tone mapping method provided by the present invention can be adaptively adjusted according to the actual scene, alleviating the problems of dark areas being too dark and highlights being overexposed that are easy to appear in the results. At the same time, a stripe removal scheme is designed, which is combined with a noise reduction algorithm to alleviate the problems of excessive noise in some scenes and the appearance of stripes in flat areas.

[0084] In the above embodiments, the inverse tone mapping method is described in detail, and the present invention also provides an embodiment corresponding to the inverse tone mapping inverse tone mapping device. It should be noted that the present invention describes the embodiments of the device part from two perspectives, one is based on the functional module perspective, and the other is based on the hardware perspective.

[0085] Figure 3 Schematic diagram of the structure of the inverse tone mapping device provided by an embodiment of the present invention. This embodiment is based on the perspective of functional modules, such as Figure 3 As shown, the device comprises:

[0086] The data conversion module 10 is used to convert the original image data into linear RGB data, and convert the linear RGB data into LUV color space to separate the brightness component and the chrominance component;

[0087] A noise reduction processing module 11 is used to perform corresponding noise reduction processing on the brightness component and the chrominance component respectively;

[0088] A mapping curve acquisition module 12 is used to determine the image brightness protection point according to the brightness component after noise reduction, and obtain the brightness mapping curve in segments with the image brightness protection point as the boundary;

[0089] The brightness histogram statistics module 13 is used to perform weighted addition of the brightness mapping curve and the historical frame brightness mapping curve, perform infinite impulse response filtering, use the filtered mapping curve to map the brightness channel, and calculate the brightness histogram;

[0090] An adjustment curve calculation module 14, used to calculate a brightness adjustment curve according to distribution data of a brightness histogram;

[0091] The brightness adjustment module 15 is used to perform weighted addition of the brightness adjustment curve and the historical frame brightness adjustment curve, perform infinite impulse response filtering, and use the filtered adjustment curve to adjust the brightness;

[0092] The non-linear conversion module 16 is used to adjust the saturation of the chroma component after noise reduction, and convert the image after brightness adjustment and chroma saturation adjustment to a non-linear HDR image of a set color gamut.

[0093] In the above-mentioned inverse tone mapping device provided in an embodiment of the present invention, the brightness mapping of the image can be adaptively adjusted through the interaction of the above-mentioned seven modules to obtain better brightness performance, reduce dead black, overexposure and brightness overflow phenomena, and effectively suppress or even eliminate the noise that may exist in the picture during the processing.

[0094] Since the embodiments of the apparatus part correspond to the embodiments of the method part, the embodiments of the apparatus part refer to the description of the embodiments of the method part, which will not be described here. And it has the same beneficial effects as the above-mentioned inverse tone mapping method.

[0095] Figure 4 Schematic diagram of the structure of the inverse tone mapping device provided by an embodiment of the present invention. This embodiment is based on the hardware perspective, such as Figure 4 As shown, the inverse tone mapping device includes:

[0096] A memory 20, used for storing computer programs;

[0097] The processor 21 is used to implement the steps of the inverse tone mapping method mentioned in the above embodiment when executing a computer program.

[0098] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU; the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0099] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the inverse tone mapping method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, the data involved in the inverse tone mapping method mentioned above, etc.

[0100] In some embodiments, the inverse tone mapping device may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26. Those skilled in the art will appreciate that Figure 4 The structure shown in does not constitute a limitation on the inverse tone mapping device, and may include more or fewer components than shown. The inverse tone mapping device provided by an embodiment of the present invention includes a memory and a processor. When the processor executes a program stored in the memory, it can implement the inverse tone mapping method mentioned above, and the effect is the same as above.

[0101] Finally, the present invention also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps recorded in the above method embodiment are implemented.

[0102] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and executes all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc. Various media that can store program codes. The computer-readable storage medium provided by the present invention can implement the above-mentioned inverse tone mapping method, and the effect is the same as above.

[0103] Finally, the present invention also provides an embodiment corresponding to a computer program product. The computer program product includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps described in the above-mentioned inverse tone mapping method embodiment are implemented. The computer program product provided by the present invention can implement the above-mentioned inverse tone mapping method, and the effect is the same as above.

[0104] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0105] The above is a detailed introduction to the inverse tone mapping method, device and storage medium provided by the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can refer to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A method for inverse tone mapping, characterized in that: The method comprises: Converting the original image data into linear RGB data, and converting the linear RGB data into a LUV color space to separate a brightness component and a chrominance component; Performing corresponding noise reduction processing on the brightness component and the chrominance component respectively; Determine the image brightness protection point according to the brightness component after noise reduction, and obtain the brightness mapping curve in segments with the image brightness protection point as the boundary; The brightness mapping curve is weightedly added to the brightness mapping curve of the historical frame, and infinite impulse response filtering is performed, and the brightness channel is mapped using the filtered mapping curve, and a brightness histogram is calculated; Calculating a brightness adjustment curve according to the distribution data of the brightness histogram; The brightness adjustment curve is weightedly added to the brightness adjustment curve of the historical frame, and infinite impulse response filtering is performed, and the brightness is adjusted using the filtered adjustment curve; The saturation of the chroma component after noise reduction is adjusted, and the image after brightness adjustment and chroma saturation adjustment is converted into a non-linear HDR image of a set color gamut.

2. The inverse tone mapping method according to claim 1, characterized in that: Converting the original image data into linear RGB data, and converting the linear RGB data into a LUV color space, including: According to the color space of the original image data, the original image data is converted into linear RGB data using an inverse photoelectric conversion curve; Performing a shift operation on the linear RGB data to increase the data bit depth; Convert the shifted linear RGB data to LUV color space.

3. The inverse tone mapping method according to claim 1, characterized in that: Determining the image brightness protection point according to the brightness component after noise reduction, comprising: Calculating the average brightness of the image according to the brightness component after noise reduction; Selecting a target brightness value between the average brightness of the image and the mid-grey brightness; The target brightness value is increased by a set percentage, and the increased brightness value is used as the image brightness protection point.

4. The inverse tone mapping method according to claim 1, characterized in that: Taking the image brightness protection point as the boundary, a brightness mapping curve is obtained segment by segment, including: The brightness component after noise reduction is linearly scaled by a set multiple to form a first curve below the image brightness protection point; Fitting a second curve passing through the image brightness protection point and saturation brightness; The first curve and the second curve are spliced ​​according to cubic splines to obtain a brightness mapping curve; the brightness mapping curve has a smooth transition at the image brightness protection point.

5. The inverse tone mapping method according to claim 1, characterized in that: Calculating a brightness adjustment curve according to the distribution data of the brightness histogram includes: Dividing the distribution data of the brightness histogram according to dead black, dark part, middle gray, highlight and overexposure; A brightness adjustment curve is obtained according to the proportions of the divided parts; the brightness adjustment curve passes through a mid-grey point.

6. The inverse tone mapping method according to claim 1, characterized in that: Also includes: Performing flat area detection on the nonlinear HDR image, and using the detected flat area as a mask; Scaling the nonlinear HDR image to create an image pyramid; Performing stripe detection and smoothing processing on the image pyramid layer by layer, and weighted fusion of processed images of different layers to obtain a stripe-free image; The nonlinear HDR image and the de-striped image are superimposed through the mask according to different weights to obtain a de-striped result.

7. The inverse tone mapping method according to claim 6, characterized in that: The stripe detection and smoothing process are performed layer by layer on the image pyramid, including: The standard deviation of the flat area of ​​each layer of the image is calculated separately to obtain the block standard deviation, and the difference between the pixel and the surrounding pixels is calculated pixel by pixel; According to the comparison result of the difference value and the block standard deviation, pixels meeting a set threshold are selected as pixels of the stripe boundary; Smooths strip boundaries.

8. The inverse tone mapping method according to claim 6, characterized in that: Also includes: Reduce the de-striping result to the target bit depth and introduce random jitter in the process of reducing the bit depth; Add random brightness noise globally to the image reduced to the target bit depth and output HDR data.

9. An inverse tone mapping device, characterized in that The device comprises: Memory for storing computer programs; A processor, configured to implement the steps of the inverse tone mapping method as described in any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the inverse tone mapping method according to any one of claims 1 to 8 are implemented.

Images