Image brightness adjustment method and apparatus thereof

Abstract

The application discloses an image brightness adjusting method and device, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a first residual image based on the brightness values of an SDR image and an HDR image, wherein the first residual image comprises brightness residual values between the SDR image and the HDR image; fitting a first brightness curve based on the first residual image and a first image, wherein the first brightness curve represents a mapping relationship between the brightness values of the first image and the brightness residual values in the first residual image, and the first image is the SDR image or the HDR image; and fitting a second brightness curve based on at least three key points in the first brightness curve and the first image, and adjusting the brightness values of the first image through the second brightness curve.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and specifically relates to an image brightness adjustment method and apparatus. Background Technology

[0002] With the development of electronic technology, most electronic devices can support the display of High Dynamic Range (HDR) images, but a small number of electronic devices only support the display of Standard Dynamic Range (SDR) images.

[0003] In the prior art, in order to display HDR images on electronic devices that only support SDR images, the electronic device can acquire an SDR image and the corresponding HDR image, and then obtain a gain image corresponding to the SDR image through the SDR image and the HDR image; thus, when displaying the image, the electronic device adjusts the brightness value of each pixel in the SDR image according to the gain image, so as to achieve the purpose of displaying HDR images on electronic devices that only support SDR images.

[0004] However, in the above method, since the electronic device needs to adjust the brightness value of the SDR image through the gain image, and the image occupies a large amount of storage space in the electronic device, the image file stored in the electronic device is large, resulting in a large storage pressure on the electronic device. Summary of the Invention

[0005] The purpose of this application is to provide an image brightness adjustment method and apparatus that can reduce the storage pressure on electronic devices when displaying HDR images.

[0006] In a first aspect, embodiments of this application provide an image brightness adjustment method, which includes: obtaining a first residual map based on the brightness values ​​of an SDR image and an HDR image, the first residual map including the brightness residual values ​​between the SDR image and the HDR image; fitting a first brightness curve based on the first residual map and a first image, the first brightness curve representing the mapping relationship between the brightness value of the first image and the brightness residual values ​​in the first residual map, the first image being an SDR image or an HDR image; fitting a second brightness curve based on at least three key points in the first brightness curve and the first image, and adjusting the brightness value of the first image using the second brightness curve.

[0007] Secondly, embodiments of this application provide an image brightness adjustment device, comprising: an acquisition module, a fitting module, and a processing module. The acquisition module is used to acquire a first residual map based on the brightness values ​​of an SDR image and an HDR image. The first residual map includes brightness residual values ​​between the SDR image and the HDR image. The fitting module is used to fit a first brightness curve based on the first residual map acquired by the acquisition module and a first image. The first brightness curve represents the mapping relationship between the brightness values ​​of the first image and the brightness residual values ​​in the first residual map. The first image is an SDR image or an HDR image. The processing module is used to fit a second brightness curve based on at least three key points in the first brightness curve obtained by the fitting module and the first image, and to adjust the brightness value of the first image using the second brightness curve.

[0008] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0009] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0010] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0011] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0012] In this embodiment, a first residual map can be obtained based on the brightness values ​​of an SDR image and an HDR image. The first residual map includes the brightness residual values ​​between the SDR image and the HDR image. Then, based on the first residual map and a first image, a first brightness curve is fitted. The first brightness curve represents the mapping relationship between the brightness value of the first image and the brightness residual values ​​in the first residual map. The first image is either an SDR image or an HDR image. Finally, based on at least three key points in the first brightness curve and the first image, a second brightness curve is fitted, and the brightness value of the first image is adjusted using the second brightness curve. In this solution, since the first residual image can characterize the brightness difference between SDR and HDR images, the electronic device can use the first brightness curve obtained by fitting the first residual image and the first image to characterize the gain image. Similarly, the second brightness curve obtained by fitting at least three key points in the first brightness curve and the first image can also characterize the gain image, thus enabling the display of HDR images on electronic devices that only support SDR images. Moreover, the electronic device only needs the first image and at least three key points to fit the second brightness curve, and adjust the brightness value of the first image using the second brightness curve. Therefore, compared to the gain image, the size of the stored file is reduced, thereby reducing the memory and performance pressure on the electronic device. In this way, the electronic device can display HDR images on electronic devices that only support SDR images while reducing the storage pressure on the electronic device. Attached Figure Description

[0013] Figure 1 This is one of the flowcharts of an image brightness adjustment method provided in the embodiments of this application;

[0014] Figure 2 This is a second flowchart of an image brightness adjustment method provided in an embodiment of this application;

[0015] Figure 3 This is the third flowchart of an image brightness adjustment method provided in the embodiments of this application;

[0016] Figure 4 This is a schematic diagram of an example of a brightness curve provided in an embodiment of this application;

[0017] Figure 5 This is the fourth flowchart of an image brightness adjustment method provided in the embodiments of this application;

[0018] Figure 6 This is the fifth flowchart of an image brightness adjustment method provided in the embodiments of this application;

[0019] Figure 7 This is the sixth flowchart of an image brightness adjustment method provided in the embodiments of this application;

[0020] Figure 8 This is the seventh flowchart of an image brightness adjustment method provided in the embodiments of this application;

[0021] Figure 9 This is the eighth flowchart of an image brightness adjustment method provided in the embodiments of this application;

[0022] Figure 10 This is the ninth flowchart of an image brightness adjustment method provided in the embodiments of this application;

[0023] Figure 11 This is one of the schematic diagrams illustrating an image display effect provided in the embodiments of this application;

[0024] Figure 12 This is a second example of an image display effect provided in an embodiment of this application;

[0025] Figure 13 This is a third example of an image display effect provided in the embodiments of this application;

[0026] Figure 14 This is a schematic diagram of the structure of an image brightness adjustment device provided in an embodiment of this application;

[0027] Figure 15 This is one of the hardware structure diagrams of an electronic device provided in the embodiments of this application;

[0028] Figure 16 This is a second schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0030] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0031] The terms "at least one," "at least one of," etc., used in the specification and claims of this application refer to any one, any two, or a combination of two or more of the included items. For example, at least one of a, b, and c can mean: "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more items, and its meaning is similar to that of "at least one."

[0032] The following provides a detailed explanation of the technical terms used in the image brightness adjustment methods, apparatus, electronic devices, storage media, and program products provided in the embodiments of this application.

[0033] HDR Images: HDR images refer to high dynamic range imaging images. An HDR file is a special graphics file format. In addition to the ordinary red (R), green (G), and blue (B) pixel information, each pixel in an HDR image also includes the actual brightness information of each pixel. This provides a wider dynamic range and more image detail. Compared with ordinary images, HDR images can provide a wider color range and more image detail, improve the contrast between light and dark areas, greatly restore the real environment, and present extremely high image quality.

[0034] SDR Images: SDR images refer to Standard Dynamic Range (SDR) imaging images. An SDR file is a special graphic file format where each pixel contains not only standard RGB pixel information but also the actual brightness information of each pixel. Compared to HDR images, SDR images do not have more comprehensive detail or a wider color gamut. When an SDR image is overexposed, information in the brighter areas will be lost; similarly, when an SDR image is underexposed, information in the darker areas will also be lost.

[0035] A histogram, also known as a bar chart, is a two-dimensional statistical chart. Its two axes represent the statistical sample and the measure of a certain attribute corresponding to that sample, respectively. In this application, the horizontal axis of the histogram represents the unique brightness value of each pixel in the image, and the vertical axis represents the number of unique brightness values ​​contained in each pixel of the image.

[0036] Reference diffuse whiteness: Reference diffuse whiteness typically refers to the brightness level of light reflected from an object's surface under standard lighting conditions. It is used to describe and measure the whiteness of an object's surface. Reference diffuse whiteness can be measured using specific instruments, such as whiteness meters. These instruments usually emit light within a certain wavelength range and measure the intensity of the light reflected from the object's surface. By comparing the measured value with a known standard value, the whiteness of the object's surface can be determined.

[0037] The image brightness adjustment method, apparatus, electronic device, storage medium, and program product provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0038] The image brightness adjustment method and apparatus provided in this application can be applied to scenes displaying HDR images. For example, displaying an HDR image corresponding to an edited SDR image, or displaying an HDR image corresponding to a captured SDR image.

[0039] Scenario 1: A user can trigger an electronic device to edit an SDR image in an editing application, such as adding filters or new image elements, resulting in an edited SDR image. The electronic device can then perform HDR image reconstruction on the edited SDR image to obtain an HDR image. Next, the electronic device can use an image processing unit to calculate the brightness ratio between the brightness value of each pixel in the edited SDR image and the brightness value of each pixel in the HDR image, obtaining a first residual image. Finally, the image processing unit in the electronic device can input each brightness residual value in the first residual image and the brightness value of each pixel in the SDR image into a cubic spline function to fit the image. A first brightness curve is obtained. When displaying an image, the image processing end in the electronic device can transmit at least three key points from the SDR image and the first brightness curve to the display end in the electronic device. The display end can input the at least three key points into a cubic spline function to fit and obtain a second brightness curve. The brightness value of each pixel of the SDR image is input into the second brightness curve to obtain the brightness residual value corresponding to each pixel of the SDR image. The brightness value of each pixel of the SDR image is multiplied by the brightness residual value corresponding to each pixel of the SDR image to adjust the brightness value of the SDR image. In this way, the electronic device can display HDR images on electronic devices that only support SDR images.

[0040] Scenario 2: A user can trigger an electronic device to capture an SDR image. The electronic device can then reconstruct an HDR image from the SDR image to obtain an HDR image. Next, the electronic device can use an image processing unit to calculate the brightness ratio between the brightness value of each pixel in the SDR image and the brightness value of each pixel in the HDR image, obtaining a first residual image. Then, the image processing unit in the electronic device can input each brightness residual value from the first residual image and the brightness value of each pixel in the SDR image into a cubic spline function to fit a first brightness curve. When displaying the image, the image processing unit in the electronic device can transmit at least three key points from the SDR image and the first brightness curve to the display unit in the electronic device. The display unit can input at least three key points into a cubic spline function to fit a second brightness curve, and input the brightness value of each pixel in the SDR image into the second brightness curve to obtain the brightness residual value corresponding to each pixel in the SDR image. Using the brightness residual value corresponding to each pixel in the SDR image, a multiplication operation is performed on the brightness value of each pixel in the SDR image to adjust the brightness value of the SDR image. In this way, the electronic device can display an HDR image on an electronic device that only supports SDR images.

[0041] It should be noted that the above scenarios are merely exemplary examples of some possible applications of the embodiments of this application. In actual implementation, the embodiments of this application can also be applied to any possible scenarios such as image transcoding and video playback. The embodiments of this application are not limited here.

[0042] In the image brightness adjustment method and apparatus provided in this application embodiment, since the first residual map can characterize the brightness difference between SDR and HDR images, the electronic device can use the first brightness curve obtained by fitting the first residual map and the first image to characterize the gain image. Similarly, the second brightness curve obtained by fitting at least three key points in the first brightness curve and the first image can also characterize the gain image, thereby enabling the display of HDR images on electronic devices that only support SDR images. Moreover, the electronic device only needs to fit the first image and at least three key points to obtain the second brightness curve, and adjust the brightness value of the first image using the second brightness curve. Therefore, compared to the gain image, the size of the file stored on the display end is reduced, thereby reducing the memory and performance pressure on the electronic device. In this way, the electronic device can display HDR images on electronic devices that only support SDR images while reducing the storage pressure on the electronic device.

[0043] The image brightness adjustment method provided in this application can be executed by an image brightness adjustment device, which can be an electronic device or a functional module within an electronic device. The following description uses an electronic device as an example to illustrate the technical solution provided in this application.

[0044] This application provides an image brightness adjustment method. Figure 1 A flowchart of an image brightness adjustment method provided in an embodiment of this application is shown. Figure 1 As shown, the image brightness adjustment method provided in this application embodiment may include the following steps 201 to 203.

[0045] Step 201: The electronic device obtains the first residual image based on the brightness values ​​of the SDR and HDR images.

[0046] In this embodiment of the application, the first residual map includes the brightness residual value between the SDR image and the HDR image.

[0047] Optionally, in this embodiment of the application, the electronic device can acquire SDR images and HDR images through the image processing terminal in the electronic device, and acquire a first residual image based on the brightness values ​​of the SDR images and HDR images.

[0048] Optionally, in this embodiment, the image processing terminal can be a shooting application, image editing application, or video playback application in an electronic device. The specific application can be determined based on actual usage requirements, and this embodiment does not impose any limitations.

[0049] In this embodiment of the application, the SDR image and HDR image mentioned above can be images from the same shooting scene.

[0050] Optionally, in the embodiments of this application, the SDR image and HDR image can be images with different bit sizes.

[0051] For example, the SDR image mentioned above can be an 8-bit SDR image, and the HDR image mentioned above can be a 10-bit HDR image.

[0052] Optionally, in this embodiment of the application, the SDR image and HDR image can be images with different frame rates.

[0053] For example, the SDR image can be a 30-frame SDR image, and the HDR image can be a 60-frame HDR image.

[0054] Optionally, in the embodiments of this application, the SDR image and HDR image described above can be images with different encoding methods.

[0055] For example, the SDR image mentioned above can be an image encoded in Gamma, and the HDR image mentioned above can be an image encoded in product quantization (PQ).

[0056] Optionally, in this embodiment, the SDR and HDR images can be images input by the user; or, the SDR and HDR images can be images acquired by the electronic device through a third-party application; or, the SDR and HDR images can be images captured by the electronic device. The specific method can be determined according to actual usage requirements, and this embodiment does not impose any limitations.

[0057] Optionally, in this embodiment of the application, the electronic device can capture an SDR image through a shooting application, and then obtain an HDR image corresponding to the SDR image through an HDR reconstruction algorithm.

[0058] In this embodiment of the application, the electronic device can calculate the brightness ratio between the brightness value of each pixel in the SDR image and the corresponding brightness value in the HDR image to obtain the first residual image mentioned above.

[0059] In this embodiment of the application, the SDR image, HDR image and the first residual image have the same image size.

[0060] For example, if the image size of the SDR image is 3×3, then the image size of the HDR image and the first residual image is also 3×3.

[0061] Optionally, in the embodiments of this application, combined with Figure 1 ,like Figure 2 As shown, step 201 can be implemented through steps 201a and 201d as described below.

[0062] Step 201a: The electronic device performs a linear domain transformation on the HDR image to obtain the transformed HDR image.

[0063] In this embodiment of the application, the electronic device can convert the HDR image to the linear domain using the PQ encoding and decoding formula, specifically using the following formula (1).

[0064] P Base_Linear =EOTF PQ (P Base / 1023) (1)

[0065] Among them, P Base_Linear For the converted HDR image, P Base For HDR images, 1023 represents the bit size of the HDR image.

[0066] Step 201b: The electronic device performs linear domain transformation on the SDR image to obtain the transformed SDR image, and aligns the brightness of the transformed SDR image with the reference diffuse white brightness to obtain a brightness-aligned SDR image.

[0067] In this embodiment, the electronic device can convert the SDR image to the linear domain using the Gamma encoding and decoding formula, and align the converted SDR image with the reference diffuse white brightness using the standard recommended value of 203 nits to obtain the brightness-aligned SDR image. Specifically, this can be achieved using the following formula (2).

[0068] P Alter_Linear =EOTF Gamma (P Alter / 255)*0.0203 (2)

[0069] Among them, P Alter_Linear For the brightness-aligned SDR image, P Alter 255 represents the bit size of the SDR image.

[0070] Step 201c: The electronic device performs luminance extraction processing on the converted HDR image to obtain a first luminance map, and performs luminance extraction processing on the luminance-aligned SDR image to obtain a second luminance map.

[0071] In this embodiment of the application, the electronic device can extract the first luminance map from the R, G, and B channels of the converted HDR image, which can be achieved by the following formula (3).

[0072] P Luma_Base =0.8*max(R,G,B)+0.05*R+0.1*G+0.05*B (3)

[0073] Among them, P Luma_Base Here is the first brightness map, where R is the pixel value of the red channel in the converted HDR image, G is the pixel value of the green channel in the converted HDR image, and B is the pixel value of the blue channel in the converted HDR image.

[0074] In this embodiment of the application, the electronic device can extract the second luminance map from the R, G, and B channels of the luminance-aligned SDR image, which can be achieved by the following formula (4).

[0075] P Luma_Alter =0.8*max(R,G,B)+0.05*R+0.1*G+0.05*B (4)

[0076] Among them, P Luma_Alter This is the second luminance map, where R is the pixel value of the red channel in the luminance-aligned SDR image, G is the pixel value of the green channel in the luminance-aligned SDR image, and B is the pixel value of the blue channel in the luminance-aligned SDR image.

[0077] Step 201d: The electronic device calculates the first residual map based on the brightness value of each pixel in the first brightness map and the brightness value of each pixel in the second brightness map.

[0078] In this embodiment of the application, the electronic device can calculate the ratio between the brightness value of each pixel in the first brightness map and the brightness value of each pixel in the second brightness map to obtain the first residual map, which can be achieved by the following formula (5).

[0079] P Residul =(P Luma_Base / P Luma_Alter (5)

[0080] Among them, P Residul For the first residual, P Luma_Base For the first brightness map, P Luma_Alter This is the second brightness map.

[0081] Optionally, in this embodiment of the application, after the electronic device calculates the ratio between the brightness value of each pixel in the first brightness map and the brightness value of each pixel in the second brightness map, it can perform a log2 operation to obtain the first residual map mentioned above, which can be achieved by the following formula (6).

[0082] P Residul =log2(P Luma_Base / P Luma_Alter (6)

[0083] Among them, P Residul For the first residual, P Luma_Base For the first brightness map, P Luma_Alter This is the second brightness map.

[0084] In this embodiment of the application, the first residual map obtained by the electronic device through log2 operation can make the brightness residual value contained in the first residual map more closely match the intensity change perceived by the human eye.

[0085] Step 202: The electronic device fits the first brightness curve based on the first residual map and the first image.

[0086] In this embodiment of the application, the first brightness curve represents the mapping relationship between the brightness value of the first image and the brightness residual value in the first residual map, wherein the first image is an SDR image or an HDR image.

[0087] For example, the first image described above can be a reference image.

[0088] Optionally, in this embodiment, the selection of the first image can be preset by the electronic device; or user-defined; or determined by the electronic device based on its display capabilities. The specific selection can be determined according to actual usage needs, and this embodiment does not impose any limitations.

[0089] In this embodiment, the electronic device can obtain a first brightness curve by fitting the brightness residual value in the first residual map with the brightness value in the first image. For details, please refer to the following embodiments; to avoid repetition, they will not be repeated here.

[0090] Optionally, in the embodiments of this application, combined with Figure 1 ,like Figure 3 As shown, step 202 above can be specifically implemented through steps 202a to 202d below.

[0091] Step 202a: The electronic device samples the brightness residual value in the first residual image to obtain at least one brightness residual value, and samples the brightness value of the pixel point in the first image corresponding to the at least one brightness residual value to obtain at least one brightness value.

[0092] Optionally, in this embodiment of the application, the electronic device can sample the brightness residual value in the first residual map through the cumulative histogram to obtain at least one brightness residual value.

[0093] For example, the electronic device can obtain the first full image histogram corresponding to the first residual map by formula (7) below, and then sample the full image histogram at the same frequency every 10% of the image size by formula (8) below to obtain the first cumulative histogram, wherein the brightness residual value included in the first cumulative histogram is at least one of the above brightness residual values.

[0094] H Global_Residual =f Hist (P Residual (7)

[0095] Among them, H Global_Residual P is the first full-plot histogram corresponding to the first residual plot. Residual This is the first residual plot.

[0096] CH Global_Residual =f CDF (H Global_Residual (8)

[0097] Among them, CH Global_Residual H is the first cumulative histogram corresponding to the first residual plot. Global_Residual This is the first full-map histogram corresponding to the first residual plot.

[0098] Optionally, in this embodiment of the application, the electronic device may perform grid sampling on the first residual map at intervals of 10% of the image size to obtain at least one brightness residual value.

[0099] Optionally, in this embodiment of the application, the electronic device may merge pixels of the first residual map at 10% of the image size to obtain at least one brightness residual value.

[0100] Optionally, in this embodiment of the application, the electronic device can sample the brightness values ​​in the first image using a cumulative histogram to obtain at least one brightness value.

[0101] For example, the electronic device can obtain the second full-image histogram corresponding to the first image by formula (9) below, and then sample the full-image histogram at equal frequency every 10% of the image size by formula (10) below to obtain the second cumulative histogram, wherein the brightness value included in the second cumulative histogram is at least one of the above brightness values.

[0102] H Global_Base =f Hist (P Luma_Base (9)

[0103] Among them, H Global_Base P is the histogram of the second full image corresponding to the first image. Luma_Base This is the brightness map corresponding to the first image.

[0104] CH Global_Base =f CDF (H Global_Base (10)

[0105] Among them, CH Global_Base H is the second cumulative histogram corresponding to the first image. Global_Base This is the histogram of the second full image corresponding to the first image.

[0106] Optionally, in this embodiment of the application, the electronic device may perform grid sampling on the brightness map corresponding to the first image, i.e., the first brightness map, at intervals of 10% of the image size, to obtain at least one brightness value.

[0107] Optionally, in this embodiment of the application, the electronic device may merge the first brightness map into pixels at 10% of the image size to obtain at least one brightness value as described above.

[0108] Step 202b: The electronic device fits a third luminance curve based on at least one luminance residual value and at least one luminance value.

[0109] In the embodiments of this application, the above-mentioned at least one brightness residual value corresponds one-to-one with at least one brightness value.

[0110] Optionally, in this embodiment of the application, the electronic device can input at least one luminance residual value and at least one luminance value into the curve model. The model can determine the best fitting parameters through the least squares algorithm, and fill the best fitting parameters into the polynomial to obtain the third luminance curve mentioned above.

[0111] Optionally, in this embodiment, the curve model can be any of the following: a polynomial model, an exponential model, or a logarithmic function model, etc. The specific model can be determined according to actual usage requirements, and this embodiment does not impose any limitations.

[0112] Optionally, in this embodiment of the application, the electronic device can calculate the ratio between each of the at least one brightness residual values ​​and each of the at least one brightness values ​​to obtain at least one third brightness residual value, which can be achieved by the following formula (11); then, the at least one brightness residual value and the at least one brightness value are input into the curve model to obtain the above-mentioned third brightness curve.

[0113] R Global[i] =CH Global_Residual[i] / CH Global_Base[i] (11)

[0114] Among them, R Global[i] For at least one third luminance residual value, CH Global_Residual[i] CH is at least one luminance residual value. Global_Base[i] It is at least one brightness value.

[0115] For example, such as Figure 4 As shown, the electronic device can obtain 7 key points through the above polynomial model. The horizontal axis of each of the 7 key points is the brightness value, and the vertical axis of each of the 7 key points is the brightness residual value. Then, the electronic device can obtain the best fitting parameters according to the least squares method. The electronic device can then fill the polynomial in the polynomial model with the best fitting parameters to obtain the third brightness curve.

[0116] Step 202c: The electronic device performs image segmentation on the first residual image to obtain N first pixel units, and performs image segmentation on the brightness image of the first image to obtain N second pixel units, where N is an integer greater than 1.

[0117] In this embodiment of the application, the area of ​​each of the above N first pixel units is the same.

[0118] Optionally, in this embodiment of the application, the electronic device can adjust the area of ​​each pixel unit in the N first pixel units according to the requirements of screen transition.

[0119] It should be noted that the area of ​​each first pixel unit is the same after the adjustment.

[0120] Optionally, in this embodiment, each of the N first pixel units can overlap.

[0121] Optionally, in this embodiment of the application, the electronic device can divide the first residual image into N first pixel units according to a preset size.

[0122] For example, the electronic device can divide the first residual map into N first pixel units by dividing it into 10% of the image size at each interval.

[0123] In this embodiment of the application, the area of ​​each of the above N second pixel units is the same.

[0124] Optionally, in this embodiment of the application, the electronic device can adjust the area of ​​each pixel unit in the N second pixel units according to the requirements of screen transition.

[0125] It should be noted that the area of ​​each second pixel unit is the same after the adjustment.

[0126] Optionally, in this embodiment, each of the N second pixel units can overlap.

[0127] Optionally, in this embodiment of the application, the electronic device can divide the first residual image into N first pixel units according to a preset size.

[0128] For example, the electronic device can divide the brightness map of the first image, i.e. the first brightness map mentioned above, into N first pixel units according to each interval of 10% of the image size.

[0129] Step 202d: The electronic device fits the first brightness curve based on N first pixel units, N second pixel units and the third brightness curve.

[0130] In this embodiment, the electronic device can obtain a first brightness curve by fitting the brightness residual value of each of the N first pixel units, the brightness value of each of the N second pixel units, and a third brightness curve. For details, please refer to the following embodiments; to avoid repetition, they will not be repeated here.

[0131] In this embodiment, the electronic device can characterize the gain image through the first brightness curve. Even without a gain image, the electronic device can adjust the brightness value of the first image through the first brightness curve, which improves the flexibility of the electronic device in adjusting the image brightness.

[0132] Optionally, in the embodiments of this application, combined with Figure 3 ,like Figure 5 As shown, step 202d above can be implemented through steps 301 and 302 below.

[0133] Step 301: The electronic device calculates the first brightness residual value corresponding to the i-th second pixel unit based on the brightness residual value of the i-th first pixel unit and the brightness value of the i-th second pixel unit, i∈[1,N].

[0134] In this embodiment of the application, the electronic device can calculate the ratio between the brightness residual value of each of the N first pixel units and the brightness value of each of the N second pixel units to obtain the N first brightness residual values ​​corresponding to each of the N second pixel units.

[0135] In this embodiment, N second pixel units correspond one-to-one with N first brightness residual values.

[0136] For example, the electronic device can calculate the ratio between the luminance residual value of the i-th first pixel unit and the luminance value of the i-th second pixel unit to obtain the first luminance residual value corresponding to the i-th second pixel unit. Then, the electronic device can calculate the ratio between the luminance residual value of the (i+1)-th first pixel unit and the luminance value of the (i+1)-th second pixel unit to obtain the first luminance residual value corresponding to the (i+1)-th second pixel unit. And so on, until the electronic device calculates the ratio between the luminance residual value of the N-th first pixel unit and the luminance value of the N-th second pixel unit to obtain the first luminance residual value corresponding to the N-th second pixel unit.

[0137] Step 302: The electronic device interpolates the third brightness curve based on the brightness values ​​of N second pixel units and the first brightness residual values ​​corresponding to the N second pixel units to obtain the first brightness curve.

[0138] In this embodiment, the electronic device can interpolate the third brightness curve using the brightness value of each of the N second pixel units and the N first brightness residual values ​​corresponding to the N second pixel units to obtain the first brightness curve. It should be noted that the specific process can be found in the following embodiments, and will not be repeated here to avoid repetition.

[0139] In the embodiment of the present application, the electronic device interpolates the third brightness curve based on the brightness values of N second pixel units and the first brightness residual values corresponding to the N second pixel units, so as to obtain the first brightness curve, which can make the first brightness curve more comprehensively represent the residual characteristics, thereby improving the accuracy of adjusting the image brightness value through the first brightness curve.

[0140] Optionally, in the embodiment of the present application, in combination with Figure 5 , as Figure 6 shown, the above step 302 can be specifically implemented through the following steps 302a to 302c.

[0141] Step 302a: The electronic device determines the brightness interval and the second brightness residual value corresponding to the brightness value of the i-th second pixel unit from the third brightness curve based on the brightness value of the i-th second pixel unit.

[0142] In the embodiment of the present application, the electronic device can obtain the brightness value of the i-th second pixel unit, and compare the brightness value of the i-th second pixel unit with the brightness values included in the third brightness curve to obtain the brightness interval corresponding to the brightness value of the i-th second pixel unit; and input the brightness value of the i-th second pixel unit into the third brightness curve to obtain the second residual value corresponding to the i-th second pixel unit.

[0143] Exemplarily, assume that the brightness value of the i-th second pixel unit is a, and this brightness value a corresponds to the brightness interval from b to c (b < c) in the third brightness curve, that is, b < a < c. Then, input the brightness value a into the third brightness curve to obtain the second brightness residual value corresponding to the brightness value a, which can be specifically implemented through the following formula (12).

[0144] R Global_a =(c - a) / (c - b)*R Global_b +(a - b) / (c - b)*R Global_c (12)

[0145] Where, R Global_a is the second residual value corresponding to the i-th second pixel unit, c and b are the brightness intervals, R Global_b is the brightness residual value corresponding to the brightness value b, and R Global_c is the brightness residual value corresponding to the brightness value c.

[0146] Step 302b: The electronic device calculates the mean value of the i-th second brightness residual value and the i-th first brightness residual value to obtain the i-th third brightness residual value.

[0147] In this embodiment, the electronic device can calculate the average of the i-th second brightness residual value and the i-th first brightness residual value to obtain the i-th third brightness residual value; then, the electronic device can calculate the average of the (i+1)-th second brightness residual value and the (i+1)-th first brightness residual value to obtain the (i+1)-th third brightness residual value; and so on, until the electronic device can calculate the average of the N-th second brightness residual value and the N-th first brightness residual value to obtain the N-th third brightness residual value, thus obtaining N third brightness residual values.

[0148] For example, the electronic device can calculate the average of the i-th second luminance residual value and the i-th first luminance residual value using the following formula (13) to obtain the i-th third luminance residual value.

[0149] C Residual_a =(R Global_a +R Global[i]_a ) / 2 (13)

[0150] Among them, C Residual_a R is the i-th third luminance residual value. Global_a R is the second luminance residual value corresponding to the i-th second pixel unit. Global[i]_a This is the first brightness residual value corresponding to the i-th second pixel unit.

[0151] Optionally, in this embodiment of the application, the electronic device can calculate the weighted average of the i-th second brightness residual value and the i-th first brightness residual value to obtain the i-th third brightness residual value, and so on, to obtain the N-th third brightness residual value, so as to obtain N third brightness residual values.

[0152] Step 302c: The electronic device interpolates the corresponding brightness interval in the third brightness curve based on each of the N third brightness residual values ​​to obtain the first brightness curve.

[0153] Optionally, in this embodiment of the application, the electronic device can linearly interpolate each of the N third brightness residual values ​​into the brightness range corresponding to the brightness value of each of the N second pixel units in the third brightness curve to obtain the first brightness curve.

[0154] In this embodiment, the electronic device interpolates the third brightness curve using the brightness values ​​of N second pixel units and the first brightness residual values ​​corresponding to the N second pixel units to obtain the first brightness curve. This allows the first brightness curve to more comprehensively characterize the residual features, thereby improving the accuracy of adjusting the image brightness value using the first brightness curve.

[0155] Step 203: The electronic device fits a second brightness curve based on at least three key points in the first brightness curve and the first image, and adjusts the brightness value of the first image using the second brightness curve.

[0156] Optionally, in this embodiment of the application, the image processing end in the electronic device can transmit at least three key points in the first brightness curve and the first image to the display end in the electronic device. Then, the display end can fit a second brightness curve based on the at least three key points in the first brightness curve and the first image, and adjust the brightness value of the first image based on the second brightness curve.

[0157] In this embodiment of the application, the electronic device can filter the key points in the first brightness curve to obtain at least three key points, and then transmit the first image and the at least three key points to the display end in the electronic device.

[0158] Optionally, in this embodiment of the application, the electronic device can calculate the slope between each pair of adjacent key points using the brightness residual value between each pair of adjacent key points among the at least three key points, so as to obtain the slope corresponding to each of the at least three key points. Then, the electronic device can transmit the first image, the at least three key points and the slope corresponding to each of the at least three key points to the display end in the electronic device.

[0159] Optionally, in this embodiment of the application, after obtaining at least three key points and the slope corresponding to each of the at least three key points, the electronic device can add the at least three key points and the slope corresponding to each of the at least three key points to the extension file or header file of the first image.

[0160] Optionally, in this embodiment of the application, the electronic device may add at least three key points and the slope corresponding to each of the at least three key points to the extension file or header file of the first image in a first order.

[0161] Optionally, in the embodiments of this application, the first order described above may be preset by the electronic device or user-defined.

[0162] For example, the electronic device can add at least three key points and the slope corresponding to each of the at least three key points to the extended file of the first image using Table 1 below.

[0163] Table 1

[0164]

[0165]

[0166] Among them, PLuma_Base[i] For each pixel in the first image, Res_max is the pixel value of the second keypoint described below, C Residual[i] Let be the brightness residual value corresponding to each of the at least three keypoints, and slope[i] be the slope corresponding to each of the at least three keypoints.

[0167] Optionally, in the embodiments of this application, the electronic device can also adaptively display and map images of multiple different dynamic contents on the browsing interface through the above embodiments, which can ensure the maximum restoration of the creator's creative intent and artistic expression while maintaining the overall page effect.

[0168] In the image brightness adjustment method provided in this application embodiment, the electronic device can obtain a first residual map based on the brightness values ​​of a standard dynamic range (SDR) image and a high dynamic range (HDR) image. The first residual map includes the brightness residual values ​​between the SDR image and the HDR image. Then, based on the first residual map and a first image, a first brightness curve is fitted. The first brightness curve represents the mapping relationship between the brightness value of the first image and the brightness residual values ​​in the first residual map. The first image is an SDR image or an HDR image. Finally, based on at least three key points in the first brightness curve and the first image, a second brightness curve is fitted, and the brightness value of the first image is adjusted using the second brightness curve. In this solution, since the first residual image can characterize the brightness difference between SDR and HDR images, the first brightness curve obtained by fitting the first residual image and the first image can be used to characterize the gain image. Similarly, the second brightness curve obtained by fitting at least three key points in the first brightness curve and the first image can also characterize the gain image, thus enabling the display of HDR images on electronic devices that only support SDR images. Moreover, the electronic device only needs to fit the first image and at least three key points to obtain the second brightness curve, and adjust the brightness value of the first image using the second brightness curve. Therefore, compared to the gain image, the size of the file stored on the display is reduced, thereby reducing the memory and performance pressure on the electronic device. In this way, the electronic device can ensure the display of HDR images on electronic devices that only support SDR images while reducing the storage pressure on the electronic device.

[0169] Optionally, in the embodiments of this application, combined with Figure 1 ,like Figure 7 As shown, prior to step 203 above, the image brightness adjustment method provided in this application embodiment further includes steps 401 and 402 as described below.

[0170] Step 401: The electronic device performs gradient processing on the first brightness curve, removes key points in the first brightness curve whose gradient is less than or equal to 0, and obtains the fourth brightness curve.

[0171] In this embodiment of the application, the electronic device can differentiate the first brightness curve to obtain the differentiated first brightness curve, and then input the brightness residual value of each key point in the first brightness curve into the differentiated first brightness curve to obtain the gradient corresponding to each key point in the first brightness curve; then compare the gradient corresponding to each key point in the first brightness curve with 0, thereby eliminating key points in the first brightness curve whose gradient is less than or equal to 0, to obtain the fourth brightness curve.

[0172] Optionally, in this embodiment of the application, the electronic device can take the partial derivative of the first brightness curve to obtain the partial derivative first brightness curve, and then input the brightness residual value of each key point in the first brightness curve into the partial derivative first brightness curve to obtain the gradient corresponding to each key point in the first brightness curve; then compare the gradient corresponding to each key point in the first brightness curve with 0, thereby eliminating key points in the first brightness curve whose gradient is less than or equal to 0, to obtain the fourth brightness curve.

[0173] Step 402: Based on the brightness value corresponding to each key point in the fourth brightness curve, the electronic device removes the first key point in the fourth brightness curve that satisfies the first condition, and obtains at least three key points.

[0174] In this embodiment of the application, the first condition is that the brightness difference between the brightness values ​​corresponding to two adjacent key points is less than or equal to a preset brightness threshold, and the first key point is the second key point among the two adjacent key points that satisfy the first condition.

[0175] In this embodiment of the application, the electronic device can calculate the brightness difference between the brightness values ​​corresponding to each two adjacent key points in the fourth brightness curve, and then compare the brightness difference between the brightness values ​​corresponding to each two adjacent key points with a preset brightness threshold. If the brightness difference is less than or equal to the preset brightness threshold, the second key point among each two adjacent key points is removed to obtain the above-mentioned at least three key points.

[0176] For example, taking a fourth brightness curve that includes five key points as an example, the electronic device can calculate the brightness difference between the first and second key points to obtain a brightness difference value 'a'. This brightness difference value 'a' is compared with a preset brightness threshold 'b'. If the brightness difference value 'a' is less than the preset brightness threshold 'b', the electronic device can eliminate the second key point. Then, the electronic device can calculate the brightness difference between the first and third key points to obtain a brightness difference value 'c'. This brightness difference value 'c' is compared with the preset brightness threshold 'b'. If the brightness difference value 'c' is less than the preset brightness threshold 'b', the electronic device can eliminate the second key point. In addition to the third key point, the electronic device can then calculate the brightness difference between the first and fourth key points, obtaining a brightness difference value d. This brightness difference value d is compared with a preset brightness threshold b. If the brightness difference value d is greater than the preset brightness threshold b, the electronic device can retain the fourth key point. Finally, the electronic device can calculate the brightness difference between the fourth and fifth key points, obtaining a brightness difference value e. This brightness difference value e is compared with a preset brightness threshold b. If the brightness difference value e is greater than the preset brightness threshold b, the electronic device can retain the fifth key point, thus obtaining three key points.

[0177] It should be noted that when removing key points in the fourth brightness curve, the electronic device needs to retain the last set of sampling points of the fourth brightness curve, that is, retain the brightness residual value corresponding to the maximum brightness value in the first brightness image, in order to limit the maximum range of the brightness residual value corresponding to the first brightness curve.

[0178] It can be understood that the electronic device needs to retain the last set of sampling points of the fourth brightness curve, which means that even if the brightness difference between the last set of sampling points of the fourth brightness curve is less than or equal to the preset brightness threshold, the electronic device will not remove the last key point in the fourth brightness curve.

[0179] In this embodiment, the electronic device checks the first brightness curve by gradient and removes key points with a gradient less than or equal to 0, ensuring that the first brightness curve is monotonically increasing and avoiding brightness reversal. By checking the brightness difference between every two adjacent points in the first brightness curve and removing key points with a brightness difference less than a preset brightness threshold, discontinuity problems caused by uneven brightness transitions can be avoided.

[0180] Optionally, in the embodiments of this application, combined with Figure 1 ,like Figure 8 As shown, the step 203 above, "fitting a second brightness curve based on at least three key points in the first brightness curve and the first image", can be specifically achieved through the following step 203a.

[0181] Step 203a: When the first ratio is less than the brightness value corresponding to the second key point, the electronic device fits the second brightness curve based on the brightness residual value of each of the at least three key points, the slope corresponding to each of the at least three key points, and the brightness value of the first image.

[0182] In this embodiment of the application, the first ratio is the ratio between the peak brightness of the screen in the electronic device and the reference diffuse white brightness, and the second key point is the key point with the largest corresponding brightness value among at least three key points.

[0183] In this embodiment of the application, the electronic device can use a cubic spline function of the form F(x)=C3*x^3+C2*x^2+C1*x+C0 to fit the second brightness curve.

[0184] For example, the electronic device can obtain the specific value of each parameter in the cubic spline function based on the brightness residual value of each of the at least three key points, the slope corresponding to each of the at least three key points, and the brightness value of the first image, so as to obtain the second brightness curve. Specifically, this can be achieved by the following formula (14).

[0185]

[0186] Where slope1 and slope2 are the slopes corresponding to every two adjacent key points in at least three key points, y1 and y2 are the brightness residual values ​​corresponding to every two adjacent key points in at least three key points, and x1 and x2 are the brightness values ​​corresponding to every two adjacent key points in the first image.

[0187] Optionally, in this embodiment of the application, when the first ratio is greater than or equal to the brightness value corresponding to the second key point and the first image is an HDR image, the electronic device can directly display the first image.

[0188] Optionally, in this embodiment of the application, when the first ratio is greater than or equal to the brightness value corresponding to the second key point and the first image is an SDR image, the electronic device can adjust the brightness value of the SDR image according to the above embodiment and then display the adjusted SDR image.

[0189] Optionally, in this embodiment of the application, the electronic device can extract the brightness value of the first image through RGB. The specific implementation process can be found in the above embodiments, and will not be repeated here to avoid repetition.

[0190] In this embodiment, the electronic device can ensure the quality of the mapped content with only a tiny increase in file size when different screen capabilities and display requirements are met, thereby reducing memory and performance pressure while ensuring rendering quality.

[0191] Optionally, in the embodiments of this application, combined with Figure 1 ,like Figure 9 As shown, the "adjusting the brightness value of the first image through the second brightness curve" in step 203 above can be specifically achieved through steps 501 to 504 below.

[0192] Step 501: The electronic device calculates the second residual map based on the brightness value of the first image and the second brightness curve.

[0193] In this embodiment of the application, the electronic device can input the brightness value of each pixel in the first image into the second brightness curve to obtain the brightness residual value corresponding to each pixel in the first image; then, according to the position information of each pixel in the first image, the brightness residual value corresponding to each pixel in the first image is filled into the corresponding position to obtain the second residual map.

[0194] It is understood that the image size of the second residual image is the same as that of the first image.

[0195] Step 502: The electronic device calculates the scaling factor based on the first ratio, the second ratio, and the second key point.

[0196] In this embodiment of the application, the second key point is the key point with the largest corresponding brightness value among at least three key points.

[0197] In this embodiment of the application, the electronic device can calculate the scaling factor using the following formula (15), which is as follows:

[0198] W = sign(H) Alter - H Base )*clip((H- H Base ) / (H Alter - H Base ),0,1) (15)

[0199] Where W is the scaling factor, H Alter H is the second key point. Base H is the second ratio, and H is the first ratio.

[0200] Step 503: The electronic device calculates the product of the scaling factor and the second residual map to obtain the scaled second residual map.

[0201] In this embodiment of the application, the electronic device can multiply the scaling factor with each brightness residual value in the second residual map to obtain the scaled second residual map, which can be achieved by the following formula (16).

[0202] P Res_Linear = PRes *W (16)

[0203] Among them, P Res_Linear The second residual plot after scaling, P Res This is the second residual plot, where W is the scaling factor.

[0204] Optionally, in this embodiment of the application, the electronic device can multiply the scaling factor with each brightness residual value in the second residual map and then perform inverse log2 processing to obtain the scaled second residual map, which can be achieved by the following formula (17).

[0205] P Res_Linear = pow(P Res *W , 2) (17)

[0206] Among them, P Res_Linear The second residual plot after scaling, P Res This is the second residual plot, where W is the scaling factor.

[0207] Step 504: The electronic device calculates the product of the brightness residual value of each pixel in the scaled second residual image and the brightness value of the corresponding pixel in the first image to obtain the first image after brightness adjustment.

[0208] In this embodiment of the application, the electronic device can multiply the brightness residual value of each pixel in the scaled second residual image with the brightness value of the corresponding pixel in the first image to obtain the first image after brightness adjustment, which can be achieved by the following formula (18).

[0209] P Result = P Res_Linear * P Base (18)

[0210] Among them, P Result The first image after brightness adjustment, P Res_Linear The second residual plot after scaling, P Base This is the first image.

[0211] Optionally, in this embodiment of the application, the electronic device can display the first image with adjusted brightness on the screen of the electronic device to achieve the display of an HDR image on the screen of an electronic device that only supports SDR.

[0212] In this embodiment of the application, the electronic device can ensure that the first image can be displayed normally on the current screen or match the display of the backup image, i.e., the non-first image, by scaling and mapping the second residual map.

[0213] The above-described method embodiments, or various possible implementations of the method embodiments, can be executed individually, or, provided there are no contradictions, they can be combined with each other. The specific implementation can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions on this.

[0214] For example, such as Figure 10 As shown below, the image brightness adjustment method provided in this application embodiment will be explained in detail through specific examples. Specifically, it can be implemented through the following steps 20 to 37.

[0215] Step 20: The image processing terminal in the electronic device acquires the first image and the backup image.

[0216] Optionally, in this embodiment of the application, the aforementioned backup image may be an HDR image or an SDR image.

[0217] Optionally, in this embodiment of the application, the first image may be an HDR image or an SDR image.

[0218] It is understandable that an electronic device can acquire an HDR image and an SDR image. Then, if the electronic device uses the HDR image as the first image, the SDR image is the backup image; or, if the electronic device uses the SDR image as the first image, the HDR image is the backup image.

[0219] Step 21: The image processing unit in the electronic device preprocesses the first image.

[0220] In this embodiment of the application, the electronic device can perform linear domain transformation on the first image and perform brightness extraction processing on the transformed first image to obtain the processed first image.

[0221] Step 22: The image processing unit in the electronic device preprocesses the backup image.

[0222] In this embodiment of the application, the electronic device can perform linear domain transformation on the backup image and perform brightness extraction processing on the transformed backup image to obtain the processed backup image.

[0223] Step 23: The image processing unit in the electronic device calculates the log2 encoded residual map.

[0224] In this embodiment of the application, the electronic device can calculate the ratio between the brightness value of each pixel in the processed reference image and the brightness value of each pixel in the processed backup image, and perform log2 calculation to obtain the residual image.

[0225] Step 24: The image processing terminal in the electronic device acquires the cumulative histogram of the processed first image and the cumulative histogram of the residual image, respectively.

[0226] Step 25: The image processing terminal in the electronic device calculates the first brightness residual value based on the cumulative histogram of the first image and the cumulative histogram of the residual image, and fits the third brightness curve.

[0227] Step 26: The image processing terminal in the electronic device performs image segmentation on the first image and the residual image respectively, and calculates the cumulative histogram of the segmented first image and the cumulative histogram of the segmented residual image respectively, as well as the second brightness residual value.

[0228] Step 27: The image processing unit in the electronic device interpolates the third brightness curve based on the first brightness residual value and the second brightness residual value to obtain the first brightness curve.

[0229] Step 28: The image processing terminal in the electronic device obtains metadata.

[0230] In this embodiment of the application, the metadata includes: multiple final key points, namely at least three key points, the maximum brightness residual value, and the slope corresponding to each key point in the first image and the first brightness curve.

[0231] Step 29: The image processing terminal in the electronic device transmits the metadata to the display terminal in the electronic device.

[0232] Step 30: The display in the electronic device determines whether to map the display.

[0233] Optionally, in this embodiment of the application, if the first ratio is greater than or equal to the brightness value corresponding to the maximum brightness residual value and the first image is an HDR image, the electronic device can execute step 37 to directly display the first image.

[0234] Optionally, in this embodiment of the application, when the first ratio is greater than or equal to the brightness value corresponding to the maximum brightness residual value and the first image is an SDR image, the electronic device may perform the following step 31.

[0235] Optionally, in this embodiment of the application, if the first ratio is less than the brightness value corresponding to the maximum brightness residual value, the electronic device may execute step 31.

[0236] Step 31: The display in the electronic device reads the metadata and obtains the second brightness curve based on the metadata.

[0237] Step 32: The display in the electronic device obtains the second residual map through the second brightness curve.

[0238] Step 33: The display in the electronic device scales the second residual map using a scaling factor to obtain the scaled second residual map.

[0239] Step 34: The display terminal in the electronic device performs a linear transformation on the scaled second residual map to obtain the transformed second residual map.

[0240] Step 35: The display terminal in the electronic device adjusts the brightness of the first image using the transformed second residual image to obtain the first image after brightness adjustment.

[0241] Step 36: The display in the electronic device shows the first image after brightness adjustment.

[0242] Step 37: The display in the electronic device displays the first image.

[0243] It should be noted that the image brightness adjustment method provided in this application embodiment can be executed by an image brightness adjustment device. This application embodiment uses an image brightness adjustment device executing the image brightness adjustment method as an example to illustrate the image brightness adjustment device provided in this application embodiment.

[0244] The image display effect achieved by the image brightness adjustment method provided in this application embodiment will be described in detail below through specific shooting scenarios.

[0245] Example 1, such as Figure 11 As shown, a user can trigger an electronic device to capture an SDR image containing a factory building. The electronic device can then perform HDR image reconstruction on the SDR image to obtain an HDR image. Next, the electronic device can use an image processing unit to calculate the brightness ratio between the brightness value of each pixel in the SDR image and the brightness value of each pixel in the HDR image, obtaining a first residual map. Then, the image processing unit in the electronic device can input each brightness residual value in the first residual map and the brightness value of each pixel in the SDR image into a cubic spline function to fit a first brightness curve. When displaying the image, the image processing unit in the electronic device can transmit at least three key points from the SDR image and the first brightness curve to the display screen of the electronic device. The display terminal can input at least three key points into a cubic spline function to fit a second brightness curve, and input the brightness value of each pixel of the SDR image into the second brightness curve to obtain the brightness residual value corresponding to each pixel of the SDR image. By using the brightness residual value corresponding to each pixel of the SDR image, a multiplication operation is performed on the brightness value of each pixel of the SDR image to adjust the brightness value of the SDR image. In this way, the electronic device can display an HDR image containing a factory on an electronic device that only supports SDR images. Compared with the HDR image obtained by mapping a fixed global curve in the prior art, the HDR image obtained by this application has better image quality and can ensure the maximum restoration of the creator's creative intention and artistic expression.

[0246] Example 2, such as Figure 12 As shown, a user can trigger an electronic device to capture an SDR image containing a landscape. The electronic device can then perform HDR image reconstruction on the SDR image to obtain an HDR image. Next, the electronic device can use an image processing unit to calculate the brightness ratio between the brightness value of each pixel in the SDR image and the brightness value of each pixel in the HDR image, obtaining a first residual map. Then, the image processing unit in the electronic device can input each brightness residual value in the first residual map and the brightness value of each pixel in the SDR image into a cubic spline function to fit a first brightness curve. When displaying the image, the image processing unit in the electronic device can transmit at least three key points from the SDR image and the first brightness curve to the display screen of the electronic device. The display device can input at least three key points into a cubic spline function to fit a second brightness curve, and input the brightness value of each pixel of the SDR image into the second brightness curve to obtain the brightness residual value corresponding to each pixel of the SDR image. By using the brightness residual value corresponding to each pixel of the SDR image, a multiplication operation is performed on the brightness value of each pixel of the SDR image to adjust the brightness value of the SDR image. In this way, the electronic device can display an HDR image containing a landscape on an electronic device that only supports SDR images. Compared with the HDR image obtained by mapping a fixed global curve in the prior art, the HDR image obtained by this application has better image quality and can ensure the maximum restoration of the creator's creative intention and artistic expression.

[0247] Example 3, such as Figure 13As shown, a user can trigger an electronic device to capture an SDR image containing a pipe. The electronic device can then perform HDR image reconstruction on the SDR image to obtain an HDR image. Next, the electronic device can use an image processing unit to calculate the brightness ratio between the brightness value of each pixel in the SDR image and the brightness value of each pixel in the HDR image, obtaining a first residual map. Then, the image processing unit in the electronic device can input each brightness residual value in the first residual map and the brightness value of each pixel in the SDR image into a cubic spline function to fit a first brightness curve. When displaying the image, the image processing unit in the electronic device can transmit at least three key points from the SDR image and the first brightness curve to the display screen of the electronic device. The display terminal can input at least three key points into a cubic spline function to fit a second brightness curve, and input the brightness value of each pixel of the SDR image into the second brightness curve to obtain the brightness residual value corresponding to each pixel of the SDR image. Then, the brightness value of each pixel of the SDR image is multiplied by the brightness residual value corresponding to each pixel of the SDR image to adjust the brightness value of the SDR image. In this way, the electronic device can display an HDR image containing a pipeline on an electronic device that only supports SDR images. Compared with the HDR image obtained by mapping a fixed global curve in the prior art, the HDR image obtained by this application has better image quality and can ensure the maximum restoration of the creator's creative intention and artistic expression.

[0248] Figure 14 A schematic diagram of a possible structure of the image brightness adjustment device involved in an embodiment of this application is shown. For example... Figure 14 As shown, the image brightness adjustment device 70 may include: an acquisition module 71, a fitting module 72, and a processing module 73.

[0249] The acquisition module 71 is used to acquire a first residual map based on the brightness values ​​of the SDR image and the HDR image. The first residual map includes the brightness residual values ​​between the SDR image and the HDR image. The fitting module 72 is used to fit a first brightness curve based on the first residual map and the first image acquired by the acquisition module 71. The first brightness curve represents the mapping relationship between the brightness values ​​of the first image and the brightness residual values ​​in the first residual map. The first image is either an SDR image or an HDR image. The processing module 73 is used to fit a second brightness curve based on at least three key points in the first brightness curve obtained by the fitting module 72 and the first image, and to adjust the brightness value of the first image using the second brightness curve.

[0250] In one possible implementation, the processing module 73 is specifically used to sample the brightness residual values ​​in the first residual map acquired by the acquisition module 71 to obtain at least one brightness residual value, and to sample the brightness values ​​of the pixels in the first image corresponding to the at least one brightness residual value to obtain at least one brightness value. The fitting module 72 is specifically used to fit a third brightness curve based on the at least one brightness residual value and the at least one brightness value. The processing module 73 is further used to perform image segmentation on the first residual map acquired by the acquisition module 71 to obtain N first pixel units, and to perform image segmentation on the brightness map of the first image to obtain N second pixel units, where N is an integer greater than 1. The fitting module 72 is further used to fit a first brightness curve based on the N first pixel units, the N second pixel units, and the third brightness curve.

[0251] In one possible implementation, the fitting module 72 is specifically used to calculate the first brightness residual value corresponding to the i-th second pixel unit based on the brightness residual value of the i-th first pixel unit and the brightness value of the i-th second pixel unit, i∈[1,N]; and to interpolate the third brightness curve based on the brightness values ​​of N second pixel units and the first brightness residual values ​​corresponding to N second pixel units to obtain the first brightness curve.

[0252] In one possible implementation, the fitting module 72 is specifically used to determine, based on the brightness value of the i-th second pixel unit, the brightness interval and the second brightness residual value corresponding to the brightness value of the i-th second pixel unit from the third brightness curve; and to calculate the average of the i-th second brightness residual value and the i-th first brightness residual value to obtain the i-th third brightness residual value; and to interpolate the corresponding brightness interval in the third brightness curve based on each of the N third brightness residual values ​​to obtain the first brightness curve.

[0253] In one possible implementation, the processing module 73 is further configured to perform gradient processing on the first brightness curve before fitting the second brightness curve based on at least three key points in the first brightness curve, thereby removing key points in the first brightness curve whose gradient is less than or equal to 0, to obtain a fourth brightness curve; and based on the brightness value corresponding to each key point in the fourth brightness curve, remove the first key point in the fourth brightness curve that satisfies a first condition, thereby obtaining at least three key points; wherein, the first condition is that the brightness difference between the brightness values ​​corresponding to two adjacent key points is less than or equal to a preset brightness threshold, and the first key point is the second key point among the two adjacent key points that satisfy the first condition.

[0254] This application provides an image brightness adjustment device. Since a first residual map can characterize the brightness difference between an SDR image and an HDR image, a first brightness curve obtained by fitting the first residual map and the first image can be used to characterize a gain image. Similarly, a second brightness curve obtained by fitting at least three key points in the first brightness curve and the first image can also characterize a gain image, thereby enabling the display of HDR images on electronic devices that only support SDR images. Moreover, the image brightness adjustment device only needs to fit the first image and at least three key points to obtain the second brightness curve, and adjust the brightness value of the first image using the second brightness curve. Therefore, compared to a gain image, the file size stored by the image brightness adjustment device is reduced, thereby reducing the memory and performance pressure on the image brightness adjustment device. In this way, the image brightness adjustment device ensures that HDR images can be displayed on electronic devices that only support SDR images while reducing the storage pressure on the image brightness adjustment device.

[0255] The image brightness adjustment device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, a mobile electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0256] The image brightness adjustment device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0257] The image brightness adjustment device provided in this application embodiment can realize the various processes implemented in the above method embodiments, and will not be described again here to avoid repetition.

[0258] Optionally, such as Figure 15 As shown, this application embodiment also provides an electronic device 90, including a processor 91 and a memory 92. The memory 92 stores a program or instructions that can run on the processor 91. When the program or instructions are executed by the processor 91, they implement the various steps of the above-described image brightness adjustment method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0259] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0260] Figure 16 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0261] The electronic device 100 includes, but is not limited to, components such as: radio frequency unit 101, network module 102, audio output unit 103, input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, and processor 110.

[0262] Those skilled in the art will understand that the electronic device 100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 16 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0263] The processor 110 is configured to: obtain a first residual map based on the brightness values ​​of an SDR image and an HDR image, the first residual map including the brightness residual values ​​between the SDR image and the HDR image; and fit a first brightness curve based on the first residual map and a first image, the first brightness curve representing the mapping relationship between the brightness value of the first image and the brightness residual values ​​in the first residual map, the first image being an SDR image or an HDR image; and fit a second brightness curve based on at least three key points in the first brightness curve and the first image, and adjust the brightness value of the first image using the second brightness curve.

[0264] Optionally, in this embodiment of the application, the processor 110 is specifically configured to sample the brightness residual values ​​in the first residual map to obtain at least one brightness residual value, and sample the brightness values ​​of the pixels in the first image corresponding to the at least one brightness residual value to obtain at least one brightness value; fit a third brightness curve based on the at least one brightness residual value and the at least one brightness value; divide the first residual map into N first pixel units, and divide the brightness map of the first image into N second pixel units, where N is an integer greater than 1; fit a first brightness curve based on the N first pixel units, the N second pixel units, and the third brightness curve.

[0265] Optionally, in this embodiment of the application, the processor 110 is specifically used to calculate the first brightness residual value corresponding to the i-th second pixel unit based on the brightness residual value of the i-th first pixel unit and the brightness value of the i-th second pixel unit, i∈[1,N]; and to interpolate the third brightness curve based on the brightness values ​​of N second pixel units and the first brightness residual values ​​corresponding to N second pixel units to obtain the first brightness curve.

[0266] Optionally, in this embodiment of the application, the processor 110 is specifically configured to: determine, based on the brightness value of the i-th second pixel unit, a brightness interval and a second residual value corresponding to the brightness value of the i-th second pixel unit from the third brightness curve; calculate the average of the i-th second brightness residual value and the i-th first brightness residual value to obtain the i-th third brightness residual value; and interpolate the corresponding brightness interval in the third brightness curve based on each of the N third brightness residual values ​​to obtain the first brightness curve.

[0267] Optionally, in this embodiment of the application, the processor 110 is further configured to perform gradient processing on the first brightness curve before fitting the second brightness curve based on at least three key points in the first brightness curve, and remove key points in the first brightness curve whose gradient is less than or equal to 0 to obtain a fourth brightness curve; based on the brightness value corresponding to each key point in the fourth brightness curve, remove the first key point in the fourth brightness curve that satisfies a first condition to obtain at least three key points; wherein, the first condition is that the brightness difference between the brightness values ​​corresponding to two adjacent key points is less than or equal to a preset brightness threshold, and the first key point is the second key point among the two adjacent key points that satisfy the first condition.

[0268] This application provides an electronic device. Since a first residual image can characterize the brightness difference between an SDR image and an HDR image, the electronic device can use the first brightness curve obtained by fitting the first residual image and the first image to characterize the gain image. Similarly, a second brightness curve obtained by fitting at least three key points in the first brightness curve and the first image can also characterize the gain image, thereby enabling the display of HDR images on electronic devices that only support SDR images. Moreover, the electronic device only needs to fit the first image and at least three key points to obtain the second brightness curve, and adjust the brightness value of the first image using the second brightness curve. Therefore, compared to the gain image, the file size stored by the electronic device is reduced, thereby reducing the memory and performance pressure on the electronic device. Thus, the electronic device can display HDR images on electronic devices that only support SDR images while reducing the storage pressure on the electronic device.

[0269] The electronic device provided in this application embodiment can implement the various processes implemented in the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0270] For details on the beneficial effects of the various implementation methods in this embodiment, please refer to the beneficial effects of the corresponding implementation methods in the above method embodiments. To avoid repetition, these will not be repeated here.

[0271] It should be understood that, in this embodiment, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 107 includes at least one of a touch panel 1071 and other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0272] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 109 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 109 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0273] Processor 110 may include one or more processing units; optionally, processor 110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 110.

[0274] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0275] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0276] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0277] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0278] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image brightness adjustment method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0279] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0280] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0281] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image brightness adjustment method, characterized by, The method comprises: obtaining a first residual map based on luminance values of a standard dynamic range (SDR) image and a high dynamic range (HDR) image, the first residual map comprising luminance residual values between the SDR image and the HDR image; fitting a first luminance curve based on the first residual map and a first image, the first luminance curve representing a mapping relationship between luminance values of the first image and the luminance residual values in the first residual map, the first image being the SDR image or the HDR image; fitting a second luminance curve based on at least three key points in the first luminance curve and the first image, and adjusting the luminance values of the first image through the second luminance curve; before fitting the second luminance curve based on the at least three key points in the first luminance curve, the method further comprises: performing gradient processing on the first luminance curve to remove key points with a gradient less than or equal to 0 in the first luminance curve, to obtain a fourth luminance curve; removing a first key point in the fourth luminance curve that satisfies a first condition based on a luminance value corresponding to each key point in the fourth luminance curve, to obtain the at least three key points; wherein the first condition is that a luminance difference between luminance values corresponding to two adjacent key points is less than or equal to a preset luminance threshold, and the first key point is a second key point of the two adjacent key points that satisfies the first condition.

2. The method of claim 1, wherein, The fitting of the first luminance curve based on the first residual map and the first image comprises: sampling luminance residual values in the first residual map to obtain at least one luminance residual value, and sampling luminance values of pixel points in the first image corresponding to the at least one luminance residual value to obtain at least one luminance value; fitting a third luminance curve based on the at least one luminance residual value and the at least one luminance value; performing image division on the first residual map to obtain N first pixel units, and performing image division on a luminance map of the first image to obtain N second pixel units, N being an integer greater than 1; fitting the first luminance curve based on the N first pixel units, the N second pixel units, and the third luminance curve.

3. The method of claim 2, wherein, The fitting of the first luminance curve based on the N first pixel units, the N second pixel units, and the third luminance curve comprises: calculating a first luminance residual value corresponding to an i-th second pixel unit based on a luminance residual value of an i-th first pixel unit and a luminance value of the i-th second pixel unit, i∈[1,N]; performing interpolation on the third luminance curve based on luminance values of the N second pixel units and the first luminance residual values corresponding to the N second pixel units, to obtain the first luminance curve.

4. The method of claim 3, wherein, The interpolation of the third luminance curve based on the luminance values of the N second pixel units and the first residual values corresponding to the N second pixel units, to obtain the first luminance curve, comprises: determine, from the third luminance curve, a luminance interval corresponding to the luminance value of the i th second pixel unit and a second luminance residual value based on the luminance value of the i th second pixel unit; calculate a mean value of the i th second luminance residual value and the i th first luminance residual value to obtain an i th third luminance residual value; perform interpolation on the corresponding luminance interval in the third luminance curve based on each of the N third luminance residual values to obtain the first luminance curve.

5. An image brightness adjusting apparatus characterized by comprising: The image luminance adjustment device comprises an acquisition module, a fitting module and a processing module. The acquisition module is configured to acquire a first residual map based on luminance values of a standard dynamic range (SDR) image and a high dynamic range (HDR) image, the first residual map comprising luminance residual values between the SDR image and the HDR image. The fitting module is configured to fit a first luminance curve based on the first residual map acquired by the acquisition module and a first image, the first luminance curve representing a mapping relationship between luminance values of the first image and the luminance residual values in the first residual map, the first image being the SDR image or the HDR image. The processing module is configured to fit a second luminance curve based on at least three key points in the first luminance curve obtained by the fitting module and the first image, and adjust luminance values of the first image through the second luminance curve. The processing module is further configured to perform gradient processing on the first luminance curve before fitting the second luminance curve based on the at least three key points, remove key points with a gradient less than or equal to 0 in the first luminance curve to obtain a fourth luminance curve, and remove a first key point satisfying a first condition in the fourth luminance curve based on a luminance value corresponding to each key point in the fourth luminance curve, to obtain the at least three key points, wherein the first condition is that a luminance difference between luminance values corresponding to two adjacent key points is less than or equal to a preset luminance threshold, and the first key point is a second key point of the two adjacent key points satisfying the first condition.

6. The apparatus of claim 5, wherein, The processing module is specifically configured to sample luminance residual values in the first residual map acquired by the acquisition module to obtain at least one luminance residual value, and sample luminance values of pixel points in the first image corresponding to the at least one luminance residual value to obtain at least one luminance value. The fitting module is specifically configured to fit a third luminance curve based on the at least one luminance residual value and the at least one luminance value. The processing module is further configured to perform image division on the first residual map acquired by the acquisition module to obtain N first pixel units, and perform image division on a luminance map of the first image to obtain N second pixel units, N being an integer greater than 1. The fitting module is further configured to fit the first luminance curve based on the N first pixel units, the N second pixel units and the third luminance curve.

7. The apparatus of claim 6, wherein, The fitting module is specifically configured to calculate a first luminance residual value corresponding to an i-th second pixel unit based on the luminance residual value of the i-th first pixel unit and the luminance value of the i-th second pixel unit, i∈[1, N]; and interpolate the third luminance curve based on the luminance values of the N second pixel units and the first luminance residual values corresponding to the N second pixel units to obtain the first luminance curve.

8. The apparatus of claim 7, wherein, The fitting module is specifically configured to determine a luminance interval corresponding to the luminance value of the i-th second pixel unit and a second luminance residual value from the third luminance curve based on the luminance value of the i-th second pixel unit; and calculate a mean value of the i-th second luminance residual value and the i-th first luminance residual value to obtain an i-th third luminance residual value; and interpolate the corresponding luminance interval in the third luminance curve based on each of the N third luminance residual values to obtain the first luminance curve.

Images