Image processing method and related device

By partitioning low dynamic range images and performing inverse tone mapping, the problem of difficulty in distinguishing between bright areas and high grayscale areas is solved, improving image display effect and processing efficiency, and realizing real-time dynamic range expansion of low dynamic range images.

WO2026232959A1PCT designated stage Publication Date: 2026-11-12HONOR DEVICE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2026/082176
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-09
Filing Date
2026-03-09
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing technologies struggle to distinguish between bright and high grayscale areas when converting low dynamic range (LVR) images to high dynamic range (LVR) images, resulting in the inability to restore detailed information and low processing efficiency, making real-time conversion difficult.

Method used

By partitioning low dynamic range (LVR) and high dynamic range (HDR) images, and using the brightness information of the HDR image to perform inverse tone mapping on each partition of the LVR image, an image with a higher dynamic range is generated.

Benefits of technology

It effectively distinguishes between bright areas and high grayscale areas, improves image display effect, and enhances image processing efficiency, enabling real-time dynamic range extension of low dynamic range videos.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2026082176_12112026_PF_FP_ABST
    Figure CN2026082176_12112026_PF_FP_ABST
Patent Text Reader

Abstract

Provided in the present application are an image processing method and a related device. The method comprises: in response to an operation of a user playing at least one first image, an electronic device generating second images corresponding to the at least one first image, and performing partition processing on the at least one first image and the second images corresponding thereto, so as to obtain N first partitions of the at least one first image, and M second partitions of at least one second image respectively corresponding to the N first partitions; on the basis of brightness information of the M second partitions, performing inverse tone mapping on each pixel in the N first partitions, so as to obtain N adjusted first partitions; and on the basis of the N adjusted first partitions, generating at least one processed first image. In the present application, partition processing is performed on a first image and a second image, and by using second partitions of the second image that comprise more brightness information as a reference, inverse tone mapping is performed on each first partition of the first image, such that the display effect of the processed first image can be effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

An image processing method and related equipment

[0001] This application claims priority to Chinese Patent Application No. 2025106002850, filed on May 9, 2025, entitled "An Image Processing Method and Related Equipment", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of electronic technology, and in particular to an image processing method and related equipment. Background Technology

[0003] Dynamic range refers to the difference in brightness between the brightest and darkest parts of an image. The greater the dynamic range, the richer the tonal range that the image can represent and the wider the color space it contains.

[0004] Currently, in order to make low dynamic range images compatible with the display capabilities of high dynamic range electronic devices, the low dynamic range image can be converted into a high dynamic range image by scaling the entire dynamic range of the low dynamic range image proportionally, and then the converted high dynamic range image can be displayed on the electronic device.

[0005] However, when using this method to convert low dynamic range images, there is a problem that the bright areas and high grayscale areas are difficult to distinguish, making it impossible to restore the details in the image and resulting in a decrease in display quality.

[0006] Therefore, there is an urgent need for a solution to address the aforementioned technical problems. Summary of the Invention

[0007] This application provides an image processing method and related equipment. To achieve the above objectives, this application adopts the following technical solution:

[0008] Firstly, this application provides an image processing method. In response to a user's playback of at least one frame of a first image, an electronic device generates a second image corresponding to the first frame. The second image has fewer pixels than the first image, and the second image includes more brightness information than the first image. Based on this, the electronic device partitions the at least one frame of the first image and the corresponding second image to obtain N first partitions of the first frame and M second partitions of the second image corresponding to each of the N first partitions. M and N are equal and are positive integers. Based on the brightness information of the M second partitions, the electronic device performs inverse tone mapping on each pixel in the N first partitions corresponding to the M second partitions to obtain N adjusted first partitions. Based on the N adjusted first partitions, at least one frame of the processed first image is generated.

[0009] In this embodiment, the electronic device performs partitioning processing on the first image and the second image, using the second partition of the second image, which includes more brightness information, as a reference, and performs inverse tone mapping on each first partition of the first image. This can effectively distinguish between bright areas and high grayscale areas, thereby avoiding the problem of difficult distinction between bright areas and high grayscale areas during the conversion of low dynamic range images. This allows the detailed information in the image to be restored, improving the display effect of the processed first image.

[0010] In one possible implementation, the electronic device can determine the metadata of N first partitions corresponding to the M second partitions based on the brightness information of the M second partitions; and perform inverse tone mapping on each pixel in the N first partitions corresponding to the M second partitions based on the metadata of the N first partitions to obtain N adjusted first partitions. In this embodiment, by performing inverse tone mapping on each first partition of the first image, the electronic device can effectively distinguish between bright areas and high grayscale areas, thereby improving the display effect of the processed first image.

[0011] In one possible implementation, the electronic device can generate inverse tone mapping curves for the N first partitions based on the metadata of the N first partitions. The inverse tone mapping curves indicate the correspondence between the tone information of each pixel in the first partition and the tone information of each pixel in the adjusted first partition. Based on the inverse tone mapping curves of the N first partitions, inverse tone mapping is performed on each pixel in the N first partitions corresponding to the M second partitions to obtain N adjusted first partitions. In this embodiment, by generating an inverse tone mapping curve for each first partition, the electronic device can perform inverse tone mapping on each first partition of the first image, effectively distinguishing between bright areas and high grayscale areas, and improving the display effect of the processed first image.

[0012] In one possible implementation, in order to match the processed first image with the display capability of the electronic device and improve the display effect of the processed first image, the electronic device can first obtain the brightness clearance capability of the electronic device; based on the brightness information of the M second partitions and the brightness clearance capability of the electronic device, inverse tone mapping is performed on each pixel in the N first partitions corresponding to the M second partitions to obtain N adjusted first partitions.

[0013] In one possible implementation, the electronic device determines the metadata of N first partitions corresponding to the M second partitions based on the brightness information of the M second partitions; based on the brightness clearance capability of the electronic device and the metadata of the N first partitions, it performs inverse tone mapping on each pixel in the N first partitions corresponding to the M second partitions to obtain N adjusted first partitions. In this embodiment, the brightness clearance capability of the electronic device is considered in the process of obtaining the adjusted first partitions to ensure that the processed first image matches the display capability of the electronic device, thereby improving the display effect of the processed first image.

[0014] In one possible implementation, the electronic device may, in response to a user's operation of playing at least one frame of a first image, downsample the at least one frame of the first image to obtain a third image corresponding to the at least one frame of the first image; and perform inverse tone mapping processing on the at least one frame of the third image to generate a second image corresponding to the at least one frame of the first image.

[0015] Since the second image is obtained based on the third image with fewer pixels, in this embodiment, the brightness information of the second image with fewer pixels is used as a reference to perform inverse tone mapping on each pixel in each first partition of the first image, which can effectively improve image processing efficiency and realize real-time dynamic range expansion processing of low dynamic range video.

[0016] In one possible implementation, the electronic device can respond to a user's video playback operation by processing at least one frame of a first image in the video frame by frame or at preset frame intervals to generate a second image corresponding to the at least one frame of the first image. In this embodiment, by setting an appropriate preset frame number and processing the first image in the video at intervals of that preset frame number, the image processing efficiency can be effectively improved while ensuring the display effect of the processed video.

[0017] In one possible implementation, the electronic device can partition the at least one first frame and the corresponding second image according to a preset number of parts, to obtain a preset number of first parts of the at least one first frame and a preset number of second parts of the at least one second frame corresponding to the preset number of first parts.

[0018] In one possible implementation, the electronic device may, in response to a user's operation of playing at least one frame of a first image, decode the at least one frame of the first image to obtain at least one frame of the decoded first image; and generate a second image corresponding to the at least one frame of the first image based on the at least one frame of the decoded first image.

[0019] In one possible implementation, after generating at least one frame of processed first image based on a plurality of the adjusted first partitions, the electronic device can display the at least one frame of processed first image on the electronic device, wherein the dynamic range supported by the electronic device is higher than that of the first image.

[0020] Secondly, this application provides an electronic device, the electronic device comprising: a processor and a memory; the memory is used to store program code and transmit the program code to the processor; the processor is used to execute the steps of an image processing method as described above according to the instructions in the program code.

[0021] Thirdly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of an image processing method as described above.

[0022] Fourth aspect: This application provides a computer program product, which, when run on a computer, enables the computer to perform the steps of an image processing method as described above.

[0023] Fifth aspect: Embodiments of this application provide a communication device, including a module for performing the method described above. Attached Figure Description

[0024] Figure 1 is a schematic diagram of an application scenario of an image processing method provided in an embodiment of this application;

[0025] Figure 2 is a flowchart of an image processing method provided in an embodiment of this application;

[0026] Figure 3 is a schematic diagram of downsampling provided in an embodiment of this application;

[0027] Figure 4 is a structural diagram of an artificial intelligence model provided in an embodiment of this application;

[0028] Figure 5 is a schematic diagram of an image partitioning provided in an embodiment of this application;

[0029] Figure 6 is a schematic diagram of an image processing method provided in an embodiment of this application;

[0030] Figure 7 is a schematic diagram of the effect of a processed first image provided in an embodiment of this application;

[0031] Figure 8 is a component example diagram of an electronic device provided in an embodiment of this application;

[0032] Figure 9 is an example diagram of the software structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0033] The terms "first," "second," and "third," etc., used in this application specification, claims, and drawings are used to distinguish different objects, not to limit a specific order.

[0034] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0035] Currently, in order to make low dynamic range images compatible with the display capabilities of high dynamic range electronic devices, the low dynamic range image can be converted into a high dynamic range image by scaling the entire dynamic range of the low dynamic range image proportionally, and then the converted high dynamic range image can be displayed on the electronic device.

[0036] Firstly, when using this method to convert low dynamic range images, there is a situation where it is difficult to distinguish between bright areas (such as diffuse white points) and high grayscale areas (such as light sources), making it impossible to restore the detailed information in the image and resulting in a reduction in display effect.

[0037] Secondly, this method requires processing a large amount of data during the conversion of low dynamic range images, resulting in low efficiency and making it difficult to achieve real-time conversion of low dynamic range images.

[0038] Based on this, embodiments of this application provide an image processing method in which an electronic device generates a second image corresponding to at least one frame of the first image in response to a user's operation of playing at least one frame of the first image. By partitioning the at least one frame of the first image and its corresponding second image, multiple first partitions of the first image and multiple second partitions of the second image can be obtained. For each second partition, based on the brightness information of the second partition, inverse tone mapping is performed on each pixel in the corresponding first partition to obtain multiple adjusted first partitions. Based on the multiple adjusted first partitions, at least one frame of the processed first image can be generated. The second image has fewer pixels than the first image, and the second image includes more brightness information than the first image.

[0039] In this embodiment, the electronic device performs partitioning processing on the first image and the second image, using the second partition of the second image, which includes more brightness information, as a reference, and performs inverse tone mapping on each first partition of the first image. This can effectively distinguish between bright areas and high grayscale areas, and improve the display effect of the processed first image.

[0040] The following describes the application scenarios of the image processing method provided in the embodiments of this application.

[0041] In this application, the electronic device can be a display device with a high dynamic range. In some embodiments, the electronic device can be a mobile phone, tablet computer, desktop computer, laptop computer, notebook computer, ultra-mobile personal computer (UMPC), handheld computer, netbook, personal digital assistant (PDA), wearable electronic device, smartwatch, etc. This application does not impose any special restrictions on the specific form of the above-mentioned electronic devices.

[0042] In one possible implementation, as shown in Figure 1, the electronic device stores a first image 1101 in its image library. When a user clicks on the first image 1101 in the image library, the electronic device, in response to the user's click on the first image 1101, generates a second image corresponding to the first image. The second image has fewer pixels than the first image, and the second image includes more brightness information than the first image.

[0043] The electronic device partitions the first image and the second image to obtain multiple first partitions of the first image and multiple second partitions of the second image, wherein each first partition corresponds to one second partition.

[0044] For ease of explanation, we will take the example of a first image containing N first partitions and a second image containing M second partitions, where M and N are equal and are positive integers.

[0045] Based on this, the electronic device can perform inverse tone mapping on each pixel in the N first partitions corresponding to the M second partitions according to the brightness information of the M second partitions, to obtain N adjusted first partitions, and generate and display the processed first image 1102 based on the N adjusted first partitions.

[0046] In another possible implementation, the electronic device stores low dynamic range video in its gallery, which consists of multiple first frames.

[0047] When a user clicks on a low dynamic range video in the gallery, the electronic device responds to the user's click on the video by generating a second image corresponding to at least one first image frame. The at least one first image frame and the second image corresponding to the at least one first image frame are partitioned to obtain N first partitions of the at least one first image frame and M second partitions of the at least one second image frame corresponding to the N first partitions respectively.

[0048] The electronic device performs inverse tone mapping on each pixel in N first partitions corresponding to the M second partitions based on the brightness information of the M second partitions, to obtain N adjusted first partitions, and generates at least one frame of processed first image based on the N adjusted first partitions, and displays it.

[0049] In one example, an electronic device can process the first image in a video frame by frame into a processed first image and then display it.

[0050] In another example, to improve image processing efficiency, the electronic device can process at least one frame of the first image in the video into a corresponding processed first image at preset frame intervals. Based on this, the electronic device can combine the processed first image with the unprocessed first image in the order of the video frames to generate a processed video for display.

[0051] For example, if the video includes 1800 first images and the preset frame number is 30, the electronic device can process the first images such as the 30th frame, the 60th frame, and the 90th frame, while the first images such as the 1st to 29th frames, the 31st to 59th frames, and the 61st to 89th frames are unprocessed first images.

[0052] It is understood that the embodiments of this application do not specifically limit the number of first partitions processed simultaneously or the processing order of different first partitions. For example, the electronic device may perform inverse tone mapping on each pixel in different first partitions sequentially, or it may perform inverse tone mapping on each pixel in different first partitions simultaneously.

[0053] Because the method provided in this application embodiment has high image processing efficiency, the electronic device can play video while processing the first image, or display the processed video after obtaining all the processed first images.

[0054] It is understood that the storage location of the first image in the electronic device is not specifically limited in the embodiments of this application. The above is only an example of the storage location as a picture library.

[0055] In another possible implementation, the electronic device can acquire the first image, or a video including multiple frames of the first image, through wireless communication technology, wired communication technology, etc. For example, when a user browses the first image on a webpage, or a video including multiple frames of the first image, clicks on the first image to view it, or clicks on the video to play it, the electronic device can generate a second image corresponding to at least one frame of the first image. It then performs partitioning processing on the at least one frame of the first image and the corresponding second image, obtaining multiple first partitions of the at least one frame of the first image and multiple second partitions of the at least one frame of the second image. Based on this, for each second partition, according to the brightness information of the second partition, inverse tone mapping is performed on each pixel in the corresponding first partition to obtain multiple adjusted first partitions. Based on these adjusted first partitions, at least one frame of the processed first image is generated and displayed.

[0056] The following describes an image processing method provided by an embodiment of this application. As shown in Figure 2, this figure is a flowchart of an image processing method provided by an embodiment of this application.

[0057] S101, In response to the user's operation of playing at least one frame of the first image, the electronic device generates a second image corresponding to at least one frame of the first image.

[0058] The second image has fewer pixels than the first image, includes more brightness information than the first image, and the electronic device supports a higher dynamic range than the first image. For example, the first image is a standard dynamic range (SDR) image, and the second image is a high dynamic range (HDR) image.

[0059] In this embodiment of the application, the electronic device can process a single frame of the first image or process a video including multiple frames of the first image.

[0060] It is understood that when a video includes multiple frames of first images, the content of the first images in different frames may be the same or different, and this application does not impose specific limitations on this.

[0061] In one example, for a video comprising multiple frames of first images, the electronic device may, in response to a user's action of playing the video, process each first image in the video frame by frame to generate a second image corresponding to at least one frame of the first images.

[0062] In another example, for a video comprising multiple frames of a first image, the electronic device may, in response to a user's operation of playing the video, process the first images in the video at preset frame intervals to generate a second image corresponding to at least one frame of the first image.

[0063] For example, if the video includes 1800 first images and the preset frame number is 30, the electronic device can process the 30th first image in response to the user's operation of playing the video to generate a second image corresponding to the 30th first image; process the 60th first image to generate a second image corresponding to the 60th first image, and so on.

[0064] It is understandable that the smaller the preset frame rate, the more first images need to be processed, and the better the display effect of the subsequently generated processed video; the larger the preset frame rate, the fewer first images need to be processed, and the faster the image processing speed.

[0065] In this embodiment, the value of the preset frame number is not specifically limited and can be set according to actual needs.

[0066] In this embodiment of the application, by setting an appropriate preset frame number and processing the first image in the video at intervals of the preset frame number, the display effect of the processed video can be guaranteed while effectively improving the image processing efficiency.

[0067] In one possible implementation, for any frame of the first image, the electronic device may generate a second image corresponding to the first image based on the following steps.

[0068] Step 11: The electronic device downsamples the first image to obtain the third image corresponding to the first image.

[0069] Downsampling aims to reduce the size of the data while preserving as much key information in the image as possible.

[0070] As shown in Figure 3, in this embodiment of the application, in order to process the first image 1201 into a second image 1203 with a higher dynamic range while ensuring image processing efficiency, the electronic device downsamples the first image to obtain a third image 1202 with fewer pixels than the first image, but retaining the key information in the first image.

[0071] The process of downsampling the first image does not improve the dynamic range of the image. For example, if the first image is a low dynamic range image, the third image obtained by downsampling the first image is also a low dynamic range image; if the first image is an SDR image, the third image is also an SDR image.

[0072] Step 12: The electronic device performs inverse tone mapping on the third image to generate the second image corresponding to the first image.

[0073] Electronic devices can reduce the amount of data processing during the inverse tone mapping process and improve image processing efficiency by performing inverse tone mapping on the downsampled third image.

[0074] In this embodiment of the application, the electronic device performs inverse tone mapping on the third image, which can improve the dynamic range of the image and generate a second image with a higher dynamic range.

[0075] In one possible implementation, the electronic device can input a third image into an artificial intelligence (AI) model, process the third image through the AI ​​model, and output a second image corresponding to the first image.

[0076] To improve image processing efficiency, this AI model can be a lightweight AI model.

[0077] In one example, the AI ​​model can use a convolutional neural network to determine a one-to-one color mapping relationship between pixels in a third image and pixels in a second image to be generated, and based on this color mapping relationship, convert the third image into a second image with high dynamic range.

[0078] In the process of converting the third image into a high dynamic range second image, in order to improve the display effect of the second image, the multi-scale color information generated by the third image through the image pyramid structure can be used as additional prior information. Based on this color mapping relationship, the AI ​​model converts the third image into a high dynamic range fourth image, and fine-tunes the fourth image based on this additional prior information to generate the second image.

[0079] In another example, as shown in Figure 4, the AI ​​model may include a color mapping module and a detail enhancement module.

[0080] The third image is input into the color mapping module, which determines a one-to-one color mapping relationship between the pixels in the third image and the pixels in the second image to be generated. Based on this color mapping relationship, the third image is converted into a fifth image with high dynamic range.

[0081] The fifth image is input into the detail enhancement module, which can perform detail enhancement processing on the fifth image, such as artifact removal and tomography removal, to generate a higher quality second image.

[0082] For example, the detail enhancement module can employ an autoencoder structure, which includes an encoder and a decoder. By combining prior information from the image pyramid and performing detail enhancement processing on the fifth image through the encoder and decoder, a higher-quality second image can be generated.

[0083] It is understood that the specific type of AI model is not specifically limited in the embodiments of this application, and the above are merely examples.

[0084] S102, The electronic device performs partitioning processing on at least one first image frame and at least one second image corresponding to the first image frame to obtain multiple first partitions of the first image frame and multiple second partitions of the second image frame.

[0085] Each first partition corresponds to one second partition.

[0086] In this embodiment of the application, the electronic device partitions the first image and the second image, so that each first partition can be subjected to inverse tone mapping in the subsequent process.

[0087] In one possible implementation, the electronic device can partition at least one frame of a first image and at least one frame of a second image corresponding to the first image according to a preset number of partitions, to obtain a preset number of first partitions of at least one frame of the first image and a preset number of second partitions of at least one frame of the second image.

[0088] In this embodiment, the partitioning method may include, but is not limited to, dividing by rectangle or by region. Taking dividing by rectangle as an example, the preset number of parts can be a×b, where a and b are positive integers, a is the number of parts horizontally, and b is the number of parts vertically. As shown in Figure 5, when a is 5 and b is 6, 30 first partitions can be divided from the first image, and 30 second partitions can be divided from the second image.

[0089] It is understood that the method of partitioning the image in this embodiment is not specifically limited, as long as each first partition corresponds to one second partition.

[0090] S103, The electronic device determines the brightness information for each second zone.

[0091] In one possible implementation, the electronic device can convert the second image to a linear space using an electro-optical conversion function (EOTF) to obtain a scene-linear image, which is used to indicate encoded values ​​proportional to the brightness or spectral radiance in the scene.

[0092] Based on the linear image of the scene, the brightness information of each second partition can be determined. This brightness information may include, but is not limited to, the maximum brightness value, minimum brightness value, and average brightness value of each pixel in the second partition.

[0093] The maximum brightness value can be represented as MaxLuminance(i); the minimum brightness value can be represented as MinLuminance(i); and the average brightness value can be represented as AverageLuminance(i). Here, i∈[1,Z], and Z is the number of the second partitions, i.e., the preset number of partitions.

[0094] S104. For each second partition, the electronic device determines the metadata of the first partition corresponding to the second partition based on the brightness information of the second partition.

[0095] It is understood that the embodiments of this application do not specifically limit the number of first partitions processed simultaneously or the processing order of different first partitions. For example, an electronic device may determine the metadata of the first partition corresponding to each of multiple second partitions sequentially based on the brightness information of different second partitions, or it may determine the metadata of the first partition corresponding to each of multiple second partitions simultaneously based on the brightness information of different second partitions.

[0096] The following example illustrates how to determine the metadata of the first partition corresponding to a second partition.

[0097] In this embodiment of the application, the metadata of the first partition may include, but is not limited to, dark area gain, gray boundary points in extended dynamic range, highlight boundary points in extended dynamic range, extended dynamic range clearance, reference white point gain, etc.

[0098] The dark zone gain can be expressed as SdrDarkZoneGain[i], which can be calculated based on MinLuminance(i), as shown in equation (1): SdrDarkZoneGain[i]=F(MinLuminance(i)) (1)

[0099] Where F(x) is a linear or nonlinear transformation function, such as F(x) = x.

[0100] The gray boundary point in the extended dynamic range can be represented as SdrExtendedMidtonePoint[i], which can be calculated based on AverageLuminance(i), as shown in Equation (2): SdrExtendedMidtonePoint[i]=F(AverageLuminance[i]) (2)

[0101] Where F(x) is a linear or nonlinear transformation function, such as F(x) = max(AverageLuminance[i], 0.5);

[0102] The extended dynamic range highlight boundary point can be represented as SdrExtendedHightlightPoint[i], which can be obtained based on MaxLuminance(i), as shown in equation (3): p1=SdrExtendedMidtonePoint[i] p2=max(p2,0.847,p1+0.10) p2=min(p2,1.0) SdrExtendedHightlightPoint[i]=p2 (3)

[0103] Where Num is the sum of the brightness of the pixels in the i-th region; p2 represents the segmentation point; N1(t) and N2(t) are the sum of the pixels on both sides of the segmentation point, and μ1(t) and μ2(t) are the average values ​​of the pixels on both sides of the segmentation point.

[0104] The extended dynamic range headroom can be expressed as SdrExtendedHeadroom, which can be obtained based on MaxHdrLuminance(i) and MaxSdrLuminance(i), as shown in equation (4): SdrExtendedHeadroom=Min(MaxHdrLuminance / MaxSdrLuminance,x) (4)

[0105] Where MaxHdrLuminance is the maximum brightness corresponding to the second partition; MaxSdrLuminance is the maximum brightness corresponding to the first partition; and x is a preset value, such as 2.5.

[0106] The reference white point gain can be represented as SdrRefWhitePointGain, which is a preset value, such as 2.0.

[0107] S105. The electronic device performs inverse tone mapping on each pixel in each first partition based on the metadata of each first partition, and obtains multiple adjusted first partitions.

[0108] In this embodiment of the application, for each first partition, the electronic device can determine the inverse tone mapping curve corresponding to the first partition based on the metadata of the first partition. The inverse tone mapping curve can indicate the correspondence between the tone information of each pixel in the first partition and the tone information of each pixel in the adjusted first partition. The tone information includes, but is not limited to, brightness value and grayscale value.

[0109] After generating the inverse tone mapping curve, the electronic device can perform inverse tone mapping on each pixel in the first partition based on the inverse tone mapping curve to obtain the adjusted first partition. The specific steps are as follows:

[0110] Step 51: The electronic device determines the slope of the inverse tone mapping curve based on the extended dynamic range highlight boundary point, the extended dynamic range gray boundary point, the dark area gain, and the extended dynamic range clearance.

[0111] In this embodiment, the slope of the inverse tone mapping curve can be expressed by equation (5) as follows: k=(headroomE-SdrDarkZoneGain[i]*wGain) / (SdrExtendedHightlightPoint[i]-SdrExtendedMidtonePoint[i]+2×(1.0-SdrExtendedHightlightPoint[i])) (5)

[0112] Where k is the slope of the inverse tone mapping curve; headroomE is the net gain factor, which can be expressed as headroomE=(SdrExtendedHeadroom*(1.0+wGain)). 1 / 2.2 wGain is the white point factor of the gain curve.

[0113] Step 52: The electronic device generates the inverse tone mapping curve corresponding to the first partition based on the slope of the inverse tone mapping curve.

[0114] The inverse tone mapping curve can be expressed by equation (6) as follows: fTonemapping(x t )=xt 2.2 *f Gain (x t (6)

[0115] Among them, fTonemapping(x t ) represents the inverse tone mapping curve.

[0116] In x t In the case of <SdrExtendedMidtonePoint[i], fGain (x t )=SdrDarkZoneGain[i]*wGain.

[0117] In the case where SdrExtendedMidtonePoint[i]<x t <SdrExtendedHightlightPoint[i], f Gain (x t )=(2·x in 3 -3·x 2 +1)·y min +(x 3 -2·x 2 +x)·(SdrExtendedHightlightPoint[i]-SdrExtendedMidtonePoint[i])·0.0001+(-2·x 3 +3·x 2 )·y max +(x 3 -x 2 )·(SdrExtendedHightlightPoint[i]-SdrExtendedMidtonePoint[i])·2·k.

[0118] where x=x t / (2*k); y min =SdrDarkZoneGain[i]*wGain; y max =SdrDarkZoneGain[i]*wGain+(SdrExtendedHightlightPoint[i]-SdrExtendedMidtonePoint[i])*wGain.

[0119] In the case where x t >SdrExtendedHightlightPoint[i], f Gain (x t )=SdrDarkZoneGain[i]*wGain+(SdrExtendedHightlightPoint[i]-SdrExtendedMidtonePoint[i])*wGain+(x t -SdrExtendedHightlightPoint[i])·2·k.

[0120] Step 53: The electronic device performs inverse tone mapping on each pixel in the first partition based on the inverse tone mapping curve to obtain the adjusted first partition.

[0121] In one possible implementation, the gain corresponding to different gray values ​​can be determined based on the inverse tone mapping curve. Based on this gain, inverse tone mapping can be performed on each pixel in the first partition to obtain the adjusted first partition.

[0122] In another possible implementation, in order to match the adjusted first partition with the display capability of the electronic device, the electronic device can obtain the brightness clearance capability of the electronic device, so as to perform inverse tone mapping on each pixel in the first partition corresponding to the second partition according to the brightness information of the second partition and the brightness clearance capability of the electronic device, and obtain the adjusted first partition.

[0123] For example, when the extended dynamic range clearance of the electronic device is greater than or equal to the extended dynamic range clearance of the first partition, the extended dynamic range clearance of the first partition is used as the brightness clearance of the electronic device; when the extended dynamic range clearance of the electronic device is less than the extended dynamic range clearance of the first partition, the extended dynamic range clearance of the electronic device is used as the brightness clearance of the electronic device.

[0124] The brightness headroom capability of an electronic device can be expressed as displayHeadroom. Based on the brightness headroom capability of the electronic device, the screen reference white point factor can be determined, as shown in equation (7).

[0125] The gain corresponding to different grayscale values ​​is gain = fTonemapping(gray).

[0126] Where gray represents the grayscale value, which can be expressed by equation (8) as follows: gray=0.299*f[i][0]+0.587[i][1]+0.114*f[i][2] (8)

[0127] Where f[i][0] represents the red channel value of the pixel in the i-th row of the first partition; f[i][1] represents the green channel value of the pixel in the i-th row of the first partition; and f[i][2] represents the blue channel value of the pixel in the i-th row of the first partition.

[0128] Based on the gain corresponding to different grayscale values ​​and the screen reference white point factor, inverse tone mapping can be performed on each pixel in the first partition to obtain the adjusted first partition.

[0129] The adjusted red, green, and blue channel values ​​of the pixels in the first partition can be represented as f, respectively. TM[i][0]=f[i][0]*refWhiteRatio(1.0+gain), f TM [i][1] = f[i][1] * refWhiteRatio(1.0 + gain), and f TM [i][2] = f[i][2] * refWhiteRatio(1.0 + gain). The adjusted first partition can be generated based on the red, green, and blue channel values ​​of the pixels in the first partition.

[0130] In this embodiment of the application, based on steps 51-53, each pixel in each first partition can be subjected to inverse tone mapping, either individually or simultaneously, to obtain multiple adjusted first partitions.

[0131] S106. The electronic device generates at least one frame of processed first image based on multiple adjusted first partitions.

[0132] For each frame of the first image being processed, after obtaining multiple adjusted first partitions, the electronic device can stitch and combine the multiple adjusted first partitions to generate a processed first image.

[0133] S107. The electronic device displays at least one frame of the processed first image.

[0134] In this embodiment of the application, by partitioning the first image and the second image, and using the second partition of the second image, which includes more brightness information, as a reference, inverse tone mapping is performed on each pixel in each first partition of the first image. This can effectively distinguish between bright areas and high grayscale areas, improve the display effect of the processed first image, and display the processed first image on an electronic device with a dynamic range that is higher than that of the first image.

[0135] For ease of understanding, the image processing method provided in the embodiments of this application will be described in general with reference to Figure 6 below.

[0136] In response to a user's operation to play an SDR video, the electronic device decodes the SDR video to obtain a decoded SDR video. The SDR video comprises multiple frames of SDR images, and the decoded SDR video comprises multiple frames of decoded SDR images.

[0137] Electronic devices can identify each frame of SDR image in the decoded SDR video as the image to be processed; or they can identify the SDR image to be processed from the decoded SDR video at preset frame intervals.

[0138] In one example, users can choose to process the decoded SDR video frame by frame, or process it at preset intervals of frames, depending on their actual needs.

[0139] In another example, the electronic device can process the decoded SDR video based on a pre-set number of frames.

[0140] After determining the SDR images to be processed, the electronic device can downsample each SDR image to obtain a downsampled SDR image. Based on an AI model, the downsampled SDR images are then converted into HDR images.

[0141] Electronic devices can convert HDR images to linear space using EOTF to obtain the corresponding linear brightness of the HDR image. Based on this, the electronic devices can perform partitioning processing on the SDR image and HDR image to be processed, resulting in multiple first partitions of the SDR image and multiple second partitions of the HDR image.

[0142] For each SDR image to be processed, the electronic device determines the brightness information of each second partition. This brightness information may include, but is not limited to, the maximum brightness value, minimum brightness value, and average brightness value of each pixel in the second partition. Based on the brightness information of the second partition, the metadata of the corresponding first partition can be determined.

[0143] It is understandable that the calculation method for metadata is the same as that in S105 above, and will not be repeated here.

[0144] The electronic device determines its luminance headroom based on the relationship between its own extended dynamic range (EPR) headroom and the EPR headroom corresponding to the first partition. Based on the luminance headroom and metadata of each first partition, the inverse tone mapping curve for each first partition is determined.

[0145] Based on the inverse tone mapping curves corresponding to each first partition, the electronic device can perform inverse tone mapping on each pixel in each first partition individually or simultaneously to generate multiple adjusted first partitions.

[0146] After generating multiple adjusted first partitions, the electronic device can stitch and combine the multiple adjusted first partitions to generate a processed SDR image corresponding to a frame of the SDR image to be processed.

[0147] Based on the above method, the electronic device can generate a corresponding processed SDR image for each frame of the SDR image to be processed. Then, based on the unprocessed SDR images in the decoded SDR video and the processed SDR images corresponding to the SDR images to be processed, the electronic device can generate a processed SDR video and display it on the electronic device.

[0148] In summary, in this embodiment of the application, by partitioning the first image and the second image, and using the second partition of the second image, which includes more brightness information, as a reference, and performing inverse tone mapping on each pixel in each first partition of the first image, the display effect of the processed first image can be effectively improved. As shown in Figure 7, compared with the display effect of image 1701 in the prior art, the processed first image 1702 obtained based on the method provided in this embodiment of the application can better distinguish between bright areas and high grayscale areas, and the color appearance is more in line with the CAM16 model, with more natural contrast between light and dark.

[0149] Meanwhile, since the brightness information of the second image with a smaller number of pixels is used as a reference in this embodiment, and inverse tone mapping is performed on each pixel in each first partition of the first image, the image processing efficiency can be effectively improved, and real-time dynamic range expansion processing of low dynamic range video can be achieved.

[0150] In this embodiment, the structure of the electronic device can be as shown in Figure 8, which is a compositional example of an electronic device provided in this application embodiment.

[0151] As shown in Figure 8, the electronic device may include a processor 810, an external memory interface 820, an internal memory 821, and a display screen 831, etc.

[0152] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0153] The processor 810 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0154] For example, in this embodiment, the processor 810 can generate a second image corresponding to at least one frame of the first image in response to a user's operation of playing at least one frame of the first image; the number of pixels in the second image is less than the number of pixels in the first image, and the brightness information included in the second image is more than the brightness information included in the first image; the processor performs partitioning processing on the at least one frame of the first image and the second image corresponding to the at least one frame of the first image to obtain N first partitions of the at least one frame of the first image and M second partitions of the at least one frame of the second image corresponding to the N first partitions respectively; M and N are equal and are positive integers; based on the brightness information of the M second partitions, the processor performs inverse tone mapping on each pixel in the N first partitions corresponding to the M second partitions to obtain N adjusted first partitions; based on the N adjusted first partitions, the processor generates at least one frame of the processed first image.

[0155] Electronic devices implement display functions through a GPU, a display screen 831, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 831 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. The processor 810 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0156] The display screen 831 is used to display images, videos, etc. The display screen 831 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include one or N displays 831, where N is a positive integer greater than 1.

[0157] In this embodiment of the application, the processed first image can be displayed on the display screen 831 of the electronic device.

[0158] The external memory interface 820 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 810 through the external memory interface 820 to perform data storage functions. For example, in this embodiment, processed files such as the first image are saved on the external memory card.

[0159] Internal memory 821 can be used to store computer executable program code, which includes instructions. Processor 810 executes various functional applications and data processing of the electronic device by running the instructions stored in internal memory 821.

[0160] In addition, an operating system runs on top of these components. Examples include Apple's iOS, Google's Android, and Microsoft's Windows. Applications can be installed and run on this operating system.

[0161] The operating system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application uses the layered architecture Android system as an example to illustrate the software structure of an electronic device.

[0162] Figure 9 is an example diagram of the software structure of an electronic device provided in an embodiment of this application.

[0163] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0164] The application layer can include a series of application packages. As shown in Figure 9, an application package can include applications such as a browser and a gallery.

[0165] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications within the application layer. The application framework layer includes predefined functions. As shown in Figure 9, the application framework layer may include modules such as image processing.

[0166] The image processing module may include a conversion module, a partitioning module, a mapping module, and a generation module.

[0167] The conversion module is used to convert the first image into the second image; the partitioning module is used to partition the first image and the second image to obtain multiple first partitions of the first image and multiple second partitions of the second image; the mapping module is used to perform inverse tone mapping on each pixel in the first partition corresponding to the second partition according to the brightness information of the second partition to obtain the adjusted first partition; and the generation module is used to generate at least one frame of the processed first image based on the multiple adjusted first partitions.

[0168] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0169] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0170] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0171] A system library can include multiple functional modules. For example, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0172] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0173] A 2D graphics engine is a graphics engine for 2D drawing.

[0174] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver.

[0175] It should be noted that although the embodiments of this application are illustrated using the Android system as an example, the basic principles are also applicable to electronic devices based on operating systems such as iOS and Windows.

[0176] This embodiment also provides a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform the aforementioned method steps to implement the method described in the above embodiment.

[0177] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, include: In response to a user's operation of playing at least one frame of a first image, a second image corresponding to at least one frame of the first image is generated; the number of pixels in the second image is less than the number of pixels in the first image, and the second image includes more brightness information than the first image includes; The at least one first image frame and the corresponding second image are partitioned to obtain N first partitions of the at least one first image frame and M second partitions of the at least one second image frame corresponding to the N first partitions respectively; M and N are equal and are positive integers. Based on the brightness information of the M second partitions, inverse tone mapping is performed on each pixel in the N first partitions corresponding to the M second partitions to obtain N adjusted first partitions; Based on N adjusted first partitions, at least one frame of processed first image is generated.

2. The method according to claim 1, characterized in that, The step of performing inverse tone mapping on each pixel in each of the N first partitions corresponding to the M second partitions based on the brightness information of the M second partitions to obtain N adjusted first partitions includes: Based on the brightness information of the M second partitions, determine the metadata of the N first partitions corresponding to the M second partitions; Based on the metadata of the N first partitions, inverse tone mapping is performed on each pixel in the N first partitions corresponding to the M second partitions to obtain N adjusted first partitions.

3. The method according to claim 2, characterized in that, The step of performing inverse tone mapping on each pixel in the N first partitions corresponding to the M second partitions based on the metadata of the N first partitions to obtain N adjusted first partitions includes: Based on the metadata of the N first partitions, inverse tone mapping curves for the N first partitions are generated; the inverse tone mapping curves are used to indicate the correspondence between the tone information of each pixel in the first partition and the tone information of each pixel in the adjusted first partition. Based on the inverse tone mapping curves of the N first partitions, inverse tone mapping is performed on each pixel in the N first partitions corresponding to the M second partitions to obtain N adjusted first partitions.

4. The method according to claim 1, characterized in that, The step of performing inverse tone mapping on each pixel in each of the N first partitions corresponding to the M second partitions based on the brightness information of the M second partitions to obtain N adjusted first partitions includes: Obtain the brightness clearance capability of electronic devices; Based on the brightness information of the M second partitions and the brightness clearance capability of the electronic device, inverse tone mapping is performed on each pixel in the N first partitions corresponding to the M second partitions to obtain N adjusted first partitions.

5. The method according to claim 4, characterized in that, The step involves performing inverse tone mapping on each pixel in the N first partitions corresponding to the M second partitions based on the brightness information of the M second partitions and the brightness clearance capability of the electronic device, to obtain N adjusted first partitions, including: Based on the brightness information of the M second partitions, determine the metadata of the N first partitions corresponding to the M second partitions; Based on the brightness clearance capability of the electronic device and the metadata of the N first partitions, inverse tone mapping is performed on each pixel in the N first partitions corresponding to the M second partitions to obtain N adjusted first partitions.

6. The method according to claim 1, characterized in that, The step of generating a second image corresponding to at least one frame of the first image in response to a user's operation of playing at least one frame of the first image includes: In response to a user's operation to play at least one frame of the first image, the at least one frame of the first image is downsampled to obtain a third image corresponding to the at least one frame of the first image; Inverse tone mapping is performed on at least one frame of the third image to generate a second image corresponding to the at least one frame of the first image.

7. The method according to claim 1, characterized in that, The step of generating a second image corresponding to at least one frame of the first image in response to a user's operation of playing at least one frame of the first image includes: In response to a user's video playback operation, at least one frame of the first image in the video is processed frame by frame or at a preset interval of frames to generate a second image corresponding to at least one frame of the first image.

8. The method according to claim 1, characterized in that, The step of partitioning the at least one first image frame and the corresponding second image to obtain N first partitions of the at least one first image frame and M second partitions of the at least one second image frame corresponding to the N first partitions, includes: According to a preset number of parts, the at least one first image frame and the corresponding second image frame are partitioned to obtain a preset number of first partitions of the at least one first image frame and a preset number of second partitions of the at least one second image frame corresponding to the preset number of first partitions.

9. The method according to claim 1, characterized in that, The step of generating a second image corresponding to at least one frame of the first image in response to a user's operation of playing at least one frame of the first image includes: In response to a user's operation to play at least one frame of the first image, the at least one frame of the first image is decoded to obtain at least one decoded first image; Based on the first image after at least one frame of decoding, a second image corresponding to the first image is generated.

10. The method according to any one of claims 1-9, characterized in that, After generating at least one frame of processed first image based on multiple adjusted first partitions, the process further includes: The first image, after processing at least one frame, is displayed on an electronic device, the electronic device supporting a dynamic range higher than that of the first image.

11. An electronic device, characterized in that, The electronic device includes: a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the steps of an image processing method as described in any one of claims 1-10 according to instructions in the program code.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of an image processing method as described in any one of claims 1-10.

13. A communication device, characterized in that, Includes a module for performing the method as described in any one of claims 1 to 10.