Image processing method and electronic device

Abstract

Embodiments of the present application relate to the technical field of image processing, and provide an image processing method and an electronic device. The method comprises: acquiring a first image and a second image, wherein the first image is a high dynamic range (HDR) image, and the second image is a standard dynamic range (SDR) image; determining a first gain range on the basis of the first image and the second image; determining metadata of first gain information on the basis of the first gain range, wherein the first gain information is calculated on the basis of the first image and the second image; and obtaining an available image on the basis of the first gain information, the metadata, and the second image. In this way, by adjusting gain ranges, the quality loss of reconstructed HDR images can be reduced, thereby improving the image quality of images displayed in an electronic device.

Description

Image processing method and electronic device

[0001] The present application claims priority to the Chinese patent application No. 202410744662.3, filed on June 7, 2024, and entitled "Image processing method and electronic device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of image processing, and in particular, to an image processing method and an electronic device. BACKGROUND

[0003] In order to make the image displayed by the electronic device more truly reflect the lighting conditions in nature, high dynamic range (HDR) technology has been widely concerned in the field of image processing technology. For example, the electronic device can display an HDR image through the display screen, thereby providing better visual experience for the user. However, some electronic devices, such as electronic devices provided with an SDR display screen, etc., may not be able to display an HDR image, or the displayed HDR image has image quality loss, etc.

[0004] Therefore, after the electronic device captures the image data, the SDR image and the gain map can be obtained according to the initial image data; then, the electronic device can display the enhanced image obtained from the SDR image and the gain map through the display screen.

[0005] However, such a method may cause the displayed image of the electronic device to have quality loss. SUMMARY

[0006] Embodiments of the present application provide an image processing method and an electronic device, which can reduce the image quality loss of the reconstructed HDR image by adjusting the gain range, and improve the image quality of the image displayed by the electronic device.

[0007] In a first aspect, an image processing method is provided. The method comprises: obtaining a first image and a second image, the first image being a high dynamic range (HDR) image, and the second image being a standard dynamic range (SDR) image; determining a first gain range according to the first image and the second image; determining metadata of first gain information according to the first gain range, the first gain information being calculated according to the first image and the second image; and obtaining a selectable image according to the first gain information, the metadata, and the second image.

[0008] The image processing method of the present application, after obtaining the first gain information, the electronic device will additionally determine a first gain range for reducing the loss of image quality according to the first image and the second image, and determine the metadata of the first gain information in combination with the first gain range, so as to adjust the first gain information according to the first gain range in the subsequent processing process, and obtain the second gain information. In this way, the gain in the second gain information can be within the first gain range, and the basic image is dynamically expanded by the second gain information, so that the reconstructed HDR image determined is closer to the first image, which helps to reduce the loss of image quality.

[0009] Optionally, the first gain range includes a maximum value and a minimum value. According to the first gain range, the metadata of the first gain information is determined, including: writing the maximum value and the minimum value into the metadata of the first gain information.

[0010] In this way, the first gain range can be quickly and efficiently obtained by analyzing the metadata.

[0011] Optionally, the first gain range is determined, including: determining the first gain range according to the linear space luminance of the first image and the linear space luminance of the second image.

[0012] Since the linear space luminance can more accurately reflect the real lighting conditions, this helps to make the HDR image reconstructed based on the first gain range meet the quality requirements.

[0013] Optionally, the first gain range is determined, including: calculating a dynamic range headroom level according to the linear space luminance of the first image and the linear space luminance of the second image; and determining the first gain range according to the dynamic range headroom level.

[0014] Since the dynamic range headroom level can reflect the brightness advantage of the first image relative to the second image, by the dynamic range headroom level, the first gain range is determined, which helps to make the HDR image reconstructed based on the first gain range closer to the first image.

[0015] Optionally, the dynamic range headroom level satisfies: the dynamic range headroom level is a first ratio, and the first ratio is a ratio of the maximum linear space luminance of the first image to the maximum linear space luminance of the second image; or, the dynamic range headroom level is a first preset headroom, and the first ratio is greater than the first preset headroom.

[0016] When the dynamic range headroom level is the first ratio, the dynamic range headroom level can reflect the advantage of the maximum linear space luminance of the first image relative to the maximum linear space luminance of the second image; when the dynamic range headroom level is the first preset headroom, the dynamic range headroom level can be matched with the performance and / or bit width of the display of the electronic device.

[0017] Optionally, determining the first gain range comprises: determining the first gain range according to the first image, the second image and the first gain information.

[0018] Since the first gain information can reflect the actual gain situation of the first image relative to the second image, and the image difference situation between the first image and the second image can be analyzed according to the first image and the second image, therefore, the first gain range determined in combination with the first image, the second image and the first gain information can be adapted to the actual scene of the current HDR image reconstruction.

[0019] Optionally, determining the first gain range comprises: determining the first gain range according to the dynamic range headroom level and the second gain range, the second gain range being a gain range of the first gain information, and the dynamic range headroom level being calculated according to the first image and the second image.

[0020] In this way, the first gain range can be determined in combination with the luminance advantage of the first image relative to the second image and the second gain range, and the first gain range can be within the second gain range, which helps to make the HDR image reconstructed based on the first gain range closer to the first image.

[0021] Optionally, determining the first gain range comprises: determining the first gain range according to the first image, the second image and the bit width.

[0022] Since the bit width can affect the color depth, detail performance and overall visual quality of the image, therefore, the first gain range is determined in combination with the first image, the second image and the bit width, which helps to make the image quality loss of the HDR image reconstructed based on the first gain range lower.

[0023] Optionally, determining the first gain range comprises: determining a first enhanced image according to the second image and the first gain information; and determining the first gain range according to the first enhanced image, the first image and the second image.

[0024] In this way, the first gain range can be calculated through the image contrast situation of the first enhanced image relative to the first image, so that the determined first gain range can realize image reconstruction of the first image with lower image loss.

[0025] Optionally, determining the first gain range comprises: determining a third gain range according to the first image and the second image; and reducing the third gain range to obtain the first gain range according to the linear space luminance of the first enhanced image and the linear space luminance of the first image.

[0026] Since the first enhanced image is the reconstructed first image (i.e., the reconstructed HDR image), the closer the first enhanced image is to the first image, the lower the image quality loss of the HDR image reconstructed based on the first gain range. Therefore, after the third gain range is initially determined, whether the third gain range can meet the image quality loss requirement of the HDR image reconstruction can be further verified based on the linear space luminance of the first enhanced image and the linear space luminance of the first image, and when not met, the third gain range is further narrowed to obtain the first gain range, so that the finally determined first gain range can effectively achieve the effect of reducing the image quality loss.

[0027] Optionally, the third gain range is narrowed, comprising: narrowing the third gain range according to the contrast of the linear space luminance of the first enhanced image and the linear space luminance of the first image.

[0028] By the contrast of the linear space luminance of the first enhanced image and the linear space luminance of the first image, the image quality loss existing in the image reconstruction based on the third gain range can be effectively measured to improve the necessity of narrowing the third gain range.

[0029] Optionally, the third gain range is narrowed, comprising: narrowing the third gain range in a case that the ratio of the number of the first pixel points to the number of the second pixel points is less than or equal to a first threshold, or the number of the first pixel points is less than or equal to a second threshold; wherein the second pixel points are pixel points in the first gain information whose gain values are in the third gain range, and the first pixel points are pixel points in the second pixel points whose contrasts satisfy a contrast sensitivity threshold.

[0030] Since when the contrast of a pixel point satisfies a corresponding contrast sensitivity threshold, it can be determined that the luminance loss of the pixel point is small, and the human eye generally cannot observe that there is image quality loss at the position.

[0031] Therefore, such a manner can make most of the pixel points in the HDR image reconstructed based on the first gain range satisfy the contrast sensitivity threshold, so that the HDR image reconstructed based on the first gain range is close to the first image, and it is helpful to realize the image reconstruction with small image quality loss.

[0032] Optionally, the first pixel points are pixel points in the second pixel points whose contrasts satisfy a contrast sensitivity function.

[0033] In this way, the number of the first pixel points or the ratio of the first pixel points in the second pixel points can evaluate the region where the human eye cannot observe the image quality loss. When the number of the first pixel points or the ratio of the first pixel points in the second pixel points is large, it can be indicated that the human eye cannot observe the difference between most regions in the first enhanced image and the first image, and it is indicated that the image quality loss of the first enhanced image is low.

[0034] Optionally, the contrast is any one of: a first difference, the first difference being a difference between the linear space luminance of the first image and the linear space luminance of the first enhanced image; a ratio of the first difference to the linear space luminance of the first image or the linear space luminance of the first enhanced image; or, a ratio of the first difference to a first sum, the first sum being a sum of the linear space luminance of the first image and the linear space luminance of the first enhanced image.

[0035] In this way, the contrast can reflect the luminance contrast of the pixel points in the first enhanced image compared with the pixel points in the first image.

[0036] Optionally, the obtaining the selectable image according to the first gain information, the metadata and the second image comprises: setting the gain smaller than the minimum value in the first gain information to the minimum value according to the minimum value recorded in the metadata; and setting the gain greater than the maximum value in the first gain information to the maximum value according to the maximum value recorded in the metadata, to obtain second gain information. The selectable image is obtained according to the second gain information and the second image.

[0037] In this way, the gain values in the first gain information can be adjusted according to the first gain range recorded in the metadata to obtain the second gain information, and then the second image is dynamically expanded based on the second gain information to obtain the selectable image for reconstructing the first image, so that the determined first gain range can be effectively applied in the reconstruction process of the HDR image. Meanwhile, in the embodiment, the gain values in the first gain information are adjusted only in the decoding stage, so that the first gain information can carry complete original gain information to the decoding stage, thereby providing more data reference for the decoding process and improving the quality of subsequent data processing.

[0038] Optionally, the method further comprises: performing normalization processing on the first gain information to obtain normalized first gain information.

[0039] The obtaining the selectable image comprises: performing denormalization processing on the normalized first gain information to obtain denormalized first gain information. The second gain information is determined according to the denormalized first gain information and the first gain range recorded in the metadata, and the selectable image is obtained according to the second gain information and the second image.

[0040] Through the normalization processing, the data amount in the data transmission process can be effectively reduced, and then through the denormalization processing, the original data can be effectively restored, so that the correctness of the data processing in the decoding process is ensured.

[0041] Optionally, the image can be selected for reconstructing the first image. For example, the image can be selected to undergo a non-linear conversion to obtain an enhanced image, which can be understood as a reconstructed HDR image, i.e., the reconstructed first image. The enhanced image is not the first image, but the enhanced image has a display effect similar to that of the first image.

[0042] In a second aspect, an embodiment of the present application provides an image processing apparatus. The image processing apparatus can be an electronic device, or a chip or chip system in the electronic device. The image processing apparatus can include a display unit and a processing unit. When the image processing apparatus is an electronic device, the display unit can be a display screen. The display unit is configured to perform the displaying, so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect.

[0043] When the image processing apparatus is an electronic device, the processing unit can be a processor. The image processing apparatus can further include a storage unit, which can be a memory. The storage unit is configured to store instructions, and the processing unit executes the instructions stored in the storage unit, so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing apparatus is a chip or chip system in the electronic device, the processing unit can be a processor. The processing unit executes the instructions stored in the storage unit, so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect. The storage unit can be a storage unit (e.g., a register, a cache, etc.) in the chip, or a storage unit (e.g., a read-only memory, a random access memory, etc.) in the electronic device and located outside the chip.

[0044] For example, the processing unit is configured to perform image fusion on part or all of the multiple frames of images to obtain the first image, and the display unit is configured to display the target image.

[0045] In a third aspect, an embodiment of the present application provides an electronic device. The electronic device includes a processor and a memory. The memory is configured to store code instructions, and the processor is configured to execute the code instructions to perform the method described in the first aspect or any possible implementation of the first aspect.

[0046] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program or instructions. When the computer program or instructions are executed on a computer, the computer performs the method described in the first aspect or any possible implementation of the first aspect.

[0047] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program, which, when executed on a computer, causes the computer to perform the method described in the first aspect or any possible implementation manner of the first aspect.

[0048] In a sixth aspect, the present application provides a chip or a chip system, which includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through a line, the at least one processor is configured to execute a computer program or instructions to perform the method described in the first aspect or any possible implementation manner of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0049] In a possible implementation, the chip or the chip system described in the present application further includes at least one memory, and the at least one memory stores instructions. The memory can be a storage unit inside the chip, for example, a register, a cache, etc., or a storage unit of the chip (for example, a read-only memory, a random access memory, etc.).

[0050] It should be understood that the second aspect to the sixth aspect of the present application correspond to the technical solution of the first aspect of the present application, and the beneficial effects obtained by each aspect and the corresponding possible implementation manner are similar, which will not be described herein again. BRIEF DESCRIPTION OF DRAWINGS

[0051] FIG. 1 is a schematic block diagram of a hardware architecture of an electronic device according to an embodiment of the present application;

[0052] FIG. 2 is a schematic diagram of an application scenario according to an embodiment of the present application;

[0053] FIG. 3 is a schematic diagram of an image processing process;

[0054] FIG. 4 is a schematic diagram of an image restoration scenario according to an embodiment of the present application;

[0055] FIG. 5 is a comparative schematic diagram of image restoration according to an embodiment of the present application;

[0056] FIG. 6 is a flowchart of an image processing method according to an embodiment of the present application;

[0057] FIG. 7 is a schematic diagram of determining metadata of first gain information according to an embodiment of the present application;

[0058] FIG. 8 is a flowchart of determining image contrast according to an embodiment of the present application;

[0059] FIG. 9 is a schematic diagram of determining a first pixel point according to an embodiment of the present application;

[0060] FIG. 10 is a curve diagram of a contrast sensitivity function according to an embodiment of the present application;

[0061] FIG. 11 is a process diagram of determining second gain information according to an embodiment of the present application;

[0062] FIG. 12 is a diagram of an image processing process according to an embodiment of the present application;

[0063] FIG. 13 is a schematic block diagram of an image processing apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to clearly describe the technical solutions of the embodiments of the present application, the following briefly introduces some terms and technologies involved in the embodiments of the present application:

[0065] 1. Dynamic range

[0066] The ratio of the maximum luminance to the minimum luminance in a video signal, or the ratio of the maximum luminance to the minimum luminance in an image.

[0067] 2. Standard dynamic range (SDR) image

[0068] Refers to an image with a traditional dynamic range, as opposed to a high dynamic range (HDR) image. Compared with an HDR image, an SDR image has a narrower dynamic range and color gamut, and is suitable for most traditional display devices and content. Moreover, an SDR image can be an image generated by tone mapping an HDR image.

[0069] 3. High dynamic range (HDR) image

[0070] Refers to an image that can display more luminance levels and a wider color gamut than an SDR image. The luminance range of an HDR image usually ranges from very dark black to very bright white, and can more realistically reflect the lighting conditions in nature. An HDR image can display higher luminance and deeper black, thereby providing richer details and a more realistic visual experience.

[0071] From the perspective of color space, an HDR image usually uses a wider color space, such as Rec. 2020 (BT. 2020) or DCI-P3, while a traditional SDR image usually uses the Rec. 709 color space. A wider color space means that an HDR image can display more colors and richer color details.

[0072] 4. Linear space brightness

[0073] Luma refers to the luminance value represented in a linear color space. Unlike non-linear color spaces (such as sRGB or Rec.709), the luminance value in a linear color space is directly proportional to the physical luminance. This means that linear space luminance can more accurately reflect the true lighting conditions when performing image processing and analysis.

[0074] Exemplarily, the processing of HDR images usually needs to be performed in a linear space in order to accurately synthesize and adjust images of different exposures.

[0075] The calculation of the linear space luminance of an HDR image usually includes the following two steps: EOTF electro-optical conversion and color space conversion. The calculation of the linear space luminance of an SDR image can be achieved by de-gamma correction.

[0076] 5、De-gamma correction

[0077] Also known as linearization, it usually refers to the process of converting an image from a gamma-encoded non-linear color space back to a linear color space.

[0078] 6、Color space conversion

[0079] Also known as color conversion, it refers to the process of converting image or video data from one color space to another color space.

[0080] A color space defines the way colors are represented and the color gamut, and the color gamut together with the color model can define a color space. Among them, the color model is an abstract mathematical model that represents colors with a set of color components. The color model may, for example, include a red green blue (RGB) three-primary color light mode, a cyan magenta yellow key plate (CMYK) four-color printing mode. The color gamut refers to the total combination of colors that a system can produce.

[0081] The Rec.2020 (BT.2020) mentioned above when introducing HDR images stands for ITU-R Recommendation BT.2020. It is a recommendation or standard for ultra-high-definition television systems developed by the International Telecommunication Union Radiocommunication Sector (ITU-R). "Rec" stands for Recommendation, and "BT" stands for Broadcasting Service. DCI-P3 stands for Digital Cinema Initiatives-P3, and Rec.709 stands for ITU-R Recommendation BT.709. In one implementation, the color gamut size of these three color spaces is: Rec.2020 > DCI-P3 > Rec.709.

[0082] 7. Gain map

[0083] Also known as gain information, it describes the brightness adjustment information for different regions in an HDR image. For example, a gain map can be an image of the same size as the HDR image, with each pixel value representing the brightness gain for that region. Brightness gain can also be called gain or gain value, and is a coefficient or scaling factor used to adjust the brightness of an image.

[0084] Gain maps are typically stored in floating-point format, representing gain values ​​in linear space.

[0085] 8. Gain Range

[0086] It is an interval, namely [minimum gain value, maximum gain value]. This interval represents the range of gain variation in different regions of the image.

[0087] The minimum gain value is found by iterating through all pixels in the gain map and identifying the smallest gain value. This typically represents the gain of the darkest area in the image.

[0088] Furthermore, the maximum gain value is the maximum gain value found across all pixels in the gain map. This typically represents the gain of the brightest region in the image.

[0089] 9. Bit depth

[0090] The number of bits used for each pixel determines the range of values ​​for that pixel. For example, an 8-bit width can represent integer values ​​from 0 to 255, while a 16-bit width can represent integer values ​​from 0 to 65535.

[0091] 10. Dynamic range headroom

[0092] Also known as headroom, it is a metric that measures the brightness advantage of HDR images over SDR images. The headroom of an HDR image can also be called the high dynamic range display headroom (HDR headroom).

[0093] 11. Perceptual quantizer (PQ)

[0094] PQ is an HDR standard and also an HDR conversion equation, determined by human visual ability. The video signal displayed by a display device is typically a PQ-encoded video signal.

[0095] 12. Barten Model

[0096] The Barten model is a model used to describe and quantify the human visual system's perception of image noise. Originally proposed by Peter GJ Barten, it has been widely applied in image quality assessment, display technology, and visual perception research. The core idea of ​​the Barten model is to use mathematical formulas to describe the visual system's sensitivity to different spatial frequencies and contrasts, thereby quantifying the human ability to perceive image noise.

[0097] 13. Contrast sensitivity function (CSF)

[0098] The core of the Barten model is the contrast sensitivity function (CSF), which describes the human eye's sensitivity to different spatial frequencies (i.e., the size of details in an image). The CSF is typically a strip filter, indicating that the human eye is most sensitive to details at medium frequencies, while its sensitivity to details at very low or very high frequencies is lower.

[0099] 14. A normal frame, also known as a normal exposure image, N-frame image, N-frame, medium frame, or medium frame image, is an image captured by a camera with an exposure of 0 EV. In other words, the exposure of a normal exposure image is 0 EV. Here, 0 EV is a relative value, not an absolute zero exposure. For example, exposure = exposure time * ISO. Assuming a normal exposure image is captured at ISO 200 and an exposure time of 50 milliseconds, the actual exposure corresponding to 0 EV is the product of 200 and 50 milliseconds.

[0100] 15. Short frames, also known as short-frame images, S-frames, or S-frame images, are images captured by a camera when the exposure is less than 0 EV.

[0101] 16. Long frames, also known as long frame images, L (long) frames, or L-frame images, are images captured by a camera when the exposure is greater than 0 EV.

[0102] 17. Electro-optical transfer function (EOTF): A conversion relationship between nonlinear primary color values ​​and linear primary color values.

[0103] 18. Optical-electro transfer function (OETF): A conversion relationship between linear primary color values ​​and nonlinear primary color values.

[0104] 19. Optical-to-Optical Transfer Curve (OOTF): A curve in video technology that converts one optical signal into another.

[0105] Of the three concepts introduced above, a primary color value is a numerical value corresponding to a specific image color component (such as R, G, B, or Y). A linear primary color value is a value that is linearly proportional to light intensity; in one optional case, its value should be normalized to [0,1], abbreviated as E. A nonlinear primary color value is a value that is non-linearly proportional to the image information; it is a normalized digital representation of the image information, proportional to the digital encoding value; in one optional case, its value should be normalized to [0,1], abbreviated as E′.

[0106] The digital code value is a digital representation of an image signal, used to represent non-linear primary color values.

[0107] 20. Other terms

[0108] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with substantially the same function and purpose. For example, "first chip" and "second chip" are used only to distinguish different chips and do not limit their order of execution. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0109] It should be noted that the terms "exemplary" and "for example" are used herein to mean "serving as an example, instance, or illustration," and not "preferred" or "advantageous over other examples." The usage of these terms in this application is not intended to convey any preference or advantage for the embodiments or examples described with such terms.

[0110] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. The relationship between the associated objects described by "and / or" indicates that there can be three kinds of relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0111] 21. An electronic device

[0112] The electronic device of the embodiments of the present application can include a handheld device with a camera function, a vehicle-mounted device, etc. For example, some electronic devices are: a mobile phone, a tablet computer, a palm computer, a notebook computer, a mobile Internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with a wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal device in a 5G network, or a terminal device in a future evolved public land mobile network (PLMN), etc., and the embodiments of the present application are not limited thereto.

[0113] By way of example and not limitation, in the embodiments of the present application, the electronic device can also be a wearable device. The wearable device can also be referred to as a wearable smart device, which is a general term for devices that are designed and developed by applying wearable technology to daily wear, such as glasses, gloves, watches, clothing, and shoes, etc. The wearable device is a portable device that is directly worn on the body or integrated into the clothes or accessories of the user. The wearable device is not only a hardware device, but also a device that realizes powerful functions through software support and data interaction and cloud interaction. The general wearable smart device includes a device with full functions and large size, which can realize complete or partial functions without relying on a smart phone, such as a smart watch or smart glasses, etc., and a device that focuses on a certain application function and needs to cooperate with other devices such as a smart phone, such as various smart wristbands and smart jewelry for monitoring vital signs, etc.

[0114] In addition, in the embodiments of the present application, the electronic device can also be a terminal device in an Internet of Things (IoT) system. The IoT is an important part of future information technology development, and its main technical feature is to connect objects through communication technology and network, so as to realize the intelligent network of man-machine interconnection and object-object interconnection.

[0115] The electronic device in the embodiments of the present application can also be referred to as a terminal device, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), an access terminal, a subscriber unit, a subscriber station, a mobile station, a mobile terminal, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent, or a user apparatus, etc.

[0116] In the embodiments of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes central processing unit (CPU), memory management unit (MMU), and memory (also known as main memory), etc. The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or windows operating system, etc. The application layer includes browser, address book, word processing software, instant messaging software, etc.

[0117] The hardware structure of the above-mentioned electronic device can be as shown in FIG. 1. FIG. 1 shows a structural schematic diagram of an electronic device 100 suitable for the present application.

[0118] The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0119] The sensor module 180 can include a pressure sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like. The audio module 170 can include a speaker, a receiver, a microphone, an earphone interface, and the like.

[0120] It should be noted that the structure shown in FIG. 1 does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than those shown in FIG. 1, or the electronic device 100 can include a combination of some of the components shown in FIG. 1, or the electronic device 100 can include sub-components of some of the components shown in FIG. 1. For example, the proximity light sensor 180G shown in FIG. 1 can be optional. The components shown in FIG. 1 can be implemented in hardware, software, or a combination of software and hardware.

[0121] The processor 110 can include one or more processing units. For example, the processor 110 can include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU). Among them, different processing units can be independent devices, or can be integrated devices.

[0122] Optionally, the electronic device 100 can further include an image encoder and an image decoder. The image encoder can be used to encode the image data obtained by the camera 193, for example, to obtain a baseline image and a gain map after processing the image data obtained by the camera 193, and to transmit an image file including the baseline image and the gain map to the image decoder. The image decoder can decode the image file from the image encoder to obtain the baseline image and the gain map, and can obtain a first enhanced image based on the baseline image and the gain map, so as to indicate the first enhanced image to the display screen 194 for display.

[0123] It can be understood that the image encoder and the image decoder can be independent devices or integrated devices. For example, in the case of the image encoder and the image decoder being integrated devices, the image encoder and the image decoder can also be referred to as an image codec.

[0124] In addition, the image encoder and the image decoder (or the image codec) can be integrated on the processor 110, or the image encoder and the image decoder (or the image codec) are independent devices, which are not limited in the present application.

[0125] In a possible implementation, the processor 110 can obtain image data by starting the camera 193. The image data can be an image in RAW format, and the processor 110 can obtain an HDR image and an SDR image from the image in RAW format. In order to facilitate the image encoder and the image decoder to determine an enhanced image based on the HDR image and the SDR image.

[0126] The controller can generate operation control signals according to the instruction operation code and the timing signal, and complete the control of fetching and executing instructions.

[0127] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0128] The electronic device 100 can realize the photographing function through the ISP, the camera 193, the GPU, the display screen 194, and the application processor.

[0129] The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture a still image or a video. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. In some embodiments, the ISP can be arranged in the camera 193. For example, the electrical signal transmitted by the camera photosensitive element to the ISP can be an image sequence, and the ISP can pre-process the image sequence. The image sequence after ISP pre-processing can be an image in RAW format.

[0130] In some embodiments, the electronic device 100 can include one or N cameras 193, and N is a positive integer greater than 1.

[0131] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information. For example, the processor 110 can call the GPU to perform image fusion and noise reduction processing on at least one frame of short frame images and at least one frame of middle frame images; the processor 110 can also call the GPU to perform color correction on a target region, such as a sky region, etc.

[0132] The display screen 194 can be used to display the enhanced image after rendering, etc.

[0133] It should be understood that the connection relationship between the modules shown in FIG. 1 is only illustrative and does not constitute a limitation on the connection relationship between the modules of the electronic device 100. Alternatively, the modules of the electronic device 100 can also use a combination of the above-mentioned various connection modes.

[0134] At present, in order to improve the shooting effect of the electronic device, part of the electronic device can take pictures through the HDR technology. Taking pictures through the HDR technology can also be understood as the electronic device taking pictures in the HDR mode. In the HDR mode, in response to a shooting operation, the camera of the electronic device can take multiple images at different exposure amounts. The multiple images include, for example, long frames, short frames, and normal frames, etc. Then, by image fusion on part or all of the multiple images, a target image can be obtained. In the case that the target image is an HDR image, the target image has more brightness levels and a wider color gamut, so that the target image displayed by the electronic device can more truly reflect the lighting conditions in nature.

[0135] However, in some cases, the electronic device may not be able to normally display the HDR image, or the image quality of the displayed HDR image is poor, etc. For example, the display screen of part of the electronic device does not support displaying the HDR image, so that the part of the electronic device cannot directly display the HDR image; the data amount of the HDR image is usually large, and the transmission and storage of the HDR image in the electronic device occupy a large bandwidth and storage space, so that the load of the electronic device is large, thereby causing the electronic device to be stuck and other abnormalities.

[0136] Therefore, at present, the shooting effect of the electronic device can be improved in the following ways. Next, the shooting process of the electronic device will be described in combination with the application scenario 200 of the embodiment of the present application shown in FIG. 2.

[0137] FIG. 2 is a schematic diagram of an application scenario 200 provided by an embodiment of the present application. The application scenario 200 can be applied to an electronic device, such as the electronic device 100, and the electronic device can support photographing by using an HDR technology.

[0138] In the HDR mode, in response to a photographing operation, the camera of the electronic device photographs image data, and based on the image data, the electronic device can obtain a base image and gain information by using an image encoder. The base image is an image with a traditional dynamic range. The gain information is a plurality of gain values stored in a floating-point number format. The image encoder can transmit an image file including the base image and the gain information to an image decoder. The image decoder can obtain an enhanced image based on the base image and the gain information, so that the display screen of the electronic device can display the enhanced image. As shown in FIG. 2, a user can view an image in an image application of the electronic device, and then the electronic device displays the enhanced image on the display screen in response to an image display command triggered by a user operation.

[0139] The enhanced image can be a non-linear domain signal, which can be obtained by performing a non-linear conversion on a linear domain signal. For example, the image decoder can calculate an alternate image based on the base image and the gain information. The alternate image is a linear domain signal.

[0140] In this way, the base image is processed by using the gain information, so that the brightness of different regions in the base image can be adjusted to expand the dynamic range of the base image. By applying different gains to different brightness regions, dark details can be made clearer, while bright details are preserved. By adjusting the local gain, the details and textures in the base image can be preserved while the overall brightness of the base image is enhanced. The image displayed by the electronic device can present an effect similar to that of an HDR image.

[0141] In addition, for the electronic device that cannot display an HDR image, the dynamic range of the enhanced image is adapted to the range of the electronic device, so that a better visual effect can be obtained on various electronic devices. That is, the enhanced image obtained by using the gain map and the base image not only has a higher dynamic range than the base image, but also has a dynamic range adapted to the electronic device.

[0142] In addition, compared with an HDR image, the bandwidth and storage space occupied by the image file transmitted from the image encoder to the image decoder can be smaller, so that the power consumption of the electronic device is smaller.

[0143] It should be understood that the image encoder above can also be referred to as an encoding module, and the image decoder can also be referred to as a decoding module. Among them, the encoding module can be used to determine a gain map and transmit an image file including the gain map and a base image to the decoding module. Then, the decoding module can determine an enhanced image based on the gain map and the base image. Moreover, the process of determining the gain map by the encoding module can also be referred to as computing a gain map, and the process of determining the enhanced image based on the gain map and the base image by the decoding module can also be referred to as applying a gain map. The name of the module for computing the gain map and the name of the module for applying the gain map are not specifically limited in the present application.

[0144] Next, the process of computing the gain map and applying the gain map will be described in detail in combination with FIG. 3.

[0145] As shown in FIG. 3, the camera of the electronic device can capture multiple images at different exposure amounts, respectively. Then, by image fusion on part or all of the multiple images, a RAW format image can be obtained. The RAW format image can also be referred to as a RAW image for short. The RAW image refers to image data directly obtained by the electronic device through an image sensor without compression and processing, which contains original pixel information captured from the image sensor.

[0146] Then, based on the RAW image, a preset HDR image and a base image can be obtained.

[0147] Among them, the preset HDR image obtained based on the RAW image can be implemented by the following manner: processing the RAW image by using the OETF transfer function of PQ to obtain the preset HDR image. The OETF transfer function of PQ represents the conversion relationship from the linear signal value of the image pixel to the nonlinear signal value in the PQ domain. Processing the RAW image by using the OETF transfer function of PQ is to convert the optical signal into an electrical signal.

[0148] In addition to the OETF transfer function of PQ, the electronic device can also process the RAW image based on the hybrid log-gamma (HLG) photoelectric transfer function to obtain the preset HDR image. The HLG photoelectric transfer function represents the conversion relationship from the linear signal value of the image pixel to the nonlinear signal value in the HLG domain.

[0149] It should be understood that the electronic device can also convert the RAW image into the preset HDR image by other manners, which are not specifically limited in the present application.

[0150] The preset HDR image can be understood as an image that the electronic device is expected to display, that is, the image to be reconstructed by the electronic device through the base image and the gain map is the preset HDR image. In other words, the electronic device can not be able to directly display the preset HDR image, so the electronic device reconstructs an image through the base image and the gain map, and the reconstructed image is an image that is close to the display effect of the preset HDR image, so that the image displayed by the electronic device has more brightness levels and a wider color gamut, and can more truly reflect the lighting conditions in nature, thereby improving the user experience.

[0151] The base image obtained based on the RAW image can be implemented in the following manner: the RAW image is subjected to gamma correction using an sRGB gamma curve (such as a 2.2 gamma), thereby obtaining the base image.

[0152] The gamma correction is used to convert linear light intensity into a non-linear signal that conforms to the response curve of the display device. For example, linear light intensity data in the RAW image can be converted into non-linear gamma coding, and then the image data is converted into a standard sRGB color space using an sRGB gamma curve, thereby obtaining the base image.

[0153] In an implementation manner, in addition to the gamma correction process, the RAW image can also be subjected to processes such as demosaicing, color correction, white balance adjustment, and tone mapping, thereby obtaining a base image with higher quality.

[0154] The demosaicing is a process of converting Bayer array data captured by a sensor into a full-color image. The white balance adjustment is used to correct color deviation under different light sources, so that the white color in the image appears real. The color correction is performed according to the color characteristic matrix of the camera, so that the color is more accurate. The tone mapping is used to compress the high dynamic range of the image to a range that can be displayed by an SDR display device.

[0155] It should be understood that the electronic device can also convert the RAW image into a base image in other manners, which are not limited in the present application.

[0156] Referring to FIG. 3, after the encoding module of the electronic device obtains the preset HDR image and the base image, the linear space brightness of the preset HDR image can be calculated based on the preset HDR image, and the linear space brightness of the base image can be calculated based on the base image. The encoding module can calculate the linear space brightness of the preset HDR image by linearizing and colour converting the preset HDR image. The encoding module can calculate the linear space brightness of the base image by linearizing the base image.

[0157] Then, the encoding module can calculate gain information, i.e., a gain map, based on the linear space luminance of the preset HDR image and the linear space luminance of the base image. In the embodiments of the present application, the gain information and the gain map can be understood as the same concept. It can be understood that the sizes of the preset HDR image, the base image and the gain information are all the same, and therefore the pixel points in the three images are one-to-one corresponding.

[0158] The encoding module calculating the gain information can be implemented by calculating the gain information according to the linear space luminance of the i-th pixel point in the preset HDR image and the linear space luminance of the i-th pixel point in the base image. The i-th pixel point in the preset HDR image and the i-th pixel point in the base image are corresponding pixel points. The corresponding pixel points can represent the same position in the same scene. For example, the position of the i-th pixel point in the preset HDR image in the preset HDR image is the same as the position of the i-th pixel point in the base image in the base image. The value range of i can be 1-n, and n is the number of pixel points in the preset HDR image. It can be understood that the number of pixel points in the base image and the gain information is also n.

[0159] The gain of the i-th pixel point recorded in the gain information can be calculated by the following formula:

[0160] wherein, Alternate represents the linear space luminance of the i-th pixel point in the preset HDR image, k Alternate represents the linear space offset of the preset HDR image, Baseline represents the linear space luminance of the i-th pixel point in the base image, k Baseline represents the linear space offset of the base image.

[0161] Since the values of k Alternate and k Baseline are usually close to 0, the above formula can also be simplified as follows:

[0162] In this way, when the i-th pixel point in the preset HDR image traverses all the pixel points in the preset HDR image, and the i-th pixel point in the base image traverses all the pixel points in the base image, the gain information can be obtained. That is, the gain information can include the gain corresponding to each pixel point in the base image.

[0163] In addition, the encoding module can also calculate the metadata of the gain information. The metadata is data describing the key information and characteristics required in the image (or video signal) processing process. It can be understood that the metadata is used to provide the information required for correctly decoding and displaying the image (or video signal).

[0164] In an implementation, the encoding module may, for example, perform corresponding data analysis according to the gain information, so as to obtain the metadata of the gain information. Alternatively, the encoding module may also perform corresponding data analysis according to the preset HDR image and the base image, so as to obtain the metadata of the gain information. The embodiments do not limit the way of calculating the metadata, and also do not limit the specific content included in the metadata, which can be set arbitrarily according to actual needs.

[0165] It can be understood with reference to FIG. 3 that, after the end of the encoding stage, the encoding module of the electronic device can generate the base image, the gain information, and the metadata of the gain information. In an implementation, the encoding module of the electronic device can save the base image, the gain information, and the metadata of the gain information as image data. The image data can also be referred to as an image file. The encoding module can transmit the image file to the decoding module.

[0166] Then in the decoding stage, the decoding module of the electronic device can perform decoding processing on the metadata of the gain information, so as to parse the gain information according to the related parameters contained in the metadata, and also obtain related data relied on by the reconstructed HDR image. Then the decoding module can perform fusion processing according to the base image and the gain information, so as to generate an enhanced image.

[0167] In an implementation, the decoding module can first generate an alternate image according to the base image and the gain information. The alternate image is a high dynamic image generated by merging the base image and the gain information, and the alternate image is a signal in a linear domain.

[0168] Therefore, the electronic device can further process the alternate image to obtain an enhanced image, where the enhanced image is an image obtained by performing nonlinear conversion on the alternate image (linear domain signal), and is a signal in a nonlinear domain. The enhanced image can be understood as a reconstructed HDR image, that is, an image restored from the preset HDR image.

[0169] It should be understood that the process of generating the alternate image by the electronic device according to the base image and the gain information is actually a process of restoring the preset HDR image. The reason why the preset HDR image that has been generated is restored again through the encoding and decoding stages is that the electronic device cannot normally display the preset HDR image on the screen, and also cannot make the display effect of the preset HDR image on the screen.

[0170] With reference to the processing procedure described above, the encoding module of the electronic device first generates the gain information according to the preset HDR image and the base image, and then the encoding module saves the base image and the gain information as image data. In the decoding stage, if the screen of the electronic device supports displaying the HDR image, the decoding module can generate the selectable image according to the base image and the gain information, and then generate the gain information on the basis of the selectable image and send the enhanced image for display, so that the electronic device can effectively display the HDR image, and the gain effect indicated by the gain information can be effectively reflected in the sent HDR image.

[0171] Alternatively, if the screen of the electronic device does not support displaying the HDR image, the decoding module can generate an enhanced image adapted to the dynamic range of the electronic device, so that better visual effects can be obtained in image display on various electronic devices. Alternatively, the decoding module can also choose to directly send the base image for display, so as to avoid the situation that the captured image cannot be displayed.

[0172] As can be understood from the above description, the gain information plays a very important role in the encoding and decoding process. The inventors have found that when the range of the gain value in the gain information is unreasonable, the enhanced image obtained by restoration will have quality loss compared with the preset HDR image.

[0173] The phenomenon can be understood with reference to FIG. 4 and FIG. 5. FIG. 4 is a schematic diagram of image restoration provided by an embodiment of the present application, and FIG. 5 is a comparative schematic diagram of image restoration provided by an embodiment of the present application.

[0174] As shown in FIG. 4, the image shown in (a) of FIG. 4 can be understood as a gain image, and the image shown in (b) of FIG. 4 can be understood as an enhanced image obtained after restoration. As described above, the base image is needed for processing when generating the enhanced image, and the base image is not shown in FIG. 4.

[0175] As can be understood from (b) of FIG. 4, the enhanced image after restoration has a banding phenomenon, wherein the banding phenomenon refers to a discontinuous color change or luminance change in the image, resulting in obvious color bands or light bands in the image, rather than smooth transition. As shown in the example of (b) of FIG. 3, there are circles of ripples around the upward area of the sun, which is the banding phenomenon described above, and can also be understood as a brightening band appearing in the enhanced image. The appearance of the brightening band is actually a manifestation of image quality loss.

[0176] The image quality loss can be understood on the basis of the comparison between the preset HDR image and the enhanced image obtained after restoration, with reference to FIG. 5.

[0177] The image shown in (a) of FIG. 5 can be understood as a preset HDR image, and the image shown in (b) of FIG. 5 can be understood as an enhanced image obtained after restoration. By comparison, it can be determined that there is actually no banding phenomenon in the preset HDR image, but the banding phenomenon appears in the enhanced image obtained after restoration, which indicates that the image quality loss occurs in the process of restoring the preset HDR image.

[0178] In order to more intuitively understand this image quality loss, the contrast images corresponding to the preset HDR image and the enhanced image are also shown in (c) of FIG. 5. The pixel points in the contrast image correspond one-to-one to the pixel points in the preset HDR image and the enhanced image. The pixel value of each pixel point in the contrast image is the contrast of the brightness value of the corresponding pixel point in the preset HDR image and the brightness value of the corresponding pixel point in the enhanced image.

[0179] The greater the contrast, the greater the difference in brightness value. Conversely, the smaller the contrast, the smaller the difference in brightness value. For example, the value range of the contrast can be 0-1. When the contrast corresponding to the pixel point is 0, the pixel point in the contrast image is black. When the contrast corresponding to the pixel point is 1, the pixel point in the contrast image is white. When the value of the contrast corresponding to the pixel point is a value between 0 and 1, the pixel point in the contrast image is gray.

[0180] It can be understood with reference to FIG. 5 that the colors corresponding to the regions of the brightening band in the contrast image are all close to white, which indicates that the regions of the brightening band are the loss regions of the enhanced image compared with the preset HDR image.

[0181] The above exemplary introduction in combination with FIG. 4 and FIG. 5 illustrates this case that the image quality loss leads to the appearance of the brightening band in the enhanced image. In fact, there are many forms of image quality loss, for example, the image quality loss can also lead to the problems such as color cast and glare of the enhanced image compared with the preset HDR image. The specific forms of the image quality loss are not limited in the embodiment, and the cases appearing when the contrast of the enhanced image compared with the preset HDR image is larger can all be understood as the cases caused by the image quality loss.

[0182] Further, it can be understood with reference to the introduction of FIG. 3 above that the current gain image is calculated according to the preset HDR image and the basic image. Therefore, various possible situations can occur in the gain value range of the gain image, which depends on the actual brightness situation of the preset HDR image and the basic image.

[0183] So as to avoid the image quality loss situation introduced above, the present application proposes the following technical concept: after the gain image is calculated, the gain value range of the gain image is adjusted, so as to limit the gain value of the gain image within a certain range, so that the contrast of the preset HDR image and the enhanced image can meet the corresponding threshold condition, so that the image quality loss situation can be effectively avoided.

[0184] The image processing method of the present application will be described in detail below in combination with FIGS. 6 to 12. The image processing method of the present application embodiment can be executed by an electronic device provided with a camera, or can be executed by a chip, chip system or processor supporting the electronic device to implement the image processing method, or can be executed by a logic module or software capable of implementing all or part of the functions of the electronic device, and the present application does not make specific limitations thereon. The image processing method of the present application embodiment will be described in detail below taking the electronic device as an example.

[0185] FIG. 6 is a flowchart of an image processing method 600 provided by the present application. The method 600 can be executed by an electronic device, and the hardware structure of the electronic device can be as shown in FIG. 1. The method 600 includes the following steps:

[0186] S601, obtaining a first image and a second image, the first image being an HDR image, and the second image being an SDR image.

[0187] The first image can also be referred to as a preset HDR image, a preset high dynamic image or an alternate image, etc. The second image can also be referred to as a basic image. The first image and the second image can be images obtained in response to an operation of instructing shooting.

[0188] It should be understood that the manner in which the electronic device obtains the first image and the second image is similar to the manner in which the encoding module obtains the preset HDR image and the basic image in FIG. 3, and reference can be made to the description above, which will not be repeated here.

[0189] S602, determining a first gain range according to the first image and the second image.

[0190] The first gain range can be a range interval calculated based on the first image and the second image. The first gain range can include a maximum value and a minimum value, and the first gain range can be represented as [minimum value, maximum value], such as [0, 10], etc.

[0191] It can be understood that the first gain range is not a gain range of the gain map calculated according to the first image and the second image (i.e., the minimum value and the maximum value of the gain values in the gain map), but a gain range determined by the electronic device based on the first image and the second image. It should be understood that the first gain range can be different from the gain range of the gain map calculated according to the first image and the second image.

[0192] In an implementation manner, the reconstruction of the HDR image can be performed based on the first gain range. In the embodiment, compared with the reconstruction of the HDR image based on the gain range of the gain map calculated according to the first image and the second image, the reconstruction of the HDR image based on the first gain range can be implemented with higher image quality, that is, the image quality loss of the HDR image reconstructed based on the first gain range is lower. In other words, the first image is dynamically range expanded by using the gain in the first gain range, and the reconstructed HDR image obtained can be closer to the first image.

[0193] S603, determine metadata of the first gain information according to the first gain range, the first gain information being calculated according to the first image and the second image.

[0194] It should be understood that the first gain information can also be referred to as a first gain map, and the first gain map can include the gain corresponding to each pixel point in the second image. Before S603, the method 600 can further include: calculating, by the electronic device, the first gain information based on the first image and the second image.

[0195] It should be understood that the manner in which the electronic device calculates the first gain information based on the first image and the second image is similar to the manner in which the encoding module calculates the gain map shown in FIG. 3, and reference can be made to the description above, which will not be repeated here.

[0196] It should also be understood that S601 to S603 can be performed by the encoding module in the electronic device, and the process of S601 to S603 can also be referred to as calculating the gain information, which is not limited in the present application.

[0197] In the embodiment, after the first gain range is determined, the metadata of the first gain information can be determined according to the first gain range. In an implementation manner, for example, the first gain range can be written into the metadata of the first gain information.

[0198] For example, the maximum value and the minimum value contained in the first gain range can be written into the metadata of the first gain information. Alternatively, the first gain range can also be written into the metadata of the first gain information in other forms, for example, in the form of the maximum value and the offset value (the maximum value and the offset value can derive the minimum value), or in the form of the minimum value and the offset value (the minimum value and the offset value can derive the maximum value).

[0199] The embodiment is not limited to the specific implementation of storing the first gain range in the metadata, as long as the first gain range can be effectively obtained by parsing the metadata.

[0200] S604, obtaining a selectable image according to the first gain information, the metadata and the second image.

[0201] In the embodiment, the first gain information is a gain image calculated according to the first image and the second image, and it can be understood that the gain range corresponding to the first gain information depends on the specific gain value in the first gain information. Meanwhile, the metadata records the first gain range, wherein the first gain range is a gain range calculated by the electronic device based on the first image and the second image, and is used to reduce the image quality loss in the process of restoring the HDR image.

[0202] Therefore, for example, part of the gain values in the first gain information can be adjusted according to the first gain range indicated in the metadata, so that the gain values in the first gain information are within the first gain range. Then, the first gain information after the gain value adjustment and the second image are fused to obtain the selectable image.

[0203] The selectable image is an image obtained by performing dynamic range expansion on the second image by using the first gain information after adjusting the gain values. The dynamic range expansion can also be referred to as dynamic range enhancement. Through the dynamic range expansion, the selectable image has more luminance levels and a wider color gamut.

[0204] The selectable image can also be understood as an image used to reconstruct the first image. The selectable image is a linear domain signal, and through nonlinear conversion and / or rendering and other operations on the selectable image, an image that can be displayed on the screen of the electronic device (i.e., the reconstructed HDR image in the foregoing) can be obtained. Since the selectable image is an image obtained by dynamic range expansion, it has a high dynamic range, so that the image displayed on the screen of the electronic device (obtained based on the selectable image) can present an effect close to the first image. Therefore, the selectable image can also be understood as an image used to reconstruct the first image.

[0205] It should be understood that S604 can be performed by a decoding module in the electronic device, and the process of S604 can also be referred to as applying gain information, which is not limited in the present application.

[0206] The image processing method of the present application, after obtaining the first gain information based on the obtained preset HDR image and the base image, additionally determines a first gain range for reducing image quality loss according to the first image and the second image, and writes the first gain range into the metadata of the first gain information, and then when performing HDR image restoration, the first gain range recorded in the metadata and the first gain information can be combined to perform dynamic range expansion on the second image, so that the determined reconstructed HDR image is closer to the first image, which helps to reduce image quality loss.

[0207] On the basis of the above introduction, the following will be combined with FIG. 7 to exemplarily introduce one implementation manner of the metadata of the determined first gain information.

[0208] It can be determined from the above introduction that the first gain range includes a maximum value and a minimum value. Exemplarily, assuming that the currently determined first gain range is [0, 8], it can be determined that the maximum value of the first gain range is 8 and the minimum value is 0.

[0209] Then, for example, the maximum value and the minimum value can be directly written into the metadata of the first gain information. As shown in FIG. 7, the minimum value (min) can be written as 0 and the maximum value (max) can be written as 8 in the metadata of the first gain information (which can also be referred to as the first gain map).

[0210] By directly writing the minimum value and the maximum value of the first gain range in the metadata, the metadata can be quickly and efficiently parsed to obtain the first gain range.

[0211] On the basis of the above introduction, the following will be combined with FIG. 7 to exemplarily introduce one implementation manner of the metadata of the determined first gain information.

[0212] Manner one, determined according to the linear space luminance of the first image and the linear space luminance of the second image.

[0213] On the basis of manner one, the first gain range can be determined according to the dynamic range headroom level, which is calculated according to the linear space luminance of the first image and the linear space luminance of the second image.

[0214] Manner two, determined according to the second gain range, which is the gain range of the first gain information.

[0215] Manner three, determined according to the first image, the second image and the first gain information.

[0216] On the basis of manner three, the first gain range can be determined according to the dynamic range headroom level and the second gain range.

[0217] Manner four, determined according to the first image, the second image and the bit width.

[0218] On the basis of the fourth mode, the first gain range can be determined according to the dynamic range headroom level and the bit width; determined according to the second gain range and the bit width; or determined according to the dynamic range headroom level, the second gain range, and the bit width.

[0219] The following will be described in detail in turn.

[0220] The first gain range can be determined according to the linear space luminance of the first image and the linear space luminance of the second image.

[0221] In an example, the electronic device can be preconfigured with a first correspondence relationship. The first correspondence relationship includes a correspondence relationship between a first linear space luminance and a first gain range.

[0222] The first linear space luminance can be an average of two linear space luminances, that is, an average of the linear space luminance of the first image and the linear space luminance of the second image. The average of the linear space luminance is a ratio of a sum of linear space luminances of each pixel point to a total number of pixel points.

[0223] Alternatively, the first linear space luminance is a range of two linear space luminances, that is, a first linear space luminance range and a second linear space luminance range. The linear space luminance range is a minimum value to a maximum value in the linear space luminance of each pixel point. The first linear space luminance range can cover the linear space luminance range of the first image, and the second linear space luminance range can cover the linear space luminance range of the second image. So that the electronic device can determine the first gain range according to the first correspondence relationship.

[0224] It can be understood that the first correspondence relationship can be preconfigured according to experience, which is not limited in the present application.

[0225] In another example, the electronic device can be preconfigured with a first model. The first model can be a neural network model. The electronic device can input the linear space luminance of the first image and the linear space luminance of the second image into the first model, and the first model can output the first gain range.

[0226] The first model can be a model learned by the first initial model through the following process. The first initial model inputs a preset image, which can include a base image 1 and a preset HDR image 1; based on the linear space luminance of the base image 1 and the linear space luminance of the preset HDR image 1, the first initial model can determine a gain range 1; the first initial model performs dynamic range expansion on the base image 1 by using a gain map with a gain range of the gain range 1, to obtain a reconstructed HDR image 1; the first initial model evaluates an image quality loss 1 of the reconstructed HDR image 1 relative to the preset HDR image 1; in the case where the image quality loss 1 does not satisfy a preset condition, the first initial model adjusts parameters used when calculating the gain range 1; a new gain range 1 is determined using the adjusted parameters, and the base image 1 is dynamically expanded using a gain map with a gain range of the new gain range 1, to obtain a new reconstructed HDR image 1; the first initial model evaluates an image quality loss 2 of the new reconstructed HDR image 1 relative to the preset HDR image 1, and in the case where the image quality loss 2 still does not satisfy the preset condition, the first initial model can continue to adjust the parameters used when calculating the new gain range 1, continue to determine an updated gain range 1, and repeat the above process. Until the first initial model evaluates the updated reconstructed HDR image 1 obtained based on the updated gain range 1 relative to the preset HDR image 1, the image quality loss satisfies the preset condition, and the first model is obtained.

[0227] Through such a learning process, the electronic device can reconstruct a reconstructed HDR image closer to the first image from the selectable image obtained by using the first gain range determined by the first model. That is, the image quality loss of the reconstructed HDR image obtained based on the selectable image is smaller.

[0228] It can be understood that the first initial model can also be trained based on base images and preset HDR images obtained by shooting in different scenes, such as different lighting environments. In this way, the first initial model can learn the parameters used when calculating the corresponding gain range in different scenes. In this way, as the scenes corresponding to the first image and the second image are different, the first model can output the first gain range based on the parameters of the scenes corresponding to the first image and the second image. In this way, the accuracy of the first model can be further improved.

[0229] On the basis of the first mode, the first gain range can be determined according to a dynamic range headroom level.

[0230] Optionally, the dynamic range headroom level is calculated according to the linear space luminance of the first image and the linear space luminance of the second image; and the first gain range is determined according to the dynamic range headroom level.

[0231] It can be understood that the dynamic range headroom level can reflect an advantage of the first image over the second image in linear space luminance, since the gain information is used to perform dynamic range (related to luminance) expansion on the second image to reconstruct an image with linear space luminance close to the first image by the second image and the gain information. Therefore, the electronic device determines the appropriate gain information in relation to the dynamic range headroom level.

[0232] In an example, the electronic device can be pre-provisioned with a second correspondence. The second correspondence includes a correspondence between a dynamic range headroom level and a first gain range; or, the second correspondence includes a correspondence between a dynamic range headroom level range and a first gain range, the dynamic range headroom level being in the dynamic range headroom level range. So that the electronic device can determine the first gain range according to the second correspondence.

[0233] It can be understood that the second correspondence can be pre-set according to experience, which is not specifically limited in the present application.

[0234] In another example, the electronic device can be pre-provisioned with a second model. The second model can be a neural network model. The electronic device can input the dynamic range headroom level into the second model, and the second model can output the first gain range.

[0235] The second model can be a model learned by a second initial model through the following process. The second initial model is input with a preset image, which can include a base image 1 and a preset HDR image 1; based on the linear space luminance of the base image 1 and the linear space luminance of the preset HDR image 1, the second initial model can calculate a dynamic range headroom level 1; based on the dynamic range headroom level 1, the second initial model can determine a gain range 1; the second initial model performs dynamic range expansion on the base image 1 by using a gain map with the gain range being the gain range 1, to obtain a reconstructed HDR image 1; the second initial model evaluates an image quality loss 1 of the reconstructed HDR image 1 relative to the preset HDR image 1; in the case that the image quality loss does not satisfy a preset condition, the second initial model adjusts parameters used when calculating the gain range 1; using the adjusted parameters and the dynamic range headroom level 1, a new gain range 1 is calculated, and dynamic range expansion is performed on the base image 1 by using a gain map with the gain range being the new gain range 1, to obtain a new reconstructed HDR image 1; the second initial model evaluates an image quality loss 2 of the new reconstructed HDR image 1 relative to the preset HDR image 1, and in the case that the image quality loss 2 still does not satisfy the preset condition, the second initial model can continue to adjust the parameters used when calculating the new gain range 1, continue to determine an updated gain range 1, and repeat the above process. Until the second initial model evaluates that the image quality loss of the updated reconstructed HDR image 1 obtained based on the updated gain range 1 relative to the preset HDR image 1 satisfies the preset condition, the second model is obtained.

[0236] Through the learning process, the electronic device can reconstruct a reconstructed HDR image closer to the first image by using the selectable image obtained by the first gain range determined by the second model. That is, the image quality loss of the reconstructed HDR image obtained based on the selectable image is smaller.

[0237] It can be understood that the second initial model can also be trained based on the baseline image and the preset HDR image obtained by shooting in different scenes, such as different lighting environments. In this way, the second initial model can learn the parameters used when calculating the gain range corresponding to different scenes. In this way, as the scenes corresponding to the input first image and the second image are different, the second model can output the first gain range based on the parameters of the scenes corresponding to the first image and the second image. In this way, the accuracy of the second model can be further improved.

[0238] On the basis of the above embodiment, the dynamic range headroom level can be calculated in the following manner.

[0239] In one possible manner, the dynamic range headroom level is a first ratio, and the first ratio is a ratio of a maximum linear space luminance of the first image to a maximum linear space luminance of the second image.

[0240] The maximum linear space luminance of the first image is a maximum value in the linear space luminance of the first image, and the maximum linear space luminance of the second image is a maximum value in the linear space luminance of the second image.

[0241] Exemplarily, the dynamic range headroom level H satisfies the following formula:

[0242] The Max(Alternate) is the maximum linear space luminance of the first image, and the Max(Baseline) is the maximum linear space luminance of the second image.

[0243] In this way, the dynamic range headroom level can reflect the advantage of the maximum linear space luminance of the first image relative to the maximum linear space luminance of the second image.

[0244] In another possible manner, the dynamic range headroom level is a second ratio, and the second ratio is a ratio of an average linear space luminance of the first image to an average linear space luminance of the second image.

[0245] In this way, the dynamic range headroom level can reflect the advantage of the average linear space luminance of the first image relative to the average linear space luminance of the second image.

[0246] It can be understood that the dynamic range headroom level can also be determined based on other parameters related to the linear space luminance of the first image (e.g., the minimum linear space luminance of the first image), or other parameters related to the linear space luminance of the second image (e.g., the minimum linear space luminance of the second image), which are not limited in the present application.

[0247] It should be noted that the dynamic range headroom level cannot be too large, otherwise it may cause the HDR image reconstructed by the selectable image to have quality loss. Therefore, in some scenarios, a preset dynamic range headroom level can be pre-set in the electronic device, and the preset dynamic range headroom level can also be referred to as a preset headroom. Details are as follows.

[0248] Optionally, on the basis of the above embodiment, in a case where the first ratio is greater than the first preset headroom, the dynamic range headroom level is the first preset headroom; or in a case where the first ratio is greater than the second preset headroom, the dynamic range headroom level is the second preset headroom.

[0249] The first preset headroom (or the second preset headroom) can be a dynamic range headroom level matched with the display of the electronic device. For example, generally, the maximum display brightness of the display of the electronic device is 2000 nits, and based on 2000 nits and the linear space luminance of the SDR image, the dynamic range headroom level can be calculated to be about 3.3, so the first preset headroom (or the second preset headroom) can be 3.3, etc.

[0250] Alternatively, the first preset headroom (or the second preset headroom) can also be a dynamic range headroom level matched with the color space used when the electronic device renders the enhanced image and the bit width of the display. The color space used when rendering the enhanced image can include RGBA1010102 / 8, scRGB-nl, and RGBA8888, etc.

[0251] The color space of RGBA1010102 / 8 uses 10 bits to represent the red, green, and blue channels, and 2 bits to represent the alpha channel, for a total of 32 bits. scRGB (standard RGB) is a high dynamic range color space, and "nl" represents non-linear. The color space of RGBA8888 uses 8 bits to represent the red, green, blue, and alpha channels, 8 bits for each channel, for a total of 32 bits.

[0252] In an implementation manner, when the color space is RGBA1010102 / 8 and the bit width of the display is 10-bit display, the first preset headroom (or the second preset headroom) can be set to 3.3.

[0253] When the color space is scRGB-nl and the bit width of the display is 10-bit display, the first preset headroom (or the second preset headroom) can be set as log2(7.59+0.6)=3.0.

[0254] When the color space is RGBA8888 and the bit width of the display is 8-bit or 10-bit display, the first preset headroom (or the second preset headroom) can be set as 2.3 to achieve better color accuracy and less banding problem.

[0255] When the bit width of the display is 8-bit, the first preset headroom (or the second preset headroom) can be set as 2.3, for example.

[0256] In actual implementation, for different color spaces and different display bit widths, the first preset headroom can be set according to actual needs, and the embodiment does not limit the specific value of the first preset aperture, which can be selected according to actual needs.

[0257] It should be understood that the first preset headroom (or the second preset headroom) can be a value preset according to experience, which is not limited in the present application.

[0258] The second mode S602 can be implemented by the following mode: determining the first gain information according to the first image and the second image; determining the first gain range based on the second gain range of the first gain information.

[0259] The second gain range is the minimum value to the maximum value in the first gain information. The first gain range can be within the second gain range. For example, the second gain range is -0.2 to 18; the first gain range is 0 to 8, etc.

[0260] In an example, the electronic device can be pre-provisioned with a third correspondence. The third correspondence includes a correspondence between the second gain range and the first gain range.

[0261] It can be understood that the third correspondence can be preset according to experience, which is not limited in the present application.

[0262] In another example, the first gain range can be calculated according to the second gain range, for example, the minimum value of the first gain range can be the sum of the minimum value of the second gain range and 2, the maximum value of the first gain range can be the difference between the maximum value of the second gain range and 3, etc.

[0263] In yet another example, the electronic device can be pre-provisioned with a third model. The third model can be a neural network model. The electronic device can input the second gain range into the third model, and the third model can output the first gain range.

[0264] The third model can be a model learned by the third initial model through the following process.

[0265] It should be understood that the training process of the third initial model is similar to that of the first initial model, except that the third initial model calculates the gain information 1 based on the basic image 1 and the preset HDR image 1, and calculates the gain range 1 based on the gain range of the gain information 1. In the case where the third initial model judges that the image quality loss 1 does not satisfy the preset condition, the third initial model adjusts the parameters used when calculating the gain range 1 (calculated based on the gain range of the gain information 1). For details, please refer to the foregoing description.

[0266] In the third mode, S602 can be implemented in the following manner: determining the first gain range according to the first image, the second image, and the first gain information.

[0267] Exemplarily, the fourth model can be pre-stored in the electronic device. The fourth model can be a neural network model. The electronic device can input the first image, the second image, and the first gain information into the fourth model, and the fourth model can output the first gain range.

[0268] The fourth model can be a model learned by the fourth initial model through the following process.

[0269] It should be understood that the training process of the fourth initial model is similar to that of the first initial model, except that the fourth initial model calculates the gain information 1 based on the basic image 1 and the preset HDR image 1, and calculates the gain range 1 based on the basic image 1, the preset HDR image 1, and the gain information 1. In the case where the fourth initial model judges that the image quality loss 1 does not satisfy the preset condition, the fourth initial model adjusts the parameters used when calculating the gain range 1 (calculated based on the basic image 1, the preset HDR image 1, and the gain information 1). For details, please refer to the foregoing description.

[0270] Based on the third mode, S602 can be implemented in the following manner: determining the first gain range according to the dynamic range headroom level and the second gain range, the second gain range being the gain range of the first gain information, and the dynamic range headroom level being calculated according to the first image and the second image.

[0271] It should be understood that the dynamic range headroom level can refer to the foregoing description, and will not be described here again.

[0272] In an example, the electronic device can prestore a fifth correspondence relationship. The fifth correspondence relationship can include a correspondence relationship between the dynamic range headroom level, the second gain range and the first gain range. Alternatively, the fifth correspondence relationship can include a correspondence relationship between the dynamic range headroom level range, the second gain range and the first gain range. Alternatively, the fifth correspondence relationship can include a correspondence relationship between the dynamic range headroom level range, the first range and the first gain range. The dynamic range headroom level is in the dynamic range headroom level range. The minimum value of the second gain range is in the first range, and the maximum value of the second gain range is in the first range.

[0273] Exemplarily, as shown in Table 1, the fifth correspondence relationship includes a correspondence relationship between the dynamic range headroom level range, the first range and the first gain range. The dynamic range headroom level range is [2, 3]; the first range can include a first sub-range and a second sub-range, the first sub-range is [-2, 2], and the second sub-range is [15, 20]; and the first gain range is [0, 8]. In this way, if the dynamic range headroom level calculated by the electronic device based on the first image and the second image is 2.4, i.e., the dynamic range headroom level is in [2, 3]; the minimum value of the second gain range calculated by the electronic device is -1.8, which is in the first sub-range [-2, 2]; and the maximum value of the second gain range calculated by the electronic device is 19, which is in the first sub-range [15, 20], the electronic device can determine that the first gain range corresponding to the dynamic range headroom level and the second gain range is [0, 8].

[0274] Table 1

[0275] It can be understood that the fifth correspondence relationship can be preset according to experience, which is not specifically limited in the present application.

[0276] In another example, the electronic device can prestore a fifth model. The fifth model can be a neural network model. The electronic device can input the dynamic range headroom level and the second gain range into the fifth model, and the fifth model can output the first gain range.

[0277] The fifth model can be a model learned by the fifth initial model through the following process.

[0278] It should be understood that the training process of the fifth initial model is similar to that of the first initial model, except that the fifth initial model is based on the base image 1 and the preset HDR image 1 to calculate the gain information 1 and the dynamic range headroom level 1, and the fifth initial model is based on the gain range of the gain information 1 and the dynamic range headroom level 1 to calculate the gain range 1. In the case where the fifth initial model judges that the image quality loss 1 does not satisfy the preset condition, the fifth initial model adjusts the parameters used when calculating the gain range 1 (calculated based on the gain range of the gain information 1 and the dynamic range headroom level 1). For details, please refer to the foregoing description.

[0279] In the fourth mode, S602 can be implemented by determining the first gain range according to the first image, the second image, and the bit width.

[0280] It can be understood that the bit width can affect the color depth, detail performance, and overall visual quality of the image, and therefore different bit widths can correspond to different gain ranges.

[0281] Exemplarily, the electronic device can be preconfigured with a sixth model. The sixth model can be a neural network model. The electronic device can input the first image, the second image, and the first gain information into the sixth model, and the sixth model can output the first gain range.

[0282] The sixth model can be a model learned by a sixth initial model through the following process.

[0283] It should be understood that the training process of the sixth initial model is similar to that of the first initial model, except that the sixth initial model is based on the base image 1, the preset HDR image 1, and the bit width 1 to calculate the gain range 1. In the case where the sixth initial model judges that the image quality loss 1 does not satisfy the preset condition, the sixth initial model adjusts the parameters used when calculating the gain range 1 (calculated based on the base image 1, the preset HDR image 1, and the bit width 1). For details, please refer to the foregoing description.

[0284] On the basis of the fourth mode, the first gain range can be determined according to the dynamic range headroom level and the bit width.

[0285] In an example, the electronic device can be preconfigured with a seventh correspondence relationship, which can include a correspondence relationship between the dynamic range headroom level, the bit width, and the first gain range; or the seventh correspondence relationship can include a correspondence relationship between a dynamic range headroom level range, a bit width, and a first gain range. The dynamic range headroom level is in the dynamic range headroom level range.

[0286] Exemplarily, as shown in Table 2, the seventh correspondence relationship includes a correspondence relationship between a dynamic range headroom level range, a bit width and a first gain range. The dynamic range headroom level range is [2, 3]; the bit width can be 8 bits; and the first gain range is [0, 8]. In this way, if the dynamic range headroom level calculated by the electronic device based on the first image and the second image is 2.4, that is, the dynamic range headroom level belongs to [2, 3], and the bit width of the electronic device is 8 bits, the electronic device can determine that the first gain range corresponding to the dynamic range headroom level and the second gain range is [0, 8].

[0287] Table 2

[0288] It can be understood that the seventh correspondence relationship can be preset according to experience, which is not limited in the present application.

[0289] In another example, the seventh model can be pre-stored in the electronic device. The seventh model can be a neural network model. The electronic device can input the dynamic range headroom level and the bit width into the seventh model, and the seventh model can output the first gain range.

[0290] The seventh model can be a model learned by the seventh initial model through the following process.

[0291] It should be understood that the training process of the seventh initial model is similar to that of the first initial model, except that the seventh initial model calculates the dynamic range headroom level 1 based on the base image 1 and the preset HDR image 1, and calculates the gain range 1 based on the bit width 1 and the dynamic range headroom level 1. In the case where the seventh initial model determines that the image quality loss 1 does not satisfy the preset condition, the seventh initial model adjusts the parameters used when calculating the gain range 1 (calculated based on the bit width 1 and the dynamic range headroom level 1). For details, please refer to the foregoing description, which will not be repeated here.

[0292] On the basis of the fourth mode, the first gain range can also be determined according to the second gain range and the bit width.

[0293] In an example, the eighth correspondence relationship can be pre-stored in the electronic device, and the eighth correspondence relationship can include a correspondence relationship between the bit width, the second gain range and the first gain range; or the eighth correspondence relationship can include a correspondence relationship between the bit width, the first range and the first gain range. The minimum value of the second gain range belongs to the first range, and the maximum value of the second gain range belongs to the first range.

[0294] For example, as shown in Table 3, the eighth correspondence includes the correspondence between bit width, first range, and first gain range. The bit width is 8 bits; the first range may include a first sub-range and a second sub-range, where the first sub-range is [-2, 2] and the second sub-range is [15, 20]; the first gain range is [0, 8]. Thus, if the bit width of the electronic device is 8 bits; the minimum value in the second gain range calculated by the electronic device is -1.8, which belongs to the first sub-range [-2, 2], and the maximum value in the second gain range calculated by the electronic device is 19, which belongs to the first sub-range [15, 20], then the electronic device can determine that the first gain range corresponding to the dynamic range clearance level and the second gain range is [0, 8].

[0295] Table 3

[0296] It is understandable that the eighth correspondence can be preset based on experience, and this application does not make any specific limitations on it.

[0297] In another example, an eighth model may be pre-configured in the electronic device. The eighth model may be a neural network model. The electronic device can input the bit width and the second gain range into the eighth model, and the eighth model can output the first gain range.

[0298] The eighth model can be a model learned from the eighth initial model through the following process.

[0299] It should be understood that the training process of the eighth initial model is similar to that of the first initial model. The difference lies in that the fourth initial model calculates gain information 1 based on the base image 1 and the preset HDR image 1; the fourth initial model calculates the gain range 1 based on the gain range and bit width 1 of the gain information 1. If the eighth initial model subsequently determines that the image quality loss 1 does not meet the preset conditions, the eighth initial model adjusts the parameters used when calculating the gain range 1 (based on the gain range and bit width 1 of the gain information 1). Refer to the description above; it will not be repeated here.

[0300] Based on method four, the first gain range can also be determined according to the dynamic range clearance level, the second gain range, and the bit width.

[0301] In one example, the electronic device may have a pre-defined ninth correspondence, which may include the correspondence between dynamic range clearance level, bit width, and the second gain range and the first gain range; or, the ninth correspondence may include the correspondence between dynamic range clearance level range, bit width, and the first range and the first gain range. Wherein, the minimum value of the second gain range belongs to the first range, and the maximum value of the second gain range belongs to the first range. The dynamic range clearance level belongs to the dynamic range clearance level range.

[0302] Exemplarily, as shown in Table 4, the ninth correspondence relationship includes a correspondence relationship between a dynamic range headroom level range, a bit width, a first range and a first gain range. The dynamic range headroom level range is [2, 3], and the bit width is 8 bits; the first range can include a first sub-range and a second sub-range, the first sub-range is [-2, 2], and the second sub-range is [15, 20]; and the first gain range is [0, 8]. In this way, if the dynamic range headroom level calculated by the electronic device based on the first image and the second image is 2.4, that is, the dynamic range headroom level belongs to [2, 3], the bit width is 8 bits, the minimum value in the second gain range calculated by the electronic device is -1.8, which belongs to the first sub-range [-2, 2], and the maximum value in the second gain range calculated by the electronic device is 19, which belongs to the first sub-range [15, 20], the electronic device can determine that the first gain range corresponding to the dynamic range headroom level and the second gain range is [0, 8].

[0303] Table 4

[0304] It can be understood that the ninth correspondence relationship can be preset according to experience, which is not limited in the present application.

[0305] In another example, the ninth model can be pre-stored in the electronic device. The ninth model can be a neural network model. The electronic device can input the bit width and the second gain range into the ninth model, and the ninth model can output the first gain range.

[0306] The ninth model can be a model learned by the ninth initial model through the following process.

[0307] It should be understood that the training process of the ninth initial model is similar to that of the first initial model, except that the fourth initial model calculates the gain information 1 and the dynamic range headroom level 1 based on the basic image 1 and the preset HDR image 1; and the fourth initial model calculates the gain range 1 based on the gain range of the gain information 1, the dynamic range headroom level 1 and the bit width 1. In the case where the ninth initial model determines that the image quality loss 1 does not satisfy the preset condition, the ninth initial model adjusts the parameters used when calculating the gain range 1 (calculated based on the gain range of the gain information 1, the dynamic range headroom level 1 and the bit width 1). For details, please refer to the description above.

[0308] It can be understood that in the embodiments of the present application, the first initial model to the ninth initial model can refer to the way of calculating the gain information shown in FIG. 3; and the way of calculating the dynamic range headroom level based on the basic image 1 and the preset HDR image 1 can refer to the way of calculating the dynamic range headroom level described above, which will not be repeated here.

[0309] The above describes various possible implementations of determining the first gain range. Since subsequent reconstruction of the HDR image needs to be based on the first gain range, in an implementation of the present application, after the first gain range is determined, the electronic device can further verify whether the first gain range can meet the image quality loss requirement of the reconstruction of the HDR image through a certain process, where the image quality loss requirement can be understood as lossless reconstruction or reconstruction with lower image quality loss.

[0310] Meanwhile, when the first gain range fails to meet the image quality loss requirement of the reconstruction of the HDR image, further adjustment can be made to the first gain range so that the finally determined first gain range can meet the image quality loss requirement of the reconstruction of the HDR image.

[0311] To distinguish the gain range before and after the adjustment, the gain range before the adjustment (which can also be understood as the initially determined first gain range) is referred to as the third gain range in the present embodiment, and the finally determined gain range that meets the image quality loss requirement is referred to as the first gain range. It should be understood that the third gain range is determined according to the first image and the second image, and the specific determination manner can refer to any one of the above-described manners of determining the first gain range, which will not be described herein again.

[0312] The verification process will be described below in combination with FIG. 8, which is a flowchart of determining the image contrast provided by an embodiment of the present application.

[0313] As shown in FIG. 8, since the effect of the HDR image reconstruction based on the third gain range is to be verified at present, in the verification stage, the gain range corresponding to the first gain information is the third gain range determined above, that is, the gain values of each pixel point in the first gain information all belong to the third gain range, so that the image reconstruction effect of the third gain range can be effectively verified.

[0314] Exemplarily, the third gain range includes a minimum value and a maximum value, for example, the gain values less than the minimum value in the first gain information can be set to the minimum value, and the gain values greater than the maximum value in the first gain information can be set to the maximum value, so as to obtain the first gain information corresponding to the third gain range.

[0315] Subsequently, for example, the first enhanced image can be determined according to the second image and the third gain information. The implementation of determining the first enhanced image according to the second image and the third gain information can refer to the implementation of determining the enhanced image according to the gain map and the basic image described in the above-described embodiment of FIG. 3, which will not be described herein again.

[0316] In the embodiment, the first enhanced image can be understood as an HDR image reconstructed based on the third gain range, and the first image is the original HDR image, i.e., the reconstruction target. Therefore, the contrast of each pixel point in the first enhanced image and the first image can be further determined according to the linear space luminance L' of each pixel point in the first enhanced image and the linear space luminance L of each pixel point in the first image. The contrast can reflect the luminance contrast of the pixel point in the first enhanced image compared with the pixel point in the first image. Accordingly, the contrast can effectively reflect the image quality loss of the reconstructed first enhanced image compared with the first image as the target.

[0317] In actual implementation, there are various possible implementation manners for determining the contrast. The following describes several possible implementation manners for determining the contrast. It can be understood that the linear space luminance and the contrast mentioned in the following various determination manners are data of a pixel point, which can be understood as the linear space luminance of any pixel point and the contrast of any pixel point.

[0318] In one implementation manner, the contrast can be a first difference, which is the difference between the linear space luminance of the first image and the linear space luminance of the first enhanced image. For example, it can be expressed by a formula as ΔL=L'-L, where ΔL is the first difference.

[0319] Alternatively, the contrast can also be the ratio of the first difference and the linear space luminance of the first image. For example, it can be expressed by a formula as

[0320] Alternatively, the contrast can also be the ratio of the first difference and the linear space luminance of the first enhanced image. For example, it can be expressed by a formula as

[0321] Alternatively, the contrast can also be the ratio of the first difference and the first sum, which is the sum of the linear space luminance of the first image and the linear space luminance of the first enhanced image. For example, it can be expressed by a formula as

[0322] In actual implementation, the specific determination manner of the contrast can be selected and extended according to actual needs, as long as the contrast is determined according to the linear space luminance of the first image and the linear space luminance of the first enhanced image, and the contrast can reflect the luminance contrast of the pixel point in the first enhanced image compared with the pixel point in the first image.

[0323] After determining the contrast, the electronic device also needs to measure whether the current image quality loss meets the requirement according to the contrast of each pixel point, and the requirement for the image quality loss is usually to require lossless reconstruction or to realize image reconstruction with small image quality loss.

[0324] For example, the contrast of the pixel point and the corresponding contrast sensitivity threshold can be compared. When the contrast of the pixel point meets the corresponding contrast sensitivity threshold, it can be determined that the luminance loss corresponding to the pixel point is small, and the human eye generally cannot observe that there is image quality loss at this position. When the contrast of the pixel point does not meet the corresponding contrast sensitivity threshold, it can be determined that the luminance loss corresponding to the pixel point is large, and the human eye can observe that there is image quality loss at this position.

[0325] Then in an implementation mode, for example, the proportion of the first pixel points meeting the contrast sensitivity threshold in the second pixel points can be counted, that is, the ratio of the number of the first pixel points to the number of the second pixel points can be determined. The second pixel point can be a pixel point in the first gain information whose gain value is in the current gain range (i.e. the third gain range) of the first gain information, that is, the second pixel point can be understood as all the pixel points in the first gain information, and then the proportion currently counted is the proportion of the first pixel points in the entire pixel points in the first gain information.

[0326] Then the ratio determined above and the first threshold can be compared to measure the reconstruction loss of the first enhanced image. For example, in the case where the ratio is less than or equal to the first threshold, it can be determined that the proportion of the first pixel points meeting the contrast sensitivity threshold does not reach the preset requirement, and therefore it can be determined that the reconstruction loss of the first enhanced image is relatively large and does not meet the expected image quality loss requirement. Or, in the case where the ratio is greater than the first threshold, it can be determined that the proportion of the first pixel points meeting the contrast sensitivity threshold has reached the preset requirement, and therefore it can be determined that the reconstruction loss of the first enhanced image is small and can be considered to have reached the expected image quality loss requirement.

[0327] The first threshold can be set to 80%, 95% and the like, which can be set according to the loss requirement of image reconstruction in actual implementation, and the specific value of the first threshold is not limited in the embodiment.

[0328] Or in another implementation mode, the number of the first pixel points meeting the contrast sensitivity threshold can also be counted, and the number of the first pixel points and the second threshold can be compared to measure the reconstruction loss of the first enhanced image.

[0329] For example, in a case where the number of the first pixel points is less than or equal to the second threshold value, it can be determined that the number of the first pixel points satisfying the contrast sensitivity threshold value does not reach the preset requirement, and thus it can be determined that the reconstruction loss of the first enhanced image is relatively large and does not meet the expected image quality loss requirement. Alternatively, in a case where the number of the first pixel points is greater than the second threshold value, it can be determined that the number of the first pixel points satisfying the contrast sensitivity threshold value has reached the preset requirement, and thus it can be determined that the reconstruction loss of the first enhanced image is relatively small and is considered to meet the expected image quality loss requirement.

[0330] The second threshold value can be set according to the loss requirement of image reconstruction in actual implementation, and the embodiment does not limit the specific value of the second threshold value.

[0331] When the reconstruction loss of the first enhanced image is relatively small, it can be determined that the current reconstruction of the HDR image based on the third gain range can meet the image quality loss requirement of the reconstructed HDR image, and thus it can be determined that the current third gain range is the final first gain range.

[0332] However, when the reconstruction loss of the first enhanced image is relatively large, it indicates that the current reconstruction of the HDR image based on the third gain range cannot meet the image quality loss requirement of the reconstructed HDR image, and thus it is still necessary to continue to adjust the third gain range until the reconstruction of the HDR image based on the third gain range can meet the image quality loss requirement of the reconstructed image.

[0333] That is, in a case where the ratio of the number of the first pixel points to the number of the second pixel points is less than or equal to the first threshold value, or the number of the first pixel points is less than or equal to the second threshold value, the third gain range is adjusted.

[0334] For example, the above-described process can be understood with reference to FIG. 9, which is a schematic diagram for determining the first pixel points provided by the embodiment of the present application.

[0335] As shown in FIG. 9, it is assumed that the first image and the first enhanced image each include 9×8 pixel points, which are one-to-one corresponding, and then the respective contrasts of the 9×8 pixel points can be determined, and then the contrast of the pixel points is compared with the respective contrast sensitivity threshold values, so as to determine the pixel points (i.e., the first pixel points, which are filled with white color in the figure) satisfying the contrast sensitivity threshold value and the pixel points (which are filled with gray color in the figure) not satisfying the contrast sensitivity threshold value.

[0336] So, referring to the example of FIG. 9, it can be determined that the number of the first pixel points satisfying the contrast sensitivity threshold is 67, and the number of the second pixel points whose gain values are in the current gain range (i.e., the third gain range) of the first gain information is 72, which is actually the total number of the pixel points.

[0337] Then in an implementation, the ratio of the number 67 of the first pixel points and the number 72 of the second pixel points can be determined as 93%, and then 93% is compared with a first threshold. Assuming that the first threshold is set as 90%, the ratio is greater than the first threshold, so it can be determined that the proportion of the first pixel points has reached the preset requirement, and thus it can be determined that the third gain range is the final used first gain range. Or assuming that the first threshold is set as 95%, the ratio is less than the first threshold, so it can be determined that the proportion of the first pixel points has not reached the preset requirement, and the third gain range needs to be further adjusted.

[0338] Or in another implementation, the number of the first pixel points can be compared with a second threshold. Assuming that the second threshold is set as 65, the number of the first pixel points is greater than the second threshold, so it can be determined that the number of the first pixel points has reached the preset requirement, and thus it can be determined that the third gain range is the final used first gain range. Or assuming that the second threshold is set as 70, the number of the first pixel points is less than 70, so it can be determined that the number of the first pixel points has not reached the preset requirement, and the third gain range needs to be further adjusted.

[0339] For example, the implementation of adjusting the third gain range can be, for example, reducing the third gain range to obtain an adjusted third gain range.

[0340] In an implementation, reducing the third gain range can be reducing the maximum value in the third gain range by a first step to obtain the first gain range, and / or increasing the minimum value in the third gain range by a second step to obtain the first gain range. The first step and the second step can both be preset values, and the specific values can be selected according to actual needs, which are not limited in the embodiment.

[0341] For example, assuming that the third gain range is [0, 10], and assuming that the first step is 2 and the second step is 1, after reducing the third gain range once, the adjusted gain range can be [1, 8].

[0342] After the adjustment for the third gain range is performed once, the verification steps introduced above can be repeatedly performed. If it is determined that the adjusted third gain range meets the loss requirement of the reconstructed image, the adjusted third gain range can be determined as the final used first gain range. If the adjusted third gain range still does not meet the loss requirement of the reconstructed image, the adjusted third gain range can continue to be reduced until the reduced third gain range meets the loss requirement of the reconstructed image, so as to obtain the final used first gain range.

[0343] In this implementation manner, whether the third gain range meets the loss requirement of the reconstructed image is verified. If it is determined that the third gain range meets the loss requirement, the third gain range is determined as the final used first gain range. If the third gain range does not meet the loss requirement, the third gain range can be further adjusted until the adjusted third gain range meets the loss requirement of the reconstructed image, so as to obtain the final used first gain range. Therefore, the HDR image reconstruction can be implemented with low image quality loss based on the first gain range, so that the image quality of the final displayed image is effectively improved.

[0344] Based on the above introduction, it can be determined that after the contrast of each pixel point is determined, the contrast of the pixel point and the corresponding contrast sensitivity threshold value need to be compared. The implementation of the contrast sensitivity threshold value will be further described below.

[0345] In an implementation manner, for example, a fixed contrast sensitivity threshold value can be uniformly set for each pixel point. Then, the contrast of each pixel point is compared with the fixed contrast sensitivity threshold value, so as to determine whether the pixel point meets the corresponding contrast sensitivity threshold value. For example, the contrast sensitivity threshold value can be a specific value. When the contrast of the pixel point is less than or equal to the contrast sensitivity threshold value, it can be considered that the pixel point meets the contrast sensitivity threshold value. Alternatively, the contrast sensitivity threshold value can be a range. When the contrast of the pixel point belongs to the range of the contrast sensitivity threshold value, it can be considered that the pixel point meets the contrast sensitivity threshold value. This depends on the specific setting manner of the contrast sensitivity threshold value, which is not limited in the embodiment.

[0346] Alternatively, the contrast sensitivity threshold value corresponding to each pixel point in the contrast sensitivity function can also be determined according to the linear space luminance corresponding to each pixel point in the first image. In this way, the contrast sensitivity threshold value corresponding to each pixel point can be adaptively determined according to the actual luminance of the pixel point, so as to improve the adaptability of the contrast sensitivity function to the actual situation of the pixel point.

[0347] For example, the contrast sensitivity function can be understood with reference to FIG. 10. FIG. 10 is a curve diagram of the contrast sensitivity function provided by the embodiment of the application.

[0348] In FIG. 10, two possible contrast sensitivity functions are illustrated. Specifically, the contrast sensitivity function can be represented by the curve shown in FIG. 10. In the illustration of FIG. 10, the horizontal axis represents luminance, and the vertical axis represents minimum detectable contrast. In the present scheme, the value on the horizontal axis corresponding to the contrast sensitivity function can be understood as the linear space luminance of a pixel in the first image, and the value on the vertical axis corresponding to the contrast sensitivity function can be understood as the contrast sensitivity threshold.

[0349] Then, for any pixel, according to the linear space luminance of the pixel in the first image, the value on the horizontal axis can be determined, and then the value on the vertical axis corresponding to the curve is determined according to the value on the horizontal axis, so as to obtain the contrast sensitivity threshold corresponding to the pixel. In other words, the part above the curve can be considered as not satisfying the contrast sensitivity function, and the part below the curve can be considered as satisfying the contrast sensitivity function.

[0350] In an implementation manner, the curve above in FIG. 10 can be understood as the function curve of the schreiber fusion model, that is, the contrast sensitivity function in the present embodiment can be the function of the schreiber fusion model. And the curve below in FIG. 10 can be understood as the function curve of the Barton model, that is, the contrast sensitivity function in the present embodiment can be the function of the Barton model.

[0351] Alternatively, the curve illustrated in FIG. 10 can also be understood as the curve obtained by fusing the function of the schreiber fusion model and the function of the Barton model. The present embodiment does not limit the specific implementation manner of the contrast sensitivity function, which can be set arbitrarily according to actual requirements, as long as the contrast sensitivity function can indicate the relationship between the linear space luminance of a pixel in the first image and the contrast sensitivity threshold corresponding to the pixel.

[0352] In an implementation manner, the contrast sensitivity function (CSF) can be as shown in the following formula:

[0353] M opt (μ), E, σ and d each have their own mathematical expression, for example, can be as follows:

[0354] 1) M opt (μ) is the optical transfer function MTK of the human eye.

[0355] 2) E represents the retinal illuminance in Td (Trolands) units.

[0356] 3)

[0357] 4)

[0358] In the above-described formulas, the meanings of the respective parameters are as follows:

[0359] m t denotes a modulation depth threshold value (also referred to as a modulation depth threshold). μ denotes a spatial frequency, i.e., the number of cycles per degree of visual angle. σ denotes a standard deviation of a line spread function resulting from convolution of different elements in the convolution process. k denotes a signal-to-noise ratio (SNR). T denotes an integration time (or integration time) of the eye, in seconds. X0 denotes an angular size of the object in the x direction, which can also be referred to as an angular dimension of the object. X max denotes a maximum angular dimension in the x direction of the integration region, in degrees. N max denotes a maximum number of cycles that the eye can integrate. η denotes a quantum efficiency of the eye. p denotes a photon conversion factor p that varies with the light source. μ0 denotes a highest spatial frequency that maintains lateral inhibition. φ0 denotes a spectral density of neural noise.

[0360] The above-described respective parameters can be, for example, Arraylike objects.

[0361] On the basis of the above-described content, the embodiment of S603 will be described in detail below.

[0362] As an optional embodiment, the first gain range includes a maximum value and a minimum value, and the above-described content can be that the first gain range can be recorded in the metadata of the first gain information, for example, the maximum value and the minimum value of the first gain range can be directly written in the metadata.

[0363] It needs to be understood that, in the encoding stage, the respective gain values included in the first gain information are still the original gain values of the first gain information. That is, the gain values in the first gain information remain as they are according to the first image and the second image, and the gain values in the first gain information are not adjusted in the encoding stage in this embodiment, but the first gain range is recorded in the metadata of the first gain information.

[0364] But the first gain range is effective, or the gain value of the first gain information needs to be adjusted, in the present application, the gain value of the first gain information can be adjusted according to the first gain range recorded in the metadata in the decoding stage. In this way, the first gain information can remain the original gain information unchanged from the encoding stage to the decoding stage, and the adjustment is made in the decoding stage to improve the information integrity that the first gain information can provide on the basis of the effective role of the first gain range.

[0365] In an implementation manner, when the first gain information is adjusted according to the first gain range recorded in the metadata, the following manner can be used: the gain less than the minimum value in the first gain information is set to the minimum value, and the gain greater than the maximum value in the first gain information is set to the maximum value. After the above adjustment is performed, part of the gain values in the first gain information actually changes, and the gain information after the gain value adjustment is called second gain information in the embodiment, so it can be understood that the second gain information can be obtained after the first gain information is adjusted according to the first gain range recorded in the metadata.

[0366] For example, it can be understood in combination with FIG. 11, (a) in FIG. 11 shows the first gain information and the metadata.

[0367] Suppose that the currently determined first gain range is [0, 10], it can be determined with reference to (a) in FIG. 7 that the maximum value 10 of the first gain range and the minimum value 0 of the first gain range are recorded in the metadata of the first gain information. It can be understood with reference to (a) in FIG. 7 that the original gain range of the first gain information is [-5, 15], and the gain value in the first gain information remains unchanged in the encoding stage.

[0368] Then in the decoding stage, the electronic device can adjust the gain value in the first gain information according to the maximum value 10 and the minimum value 0 recorded in the metadata, with reference to FIG. 10, the electronic device can set the gain less than 0 in the first gain information to 0, and set the gain greater than 10 to 10, thereby obtaining the second gain information. The second gain information is shown in (b) in FIG. 7.

[0369] After the second gain information is obtained in the decoding stage, the gain range corresponding to the second gain information is the first gain range, so the selectable image can be obtained according to the second gain information and the second image. Therefore, the reconstruction of the HDR image can be effectively realized based on the first gain range, so as to effectively reduce the image quality loss in the process of reconstructing the HDR image.

[0370] It is illustrated above that the electronic device can calculate the first gain information according to the first image and the second image. In some possible implementation manners, after the first gain information is determined, the method 600 further includes: the electronic device performs normalization processing on the first gain information to obtain normalized first gain information. Correspondingly, S604 can be implemented in the following manner: according to the normalized first gain information, the metadata, and the second image, the selectable image is obtained.

[0371] An implementation manner of the normalization processing on the first gain information is introduced below. Exemplarily, the electronic device can perform normalization processing on the first gain information in the following manner.

[0372] For the gain of each pixel point in the first gain information, for example, the gain (G) of the jth pixel point, the normalization processing can be performed in the following formula:

[0373] wherein G represents the gain value of the ith pixel point, Min(G) represents the minimum value in the gain values of the pixel points in the first gain information, Max(G) represents the maximum value in the gain values of the pixel points in the first gain information, and G Normalized represents the gain value of the ith pixel point after normalization.

[0374] The value range of j can be 1 to m, and m is the number of pixel points in the first gain information. In this way, when the jth pixel point in the first gain information traverses all the pixel points in the first gain information, the normalized first gain information can be obtained.

[0375] It can be understood that the calculation of the normalized first gain information can be performed by the encoding module, and the calculation of the normalized first gain information also belongs to the process of calculating the gain information. In this way, the image file transmitted by the encoding module to the decoding module can include the normalized first gain information, the metadata of the first gain information, and the second image. Therefore, the data amount of the image file transmitted by the encoding module to the decoding module is small.

[0376] Correspondingly, in the decoding stage, the electronic device can first perform denormalization processing on the normalized first gain information, and then perform the gain value adjustment introduced above according to the denormalized first gain information, so as to obtain the second gain information. Therefore, it can be understood that the second gain information in the present application is denormalized.

[0377] Correspondingly, obtaining the selectable image according to the normalized first gain information and the second image can be performed by the decoding module, and the process belongs to the process of applying the gain information. Specifically, the decoding module can first perform a denormalization process on the normalized first gain information to obtain denormalized first gain information, which can be the same as the first gain information. Then, the decoding module performs the gain value adjustment according to the denormalized first gain information, thereby obtaining the second gain information. Then, the selectable image is obtained according to the second gain information and the second image.

[0378] It should be understood that the denormalization process can be the inverse process of the normalization process, and the normalization process of the first gain information can be referred to the above description, which will not be repeated here.

[0379] The process of obtaining the selectable image in the method 600 will be further described below in combination with FIG. 12.

[0380] FIG. 12 is a schematic diagram of an image processing process 900 provided by an embodiment of the present application. The process 900 includes two sub-processes of calculating gain information and applying gain information. The calculating gain information can be performed by the encoding module of the electronic device, and the applying gain information can be performed by the decoding module of the electronic device.

[0381] For the process of calculating the gain information: the encoding module obtains the first image (the preset HDR image) and the second image (the basic image). The encoding module calculates the linear space brightness of the first image and the linear space brightness of the second image. Then, based on the linear space brightness of the first image, the encoding module calculates the maximum linear space brightness of the first image, and based on the linear space brightness of the second image, the encoding module calculates the maximum linear space brightness of the second image. The encoding module calculates the dynamic range headroom level according to the maximum linear space brightness of the first image and the maximum linear space brightness of the second image, and calculates the first gain information based on the linear space brightness of the first image and the linear space brightness of the second image. Then, the encoding module determines the first gain range based on the dynamic range headroom level, the bit width, and the gain range of the first gain information (i.e., the second gain range).

[0382] After determining the first gain range, the encoding module can write the first gain range into the metadata of the first gain information, and the gain value of the first gain information remains unchanged. Then, the encoding module performs a normalization process on the first gain information to obtain normalized first gain information. The encoding module transmits the normalized first gain information, the metadata of the first gain information, and the second image to the process of applying the gain information. For example, the encoding module can transmit an image file to the decoding module, and the image file includes the first gain information, the metadata of the first gain information, and the second image.

[0383] For the process of applying the gain information: the decoding module obtains the image file, denormalizes the normalized first gain information to obtain denormalized first gain information. Then, the encoding module adjusts the gain value of the denormalized first gain information based on the first gain range recorded in the metadata to obtain second gain information (the second gain information is also denormalized), and then uses the denormalized second gain information to perform dynamic range expansion on the second image to obtain the selectable image.

[0384] It can be understood that, in the embodiment of the present application, after the electronic device calculates the selectable image, the selectable image can be subjected to non-linear conversion to obtain an enhanced image, which can be understood as a reconstructed HDR image, i.e., a reconstructed first image. The closer the image is to the first image, the lower the loss of image quality of the reconstructed HDR image.

[0385] It should be noted that, in the embodiment of the present application, the encoding module and the decoding module described above include a plurality of steps, but the steps described in the above embodiment do not constitute an absolute limitation on the steps specifically included in the encoding module and the decoding module. That is, between any two operation steps in the encoding module and the decoding module described above, any intermediate step can be added according to actual needs, as long as the ultimate purpose of reconstructing the HDR image can be achieved.

[0386] In addition, each step included in the encoding module and the decoding module is not explicitly explained in the above embodiment, and can be understood with reference to related implementations, for example, the understanding of determining the linear space brightness of the image described above can be understood according to the current related implementation.

[0387] It should be noted that the module names involved in the embodiments of the present application can be defined as other names, as long as the functions of the modules can be achieved, and the names of the modules are not specifically limited. In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to select authorization or refusal.

[0388] The image processing method of the embodiments of the present application has been described above, and the device provided by the embodiments of the present application for executing the above method will be described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the related device provided by the embodiments of the present application can execute the steps in the method of listing sorting.

[0389] FIG. 13 is a schematic block diagram of an image processing apparatus 1300 according to an embodiment of the present application. The apparatus 1300 includes a processor 1301, a communication interface 1302, and a memory 1303. The processor 1301, the communication interface 1302, and the memory 1303 communicate with each other through an internal connection path. The memory 1303 is configured to store instructions, and the processor 1301 is configured to execute the instructions stored in the memory 1303. The communication interface 1302 can be configured to send signals to other apparatuses (for example, the processor 1301 or a touch screen of an electronic device) and receive signals from other apparatuses (for example, the memory 1303). For example, the communication interface 1302 reads the instructions stored in the memory 1303 and sends the instructions to the processor 1301.

[0390] It should be understood that the apparatus 1300 can be specifically an electronic device in the above-described embodiments, and can be configured to perform each step and / or process corresponding to the electronic device in the above-described method embodiments. Alternatively, the memory 1303 can include a read-only memory and a random access memory, and provide instructions and data for the processor. A part of the memory can also include a non-volatile random access memory. For example, the memory can also store device type information. The processor 1301 can be configured to execute the instructions stored in the memory, and when the processor 1301 executes the instructions stored in the memory, the processor 1301 is configured to perform each step and / or process of the above-described method embodiments.

[0391] It should be understood that in the embodiments of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0392] In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor or the instructions in the form of software. The steps of the method disclosed in the embodiments of the present application can be directly embodied as hardware processor execution, or executed by the combination of hardware and software modules in the processor. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, or other mature storage media in the art. The storage medium is located in the memory, and the processor executes the instructions in the memory to complete the steps of the above method in combination with the hardware. To avoid repetition, it will not be described in detail here.

[0393] The image processing method provided by the embodiments of the present application can be applied to an electronic device with a communication function. The electronic device includes a terminal device, and the specific device form of the terminal device can refer to the related description above, which will not be repeated here.

[0394] The embodiments of the present application provide a terminal device, which includes: a processor and a memory; the memory stores computer execution instructions; and the processor executes the computer execution instructions stored in the memory, so that the terminal device executes the above method.

[0395] The embodiments of the present application provide a chip. The chip includes a processor, which is configured to invoke a computer program in a memory to execute the technical solutions in the above embodiments. The implementation principle and technical effects are similar to those of the above related embodiments, which will not be repeated here.

[0396] The embodiments of the present application also provide a computer readable storage medium. The computer readable storage medium stores a computer program. The computer program is executed by the processor to implement the above method. The method described in the above embodiments can be implemented by software, hardware, firmware or any combination thereof, in whole or in part. If implemented in software, the functions can be stored as one or more instructions or codes on a computer readable medium or transmitted on a computer readable medium. The computer readable medium can include computer storage medium and communication medium, and can also include any medium that can carry computer programs from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0397] In a possible implementation, the computer readable medium can include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that is targeted at carrying or storing desired program codes in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately called a computer readable medium. For example, if software is transmitted from a website, server or other remote source using a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave), the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave) is included in the definition of medium. As used herein, magnetic disks and optical disks include compact disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, in which magnetic disks usually reproduce data magnetically, and optical disks reproduce data optically with laser. The above combinations should also be included in the scope of computer readable medium.

[0398] The embodiment of the present application provides a computer program product, which comprises a computer program, and when the computer program is executed, the computer executes the above method.

[0399] The embodiment of the present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiment of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing devices produce a device for implementing the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0400] The above detailed description is further detailed for the purpose of the present application, technical solutions, and beneficial effects, and it should be understood that the above is only a specific embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application should be included in the protection scope of the present application.

Claims

1. An image processing method, characterized by, The method comprises: obtaining a first image and a second image, the first image being a high dynamic range (HDR) image, and the second image being a standard dynamic range (SDR) image; determining a first gain range according to the first image and the second image; determining metadata of first gain information according to the first gain range, the first gain information being calculated according to the first image and the second image; obtaining a selectable image according to the first gain information, the metadata and the second image.

2. The method of claim 1, wherein, The first gain range comprises a maximum value and a minimum value. The determining the metadata of the first gain information according to the first gain range comprises: writing the maximum value and the minimum value into the metadata of the first gain information.

3. The method according to claim 1 or 2, characterized in that, The determining the first gain range comprises: determining the first gain range according to linear space luminance of the first image and linear space luminance of the second image.

4. The method of claim 3, wherein, The determining the first gain range comprises: calculating a dynamic range headroom level according to linear space luminance of the first image and linear space luminance of the second image; determining the first gain range according to the dynamic range headroom level.

5. The method of claim 4, wherein, The calculating the dynamic range headroom level comprises: calculating the dynamic range headroom level according to linear space luminance of the first image and linear space luminance of the second image.

6. The method of claim 5, wherein, The dynamic range headroom level satisfies: the dynamic range headroom level is a first ratio, the first ratio being a ratio of maximum linear space luminance of the first image to maximum linear space luminance of the second image; or, the dynamic range headroom level is a first preset headroom, and the first ratio is greater than the first preset headroom.

7. The method according to any one of claims 1 to 6, characterized in that, The determining the first gain range comprises: determining the first gain range according to the first image, the second image and the first gain information.

8. The method of claim 7, wherein, The determining the first gain range comprises: determining the first gain range according to a dynamic range headroom level and a second gain range, the second gain range being a gain range of the first gain information, and the dynamic range headroom level being calculated according to the first image and the second image.

9. The method according to any one of claims 1 to 8, characterized in that, The determining the first gain range comprises: determining the first gain range according to the first image, the second image and a bit width.

10. The method according to any one of claims 1 to 9, characterized in that, The determining the first gain range comprises: determining a first enhanced image according to the second image and the first gain information; determining the gain range according to the first enhanced image, the first image and the second image.

11. The method of claim 10, wherein, The determining the first gain range comprises: determining a third gain range according to the first image and the second image; reducing the third gain range to obtain the first gain range according to linear space luminance of the first enhanced image and linear space luminance of the first image.

12. The method of claim 11, wherein, The reducing the third gain range comprises: reducing the third gain range according to contrast of linear space luminance of the first enhanced image and linear space luminance of the first image.

13. The method of claim 12, wherein, The reducing the third gain range comprises: In a case where a ratio of a number of the first pixel points to a number of the second pixel points is less than or equal to a first threshold value, or the number of the first pixel points is less than or equal to a second threshold value, the third gain range is reduced. The second pixel points are pixel points in the first gain information whose gain values are in the third gain range, and the first pixel points are pixel points in the second pixel points whose contrast meets a contrast sensitivity threshold value.

14. The method of claim 13, wherein, The first pixel points are pixel points in the second pixel points whose contrast meets a contrast sensitivity function.

15. The method according to any one of claims 12 to 14, characterized in that, The contrast is any one of the following: a first difference value, the first difference value being a difference between a linear space luminance of the first image and a linear space luminance of the first enhanced image; a ratio of the first difference value to the linear space luminance of the first image or the linear space luminance of the first enhanced image; or a ratio of the first difference value to a first sum, the first sum being a sum of the linear space luminance of the first image and the linear space luminance of the first enhanced image.

16. The method according to any one of claims 2 to 15, characterized in that, The obtaining of the selectable image according to the first gain information, the metadata, and the second image includes: setting, according to a minimum value recorded in the metadata, a gain smaller than the minimum value in the first gain information as the minimum value, and setting, according to a maximum value recorded in the metadata, a gain greater than the maximum value in the first gain information as the maximum value, to obtain second gain information; obtaining a selectable image according to the second gain information and the second image.

17. The method of any one of claims 1 to 16, wherein, The first gain information is normalized gain information, and the second gain information is denormalized gain information.

18. The method of any one of claims 1 to 17, wherein, The selectable image is used for reconstructing the first image.

19. An electronic device, comprising: The electronic device includes one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is configured to store computer program code including computer instructions, and the one or more processors are configured to invoke the computer instructions to cause the electronic device to perform the method in any one of claims 1 to 18.

20. A chip system, characterized by The chip system is applied to an electronic device, and the chip system includes one or more processors configured to invoke computer instructions to cause the electronic device to perform the method in any one of claims 1 to 18.

21. A computer-readable storage medium, characterized in that, The computer readable storage medium includes computer instructions configured to cause the electronic device to perform the method in any one of claims 1 to 18 when the computer instructions run on the electronic device.

22. A computer program product, characterised in that, The computer program product includes computer program code configured to cause the electronic device to perform the method in any one of claims 1 to 18 when the computer program code runs on the electronic device.

Images