Image processing method and electronic device

Abstract

The present application relates to the technical field of image processing. Provided in the embodiments are an image processing method and an electronic device. The method comprises: acquiring a first image and a second image, the first image being a high dynamic range (HDR) image, and the second image being a standard dynamic range (SDR) image; on the basis of the first image and the second image, determining a first gain range; on the basis of first gain information and the first gain range, determining second gain information, the first gain information being calculated on the basis of the first image and the second image; and, on the basis of the second gain information and the second image, obtaining a selectable image. In this way, adjusting gain ranges can reduce the quality loss of reconstructed HDR images, thus improving the image quality of images displayed by electronic devices.

Description

Image processing method and electronic device

[0001] The present application claims priority from the Chinese patent application No. 202410745365.0 filed on June 7, 2024, and entitled "Image processing method and electronic device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of image processing, and in particular, to an image processing method and an electronic device. BACKGROUND

[0003] In order to make the image displayed by the electronic device more truly reflect the lighting conditions in nature, high dynamic range (HDR) technology has been widely concerned in the field of image processing technology. For example, the electronic device can display an HDR image through the display screen, thereby providing better visual experience for the user. However, some electronic devices, such as electronic devices provided with an SDR display screen, etc., may not be able to display an HDR image, or the displayed HDR image has image quality loss, etc.

[0004] Therefore, after the electronic device captures the image data, the SDR image and the gain map can be obtained according to the initial image data; then, the electronic device can display the enhanced image obtained from the SDR image and the gain map through the display screen.

[0005] However, such a method may cause the displayed image of the electronic device to have image quality loss. SUMMARY

[0006] The embodiments of the present application provide an image processing method and an electronic device, which can reduce the quality loss of the reconstructed HDR image by adjusting the gain range, and improve the image quality of the image displayed by the electronic device.

[0007] In a first aspect, an image processing method is provided. The method comprises: obtaining a first image and a second image, the first image being a high dynamic range (HDR) image, and the second image being a standard dynamic range (SDR) image; determining a first gain range according to the first image and the second image; determining second gain information according to the first gain information and the first gain range, the first gain information being calculated according to the first image and the second image; and obtaining a selectable image according to the second gain information and the second image.

[0008] The image processing method of the present application, after obtaining the first gain information, the electronic device will additionally determine a first gain range for reducing the image quality loss according to the first image and the second image, and adjust the first gain information in combination with the first gain range to obtain the second gain information. In this way, the gain in the second gain information can be within the first gain range, and the basic image is dynamically expanded by the second gain information, so that the reconstructed HDR image is closer to the first image, which helps to reduce the image quality loss.

[0009] Optionally, the first gain range is determined according to the linear space brightness of the first image and the linear space brightness of the second image.

[0010] Since the linear space brightness can more accurately reflect the real lighting conditions, it is helpful to make the HDR image reconstructed based on the first gain range meet the quality requirements.

[0011] Optionally, the first gain range is determined according to the linear space brightness of the first image and the linear space brightness of the second image.

[0012] Since the dynamic range headroom level can reflect the brightness advantage of the first image relative to the second image, by the dynamic range headroom level, the first gain range is determined, which helps to make the HDR image reconstructed based on the first gain range closer to the first image.

[0013] Optionally, the dynamic range headroom level satisfies: the dynamic range headroom level is a first ratio, the first ratio is a ratio of the maximum linear space brightness of the first image to the maximum linear space brightness of the second image; or, the dynamic range headroom level is a first preset headroom, and the first ratio is greater than the first preset headroom.

[0014] When the dynamic range headroom level is the first ratio, the dynamic range headroom level can reflect the advantage of the maximum linear space brightness of the first image relative to the maximum linear space brightness of the second image; when the dynamic range headroom level is the first preset headroom, the dynamic range headroom level can be matched with the performance and / or bit width of the display of the electronic device.

[0015] Optionally, the first gain range is determined according to the first image, the second image and the first gain information.

[0016] Since the first gain information can reflect the actual gain condition of the first image relative to the second image, and the image difference condition between the first image and the second image can be analyzed according to the first image and the second image, the first gain range can be determined by combining the first image, the second image and the first gain information, so that the obtained first gain range is adapted to the actual scene of the current HDR image reconstruction.

[0017] Optionally, the first gain range is determined according to the dynamic range headroom level and the second gain range, the second gain range being a gain range of the first gain information, and the dynamic range headroom level being calculated according to the first image and the second image.

[0018] In this way, the first gain range can be determined by combining the luminance advantage of the first image relative to the second image and the second gain range, and the first gain range can be within the second gain range, which helps to make the HDR image reconstructed based on the first gain range closer to the first image.

[0019] Optionally, the first gain range is determined according to the first image, the second image and the bit width.

[0020] Since the bit width can affect the color depth, detail performance and overall visual quality of the image, the first gain range is determined by combining the first image, the second image and the bit width, which helps to make the image quality loss of the HDR image reconstructed based on the first gain range lower.

[0021] Optionally, the first gain range is determined according to the second image and the first gain information, the first enhanced image is determined according to the first gain information, and the first gain range is determined according to the first enhanced image, the first image and the second image.

[0022] In this way, the first gain range can be calculated by the image contrast condition of the first enhanced image relative to the first image, so that the obtained first gain range can realize image reconstruction of the first image with lower image loss.

[0023] Optionally, the first gain range is determined according to the first image and the second image, the third gain range is determined according to the first image and the second image, and the first gain range is obtained by reducing the third gain range according to the linear space luminance of the first enhanced image and the linear space luminance of the first image.

[0024] Since the first enhanced image is the reconstructed first image (i.e. the reconstructed HDR image), the closer the first enhanced image is to the first image, the lower the image quality loss of the HDR image reconstructed based on the first gain range.

[0025] Therefore, the image quality loss of the HDR image reconstructed based on the first gain range can be low.

[0026] Optionally, the third gain range is reduced by reducing the third gain range according to a contrast between the linear space luminance of the first enhanced image and the linear space luminance of the first image.

[0027] Since the contrast can reflect the luminance contrast of the pixel points in the first enhanced image compared with the pixel points in the first image, the image quality loss of the reconstructed first enhanced image compared with the target first image can be effectively reflected by the contrast.

[0028] Therefore, whether the image quality loss of the HDR image reconstructed based on the third gain range meets the requirement can be determined based on the contrast.

[0029] Optionally, the third gain range is reduced by reducing the third gain range in a case that a ratio of the number of the first pixel points to the number of the second pixel points is less than or equal to a first threshold, or the number of the first pixel points is less than or equal to a second threshold; wherein the second pixel points are pixel points with gain values in the third gain range in the first gain information, and the first pixel points are pixel points in the second pixel points with contrasts meeting a contrast sensitivity threshold.

[0030] Since the luminance loss of the pixel points can be determined to be small when the contrast of the pixel points meets the corresponding contrast sensitivity threshold, the image quality loss at the position cannot be observed by the human eye in general.

[0031] Therefore, most of the pixel points in the HDR image reconstructed based on the first gain range can meet the contrast sensitivity threshold, that is, the HDR image reconstructed based on the first gain range is close to the first image, which helps to realize image reconstruction with small image quality loss.

[0032] Optionally, the first pixel points are pixel points in the second pixel points with contrasts meeting a contrast sensitivity function.

[0033] In this way, the number of the first pixel points or the ratio of the first pixel points in the second pixel points can evaluate the region where the image quality loss cannot be observed by the human eye. When the number of the first pixel points or the ratio of the first pixel points in the second pixel points is large, it can be indicated that the human eye cannot observe the difference between most regions in the first enhanced image and the first image, and it is indicated that the image quality loss of the first enhanced image is low.

[0034] Optionally, the contrast is any one of the following: a first difference, the first difference being a difference between the linear space luminance of the first image and the linear space luminance of the first enhanced image; a ratio of the first difference to the linear space luminance of the first image or the linear space luminance of the first enhanced image; or, a ratio of the first difference to a first sum, the first sum being a sum of the linear space luminance of the first image and the linear space luminance of the first enhanced image.

[0035] In this way, the contrast can reflect the luminance contrast of the pixel points in the first enhanced image compared with the pixel points in the first image.

[0036] Optionally, the first gain range includes a maximum value and a minimum value; and determining the second gain information includes: setting the gain smaller than the minimum value in the first gain information to the minimum value, and setting the gain greater than the maximum value in the first gain information to the maximum value.

[0037] In this way, the gain in the second gain information can be in the first gain range. This helps to reconstruct an image with low image quality loss by using the second gain information.

[0038] Optionally, the method further includes: performing normalization processing on the second gain information to obtain normalized second gain information; and obtaining the selectable image includes: obtaining the selectable image according to the normalized second gain information and the second image.

[0039] In this way, the encoding module of the electronic device can transmit the normalized second gain information to the decoding module, which helps to reduce the amount of data transmitted, so that the power consumption of the electronic device is small.

[0040] Optionally, the selectable image is used to reconstruct the first image.

[0041] For example, the selectable image is subjected to non-linear conversion to obtain an enhanced image, which can be understood as a reconstructed HDR image, i.e., a reconstructed first image. The enhanced image is not the first image, but the display effect of the enhanced image is similar to that of the first image.

[0042] Optionally, the first gain range is included in the metadata of the second gain information.

[0043] In a second aspect, an image processing apparatus is provided. The image processing apparatus can be an electronic device, or a chip or chip system in the electronic device. The image processing apparatus can include a display unit and a processing unit. When the image processing apparatus is an electronic device, the display unit can be a display screen. The display unit is configured to perform the displaying, so that the electronic device implements an image processing method described in the first aspect or any possible implementation manner of the first aspect.

[0044] When the image processing apparatus is an electronic device, the processing unit can be a processor. The image processing apparatus can further include a storage unit, which can be a memory. The storage unit is configured to store instructions, and the processing unit is configured to execute the instructions stored in the storage unit, so that the electronic device implements an image processing method described in the first aspect or any possible implementation manner of the first aspect. When the image processing apparatus is a chip or a chip system in an electronic device, the processing unit can be a processor. The processing unit executes the instructions stored in the storage unit, so that the electronic device implements an image processing method described in the first aspect or any possible implementation manner of the first aspect. The storage unit can be a storage unit (for example, a register, a cache, etc.) in the chip, or a storage unit (for example, a read-only memory, a random access memory, etc.) outside the chip in the electronic device.

[0045] For example, the processing unit is configured to perform image fusion on part or all of the multiple frames of images to obtain a first image; and the display unit is configured to display a target image.

[0046] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory. The memory is configured to store code instructions, and the processor is configured to execute the code instructions to perform the method described in the first aspect or any possible implementation manner of the first aspect.

[0047] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores computer programs or instructions. When the computer programs or instructions are executed on a computer, the computer is caused to perform the method described in the first aspect or any possible implementation manner of the first aspect.

[0048] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program is executed on a computer, the computer is caused to perform the method described in the first aspect or any possible implementation manner of the first aspect.

[0049] In a sixth aspect, the present application provides a chip or a chip system, which includes at least one processor and a communication interface. The communication interface and the at least one processor are interconnected through a line. The at least one processor is configured to execute computer programs or instructions to perform the method described in the first aspect or any possible implementation manner of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0050] In a possible implementation, the chip or the chip system described above in the present application further includes at least one memory in which instructions are stored. The memory can be a storage unit inside the chip, for example, a register, a cache, etc., or a storage unit of the chip (for example, a read-only memory, a random access memory, etc.).

[0051] It should be understood that the second aspect to the sixth aspect of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects obtained by each aspect and the corresponding feasible implementation manners are similar, which will not be described again. BRIEF DESCRIPTION OF DRAWINGS

[0052] FIG. 1 is a schematic block diagram of a hardware architecture of an electronic device according to an embodiment of the present application;

[0053] FIG. 2 is a schematic diagram of an application scenario according to an embodiment of the present application;

[0054] FIG. 3 is a schematic diagram of an image processing process;

[0055] FIG. 4 is a schematic diagram of an image restoration scenario according to an embodiment of the present application;

[0056] FIG. 5 is a comparative schematic diagram of image restoration according to an embodiment of the present application;

[0057] FIG. 6 is a flowchart of an image processing method according to an embodiment of the present application;

[0058] FIG. 7 is a schematic diagram of determining second gain information according to an embodiment of the present application;

[0059] FIG. 8 is a flowchart of determining image contrast according to an embodiment of the present application;

[0060] FIG. 9 is a schematic diagram of determining a first pixel point according to an embodiment of the present application;

[0061] FIG. 10 is a curve diagram of a contrast sensitivity function according to an embodiment of the present application;

[0062] FIG. 11 is a schematic diagram of an image processing process according to an embodiment of the present application;

[0063] FIG. 12 is a schematic block diagram of an image processing apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to clearly describe the technical solutions of the embodiments of the present application, the following briefly introduces some terms and technologies involved in the embodiments of the present application:

[0065] 1. Dynamic range (dynamic range)

[0066] Ratio of maximum luminance to minimum luminance in a video signal.

[0067] 2、Standard dynamic range (SDR) image

[0068] Refers to an image with a traditional dynamic range, as opposed to a high dynamic range (HDR) image. Compared to an HDR image, an SDR image has a narrower dynamic range and color gamut, suitable for most traditional display devices and content. And an SDR image can be an image generated by tone mapping an HDR image.

[0069] 3、High dynamic range (HDR) image

[0070] Refers to an image that can display more luminance levels and a wider color gamut than an SDR image. The luminance range of an HDR image typically ranges from very dark black to very bright white, and can more realistically reflect the lighting conditions in nature.

[0071] HDR images can display higher brightness and deeper black, providing richer details and more realistic visual experiences.

[0072] From the perspective of color space, HDR images typically use a wider color space, such as Rec. 2020 (BT. 2020) or DCI-P3, while traditional SDR images typically use Rec. 709 color space. A wider color space means that an HDR image can display more colors and richer color details.

[0073] 4、Linear space brightness

[0074] Refers to the luminance value represented in a linear color space. Unlike non-linear color spaces such as sRGB or Rec. 709, the luminance value in a linear color space is directly proportional to the physical luminance. This means that when performing image processing and analysis, linear space brightness can more accurately reflect the true lighting conditions.

[0075] For example, the processing of an HDR image is usually performed in a linear space to accurately combine and adjust images of different exposures.

[0076] Calculating the linear space brightness of an HDR image typically includes the following two steps: EOTF electro-optical conversion and color space conversion. The linear space brightness of an SDR image can be calculated by de-gamma correction.

[0077] 5、De-gamma correction

[0078] Also referred to as linearization, generally refers to the process of converting an image from a gamma-encoded non-linear color space back to a linear color space.

[0079] 6、Color space conversion

[0080] Also referred to as color conversion, refers to the process of converting image or video data from one color space to another color space.

[0081] A color space defines a method of representing colors and a color gamut, and a color gamut together with a color model can define a color space. A color model is an abstract mathematical model that represents colors using a set of color components. Color models can include, for example, red green blue (RGB), cyan magenta yellow key plate (CMYK). A color gamut refers to the total combination of colors that a system can produce.

[0082] Rec. 2020 (BT. 2020) mentioned above in the introduction of HDR image is the full name of ITU-R Recommendation BT. 2020, which is a recommendation or standard for ultra-high-definition television system formulated by International Telecommunication Union Radio Communication Department (ITU-R). The meaning of Rec is Recommendation, and the meaning of BT is Broadcasting Service. The full name of DCI-P3 is Digital Cinema Initiatives-P3. The full name of Rec. 709 is ITU-R Recommendation BT. 709. In an implementation mode, the color gamut sizes of the three color spaces are color gamut size: Rec. 2020 > DCI-P3 > Rec. 709.

[0083] 7、Gain map

[0084] Also referred to as gain information or gain image, is used to describe the brightness adjustment information of different regions in the HDR image. Exemplarily, the gain information can be an image with the same size as the HDR image, and each pixel value represents the brightness gain of the region. The brightness gain can also be referred to as gain or gain value, which is a coefficient or scaling factor for adjusting the brightness of the image.

[0085] The gain information is usually stored in a floating-point number format, representing the gain value in a linear space.

[0086] 8、Gain range

[0087] It is an interval, namely [minimum gain value, maximum gain value]. This interval represents the range of gain variation in different regions of the image.

[0088] The minimum gain value is found by iterating through all pixels in the gain information and identifying the smallest gain value. This typically represents the gain of the darkest area in the image.

[0089] Furthermore, the maximum gain value is the maximum gain value found across all pixels in the gain information. This typically represents the gain of the brightest area in the image.

[0090] 9. Bit depth

[0091] The number of bits used for each pixel determines the range of values ​​for that pixel. For example, an 8-bit width can represent integer values ​​from 0 to 255, while a 16-bit width can represent integer values ​​from 0 to 65535.

[0092] Furthermore, bit width can affect the color depth, detail representation, and overall visual quality of an image.

[0093] 10. Dynamic range headroom

[0094] Also known as headroom, it is a metric that measures the brightness advantage of HDR images over SDR images. The headroom of an HDR image can also be called the high dynamic range display headroom (HDR headroom).

[0095] 11. Perceptual quantizer (PQ)

[0096] PQ is an HDR standard and also an HDR conversion equation, determined by human visual ability. The video signal displayed by a display device is typically a PQ-encoded video signal.

[0097] 12. Barten Model

[0098] The Barten model is a model used to describe and quantify the human visual system's perception of image noise. Originally proposed by Peter GJ Barten, it has been widely applied in image quality assessment, display technology, and visual perception research. The core idea of ​​the Barten model is to use mathematical formulas to describe the visual system's sensitivity to different spatial frequencies and contrasts, thereby quantifying the human ability to perceive image noise.

[0099] 13. Contrast sensitivity function (CSF)

[0100] The core of the Barten model is the contrast sensitivity function, which describes the sensitivity of the human eye to different spatial frequencies (i.e. the size of the details in an image). The CSF is typically a bandpass filter, indicating that the human eye is most sensitive to details of medium frequency, and less sensitive to details of very low or very high frequency.

[0101] 14、Normal frame, which can also be referred to as normal exposure image, N frame image, N frame, mid frame or mid frame image, etc., is an image captured by a camera under the condition that the exposure is 0 EV. That is, the exposure of the normal exposure image is 0 EV. Here, 0 EV is a relative value, rather than saying that the exposure is 0. Exemplarily, exposure = exposure time * ISO. Assuming that the normal exposure image is captured under the condition that ISO is 200 and the exposure time is 50 ms, 0 EV actually corresponds to the exposure of the product of 200 and 50 ms.

[0102] 15、Short frame, which can also be referred to as short frame image, S frame or S frame image, is an image captured by a camera under the condition that the exposure is less than 0 EV.

[0103] 16、Long frame, which can also be referred to as long frame image, L frame or L frame image, is an image captured by a camera under the condition that the exposure is greater than 0 EV.

[0104] 17、Electro-optical transfer function (EOTF): a conversion relationship between nonlinear color values and linear color values.

[0105] 18、Optical-electro transfer function (OETF): a conversion relationship between linear color values and nonlinear color values.

[0106] 19、Optical-optical transfer curve (OOTF): a curve for converting one optical signal into another optical signal in video technology.

[0107] In the three concepts introduced above, the color value is a numerical value corresponding to a specific image color component (for example, R, G, B or Y). The linear color value refers to a linear color value, which is proportional to the light intensity, and in an optional case, the value should be normalized to [0, 1], abbreviated as E. The nonlinear color value refers to a nonlinear color value, which is a normalized digital representation value of image information, and is proportional to the digital coding value, and in an optional case, the value should be normalized to [0, 1], abbreviated as E'.

[0108] wherein the digital code value is a digital representation of an image signal, and the digital code value is used to represent the non-linear primary color value.

[0109] 20、Other terms

[0110] In the embodiments of the present application, the terms "first", "second" and the like are used to distinguish between similar or identical items or elements having the same function or role. For example, the first chip and the second chip are merely used to distinguish between different chips, and do not limit the sequence. Those skilled in the art can understand that the terms "first", "second" and the like do not limit the number and execution sequence, and the terms "first", "second" and the like do not necessarily mean different.

[0111] It should be noted that in the embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design scheme described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or advantageous than other embodiments or design schemes. Rather, the use of the words "exemplary" or "for example" is intended to present the relevant concept in a specific manner.

[0112] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. The association relationship between the associated objects is described by "and / or", which means that there can be three kinds of relationships, for example, A and / or B, which can represent the following three cases: A exists alone, A and B exist together, and B exists alone, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, and c can be single or multiple.

[0113] 21、Electronic device

[0114] The electronic device of the embodiments of the present application can include a handheld device with a camera function, a vehicle-mounted device, etc. For example, some electronic devices are: a mobile phone, a tablet computer, a palm computer, a notebook computer, a mobile Internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with a wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal device in a 5G network, or a terminal device in a future evolved public land mobile network (PLMN), etc., and the embodiments of the present application are not limited thereto.

[0115] By way of example and not limitation, in the embodiments of the present application, the electronic device can also be a wearable device. The wearable device can also be referred to as a wearable smart device, which is a general term for devices that are designed and developed by applying wearable technology to daily wear, such as glasses, gloves, watches, clothing, and shoes, etc. The wearable device is a portable device that is directly worn on the body or integrated into the clothes or accessories of the user. The wearable device is not only a hardware device, but also a device that realizes powerful functions through software support and data interaction and cloud interaction. The general wearable smart device includes a device with full functions and large size, which can realize complete or partial functions without relying on a smart phone, such as a smart watch or smart glasses, etc., and a device that focuses on a certain application function and needs to cooperate with other devices such as a smart phone, such as various smart wristbands and smart jewelry for monitoring vital signs, etc.

[0116] In addition, in the embodiments of the present application, the electronic device can also be a terminal device in an Internet of Things (IoT) system. The IoT is an important part of future information technology development, and its main technical feature is to connect objects through communication technology and network, so as to realize the intelligent network of man-machine interconnection and object-object interconnection.

[0117] The electronic device in the embodiments of the present application can also be referred to as a terminal device, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), an access terminal, a subscriber unit, a subscriber station, a mobile station, a mobile terminal, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent, or a user apparatus, etc.

[0118] In the embodiments of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes central processing unit (CPU), memory management unit (MMU), and memory (also known as main memory), etc. The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or windows operating system, etc. The application layer includes browser, address book, word processing software, instant messaging software, etc.

[0119] The hardware structure of the above-mentioned electronic device can be as shown in FIG. 1. FIG. 1 shows a structural schematic diagram of an electronic device 100 suitable for the present application.

[0120] The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0121] The sensor module 180 can include a pressure sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like. The audio module 170 can include a speaker, a receiver, a microphone, an earphone interface, and the like.

[0122] It should be noted that the structure shown in FIG. 1 does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than those shown in FIG. 1, or the electronic device 100 can include a combination of some of the components shown in FIG. 1, or the electronic device 100 can include sub-components of some of the components shown in FIG. 1. For example, the proximity light sensor 180G shown in FIG. 1 can be optional. The components shown in FIG. 1 can be implemented in hardware, software, or a combination of software and hardware.

[0123] The processor 110 can include one or more processing units. For example, the processor 110 can include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU). Different processing units can be independent devices or integrated devices.

[0124] Optionally, the electronic device 100 can further include an image encoder and an image decoder. The image encoder can be configured to encode image data obtained by the camera 193, for example, to obtain a baseline image and gain information by processing image data obtained by the camera 193, and to transmit an image file including the baseline image and the gain information to the image decoder. The image decoder can decode the image file from the image encoder to obtain the baseline image and the gain information, and can obtain an enhanced image based on the baseline image and the gain information, so as to indicate the enhanced image to the display screen 194 for display.

[0125] It should be understood that in the embodiments of the present application, the gain information can also be referred to as a gain map, which is not specifically limited in the present application.

[0126] It can be understood that the image encoder and the image decoder can be independent devices or integrated devices. For example, in the case of the image encoder and the image decoder being integrated devices, the image encoder and the image decoder can also be referred to as an image codec.

[0127] In addition, the image encoder and the image decoder (or the image codec) can be integrated on the processor 110, or the image encoder and the image decoder (or the image codec) are independent devices, which are not limited in the present application.

[0128] In a possible implementation, the processor 110 can obtain image data by starting the camera 193. The image data can be an image in RAW format, and the processor 110 can obtain an HDR image and an SDR image from the image in RAW format. In order to facilitate the image encoder and the image decoder to determine an enhanced image based on the HDR image and the SDR image.

[0129] The controller can generate operation control signals according to the instruction operation code and the timing signal, and complete the control of fetching and executing instructions.

[0130] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0131] The electronic device 100 can realize the photographing function through the ISP, the camera 193, the GPU, the display screen 194, and the application processor.

[0132] The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture a still image or a video. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. In some embodiments, the ISP can be arranged in the camera 193. For example, the electrical signal transmitted by the camera photosensitive element to the ISP can be an image sequence, and the ISP can pre-process the image sequence. The image sequence after ISP pre-processing can be an image in RAW format.

[0133] In some embodiments, the electronic device 100 can include one or N cameras 193, and N is a positive integer greater than 1.

[0134] The electronic device 100 implements the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, connected with the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information. For example, the processor 110 can call the GPU to perform image fusion and noise reduction processing on at least one frame of short frame image and at least one frame of middle frame image; the processor 110 can also call the GPU to perform color correction on a target region, such as a sky region, etc.

[0135] The display screen 194 can be used to display the enhanced image after rendering, etc.

[0136] It should be understood that the connection relationship between the modules shown in FIG. 1 is only illustrative and does not constitute a limitation on the connection relationship between the modules of the electronic device 100. Alternatively, the modules of the electronic device 100 can also use a combination of the above-mentioned various connection modes.

[0137] At present, in order to improve the shooting effect of the electronic device, part of the electronic device can take pictures through the HDR technology. Taking pictures through the HDR technology can also be understood as the electronic device taking pictures in the HDR mode. In the HDR mode, in response to the shooting operation, the camera of the electronic device can take multiple images under different exposure amounts. The multiple images include, for example, long frame, short frame and normal frame, etc. Then, by image fusion on part or all of the multiple images, a target image can be obtained. In the case that the target image is an HDR image, the target image has more brightness levels and a wider color gamut, so that the target image displayed by the electronic device can more truly reflect the lighting conditions in nature.

[0138] However, in some cases, the electronic device may not be able to normally display the HDR image, or the image quality of the displayed HDR image is poor, etc. For example, the display screen of part of the electronic device does not support displaying the HDR image, so that the part of the electronic device cannot directly display the HDR image; the data amount of the HDR image is usually large, and the bandwidth and storage space occupied by the internal transmission and storage of the HDR image in the electronic device are large, so that the load of the electronic device is large, thereby causing the electronic device to be stuck and other abnormalities.

[0139] Therefore, at present, the shooting effect of the electronic device can be improved in the following way. Next, the shooting process of the electronic device will be described in combination with the application scenario 200 of the embodiment of the present application shown in FIG. 2.

[0140] FIG. 2 is a schematic diagram of an application scenario 200 provided by an embodiment of the present application. The application scenario 200 can be applied to an electronic device, such as the electronic device 100, and the electronic device can support photographing by using an HDR technology.

[0141] In the HDR mode, in response to a photographing operation, the camera of the electronic device photographs image data, and based on the image data, the electronic device can obtain a base image and gain information by using an image encoder. The base image is an image with a traditional dynamic range. The gain information is a plurality of gain values stored in a floating-point number format. The image encoder can transmit an image file including the base image and the gain information to an image decoder. The image decoder can obtain an enhanced image based on the base image and the gain information, so that the display screen of the electronic device can display the enhanced image. As shown in FIG. 2, a user can view an image in an image application of the electronic device, and then the electronic device displays the enhanced image on the display screen in response to an image display command triggered by a user operation.

[0142] The enhanced image can be a non-linear domain signal and can be obtained by performing a non-linear conversion on a linear domain signal. For example, the image decoder can calculate an alternate image based on the base image and the gain information. The alternate image is a linear domain signal.

[0143] In this way, the base image is processed by using the gain information, so that the brightness of different regions in the base image can be adjusted to expand the dynamic range of the base image. By applying different gains to different brightness regions, dark details can be made clearer while bright details are preserved. By adjusting the local gain, the details and textures in the base image can be preserved while the overall brightness of the base image is enhanced. The image displayed by the electronic device can present an effect similar to that of an HDR image.

[0144] Furthermore, for an electronic device that cannot display an HDR image, the dynamic range of the enhanced image is adapted to the range of the electronic device, so that a better visual effect can be obtained on various electronic devices. That is, the enhanced image obtained by using the gain information and the base image not only has a higher dynamic range than the base image, but also has a dynamic range adapted to the electronic device.

[0145] In addition, compared with an HDR image, the bandwidth and storage space occupied by the image file transmitted from the image encoder to the image decoder can be smaller, so that the power consumption of the electronic device is smaller.

[0146] It should be understood that the image encoder above can also be referred to as an encoding module, and the image decoder can also be referred to as a decoding module. Among them, the encoding module can be used to determine the gain information, and transmit an image file including the gain information and the base image to the decoding module. Then, the decoding module can determine the enhanced image based on the gain information and the base image. Moreover, the process of determining the gain information by the encoding module can also be referred to as computing a gain map, and the process of determining the enhanced image based on the gain information and the base image by the decoding module can also be referred to as applying a gain map. The name of the module for computing the gain map and the name of the module for applying the gain map are not specifically limited in the present application.

[0147] Next, the process of computing the gain information and applying the gain information will be described in detail in combination with FIG. 3.

[0148] As shown in FIG. 3, the camera of the electronic device can capture multiple images at different exposure amounts, respectively. Then, by image fusion on part or all of the multiple images, a RAW format image can be obtained. The RAW format image can also be referred to as a RAW image for short. The RAW image refers to image data directly obtained by the electronic device through the image sensor without compression and processing, which contains the original pixel information captured from the image sensor.

[0149] Then, based on the RAW image, a preset HDR image and a base image can be obtained.

[0150] Among them, obtaining the preset HDR image based on the RAW image can be implemented in the following manner: processing the RAW image by using the OETF transfer function of PQ to obtain the preset HDR image. The OETF transfer function of PQ represents the conversion relationship from the linear signal value of the image pixel to the nonlinear signal value in the PQ domain. Processing the RAW image by using the OETF transfer function of PQ is to convert the optical signal into an electrical signal.

[0151] In addition to the PQ photoelectric conversion function, the electronic device can also process the RAW image based on the OETF transfer function of hybrid log-gamma (HLG) to obtain the preset HDR image. The OETF transfer function of HLG represents the conversion relationship from the linear signal value of the image pixel to the nonlinear signal value in the HLG domain.

[0152] It should be understood that the electronic device can also convert the RAW image into the preset HDR image in other manners, which are not specifically limited in the present application.

[0153] The preset HDR image can be understood as an image that the electronic device is expected to display, i.e., the image to be reconstructed by the electronic device through the basic image and the gain information is the preset HDR image. In other words, the electronic device can not be able to directly display the preset HDR image, so the electronic device reconstructs an image through the basic image and the gain information, the reconstructed image is to achieve a display effect close to the preset HDR image, so that the image displayed by the electronic device has more brightness levels and a wider color gamut, and can more truly reflect the lighting conditions in nature, thereby improving the user experience.

[0154] The basic image obtained based on the RAW image can be implemented in the following manner: the RAW image is subjected to gamma correction using an sRGB gamma curve (such as a 2.2 gamma), thereby obtaining the basic image.

[0155] The gamma correction is used to convert linear light intensity into a non-linear signal conforming to the response curve of the display device. For example, linear light intensity data in the RAW image can be converted into non-linear gamma coding, and then the image data is converted into a standard sRGB color space using an sRGB gamma curve, thereby obtaining the basic image.

[0156] In an implementation manner, in addition to the gamma correction processing, the RAW image can also be subjected to processing such as demosaicing, color correction, white balance adjustment, and tone mapping, thereby obtaining a basic image with higher quality.

[0157] The demosaicing is a process of converting Bayer array data captured by a sensor into a full-color image. The white balance adjustment is used to correct color deviation under different light sources, so that the white color in the image appears real. The color correction is performed according to the color characteristic matrix of the camera, so that the color is more accurate. The tone mapping is used to compress the high dynamic range of the image into a range that can be displayed by an SDR display device.

[0158] It should be understood that the electronic device can also convert the RAW image into a basic image in other manners, which are not limited in the present application.

[0159] Referring to FIG. 3, after the encoding module of the electronic device obtains the preset HDR image and the basic image, the linear space brightness of the preset HDR image can be calculated based on the preset HDR image, and the linear space brightness of the basic image can be calculated based on the basic image. The encoding module can calculate the linear space brightness of the preset HDR image by linearizing and colour converting the preset HDR image. The encoding module can calculate the linear space brightness of the basic image by linearizing the basic image.

[0160] Then, the encoding module can calculate the gain information based on the linear space luminance of the preset HDR image and the linear space luminance of the base image. It can be understood that the sizes of the preset HDR image, the base image and the gain information are all the same, so the pixel points in the three images are one-to-one corresponding.

[0161] The encoding module calculating the gain information can be implemented by calculating the gain information according to the linear space luminance of the i-th pixel point in the preset HDR image and the linear space luminance of the i-th pixel point in the base image. The i-th pixel point in the preset HDR image and the i-th pixel point in the base image are corresponding pixel points. The corresponding pixel points can represent the same position in the same scene. For example, the position of the i-th pixel point in the preset HDR image in the preset HDR image is the same as the position of the i-th pixel point in the base image in the base image. Wherein, the value range of i can be 1-n, n is the number of pixel points in the preset HDR image, it can be understood that the number of pixel points in the base image and the gain information is also n.

[0162] The gain of the i-th pixel point recorded in the gain information can be calculated by the following formula:

[0163] Wherein, Alternate represents the linear space luminance of the i-th pixel point in the preset HDR image, k Alternate represents the linear space offset of the preset HDR image, Baseline represents the linear space luminance of the i-th pixel point in the base image, k Baseline represents the linear space offset of the base image.

[0164] Since the values of k Alternate and k Baseline are usually close to 0, the above formula can also be simplified to the following formula:

[0165] In this way, when the i-th pixel point in the preset HDR image traverses all the pixel points in the preset HDR image, and the i-th pixel point in the base image traverses all the pixel points in the base image, the gain information can be obtained. That is, the gain information can include the gain corresponding to each pixel point in the base image.

[0166] In addition, the encoding module can also calculate the metadata of the gain information. The metadata is data describing the key information and characteristics required in the image (or video signal) processing process. It can be understood that the metadata is used to provide the information required for correct decoding and displaying the image (or video signal).

[0167] In an implementation, the encoding module may, for example, perform corresponding data analysis according to the gain information, so as to obtain the metadata of the gain information. Alternatively, the encoding module may also perform corresponding data analysis according to the preset HDR image and the base image, so as to obtain the metadata of the gain information. The embodiments do not limit the way of calculating the metadata, and also do not limit the specific content included in the metadata, which can be set arbitrarily according to actual needs.

[0168] It can be understood with reference to FIG. 3 that, after the end of the encoding stage, the encoding module of the electronic device can generate the base image, the gain information, and the metadata of the gain information. In an implementation, the encoding module of the electronic device can save the base image, the gain information, and the metadata of the gain information as image data. The image data can also be referred to as an image file. The encoding module can transmit the image file to the decoding module.

[0169] Then in the decoding stage, the decoding module of the electronic device can perform decoding processing on the metadata of the gain information, so as to parse the gain information according to the related parameters contained in the metadata, and also obtain related data relied on by the reconstructed HDR image. Then the decoding module can perform fusion processing according to the base image and the gain information, so as to generate an enhanced image.

[0170] In an implementation, the decoding module can first generate an alternate image according to the base image and the gain information. The alternate image is a high dynamic image generated by merging the base image and the gain information, and the alternate image is a signal in a linear domain.

[0171] Therefore, the electronic device can further process the alternate image to obtain an enhanced image, where the enhanced image is an image obtained by performing nonlinear conversion on the alternate image (linear domain signal), and is a signal in a nonlinear domain. The enhanced image can be understood as a reconstructed HDR image, that is, an image restored from the preset HDR image.

[0172] It should be understood that the process of generating the alternate image according to the base image and the gain information by the electronic device is actually a process of restoring the preset HDR image. The reason why the preset HDR image that has been generated is restored again through the encoding and decoding stages is that the electronic device cannot guarantee that the preset HDR image can be normally displayed on the screen, and also cannot guarantee the display effect of the preset HDR image on the screen.

[0173] With reference to the processing procedure described above, the encoding module of the electronic device first generates the gain information according to the preset HDR image and the base image, and then the encoding module saves the base image and the gain information as image data. In the decoding stage, if the screen of the electronic device supports displaying the HDR image, the decoding module can generate the selectable image according to the base image and the gain information, and then generate the gain information on the basis of the selectable image and send the enhanced image for display, so that the electronic device can effectively display the HDR image, and the gain effect indicated by the gain information can be effectively reflected in the sent HDR image.

[0174] Alternatively, if the screen of the electronic device does not support displaying the HDR image, the decoding module can generate an enhanced image adapted to the dynamic range of the electronic device, so that better visual effects can be obtained in image display on various electronic devices. Alternatively, the decoding module can also choose to directly send the base image for display to avoid the situation that the captured image cannot be displayed.

[0175] As can be understood from the above description, the gain information plays a very important role in the encoding and decoding process. The inventors have found that when the range of the gain value in the gain information is unreasonable, the enhanced image obtained by restoration will have quality loss compared with the preset HDR image.

[0176] The phenomenon can be understood with reference to FIG. 4 and FIG. 5. FIG. 4 is a schematic diagram of image restoration provided by an embodiment of the present application, and FIG. 5 is a comparative schematic diagram of image restoration provided by an embodiment of the present application.

[0177] As shown in FIG. 4, the image shown in (a) of FIG. 4 can be understood as the gain information, and the image shown in (b) of FIG. 4 can be understood as the enhanced image obtained after restoration. As described above, the base image needs to be processed according to the base image and the gain information when the enhanced image is generated, and the base image is not shown in FIG. 4.

[0178] As can be understood from (b) of FIG. 4, the enhanced image after restoration has a banding phenomenon, where the banding phenomenon refers to the occurrence of discontinuous color change or luminance change in the image, resulting in obvious color bands or light bands in the image, rather than smooth transition. As shown in the example of (b) of FIG. 3, there are circles of ripples around the upward area of the sun, which is the banding phenomenon described above, and can also be understood as the occurrence of a brightening band in the enhanced image. The occurrence of such a brightening band is actually a manifestation of image quality loss.

[0179] The image quality loss can be understood on the basis of the comparison between the preset HDR image and the enhanced image obtained after restoration with reference to FIG. 5.

[0180] The image shown in (a) of FIG. 5 can be understood as a preset HDR image, and the image shown in (b) of FIG. 5 can be understood as an enhanced image obtained after restoration. By comparison, it can be determined that there is actually no banding phenomenon in the preset HDR image, but the banding phenomenon appears in the enhanced image obtained after restoration, which indicates that the image quality loss occurs in the process of restoring the preset HDR image.

[0181] In order to more intuitively understand this image quality loss, the contrast images corresponding to the preset HDR image and the enhanced image are also shown in (c) of FIG. 5. The pixel points in the contrast image correspond one-to-one to the pixel points in the preset HDR image and the enhanced image. The pixel value of each pixel point in the contrast image is the contrast of the brightness value of the corresponding pixel point in the preset HDR image and the brightness value of the corresponding pixel point in the enhanced image.

[0182] The greater the contrast, the greater the difference in brightness value. Conversely, the smaller the contrast, the smaller the difference in brightness value. For example, the value range of the contrast can be 0-1. When the contrast corresponding to the pixel point is 0, the pixel point in the contrast image is black. When the contrast corresponding to the pixel point is 1, the pixel point in the contrast image is white. When the value of the contrast corresponding to the pixel point is a value between 0 and 1, the pixel point in the contrast image is gray.

[0183] Referring to FIG. 5, it can be understood that the colors corresponding to the regions of the brightening band in the contrast image are close to white, which indicates that the regions of the brightening band are the loss regions of the enhanced image compared with the preset HDR image.

[0184] The above exemplary introduction in combination with FIG. 4 and FIG. 5 illustrates this case that the image quality loss leads to the appearance of the brightening band in the enhanced image. In fact, there are many forms of image quality loss, for example, the image quality loss can also lead to the problems such as color cast and glare in the enhanced image compared with the preset HDR image. The specific forms of the image quality loss are not limited in the embodiment, and the cases appearing when the contrast of the enhanced image compared with the preset HDR image is larger can be understood as the cases caused by the image quality loss.

[0185] Further, referring to the introduction of FIG. 3 above, it can be understood that the current gain information is calculated according to the preset HDR image and the basic image. Therefore, the gain value range of the gain information can appear various possible cases, which depends on the actual brightness situation of the preset HDR image and the basic image.

[0186] So as to avoid the image quality loss situation introduced above, the present application proposes the following technical concept: after the gain information is calculated, the gain value range of the gain information is adjusted, so as to limit the gain value of the gain information within a certain range, so that the contrast of the preset HDR image and the enhanced image can meet the corresponding threshold condition, so that the image quality loss situation can be effectively avoided.

[0187] The image processing method of the present application will be described in detail below in combination with FIGS. 6 to 12. The image processing method of the present application embodiment can be executed by an electronic device provided with a camera, or can be executed by a chip, chip system or processor supporting the electronic device to implement the image processing method, or can be executed by a logic module or software capable of implementing all or part of the functions of the electronic device, and the present application does not make specific limitations thereon. The image processing method of the present application embodiment will be described in detail below taking the electronic device as an example.

[0188] FIG. 6 is a flowchart of an image processing method 600 provided by the present application. The method 600 can be executed by an electronic device, and the hardware structure of the electronic device can be as shown in FIG. 1. The method 600 includes the following steps:

[0189] S601, obtaining a first image and a second image, the first image being an HDR image, and the second image being an SDR image.

[0190] The first image can also be referred to as a preset HDR image, a preset high dynamic image or an alternate image, etc. The second image can also be referred to as a basic image. The first image and the second image can be images obtained in response to an operation of instructing shooting.

[0191] It should be understood that the manner in which the electronic device obtains the first image and the second image is similar to the manner in which the encoding module obtains the preset HDR image and the basic image in FIG. 3, and reference can be made to the description above, which will not be repeated here.

[0192] S602, determining a first gain range according to the first image and the second image.

[0193] The first gain range can be a range interval calculated based on the first image and the second image. The first gain range can include a maximum value and a minimum value, and the first gain range can be represented as [minimum value, maximum value], such as [0, 10], etc.

[0194] It can be understood that the first gain range is not a gain range of the gain information calculated according to the first image and the second image (i.e., the minimum value and the maximum value in the gain information), but a gain range additionally determined by the electronic device based on the first image and the second image. It should be understood that the first gain range can be different from the gain range of the gain information calculated according to the first image and the second image.

[0195] In an implementation manner, the reconstruction of the HDR image can be performed based on the first gain range. In this embodiment, compared with the reconstruction of the HDR image based on the gain range of the gain information calculated according to the first image and the second image, the reconstruction of the HDR image based on the first gain range can be implemented with higher image quality, that is, the image quality loss of the HDR image reconstructed based on the first gain range is lower. In other words, the first image is dynamically range expanded by using the gain in the first gain range, and the reconstructed HDR image obtained can be closer to the first image.

[0196] S603, determining second gain information according to the first gain information and the first gain range, the first gain information being calculated according to the first image and the second image.

[0197] It should be understood that the first gain information can also be referred to as a first gain map, and can include gains corresponding to each pixel point in the second image. Before S603, the method 600 can further include: calculating, by the electronic device, the first gain information based on the first image and the second image.

[0198] It should be understood that the manner in which the electronic device calculates the first gain information based on the first image and the second image is similar to the manner in which the encoding module calculates the gain information shown in FIG. 3, and reference can be made to the description above, which will not be repeated here.

[0199] It should also be understood that S601 to S603 can be performed by an encoding module in the electronic device, and the process of S601 to S603 can also be referred to as calculating gain information, which is not specifically limited in the present application.

[0200] S604, obtaining a selectable image according to the second gain information and the second image.

[0201] The selectable image is an image obtained by dynamically range expanding the second image by using the second gain information. The dynamic range expansion can also be referred to as dynamic range enhancement. Through the dynamic range expansion, the selectable image has more luminance levels and a wider color gamut.

[0202] The selectable image can also be understood as an image used to reconstruct the first image. The selectable image is a signal in the linear domain, and through operations such as non-linear conversion and / or rendering on the selectable image, an image that can be displayed on the screen of the electronic device (that is, the reconstructed HDR image in the foregoing) can be obtained. Since the selectable image is an image obtained through dynamic range expansion, it has a high dynamic range, so that the image displayed on the screen of the electronic device (obtained based on the selectable image) can present an effect close to the first image. Therefore, the selectable image can also be understood as an image used to reconstruct the first image.

[0203] It should be understood that S604 can be performed by a decoding module in the electronic device, and the process of S604 can also be referred to as applying gain information, which is not limited in the present application.

[0204] The image processing method of the present application, after obtaining the first gain information based on the obtained preset HDR image and the base image, additionally determines a first gain range for reducing image quality loss according to the first image and the second image, and adjusts the first gain information in combination with the first gain range to obtain the second gain information. In this way, the gain in the second gain information can be within the first gain range, and the base image is dynamically range expanded through the second gain information, so that the reconstructed HDR image obtained is closer to the first image, which helps to reduce image quality loss.

[0205] In the method 600, the first gain range can be determined in the following ways.

[0206] Way one, the first gain range is determined according to the linear space luminance of the first image and the linear space luminance of the second image.

[0207] On the basis of way one, the first gain range can be determined according to a dynamic range headroom level, and the dynamic range headroom level is calculated according to the linear space luminance of the first image and the linear space luminance of the second image.

[0208] Way two, the first gain range is determined according to a second gain range, and the second gain range is a gain range of the first gain information.

[0209] Way three, the first gain range is determined according to the first image, the second image and the first gain information.

[0210] On the basis of way three, the first gain range can be determined according to a dynamic range headroom level and a second gain range.

[0211] Way four, the first gain range is determined according to the first image, the second image and a bit width.

[0212] On the basis of the fourth mode, the first gain range can be determined according to the dynamic range headroom level and the bit width; determined according to the second gain range and the bit width; or determined according to the dynamic range headroom level, the second gain range, and the bit width.

[0213] The following will be described in detail.

[0214] The first gain range can be determined according to the linear space brightness of the first image and the linear space brightness of the second image.

[0215] In an example, the electronic device can be preconfigured with a first correspondence relationship. The first correspondence relationship includes a correspondence relationship between the first linear space brightness and the first gain range.

[0216] The first linear space brightness can be an average of two linear space brightnesses, i.e., an average of the linear space brightness of the first image and an average of the linear space brightness of the second image. The average of the linear space brightness is a ratio of a sum of linear space brightnesses of each pixel point to a total number of pixel points.

[0217] Alternatively, the first linear space brightness is a range of two linear space brightnesses, i.e., a first linear space brightness range and a second linear space brightness range. The linear space brightness range is a minimum value to a maximum value in the linear space brightness of each pixel point. The first linear space brightness range can cover the linear space brightness range of the first image, and the second linear space brightness range can cover the linear space brightness range of the second image. So that the electronic device can determine the first gain range according to the first correspondence relationship.

[0218] It can be understood that the first correspondence relationship can be preconfigured according to experience, which is not limited in the present application.

[0219] In another example, the electronic device can be preconfigured with a first model. The first model can be a neural network model. The electronic device can input the linear space brightness of the first image and the linear space brightness of the second image into the first model, and the first model can output the first gain range.

[0220] The first model can be a model learned by the first initial model through the following process. The first initial model inputs a preset image, which can include a basic image 1 and a preset HDR image 1; based on the linear space luminance of the basic image 1 and the linear space luminance of the preset HDR image 1, the first initial model can determine a gain range 1; the first initial model performs dynamic range expansion on the basic image 1 by using gain information with a gain range being the gain range 1, to obtain a reconstructed HDR image 1; the first initial model evaluates an image quality loss 1 of the reconstructed HDR image 1 relative to the preset HDR image 1; in the case where the image quality loss 1 does not satisfy a preset condition, the first initial model adjusts parameters used when calculating the gain range 1; a new gain range 1 is determined by using the adjusted parameters, and gain information with a gain range being the new gain range 1 is used to perform dynamic range expansion on the basic image 1, to obtain a new reconstructed HDR image 1; the first initial model evaluates an image quality loss 2 of the new reconstructed HDR image 1 relative to the preset HDR image 1, and in the case where the image quality loss 2 still does not satisfy the preset condition, the first initial model can continue to adjust the parameters used when calculating the new gain range 1, continue to determine an updated gain range 1, and repeat the above process. Until the first initial model evaluates an updated reconstructed HDR image 1 obtained based on the updated gain range 1 relative to the preset HDR image 1, the image quality loss satisfies the preset condition, and then the first model is obtained.

[0221] Through such a learning process, the electronic device can reconstruct a reconstructed HDR image closer to the first image by using the first gain range determined by the first model based on the selectable image. That is, the image quality loss of the reconstructed HDR image based on the selectable image is smaller.

[0222] It can be understood that the first initial model can also be trained based on basic images and preset HDR images obtained by shooting in different scenes, such as different lighting environments. In this way, the first initial model can learn the parameters used when calculating the corresponding gain range in different scenes. In this way, as the scenes corresponding to the first image and the second image are different, the first model can output the first gain range based on the parameters of the scenes corresponding to the first image and the second image. In this way, the accuracy of the first model can be further improved.

[0223] On the basis of the first mode, the first gain range can be determined according to a dynamic range headroom level.

[0224] Optionally, the dynamic range headroom level is calculated according to the linear space luminance of the first image and the linear space luminance of the second image; and the first gain range is determined according to the dynamic range headroom level.

[0225] It can be understood that the dynamic range headroom level can reflect an advantage of the first image over the second image in linear space luminance, since the gain information is used to perform dynamic range (related to luminance) expansion on the second image, so as to reconstruct an image with linear space luminance close to the first image by the second image and the gain information. Therefore, the electronic device determines the appropriate gain information in relation to the dynamic range headroom level.

[0226] In an example, the electronic device can be pre-provisioned with a second correspondence. The second correspondence includes a correspondence between the dynamic range headroom level and the first gain range; or, the second correspondence includes a correspondence between a dynamic range headroom level range and the first gain range, the dynamic range headroom level being in the dynamic range headroom level range. So that the electronic device can determine the first gain range according to the second correspondence.

[0227] It can be understood that the second correspondence can be pre-set according to experience, which is not specifically limited in the present application.

[0228] In another example, the electronic device can be pre-provisioned with a second model. The second model can be a neural network model. The electronic device can input the dynamic range headroom level into the second model, and the second model can output the first gain range.

[0229] The second model can be a model learned by the second initial model through the following process. The second initial model inputs a preset image, which can include the base image 1 and a preset HDR image 1; based on the linear space luminance of the base image 1 and the linear space luminance of the preset HDR image 1, the second initial model can calculate a dynamic range headroom level 1; based on the dynamic range headroom level 1, the second initial model can determine a gain range 1; the second initial model performs dynamic range expansion on the base image 1 by using gain information with a gain range being the gain range 1 to obtain a reconstructed HDR image 1; the second initial model evaluates an image quality loss 1 of the reconstructed HDR image 1 relative to the preset HDR image 1; in the case where the image quality loss does not satisfy a preset condition, the second initial model adjusts parameters used when calculating the gain range 1; using the adjusted parameters and the dynamic range headroom level 1, a new gain range 1 is calculated, and the base image 1 is dynamically expanded by using gain information with a gain range being the new gain range 1 to obtain a new reconstructed HDR image 1; the second initial model evaluates an image quality loss 2 of the new reconstructed HDR image 1 relative to the preset HDR image 1, and in the case where the image quality loss 2 still does not satisfy the preset condition, the second initial model can continue to adjust the parameters used when calculating the new gain range 1, continue to determine an updated gain range 1, and repeat the above process. Until the second initial model evaluates an updated reconstructed HDR image 1 obtained based on the updated gain range 1 relative to the preset HDR image 1, the image quality loss satisfies the preset condition, and then the second model is obtained.

[0230] Through such a learning process, the electronic device can reconstruct a reconstructed HDR image closer to the first image from the selectable image obtained by using the first gain range determined by the second model. That is, the image quality loss of the reconstructed HDR image obtained based on the selectable image is smaller.

[0231] It can be understood that the second initial model can also be trained based on base images and preset HDR images obtained under different scenes, such as different lighting environments. In this way, the second initial model can learn the parameters used when calculating the gain range corresponding to different scenes. In this way, as the scenes corresponding to the first image and the second image are different, the second model can output the first gain range based on the parameters of the scenes corresponding to the first image and the second image. In this way, the accuracy of the second model can be further improved.

[0232] On the basis of the above embodiment, the dynamic range headroom level can be calculated in the following manner.

[0233] In one possible manner, the dynamic range headroom level is a first ratio, and the first ratio is a ratio of the maximum linear space luminance of the first image and the maximum linear space luminance of the second image.

[0234] wherein the maximum linear-space luminance of the first image is a maximum value among the linear-space luminances of the first image; and the maximum linear-space luminance of the second image is a maximum value among the linear-space luminances of the second image.

[0235] Exemplarily, the dynamic range headroom level H satisfies the following formula:

[0236] wherein Max(Alternate) is the maximum linear-space luminance of the first image; and Max(Baseline) is the maximum linear-space luminance of the second image.

[0237] In this way, the dynamic range headroom level can reflect the advantage of the maximum linear-space luminance of the first image relative to the maximum linear-space luminance of the second image.

[0238] In another possible manner, the dynamic range headroom level is a second ratio, and the second ratio is a ratio of an average of the linear-space luminances of the first image to an average of the linear-space luminances of the second image.

[0239] In this way, the dynamic range headroom level can reflect the advantage of the average linear-space luminance of the first image relative to the average linear-space luminance of the second image.

[0240] It can be understood that the dynamic range headroom level can also be determined based on other parameters related to the linear-space luminances of the first image (for example, a minimum linear-space luminance of the first image) and other parameters related to the linear-space luminances of the second image (for example, a minimum linear-space luminance of the second image), which are not limited in the present application.

[0241] It should be noted that the dynamic range headroom level cannot be too large, otherwise it can cause the HDR image reconstructed by using the selectable image to have image quality loss. Therefore, in some scenarios, a preset dynamic range headroom level can be preconfigured in the electronic device, and the preset dynamic range headroom level can also be referred to as a preset headroom. Details are as follows.

[0242] Optionally, on the basis of the above embodiment, in a case where the first ratio is greater than a first preset headroom, the dynamic range headroom level is the first preset headroom; or in a case where the first ratio is greater than a second preset headroom, the dynamic range headroom level is the second preset headroom.

[0243] The first preset headroom (or the second preset headroom) can be a dynamic range headroom level matched with the electronic device display. For example, generally, the display brightness of the electronic device display is at most 2000 nits, and based on 2000 nits and the linear space brightness of the SDR image, the dynamic range headroom level can be calculated to be about 3.3. Therefore, the first preset headroom (or the second preset headroom) can be 3.3, and the like.

[0244] Alternatively, the first preset headroom (or the second preset headroom) can also be a dynamic range headroom level matched with the color space used when the electronic device renders the enhanced image and the bit width of the display. The color space used when the enhanced image is rendered can include RGBA1010102 / 8, scRGB-nl, and RGBA8888, for example.

[0245] The color space of RGBA1010102 / 8 uses 10 bits to represent the red, green, and blue channels, and 2 bits to represent the alpha channel, for a total of 32 bits. scRGB (standard RGB) is a high dynamic range color space, and "nl" represents non-linear. The color space of RGBA8888 uses 8 bits to represent the red, green, blue, and alpha channels, 8 bits for each channel, for a total of 32 bits.

[0246] In an implementation, when the color space is RGBA1010102 / 8 and the bit width of the display is 10-bit display, the first preset headroom (or the second preset headroom) can be set to 3.3.

[0247] When the color space is scRGB-nl and the bit width of the display is 10-bit display, the first preset headroom (or the second preset headroom) can be set to log2(7.59+0.6) = 3.0.

[0248] When the color space is RGBA8888 and the bit width of the display is 8-bit or 10-bit display, the first preset headroom (or the second preset headroom) can be set to 2.3 to present better color accuracy and less banding problems.

[0249] When the bit width of the display is 8-bit, the first preset headroom (or the second preset headroom) can be set to 2.3, for example.

[0250] In actual implementation, for different color spaces and different display bit widths, the first preset headroom can be set according to actual needs, and the embodiment does not limit the specific value of the first preset headroom, which can be selected according to actual needs.

[0251] It should be understood that the first preset headroom (or the second preset headroom) can be a value preset according to experience, which is not specifically limited in the present application.

[0252] The second manner, S602 can be implemented by: determining the first gain information according to the first image and the second image; determining the first gain range based on a second gain range of the first gain information.

[0253] The second gain range is the minimum value to the maximum value in the first gain information. The first gain range can be within the second gain range. For example, the second gain range is -0.2 to 18; the first gain range is 0 to 8, etc.

[0254] In an example, the electronic device can be pre-provisioned with a third correspondence. The third correspondence includes a correspondence between the second gain range and the first gain range.

[0255] It can be understood that the third correspondence can be preset according to experience, which is not specifically limited in the present application.

[0256] In another example, the first gain range can be calculated according to the second gain range, for example, the minimum value of the first gain range can be the sum of the minimum value of the second gain range and 2, the maximum value of the first gain range can be the difference between the maximum value of the second gain range and 3, etc.

[0257] In yet another example, the electronic device can be pre-provisioned with a third model. The third model can be a neural network model. The electronic device can input the second gain range into the third model, and the third model can output the first gain range.

[0258] The third model can be a model learned by the third initial model through the following process.

[0259] It should be understood that the training process of the third initial model is similar to that of the first initial model, except that the third initial model calculates the gain information 1 based on the basic image 1 and the preset HDR image 1, and calculates the gain range 1 based on the gain range of the gain information 1. In the case where the third initial model determines that the image quality loss 1 does not satisfy the preset condition, the third initial model adjusts the parameters used when calculating the gain range 1 (calculated based on the gain range of the gain information 1). Please refer to the description above, which will not be repeated here.

[0260] The third manner, S602 can be implemented by: determining the first gain range according to the first image, the second image and the first gain information.

[0261] Exemplarily, the fourth model can be pre-stored in the electronic device. The fourth model can be a neural network model. The electronic device can input the first image, the second image, and the first gain information into the fourth model, and the fourth model can output the first gain range.

[0262] The fourth model can be a model learned by the fourth initial model through the following process.

[0263] It should be understood that the training process of the fourth initial model is similar to that of the first initial model, except that the fourth initial model calculates the gain information 1 based on the basic image 1 and the preset HDR image 1, and calculates the gain range 1 based on the basic image 1, the preset HDR image 1, and the gain information 1. In the case where the fourth initial model determines that the image quality loss 1 does not satisfy the preset condition, the fourth initial model adjusts the parameters used when calculating the gain range 1 (calculated based on the basic image 1, the preset HDR image 1, and the gain information 1). For details, please refer to the foregoing description.

[0264] Based on the third mode, S602 can be implemented in the following manner: determining the first gain range according to the dynamic range headroom level and the second gain range, the second gain range being the gain range of the first gain information, and the dynamic range headroom level being calculated according to the first image and the second image.

[0265] It should be understood that the dynamic range headroom level can be understood with reference to the foregoing description, which will not be repeated here.

[0266] In an example, the electronic device can pre-store a fifth correspondence relationship, which can include a correspondence relationship between the dynamic range headroom level, the second gain range, and the first gain range; or the fifth correspondence relationship can include a correspondence relationship between the dynamic range headroom level range, the second gain range, and the first gain range; or the fifth correspondence relationship can include a correspondence relationship between the dynamic range headroom level range, the first range, and the first gain range. The dynamic range headroom level is in the dynamic range headroom level range. The minimum value of the second gain range belongs to the first range, and the maximum value of the second gain range belongs to the first range.

[0267] For example, as shown in Table 1, the fifth correspondence includes the correspondence between the dynamic range net level range, the first range, and the first gain range. The dynamic range net level range is [2,3]; the first range may include a first sub-range and a second sub-range, the first sub-range being [-2,2] and the second sub-range being [15,20]; the first gain range is [0,8]. Thus, if the dynamic range net level calculated by the electronic device based on the first and second images is 2.4, that is, the dynamic range net level belongs to [2,3]; the minimum value in the second gain range calculated by the electronic device is -1.8, belonging to the first sub-range [-2,2], and the maximum value in the second gain range calculated by the electronic device is 19, belonging to the first sub-range [15,20], then the electronic device can determine that the first gain range corresponding to the dynamic range net level and the second gain range is [0,8].

[0268] Table 1

[0269] It is understandable that the fifth correspondence can be pre-set based on experience, and this application does not make any specific limitations on it.

[0270] In another example, the electronic device may have a pre-configured fifth model. This fifth model could be a neural network model. The electronic device can input the dynamic range headroom level and the second gain range into the fifth model, and the fifth model can output the first gain range.

[0271] The fifth model can be a model learned from the fifth initial model through the following process.

[0272] It should be understood that the training process of the fifth initial model is similar to that of the first initial model. The difference lies in that the fifth initial model calculates gain information 1 and dynamic range clearance level 1 based on the base image 1 and the preset HDR image 1; the fifth initial model calculates the gain range 1 based on the gain range of gain information 1 and the dynamic range clearance level 1. If the fifth initial model subsequently determines that the image quality loss 1 does not meet the preset conditions, it adjusts the parameters used when calculating the gain range 1 (based on the gain range of gain information 1 and the dynamic range clearance level 1). Refer to the description above; it will not be repeated here.

[0273] Method 4, S602 can be implemented in the following way: determine the first gain range based on the first image, the second image, and the bit width.

[0274] It is understandable that bit width can affect the color depth, detail representation, and overall visual quality of an image; therefore, different bit widths can correspond to different gain ranges.

[0275] Exemplarily, the electronic device can pre-store a sixth model. The sixth model can be a neural network model. The electronic device can input the first image, the second image, and the first gain information into the sixth model, and the sixth model can output the first gain range.

[0276] The sixth model can be a model learned by the sixth initial model through the following process.

[0277] It should be understood that the training process of the sixth initial model is similar to that of the first initial model, except that the sixth initial model calculates the gain range 1 based on the basic image 1, the preset HDR image 1, and the bit width 1. In the case where the subsequent sixth initial model determines that the image quality loss 1 does not satisfy the preset condition, the sixth initial model adjusts the parameters used when calculating the gain range 1 (calculated based on the basic image 1, the preset HDR image 1, and the bit width 1). For details, please refer to the foregoing description.

[0278] Based on the fourth mode, the first gain range can be determined according to the dynamic range headroom level and the bit width.

[0279] In an example, the electronic device can pre-store a seventh correspondence relationship. The seventh correspondence relationship can include a correspondence relationship between the dynamic range headroom level, the bit width, and the first gain range; or the seventh correspondence relationship can include a correspondence relationship between the dynamic range headroom level range, the bit width, and the first gain range. The dynamic range headroom level is in the dynamic range headroom level range.

[0280] Exemplarily, as shown in Table 2, the seventh correspondence relationship includes a correspondence relationship between the dynamic range headroom level range, the bit width, and the first gain range. The dynamic range headroom level range is [2, 3]; the bit width can be 8 bits; and the first gain range is [0, 8]. In this way, if the dynamic range headroom level calculated by the electronic device based on the first image and the second image is 2.4, that is, the dynamic range headroom level belongs to [2, 3], and the bit width of the electronic device is 8 bits, the electronic device can determine that the first gain range corresponding to the dynamic range headroom level and the second gain range is [0, 8].

[0281] Table 2

[0282] It can be understood that the seventh correspondence relationship can be pre-set according to experience, which is not limited in the present application.

[0283] In another example, the electronic device can pre-store a seventh model. The seventh model can be a neural network model. The electronic device can input the dynamic range headroom level and the bit width into the seventh model, and the seventh model can output the first gain range.

[0284] The seventh model can be a model learned by the seventh initial model through the following process.

[0285] It should be understood that the training process of the seventh initial model is similar to that of the first initial model, except that the seventh initial model calculates the dynamic range headroom level 1 based on the base image 1 and the preset HDR image 1, and calculates the gain range 1 based on the bit width 1 and the dynamic range headroom level 1. In the case where the seventh initial model determines that the image quality loss 1 does not satisfy the preset condition, the seventh initial model adjusts the parameters used in calculating the gain range 1 (calculated based on the bit width 1 and the dynamic range headroom level 1). For details, please refer to the foregoing description.

[0286] On the basis of the fourth mode, the first gain range can also be determined according to the second gain range and the bit width.

[0287] In an example, the electronic device can be preconfigured with an eighth correspondence relationship. The eighth correspondence relationship can include a correspondence relationship between the bit width, the second gain range, and the first gain range. Alternatively, the eighth correspondence relationship can include a correspondence relationship between the bit width, the first range, and the first gain range. The minimum value of the second gain range belongs to the first range, and the maximum value of the second gain range belongs to the first range.

[0288] For example, as shown in Table 3, the eighth correspondence relationship includes a correspondence relationship between the bit width, the first range, and the first gain range. The bit width is 8 bits; the first range can include a first sub-range and a second sub-range, the first sub-range is [-2, 2], and the second sub-range is [15, 20]; and the first gain range is [0, 8]. In this way, if the bit width of the electronic device is 8 bits, the minimum value of the second gain range calculated by the electronic device is -1.8, which belongs to the first sub-range [-2, 2], and the maximum value of the second gain range calculated by the electronic device is 19, which belongs to the first sub-range [15, 20], the electronic device can determine that the first gain range corresponding to the dynamic range headroom level and the second gain range is [0, 8].

[0289] Table 3

[0290] It can be understood that the eighth correspondence relationship can be preconfigured according to experience, which is not limited in the present application.

[0291] In another example, the electronic device can be preconfigured with an eighth model. The eighth model can be a neural network model. The electronic device can input the bit width and the second gain range into the eighth model, and the eighth model can output the first gain range.

[0292] The eighth model can be a model learned by the eighth initial model through the following process.

[0293] It should be understood that the training process of the eighth initial model is similar to that of the first initial model, except that the fourth initial model is based on the base image 1 and the preset HDR image 1 to calculate the gain information 1, and the fourth initial model is based on the gain range of the gain information 1 and the bit width 1 to calculate the gain range 1. In the case where the eighth initial model judges that the image quality loss 1 does not satisfy the preset condition, the eighth initial model adjusts the parameters used when calculating the gain range 1 (calculated based on the gain range of the gain information 1 and the bit width 1). For details, please refer to the foregoing description.

[0294] On the basis of the fourth mode, the first gain range can also be determined according to the dynamic range headroom level, the second gain range, and the bit width.

[0295] In an example, the electronic device can be preconfigured with a ninth correspondence relationship, which can include a correspondence relationship between the dynamic range headroom level, the bit width, the second gain range, and the first gain range; or the ninth correspondence relationship can include a correspondence relationship between the dynamic range headroom level range, the bit width, the first range, and the first gain range. The minimum value of the second gain range belongs to the first range, and the maximum value of the second gain range belongs to the first range. The dynamic range headroom level belongs to the dynamic range headroom level range.

[0296] For example, as shown in Table 4, the ninth correspondence relationship includes a correspondence relationship between the dynamic range headroom level range, the bit width, the first range, and the first gain range. The dynamic range headroom level range is [2, 3], and the bit width is 8 bits; the first range can include a first sub-range and a second sub-range, the first sub-range is [-2, 2], and the second sub-range is [15, 20]; and the first gain range is [0, 8]. In this way, if the dynamic range headroom level calculated by the electronic device based on the first image and the second image is 2.4, that is, the dynamic range headroom level belongs to [2, 3], the bit width is 8 bits, the minimum value of the second gain range calculated by the electronic device is -1.8, which belongs to the first sub-range [-2, 2], and the maximum value of the second gain range calculated by the electronic device is 19, which belongs to the first sub-range [15, 20], the electronic device can determine that the first gain range corresponding to the dynamic range headroom level and the second gain range is [0, 8].

[0297] Table 4

[0298] It can be understood that the ninth correspondence relationship can be preconfigured according to experience, which is not limited in the present application.

[0299] In another example, the electronic device can be pre-installed with a ninth model. The ninth model can be a neural network model. The electronic device can input the bit width and the second gain range into the ninth model, and the ninth model can output the first gain range.

[0300] The ninth model can be a model learned by the ninth initial model through the following process.

[0301] It should be understood that the training process of the ninth initial model is similar to that of the first initial model, except that the fourth initial model calculates the gain information 1 and the dynamic range headroom level 1 based on the base image 1 and the preset HDR image 1, and calculates the gain range 1 based on the gain range of the gain information 1, the dynamic range headroom level 1, and the bit width 1. In the case where the ninth initial model determines that the image quality loss 1 does not satisfy the preset condition, the ninth initial model adjusts the parameters used when calculating the gain range 1 (calculated based on the gain range of the gain information 1, the dynamic range headroom level 1, and the bit width 1). For details, please refer to the description above.

[0302] It can be understood that in the embodiments of the present application, the first initial model to the ninth initial model can refer to the way of calculating the gain information 1 shown in FIG. 3, and the way of calculating the dynamic range headroom level 1 based on the base image 1 and the preset HDR image 1 can refer to the way of calculating the dynamic range headroom level described above, which will not be repeated here.

[0303] Based on the determination of the first gain range, the following describes the way of determining the second gain information based on the first gain range (an embodiment of S603).

[0304] As an optional embodiment, the first gain range includes a maximum value and a minimum value; S603 can be implemented by the following way: setting the gain less than the minimum value in the first gain information to the minimum value, and setting the gain greater than the maximum value in the first gain information to the maximum value.

[0305] The gain can be understood as the gain coefficient corresponding to each pixel point in the gain information, which can also be referred to as a scaling factor or a gain value, etc. The first gain information is calculated based on the first image and the second image, and the gain range of the first gain information is the second gain range.

[0306] The first gain range can be smaller than the second gain range, i.e., a maximum value in the first gain range can be smaller than or equal to a maximum value in the second gain range; a minimum value in the first gain range can be greater than or equal to a minimum value in the second gain range. Therefore, the first gain information can include one or more gains greater than the maximum value in the first gain range; and can also include one or more gains smaller than the minimum value in the first gain range.

[0307] It should be understood that metadata is data describing key information and features required in the process of image (or video signal) processing, and it can be understood that the metadata is used to provide information required for correct decoding and display of the image (or video signal). The metadata of the first gain information can include the gain range of the first gain information, i.e., the second gain range; and the metadata of the second gain information can include the gain range of the second gain information, i.e., the first gain range.

[0308] In combination with FIG. 7, (a) in FIG. 7 shows the first gain information and the metadata of the first gain information. The metadata of the first gain information can include the gain range of the first gain information, i.e., the second gain range. The second gain range is [-5, 15], i.e., the minimum gain in the first gain information is -5 and the maximum gain is 15. Assuming that the first gain range calculated by the electronic device is [0, 10], the electronic device sets the gain smaller than 0 in the first gain information to 0 and sets the gain greater than 10 to 10, thereby obtaining the second gain information. The second gain information is shown in (b) in FIG. 7. Correspondingly, the electronic device writes the first gain range into the metadata of the second gain information, so that the metadata of the second gain information includes 0 and 10.

[0309] The above describes various possible implementation manners of determining the first gain range. Because the subsequent reconstruction of the HDR image needs to be based on the first gain range, in an implementation of the present application, after the first gain range is determined, the electronic device can also verify whether the first gain range can meet the image quality loss requirement of the reconstruction of the HDR image through a certain process, wherein the image quality loss requirement can be understood as lossless reconstruction or reconstruction with a lower image quality loss.

[0310] Meanwhile, when the first gain range fails to meet the image quality loss requirement of the reconstruction of the HDR image, further adjustment can be made to the first gain range so that the finally determined first gain range can meet the image quality loss requirement of the reconstruction of the HDR image.

[0311] In order to distinguish the gain range before and after the adjustment, the gain range before the adjustment (which can also be understood as the initially determined first gain range) is referred to as a third gain range in this embodiment, and the finally determined gain range satisfying the image quality loss requirement is referred to as a first gain range. It should be understood that the third gain range is determined according to the first image and the second image, and the specific determination manner can refer to any one of the determination manners of the first gain range described above, which will not be described here again.

[0312] The verification process will be described below in combination with FIG. 8, which is a flowchart for determining image contrast provided by an embodiment of the present application.

[0313] As shown in FIG. 8, because the effect of HDR image reconstruction based on the third gain range is to be verified at present, the gain range corresponding to the third gain information is the third gain range determined above in the verification stage, that is, the gain values of each pixel point in the first gain information all belong to the third gain range, so that the image reconstruction effect of the third gain range can be effectively verified.

[0314] It can be understood that the third gain information is calculated according to the third gain range and the first gain information. The manner of calculating the third gain information can refer to the manner of calculating the second gain information described above, which will not be described here again.

[0315] For example, the minimum value and the maximum value are included in the third gain range, for example, the gain values less than the minimum value in the first gain information can be set to the minimum value, and the gain values greater than the maximum value in the first gain information can be set to the maximum value, so as to obtain the third gain information corresponding to the third gain range.

[0316] Then, for example, the first enhanced image can be determined according to the second image and the third gain information. The implementation of determining the first enhanced image according to the second image and the third gain information can refer to the implementation of determining the enhanced image according to the gain information and the basic image described in the above embodiment of FIG. 3, which will not be described here again.

[0317] In this embodiment, the first enhanced image can be understood as the HDR image reconstructed based on the third gain range, and the first image is the original HDR image, that is, the target of reconstruction. Therefore, the contrast of each pixel point in the first enhanced image and the first image can be further determined according to the linear space luminance L' of each pixel point in the first enhanced image and the linear space luminance L of each pixel point in the first image, wherein the contrast can reflect the luminance contrast of the pixel point in the first enhanced image compared with the pixel point in the first image, and then the corresponding contrast can effectively reflect the image quality loss of the first enhanced image after reconstruction compared with the first image as the target.

[0318] In actual implementation, there are various possible implementation manners for determining the contrast, and the following describes several possible implementation manners for determining the contrast. It can be understood that the linear space luminance and the contrast mentioned in the following description are data of a pixel point, which can be understood as the linear space luminance of any pixel point and the contrast of any pixel point.

[0319] In an implementation manner, the contrast can be a first difference value, and the first difference value is a difference value between the linear space luminance of the first image and the linear space luminance of the first enhanced image. For example, it can be expressed by a formula as ΔL=L`-L, where ΔL is the first difference value.

[0320] Alternatively, the contrast can also be a ratio of the first difference value to the linear space luminance of the first image. For example, it can be expressed by a formula as

[0321] Alternatively, the contrast can also be a ratio of the first difference value to the linear space luminance of the first enhanced image. For example, it can be expressed by a formula as

[0322] Alternatively, the contrast can also be a ratio of the first difference value to a first sum, and the first sum is a sum of the linear space luminance of the first image and the linear space luminance of the first enhanced image. For example, it can be expressed by a formula as

[0323] In actual implementation, the specific determination manner of the contrast can be selected and extended according to actual needs, as long as the contrast is determined according to the linear space luminance of the first image and the linear space luminance of the first enhanced image, and the contrast can reflect the luminance contrast of the pixel point in the first enhanced image compared with the pixel point in the first image.

[0324] After the contrast is determined, the electronic device also needs to measure whether the current image quality loss meets the requirement according to the contrast of each pixel point, and the requirement for the image quality loss is usually to require lossless reconstruction or to realize image reconstruction with small image quality loss.

[0325] For example, the contrast of the pixel point and the corresponding contrast sensitivity threshold can be compared, when the contrast of the pixel point meets the corresponding contrast sensitivity threshold, it can be determined that the luminance loss corresponding to the pixel point is small, and the human eye generally cannot observe that there is image quality loss at the position. When the contrast of the pixel point does not meet the corresponding contrast sensitivity threshold, it can be determined that the luminance loss corresponding to the pixel point is large, and the human eye can observe that there is image quality loss at the position.

[0326] In an implementation, for example, the proportion of the first pixel points satisfying the contrast sensitivity threshold in the second pixel points can be counted, that is, the ratio of the number of the first pixel points to the number of the second pixel points can be determined. The second pixel points can be pixel points in the first gain information whose gain values are in the current gain range (i.e., the third gain range) of the first gain information, that is, the second pixel points can be understood as all the pixel points in the first gain information, and the proportion currently counted is the proportion of the first pixel points in the entire pixel points in the first gain information.

[0327] The ratio determined above can be compared with the first threshold value, so as to measure the reconstruction loss of the first enhanced image. For example, in the case where the ratio is less than or equal to the first threshold value, it can be determined that the proportion of the first pixel points satisfying the contrast sensitivity threshold does not reach the preset requirement, and thus it can be determined that the reconstruction loss of the first enhanced image is relatively large and does not meet the expected image quality loss requirement. Or, in the case where the ratio is greater than the first threshold value, it can be determined that the proportion of the first pixel points satisfying the contrast sensitivity threshold has reached the preset requirement, and thus it can be determined that the reconstruction loss of the first enhanced image is relatively small and can be considered to have reached the expected image quality loss requirement.

[0328] The first threshold value can be set to 80%, 95%, etc., which can be set according to the loss requirement of image reconstruction in actual implementation, and the specific value of the first threshold value is not limited in the embodiment.

[0329] Or in another implementation, the number of the first pixel points satisfying the contrast sensitivity threshold can also be counted, and the number of the first pixel points is compared with the second threshold value, so as to measure the reconstruction loss of the first enhanced image.

[0330] For example, in the case where the number of the first pixel points is less than or equal to the second threshold value, it can be determined that the number of the first pixel points satisfying the contrast sensitivity threshold does not reach the preset requirement, and thus it can be determined that the reconstruction loss of the first enhanced image is relatively large and does not meet the expected image quality loss requirement. Or, in the case where the number of the first pixel points is greater than the second threshold value, it can be determined that the number of the first pixel points satisfying the contrast sensitivity threshold has reached the preset requirement, and thus it can be determined that the reconstruction loss of the first enhanced image is relatively small and can be considered to have reached the expected image quality loss requirement.

[0331] The second threshold value can be set according to the loss requirement of image reconstruction in actual implementation, and the specific value of the second threshold value is not limited in the embodiment.

[0332] When the reconstruction loss of the first enhanced image is small, it can be determined that the current HDR image reconstructed based on the third gain range can meet the image quality loss requirement of the reconstructed HDR image, and thus it can be determined that the current third gain range is the final first gain range.

[0333] However, when the reconstruction loss of the first enhanced image is large, it indicates that the current HDR image reconstructed based on the third gain range cannot meet the image quality loss requirement of the reconstructed HDR image, and thus the third gain range needs to be further adjusted until the HDR image reconstructed based on the third gain range can meet the image quality loss requirement of the reconstructed image.

[0334] That is, when the ratio of the number of the first pixel points to the number of the second pixel points is less than or equal to the first threshold value, or the number of the first pixel points is less than or equal to the second threshold value, the third gain range is adjusted.

[0335] Here, the above process can be understood with reference to FIG. 9, which is a schematic diagram for determining the first pixel points provided by an embodiment of the present application.

[0336] As shown in FIG. 9, it is assumed that the first image and the first enhanced image each include 9x8 pixel points, which are one-to-one corresponding, and then the respective contrast of the 9x8 pixel points can be determined, and then the contrast of the pixel points is compared with the respective contrast sensitivity threshold value, so as to determine the pixel points (i.e., the first pixel points, which are filled with white color in the figure) that meet the contrast sensitivity threshold value and the pixel points (which are filled with gray color in the figure) that do not meet the contrast sensitivity threshold value.

[0337] Then, referring to the example of FIG. 9, it can be determined that the number of the first pixel points that meet the contrast sensitivity threshold value is 67, and the number of the second pixel points whose gain values are in the current gain range (i.e., the third gain range) of the first gain information is 72, which is actually the total number of the pixel points.

[0338] Then, in an implementation manner, the ratio of the number of the first pixel points 67 to the number of the second pixel points 72 can be determined as 93%, and then 93% is compared with the first threshold value. It is assumed that the first threshold value is set as 90%, and thus the ratio is greater than the first threshold value, and thus it can be determined that the proportion of the first pixel points has met the preset requirement, and thus it can be determined that the third gain range is the final first gain range. Or it is assumed that the first threshold value is set as 95%, and thus the ratio is less than the first threshold value, and thus it can be determined that the proportion of the first pixel points has not met the preset requirement, and thus the third gain range needs to be further adjusted.

[0339] Or in another implementation, the number of first pixel points can be compared with a second threshold. Assuming that the second threshold is set to 65, the number of first pixel points is greater than the second threshold, so it can be determined that the number of first pixel points has reached the preset requirement, and therefore it can be determined that the third gain range is the first gain range for final use. Or assuming that the second threshold is set to 70, the number of first pixel points is less than 70, so it can be determined that the number of first pixel points has not reached the preset requirement, and the third gain range needs to be further adjusted.

[0340] Exemplarily, the implementation of adjusting the third gain range can be, for example, reducing the third gain range, so as to obtain the adjusted third gain range.

[0341] In an implementation, reducing the third gain range can be reducing the maximum value in the third gain range by a first step, so as to obtain the first gain range; and / or increasing the minimum value in the third gain range by a second step, so as to obtain the first gain range. The first step and the second step can both be preset values, and the specific values can be selected according to actual needs, which are not limited in the embodiment.

[0342] Exemplarily, assuming that the third gain range is [0, 10], and assuming that the first step is 2 and the second step is 1, after reducing the third gain range once, the adjusted gain range can be [1, 8].

[0343] After adjusting the third gain range once, the verification steps introduced above can be repeatedly performed. If it is determined that the adjusted third gain range meets the loss requirement of the reconstructed image, it can be determined that the adjusted third gain range can be used as the first gain range for final use. If the adjusted third gain range still does not meet the loss requirement of the reconstructed image, the adjusted third gain range can be further reduced until the reduced third gain range meets the loss requirement of the reconstructed image, so as to obtain the first gain range for final use.

[0344] In this implementation, by verifying whether the third gain range meets the loss requirement of the reconstructed image, if it is determined that it meets the requirement, it is determined that the third gain range is the first gain range for final use. If it does not meet the requirement, the third gain range can be further adjusted until the adjusted third gain range can meet the loss requirement of the reconstructed image, so as to obtain the first gain range for final use. Therefore, based on the first gain range, the HDR image reconstruction can be realized with low image quality loss, so as to effectively improve the image quality of the final display.

[0345] Based on the above description, it can be determined that after determining the contrast of each pixel point, the contrast of the pixel point and the corresponding contrast sensitivity threshold need to be compared. The implementation of the contrast sensitivity threshold will be further described below.

[0346] In an implementation manner, for example, a fixed contrast sensitivity threshold can be uniformly set for each pixel point, and then the contrast of each pixel point is compared with the fixed contrast sensitivity threshold, so as to determine whether the pixel point meets the corresponding contrast sensitivity threshold. For example, the contrast sensitivity threshold can be a specific value, and when the contrast of the pixel point is less than or equal to the contrast sensitivity threshold, it can be considered that the pixel point meets the contrast sensitivity threshold. Alternatively, the contrast sensitivity threshold can be a range, and when the contrast of the pixel point belongs to the range of the contrast sensitivity threshold, it can be considered that the pixel point meets the contrast sensitivity threshold. It depends on the specific setting manner of the contrast sensitivity threshold, and the embodiment does not limit this.

[0347] Alternatively, the contrast sensitivity threshold corresponding to each pixel point can also be determined in the contrast sensitivity function according to the linear space brightness corresponding to each pixel point in the first image.

[0348] For example, the contrast sensitivity function can be understood with reference to FIG. 10, which is a curve diagram of the contrast sensitivity function provided by the embodiment of the application.

[0349] In FIG. 10, two possible contrast sensitivity functions are shown. Specifically, the contrast sensitivity function can be represented as the curve shown in FIG. 10. In the diagram of FIG. 10, the horizontal axis is the brightness, and the vertical axis is the minimum detectable contrast. Corresponding to the present scheme, the value on the horizontal axis corresponding to the contrast sensitivity function can be understood as the linear space brightness of the pixel point in the first image, and the value on the vertical axis corresponding to the contrast sensitivity function can be understood as the contrast sensitivity threshold.

[0350] Then for any pixel point, the value on the horizontal axis can be determined according to the linear space brightness of the pixel point in the first image, and then the value on the vertical axis corresponding to the curve is determined according to the value on the horizontal axis, so as to obtain the contrast sensitivity threshold corresponding to the pixel point. In other words, the part above the curve can be considered as not meeting the contrast sensitivity function, and the part below the curve is considered as meeting the contrast sensitivity function.

[0351] In an implementation, the upper curve in FIG. 10 can be understood as a function curve of the schreiber fusion model, that is, the contrast sensitivity function in the embodiment can be a function of the schreiber fusion model. The lower curve in FIG. 10 can be understood as a function curve of the Barton model, that is, the contrast sensitivity function in the embodiment can be a function of the Barton model.

[0352] Alternatively, the curves shown in FIG. 10 can also be understood as curves obtained by fusing the function of the schreiber fusion model and the function of the Barton model. The embodiment does not limit the specific implementation of the contrast sensitivity function, which can be set arbitrarily according to actual needs, as long as the contrast sensitivity function can indicate the relationship between the linear space luminance of the pixel point in the first image and the contrast sensitivity threshold corresponding to the pixel point.

[0353] In an implementation, the contrast sensitivity function (CSF) can be as follows:

[0354] M opt (μ), E, σ, and d each have their own mathematical expressions, for example, can be as follows:

[0355] 1) M opt (μ) is the optical transfer function MTK of the human eye.

[0356] 2) E represents the retinal illuminance measured in Td (Trolands).

[0357] 3)

[0358] 4) d = 5 - 3 x tanh (0.4 x log x (L x X0 2 / 40 2 )).

[0359] In the above-mentioned multiple formulas, the meanings of the respective parameters are as follows:

[0360] m t represents the modulation depth threshold (also referred to as the modulation threshold). μ represents the spatial frequency, that is, the number of cycles per degree of visual angle. σ represents the standard deviation of the line spread function generated by the convolution of different elements in the convolution process. k represents the signal-to-noise ratio (SNR). T represents the integration time (or integration time) of the eye, measured in seconds. X0 represents the angular size of the object in the x direction, which can also be referred to as the angular size of the object. X maxdenotes the maximum angular size of the integration area in the x-direction in degrees. N max denotes the maximum number of cycles that the eye can integrate information. η denotes the quantum efficiency of the eye. p denotes a photon conversion factor p that varies with the light source. μ0 denotes the highest spatial frequency that preserves lateral inhibition. φ0 denotes the spectral density of neural noise.

[0361] The above-described various parameters may, for example, be Arraylike.

[0362] The manner of determining the first gain range and calculating the second gain information according to the first gain range is described above.

[0363] In some possible implementation manners, after the second gain information is determined, the method 600 further includes: the electronic device performing normalization processing on the second gain information to obtain normalized second gain information; correspondingly, S604 can be implemented by the following manner: obtaining the selectable image according to the normalized second gain information and the second image.

[0364] The normalization processing on the second gain information is implemented based on the first gain range and the second gain information. For example, the electronic device can perform the normalization processing on the second gain information by the following manner.

[0365] For the gain of each pixel point in the second gain information, for example, the gain (G) of the jth pixel point, the normalization processing can be performed by the following formula:

[0366] wherein G Normalized is the normalized gain of the jth pixel point, Min(G) is the minimum value in the first gain range, and Max(G) is the maximum value in the first gain range.

[0367] The value range of j can be 1 to m, and m is the number of pixel points in the second gain information. In this way, when the jth pixel point in the second gain information traverses all the pixel points in the second gain information, the normalized second gain information can be obtained.

[0368] It can be understood that the calculation of the normalized second gain information can be performed by the encoding module, and the calculation of the normalized second gain information also belongs to the process of calculating the gain information. In this way, the image file transmitted by the encoding module to the decoding module can include the normalized second gain information, the metadata of the second gain information, and the second image. Therefore, the data amount of the image file transmitted by the encoding module to the decoding module is small.

[0369] Correspondingly, obtaining the selectable image according to the normalized second gain information and the second image can be performed by the decoding module, and the process belongs to the process of applying the gain information. Specifically, the decoding module can first perform a denormalization process on the normalized second gain information to obtain denormalized second gain information. The denormalized second gain information can be the same as the second gain information; then, the decoding module obtains the selectable image based on the denormalized second gain information and the second image.

[0370] It should be understood that the denormalization process can be the inverse process of the normalization process, and the normalization process of the second gain information can be referred to the above description, which will not be repeated here.

[0371] The process of obtaining the selectable image in the method 600 will be further described below in combination with FIG. 11.

[0372] FIG. 11 is a schematic diagram of an image processing process 900 provided by an embodiment of the present application. The process 900 includes two sub-processes of calculating gain information and applying gain information. The calculating gain information can be performed by the encoding module of the electronic device, and the applying gain information can be performed by the decoding module of the electronic device.

[0373] For the process of calculating the gain information: the encoding module obtains the first image (the preset HDR image) and the second image (the basic image). The encoding module calculates the linear space brightness of the first image, and calculates the linear space brightness of the second image. Then, based on the linear space brightness of the first image, the encoding module calculates the maximum linear space brightness of the first image, and based on the linear space brightness of the second image, the encoding module calculates the maximum linear space brightness of the second image. The encoding module calculates the dynamic range headroom level according to the maximum linear space brightness of the first image and the maximum linear space brightness of the second image, and calculates the first gain information based on the linear space brightness of the first image and the linear space brightness of the second image. Then, the encoding module determines the first gain range based on the dynamic range headroom level, the bit width, and the gain range of the first gain information (i.e., the second gain range). The encoding module calculates the second gain information based on the first gain range and the first gain information, and the gain range of the second gain information is the first gain range. Therefore, the encoding module also writes the first gain range into the metadata of the second gain information. Then, the encoding module performs a normalization process on the second gain information to obtain normalized second gain information. The encoding module transmits the normalized second gain information, the metadata of the second gain information, and the second image to the applying gain information. Exemplarily, the encoding module can transmit an image file to the decoding module, and the image file includes the second gain information, the metadata of the second gain information, and the second image.

[0374] For the process of applying the gain information: the decoding module obtains the image file, denormalizes the normalized second gain information to obtain denormalized second gain information. Then, the second image is dynamically range expanded using the denormalized second gain information to obtain the selectable image.

[0375] It can be understood that, in the embodiment of the present application, after the electronic device calculates the selectable image, the selectable image can be subjected to non-linear conversion to obtain an enhanced image, which can be understood as a reconstructed HDR image, i.e., the reconstructed first image. The closer the image is to the first image, the lower the loss of image quality of the reconstructed HDR image.

[0376] It should be noted that, in the embodiment of the present application, in the process of calculating the gain information, the electronic device includes a plurality of steps in the encoding module and the decoding module described above, but the steps described in the above embodiment do not constitute an absolute limitation on the steps specifically included in the encoding module and the decoding module. That is, between any two operation steps in the encoding module and the decoding module described above, any intermediate step can be added according to actual needs, as long as the ultimate purpose of reconstructing the HDR image can be achieved.

[0377] In addition, each step included in the encoding module and the decoding module is not explicitly explained in the above embodiment, and can be understood with reference to related implementations, for example, the understanding of determining the linear space brightness of the image described above can be understood according to the current related implementation.

[0378] It should be noted that the module names involved in the embodiments of the present application can be defined as other names, as long as the functions of the modules can be achieved, and the names of the modules are not specifically limited.

[0379] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0380] The image processing method of the embodiment of the present application has been described above, and the device provided by the embodiment of the present application for executing the above method will be described below. Those skilled in the art can understand that the method and the device can be combined and referenced, and the related device provided by the embodiment of the present application can execute the steps in the above list sorting method.

[0381] FIG. 12 is a schematic block diagram of an image processing apparatus 1200 according to an embodiment of the present application. The apparatus 1200 includes a processor 1201, a communication interface 1202, and a memory 1203. The processor 1201, the communication interface 1202, and the memory 1203 communicate with each other through an internal connection path. The memory 1203 is configured to store instructions, and the processor 1201 is configured to execute the instructions stored in the memory 1203. The communication interface 1202 can be configured to send signals to other apparatuses (for example, the processor 1201 or a touch screen of an electronic device) and receive signals from other apparatuses (for example, the memory 1203). For example, the communication interface 1202 reads the instructions stored in the memory 1203 and sends the instructions to the processor 1201.

[0382] It should be understood that the apparatus 1200 can be specifically an electronic device in the above-described embodiments, and can be configured to perform each step and / or process corresponding to the electronic device in the above-described method embodiments. Alternatively, the memory 1203 can include a read-only memory and a random access memory, and provide instructions and data for the processor. A part of the memory can also include a non-volatile random access memory. For example, the memory can also store device type information. The processor 1201 can be configured to execute the instructions stored in the memory, and when the processor 1201 executes the instructions stored in the memory, the processor 1201 is configured to perform each step and / or process of the above-described method embodiments.

[0383] It should be understood that in the embodiments of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0384] In the implementation process, each step of the above-described method can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software. The steps of the method disclosed in the embodiments of the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, or other mature storage media in the art. The storage medium is located in the memory, and the processor executes the instructions in the memory to combine the hardware to complete the steps of the above-described method. To avoid repetition, it will not be described in detail here.

[0385] The image processing method provided by the embodiments of the present application can be applied to an electronic device with a communication function. The electronic device includes a terminal device, and the specific device form of the terminal device can refer to the related description above, which will not be repeated here.

[0386] The embodiments of the present application provide a terminal device, which includes: a processor and a memory; the memory stores computer execution instructions; and the processor executes the computer execution instructions stored in the memory, so that the terminal device executes the above method.

[0387] The embodiments of the present application provide a chip. The chip includes a processor, and the processor is configured to invoke a computer program in a memory to execute the technical solutions in the above embodiments. The implementation principle and technical effects are similar to those of the above related embodiments, which will not be repeated here.

[0388] The embodiments of the present application also provide a computer readable storage medium. The computer readable storage medium stores a computer program. The computer program is executed by the processor to implement the above method. The method described in the above embodiments can be implemented by software, hardware, firmware or any combination thereof, in whole or in part. If implemented in software, the functions can be stored as one or more instructions or codes on a computer readable medium or transmitted on a computer readable medium. The computer readable medium can include computer storage medium and communication medium, and can also include any medium that can carry computer programs from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0389] In a possible implementation, the computer readable medium can include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that is targeted to carry the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately called a computer readable medium. For example, if software is transmitted from a website, server or other remote source using a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave), the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave) is included in the definition of medium. As used herein, magnetic disks and optical disks include compact disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, in which magnetic disks usually reproduce data magnetically, and optical disks reproduce data optically with laser. The above combinations should also be included in the scope of computer readable medium.

[0390] The embodiment of the present application provides a computer program product, which comprises a computer program, and when the computer program is executed, the computer executes the above method.

[0391] The embodiment of the present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiment of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing devices produce a device for implementing the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0392] The above detailed description is further detailed for the purpose of the present application, technical solutions, and beneficial effects, and it should be understood that the above is only a specific embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application should be included in the protection scope of the present application.

Claims

1. An image processing method, characterized by, The method comprises: obtaining a first image and a second image, the first image being a high dynamic range (HDR) image, and the second image being a standard dynamic range (SDR) image; determining a first gain range according to the first image and the second image; determining second gain information according to first gain information and the first gain range, the first gain information being calculated according to the first image and the second image; obtaining a selectable image according to the second gain information and the second image.

2. The method of claim 1, wherein, The determining of the first gain range comprises: determining the first gain range according to linear space luminance of the first image and linear space luminance of the second image.

3. The method of claim 2, wherein, The determining of the first gain range comprises: calculating a dynamic range headroom level according to linear space luminance of the first image and linear space luminance of the second image; determining the first gain range according to the dynamic range headroom level.

4. The method of claim 3, wherein, The dynamic range headroom level satisfies: the dynamic range headroom level is a first ratio, the first ratio being a ratio of maximum linear space luminance of the first image and maximum linear space luminance of the second image; or, the dynamic range headroom level is a first preset headroom, and the first ratio is greater than the first preset headroom.

5. The method according to any one of claims 1 to 4, characterized in that, The determining of the first gain range comprises: determining the first gain range according to the first image, the second image and the first gain information.

6. The method of claim 5, wherein, The determining of the first gain range comprises: determining the first gain range according to a dynamic range headroom level and a second gain range, the second gain range being a gain range of the first gain information, and the dynamic range headroom level being calculated according to the first image and the second image.

7. The method according to any one of claims 1 to 6, characterized in that, The determining of the first gain range comprises: determining the first gain range according to the first image, the second image and a bit width.

8. The method according to any one of claims 1 to 7, characterized in that, The determining of the first gain range comprises: determining a first enhanced image according to the second image and the first gain information; determining the first gain range according to the first enhanced image, the first image and the second image.

9. The method of claim 8, wherein, The determining of the first gain range comprises: determining a third gain range according to the first image and the second image; reducing the third gain range to obtain the first gain range according to linear space luminance of the first enhanced image and linear space luminance of the first image.

10. The method of claim 9, wherein, The reducing of the third gain range comprises: reducing the third gain range according to a contrast of linear space luminance of the first enhanced image and linear space luminance of the first image.

11. The method of claim 10, wherein, The reducing of the third gain range comprises: reducing the third gain range in a case that a ratio of a number of first pixel points to a number of second pixel points is less than or equal to a first threshold, or the number of first pixel points is less than or equal to a second threshold; wherein the second pixel points are pixel points in the first gain information whose gain values are in the third gain range, and the first pixel points are pixel points in the second pixel points whose contrasts satisfy a contrast sensitivity threshold.

12. The method of claim 11, wherein, The first pixel point is a pixel point in the second pixel point that satisfies the contrast sensitivity function.

13. The method according to any one of claims 10 to 12, characterized in that, The contrast is any one of the following: A first difference value, the first difference value being a difference value between a linear space luminance of the first image and a linear space luminance of the first enhanced image; A ratio of the first difference value and the linear space luminance of the first image or the linear space luminance of the first enhanced image; Or, A ratio of the first difference value and a first sum, the first sum being a sum of the linear space luminance of the first image and the linear space luminance of the first enhanced image.

14. The method according to any one of claims 1 to 13, characterized in that, The first gain range includes a maximum value and a minimum value; The determining the second gain information includes: Setting a gain smaller than the minimum value in the first gain information to the minimum value, and setting a gain greater than the maximum value in the first gain information to the maximum value.

15. The method according to any one of claims 1 to 14, characterized in that, The method further includes: Normalizing the second gain information to obtain normalized second gain information; The obtaining the selectable image includes: Obtaining the selectable image according to the normalized second gain information and the second image.

16. The method according to any one of claims 1 to 15, characterized in that, The selectable image is used to reconstruct the first image.

17. The method of any one of claims 1 to 16, wherein, The metadata of the second gain information includes the first gain range.

18. An electronic device, comprising: The electronic device includes one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is configured to store computer program code including computer instructions, and the one or more processors are configured to invoke the computer instructions to cause the electronic device to perform the method according to any one of claims 1 to 17.

19. A chip system, characterized by The chip system is applied to an electronic device, and the chip system includes one or more processors configured to invoke computer instructions to cause the electronic device to perform the method according to any one of claims 1 to 17.

20. A computer-readable storage medium, characterized in that, The computer readable storage medium includes computer instructions, and when the computer instructions run on an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 17.

21. A computer program product, characterised in that, The computer program product includes computer program code, and when the computer program code runs on an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 17.

Images