Terminal device and method of displaying an image

Abstract

The application provides a terminal device and a method for displaying an image, and relates to the technical field of image processing. The HDR image can have a higher dynamic range when displayed on a device with a higher dynamic range. The terminal device is configured to: obtain a plurality of images under a plurality of exposure parameters, perform tone mapping on the plurality of images to obtain a first image, and perform encoding operations on the luminance information and the pixel value of each pixel point in the first image under a target exposure parameter to obtain an image file; then, when the image file is displayed, perform decoding operations on the image file to obtain the pixel value of each pixel point in the first image and the luminance information of each pixel point; divide the first image into a dark region and a highlight region; adjust the pixel value of the pixel point in the dark region and the highlight region based on a first adjustment coefficient and a second adjustment coefficient respectively to obtain an expanded image; and increase the screen brightness and display the expanded image.

Description

[0001] This application is a divisional application. The original application has the application number 202210675917.6 and the original application date is June 15, 2022. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of image processing technology, and in particular to a terminal device and a method for displaying images. Background Technology

[0003] With the development of the terminal industry, most terminal devices support high dynamic range (HDR) photography technology. HDR photography technology can capture and display the vast dynamic range of the real world, even when the dynamic range available in typical imaging sensors and display devices is limited.

[0004] Currently, HDR photography is typically implemented by capturing multiple images of the same scene at different exposure levels, fusing these images into a single image, and then applying some form of tone mapping to bring the fused image into the dynamic range of a standard dynamic range (SDR) display. The fused image is generally stored as an 8-bit image, with a final dynamic range of approximately 255:1.

[0005] However, with the continuous improvement of display technology, most current display devices can provide a dynamic range higher than 255:1. However, when displaying the fused image, these devices cannot display the image with a higher dynamic range. Summary of the Invention

[0006] In view of this, this application provides a terminal device and a method for displaying images, which can enable HDR images to have higher contrast in highlight areas and higher dynamic range when displayed on a device with a higher dynamic range.

[0007] In a first aspect, this application provides a terminal device, which includes a display screen, a memory, and one or more processors coupled to the memory; wherein the memory stores computer program code, which includes computer instructions; the one or more processors are configured to execute the computer instructions and perform the following operations: in response to a user viewing an image file, decoding the image file to obtain the pixel value of each pixel in a first image and the brightness information of each pixel in the first image under target exposure parameters; dividing the first image into multiple image regions based on the brightness information of each pixel, the multiple image regions including a first region and a second region, the first region being a dark region in the first image and the second region being a bright region in the first image; determining a target screen brightness based on the current screen brightness and the maximum screen brightness of the display screen, wherein the target screen brightness is greater than the current screen brightness and less than or equal to the maximum screen brightness; adjusting the pixel values ​​of pixels in the first region based on a first adjustment coefficient and adjusting the pixel values ​​of pixels in the second region based on a second adjustment coefficient to obtain an extended image, wherein the first adjustment coefficient is the ratio of the current screen brightness to the target screen brightness and the second adjustment coefficient is greater than or equal to 1; adjusting the screen brightness of the display screen to the target screen brightness and displaying the extended image.

[0008] During the display of the first image, the brightness of each pixel is increased by increasing the screen brightness of the phone; simultaneously, the brightness of pixels in the first region is decreased by decreasing the pixel value of pixels in the first region, and the brightness of pixels in the second region is increased or maintained by increasing the pixel value of pixels in the second region. This achieves the effect of increasing the brightness of pixels in the second region without changing their overall brightness. In other words, during the display of the HDR image (first image), bright areas in the HDR image can be made brighter, and dark areas can be made darker or remain unchanged, increasing the contrast and expanding the dynamic range of the HDR image.

[0009] In one embodiment of the first aspect, one or more processors are further configured to perform the following operations: obtain multiple frames of images under multiple exposure parameters, wherein the multiple frames of images correspond one-to-one with the multiple exposure parameters; perform tone mapping on the multiple frames of images to obtain a first image; obtain the pixel value of each pixel in the first image; group the pixels in the first image according to a preset pixel grouping strategy, and obtain the brightness information of each group of pixels under a target exposure parameter, wherein the target exposure parameter is one of the multiple exposure parameters, or is a parameter determined based on at least two of the multiple exposure parameters; and encode the brightness information and the pixel value of each pixel to obtain an image file.

[0010] The image file obtained based on the embodiments of this application includes both the pixel values ​​of the first image and the brightness information before tone mapping. This brightness information can better reflect the true brightness of the first image. Thus, by adjusting the dynamic range of the first image based on this brightness information during the display process, the displayed image can be closer to the true brightness.

[0011] In one embodiment of the first aspect, one or more processors are further configured to perform the following operations: acquire a target exposure image corresponding to a target exposure parameter, wherein if the target exposure parameter is a first parameter among a plurality of exposure parameters, the target exposure image is an image frame corresponding to the first parameter; if the target exposure parameter is a parameter determined based on at least two exposure parameters among a plurality of exposure parameters, the target exposure image is an image obtained by fusing image frames corresponding to at least two exposure parameters; acquire the brightness value of each pixel in each group of pixels in the target exposure image; and use the average of the brightness values ​​of all pixels in each group of pixels in the target exposure image as the brightness information of that group of pixels under the target exposure parameter. By grouping the pixels, the amount of brightness information can be reduced, thereby reducing the file size.

[0012] In one embodiment of the first aspect, one or more processors are further configured to perform the following operations: encode the pixel value of each pixel to obtain first encoded data; encode the brightness information to obtain second encoded data; and write the second encoded data into the first encoded data to obtain an image file. That is, brightness information can be written into an image, resulting in only one image file.

[0013] In one embodiment of the first aspect, the second region includes M pixels, the second adjustment coefficient includes M sub-adjustment coefficients, and the M pixels correspond one-to-one with the M sub-adjustment coefficients. One or more processors are further configured to perform the following operations: determine the brightness value of the j-th pixel based on the brightness information of the j-th pixel, where j≤M; calculate the sub-adjustment coefficient of the j-th pixel based on the brightness value of the j-th pixel, wherein the sub-adjustment coefficient of the j-th pixel is positively correlated with the brightness value of the j-th pixel and the sub-adjustment coefficient of the j-th pixel is greater than 1; and adjust the pixel value of the j-th pixel based on the sub-adjustment coefficient of the j-th pixel.

[0014] In other words, the second adjustment coefficient can be different for each pixel. By adjusting the pixel values ​​of the second region in this way, the brightness and contrast of each pixel within the second region can be preserved, making the display result more realistic.

[0015] In one embodiment of the first aspect, the second adjustment factor is 1.

[0016] In one embodiment of the first aspect, one or more processors are further configured to perform the following operations: determine the brightness value of a corresponding pixel based on the brightness information of each pixel; calculate the average brightness value of each pixel; determine a first threshold based on the average brightness value, wherein the first threshold is greater than the average brightness value and is positively correlated with the average brightness value; if the brightness value of any pixel is less than the first threshold, assign the pixel to a first region; if the brightness value of any pixel is greater than or equal to the first threshold, assign the pixel to a second region.

[0017] In one embodiment of the first aspect, the plurality of image regions further includes a third region, and one or more processors are further configured to perform the following operations: determine the brightness value of a corresponding pixel based on the brightness information of each pixel; calculate the average brightness value of each pixel; determine a second threshold and a third threshold based on the average brightness value, wherein the second threshold is less than the average brightness value, the third threshold is greater than the average brightness value, and the second threshold and the third threshold are positively correlated with the average brightness value; if the brightness value of any pixel is less than the second threshold, assign the pixel to the first region; if the brightness value of any pixel is greater than or equal to the second threshold and less than the third threshold, assign the pixel to the third region; if the brightness value of any pixel is greater than or equal to the third threshold, assign the pixel to the second region.

[0018] In one embodiment of the first aspect, the target screen brightness is the maximum screen brightness.

[0019] In one embodiment of the first aspect, the target screen brightness is the larger of the maximum screen brightness and a first brightness threshold, whereby the first brightness threshold is the product of the current screen brightness and a preset multiple.

[0020] Secondly, embodiments of this application provide a terminal device, which includes a display screen, a memory, and one or more processors coupled to the memory. The memory stores computer program code, including computer instructions. When the one or more processors are configured to execute the computer instructions, they perform the following operations: obtaining multiple frames of images under multiple exposure parameters, each frame corresponding to one of the multiple exposure parameters; performing tone mapping on the multiple frames to obtain a first image; obtaining the pixel value of each pixel in the first image; grouping the pixels in the first image according to a preset pixel grouping strategy, and obtaining the brightness information of each group of pixels under a target exposure parameter, where the target exposure parameter is one of the multiple exposure parameters, or a parameter determined based on at least two of the multiple exposure parameters; and encoding the brightness information and the pixel value of each pixel to obtain an image file.

[0021] As can be seen, the image file obtained based on the embodiments of this application includes not only the pixel values ​​of the first image, but also the brightness information before tone mapping. This brightness information can better reflect the true brightness of the first image. Thus, by adjusting the dynamic range of the first image based on this brightness information during the display process, the displayed image can be closer to the true brightness.

[0022] In one embodiment of the second aspect, one or more processors are further configured to perform the following operations: acquire multiple frames of images under multiple exposure parameters, wherein each frame corresponds one-to-one with a different exposure parameter; perform tone mapping on the multiple frames to obtain a first image; acquire the pixel value of each pixel in the first image; group the pixels in the first image according to a preset pixel grouping strategy, acquire the brightness information of each group of pixels under a target exposure parameter, wherein the target exposure parameter is one of the multiple exposure parameters, or a parameter determined based on at least two of the multiple exposure parameters; and encode the brightness information and the pixel value of each pixel to obtain an image file. By grouping the pixels, the amount of brightness information can be reduced, thereby reducing the file size.

[0023] In one embodiment of the second aspect, one or more processors are further configured to perform the following operations: encode the pixel value of each pixel to obtain first encoded data; encode the brightness information to obtain second encoded data; and write the second encoded data into the first encoded data to obtain an image file. That is, brightness information can be written into an image, resulting in only one image file.

[0024] Thirdly, embodiments of this application provide a method for creating HDR images, applied to a terminal device. The method includes: obtaining multiple frames of images under multiple exposure parameters, wherein each frame corresponds one-to-one with a multiple exposure parameter; performing tone mapping on the multiple frames to obtain a first image; obtaining the pixel value of each pixel in the first image; grouping the pixels in the first image according to a preset pixel grouping strategy, and obtaining the brightness information of each group of pixels under a target exposure parameter, wherein the target exposure parameter is one of the multiple exposure parameters, or a parameter determined based on at least two of the multiple exposure parameters; and encoding the brightness information and the pixel value of each pixel to obtain an image file.

[0025] In one embodiment of the third aspect, obtaining the brightness information of each group of pixels under the target exposure parameters includes: obtaining a target exposure image corresponding to the target exposure parameters, wherein if the target exposure parameters are a first parameter among a plurality of exposure parameters, the target exposure image is an image frame corresponding to the first parameter; if the target exposure parameters are parameters determined according to at least two exposure parameters among a plurality of exposure parameters, the target exposure image is an image obtained by fusing the image frames corresponding to at least two exposure parameters; obtaining the brightness value of each pixel in each group of pixels in the target exposure image; and taking the average of the brightness values ​​of all pixels in each group of pixels in the target exposure image as the brightness information of that group of pixels under the target exposure parameters.

[0026] In one embodiment of the third aspect, encoding the brightness information and the pixel value of each pixel to obtain an image file includes: encoding the pixel value of each pixel to obtain first encoded data; encoding the brightness information to obtain second encoded data; and writing the second encoded data into the first encoded data to obtain an image file.

[0027] Fourthly, embodiments of this application also provide a method for displaying HDR images, applied to a terminal device. The method includes: in response to a user's operation of viewing an image file, decoding the image file to obtain the pixel value of each pixel in a first image and the brightness information of each pixel in the first image under target exposure parameters; dividing the first image into multiple image regions based on the brightness information of each pixel, the multiple image regions including a first region and a second region, the first region being a dark area in the first image and the second region being a bright area in the first image; determining a target screen brightness based on the current screen brightness and the maximum screen brightness of the display screen, the target screen brightness being greater than the current screen brightness and less than or equal to the maximum screen brightness; adjusting the pixel values ​​of pixels in the first region based on a first adjustment coefficient and adjusting the pixel values ​​of pixels in the second region based on a second adjustment coefficient to obtain an extended image, wherein the first adjustment coefficient is the ratio of the current screen brightness to the target screen brightness, and the second adjustment coefficient is greater than or equal to 1; adjusting the screen brightness of the display screen to the target screen brightness and displaying the extended image.

[0028] In one embodiment of the fourth aspect, the second region includes M pixels, and the second adjustment coefficient includes M sub-adjustment coefficients. The M pixels correspond one-to-one with the M sub-adjustment coefficients. Adjusting the pixel value of the pixels in the second region based on the second adjustment coefficient includes: determining the brightness value of the j-th pixel based on the brightness information of the j-th pixel, where j≤M; calculating the sub-adjustment coefficient of the j-th pixel based on the brightness value of the j-th pixel, wherein the sub-adjustment coefficient of the j-th pixel is positively correlated with the brightness value of the j-th pixel and the sub-adjustment coefficient of the j-th pixel is greater than 1; and adjusting the pixel value of the j-th pixel based on the sub-adjustment coefficient of the j-th pixel.

[0029] In one embodiment of the fourth aspect, the second adjustment factor is 1.

[0030] In one embodiment of the fourth aspect, dividing the first image into multiple image regions based on the brightness information of each pixel includes: determining the brightness value of the corresponding pixel based on the brightness information of each pixel; calculating the average brightness value of each pixel; determining a first threshold based on the average brightness value, wherein the first threshold is greater than the average brightness value and is positively correlated with the average brightness value; if the brightness value of any pixel is less than the first threshold, assigning the pixel to the first region; and if the brightness value of any pixel is greater than or equal to the first threshold, assigning the pixel to the second region.

[0031] In one embodiment of the fourth aspect, the multiple image regions further include a third region. Dividing the first image into multiple image regions based on the brightness information of each pixel includes: determining the brightness value of the corresponding pixel based on the brightness information of each pixel; calculating the average brightness value of each pixel; determining a second threshold and a third threshold based on the average brightness value, wherein the second threshold is less than the average brightness value and the third threshold is greater than the average brightness value, and the second threshold and the third threshold are positively correlated with the average brightness value; if the brightness value of any pixel is less than the second threshold, the pixel is assigned to the first region; if the brightness value of any pixel is greater than or equal to the second threshold and less than the third threshold, the pixel is assigned to the third region; if the brightness value of any pixel is greater than or equal to the third threshold, the pixel is assigned to the second region.

[0032] In one embodiment of the fourth aspect, the target screen brightness is the maximum screen brightness.

[0033] In one embodiment of the fourth aspect, the target screen brightness is the larger of the maximum screen brightness and the first brightness threshold, where the first brightness threshold is the product of the current screen brightness and a preset multiple.

[0034] Fifthly, embodiments of this application also provide a computer-readable storage medium, including computer instructions that, when executed on a terminal, cause the terminal to perform the method of any one of the third and fourth aspects.

[0035] Sixthly, embodiments of this application also provide a computer program product, including computer instructions, which, when executed on a terminal, cause the terminal to perform the method of any one of the third and fourth aspects.

[0036] Understandably, the beneficial effects that the computer-readable storage medium of the fifth aspect and the computer program product of the sixth aspect can achieve can be referred to the beneficial effects of the first aspect and any of its possible design embodiments, which will not be repeated here. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application;

[0038] Figure 2A A flowchart illustrating a method for creating an HDR image, as provided in this application embodiment;

[0039] Figure 2B This is a schematic diagram of a synthesized first image provided in an embodiment of this application;

[0040] Figure 3 This is a schematic diagram of a shooting process;

[0041] Figure 4 This is a schematic diagram of a pixel grouping strategy;

[0042] Figure 5 A schematic diagram illustrating another pixel grouping strategy;

[0043] Figure 6 A flowchart illustrating a method for displaying HDR images provided in an embodiment of this application;

[0044] Figure 7 A schematic diagram illustrating different methods of displaying an image, as provided in an embodiment of this application;

[0045] Figure 8 This is a schematic diagram of a chip system provided in an embodiment of this application. Detailed Implementation

[0046] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. In the description of this application, unless otherwise stated, "at least one" refers to one or more, and "more than one" refers to two or more. Furthermore, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.

[0047] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the relevant concepts or technologies is given first:

[0048] HDR refers to a signal whose dynamic range (the ratio between its highest and lowest values) is greater than or equal to a preset first threshold. In an image, HDR can be understood as an image containing highlight and shadow areas, where the ratio of the brightness values ​​of the highlight areas to the brightness values ​​of the shadow areas exceeds the first threshold. Such an image can also be called an HDR image.

[0049] SDR (Simultaneous Dynamic Range) refers to a signal whose dynamic range is less than a preset first threshold and greater than or equal to a preset second threshold (the second threshold being less than the first threshold). In an image, SDR can be understood as the presence of highlight and shadow areas where the ratio of the brightness values ​​of the highlight areas to those of the shadow areas is less than the preset first threshold and greater than or equal to the preset second threshold. Such an image can also be called an SDR image.

[0050] Exposure values ​​(EV) reflect the exposure level of an image. By adjusting the camera's exposure parameters, images can have different EV values. Exposure parameters refer to those that affect the amount of exposure by the camera, including but not limited to aperture, shutter speed, exposure time, and ISO sensitivity. In this embodiment, a normally exposed exposure value is represented as EV0, an underexposed exposure value as EV-, and an overexposed exposure value as EV+. Furthermore, the exposure value EV0 × 2... n It is represented as EVn. For example, EV-1 means half the exposure value of EV0, and EV-2 means half the exposure value of EV-1; as another example, EV1 means twice the exposure value of EV0, and EV2 means twice the exposure value of EV1.

[0051] In one existing technology, a terminal device can capture multiple images of the same scene at different exposure values ​​and then fuse these images together to create an "HDR" image. However, this type of "HDR" image is typically an 8-bit image, with a final dynamic range of approximately 255:1. This means that the dynamic range of the "HDR" image remains within the standard dynamic range. In other words, this type of "HDR" image is actually an SDR image.

[0052] When displaying an "HDR" image on a device with a higher dynamic range, the brightness of the "HDR" image can increase or decrease with the brightness of the device's screen. However, during this change, the brightness of the highlight and shadow areas of the "HDR" image can change proportionally, meaning the contrast between the highlight and shadow areas does not change. In other words, the dynamic range of the "HDR image" displayed by the device does not change, and the device's higher dynamic range is not fully utilized.

[0053] In view of this, embodiments of this application provide a method for generating / displaying HDR images, applied to a terminal device. During the generation of an HDR image, the terminal device can encode brightness information reflecting the true brightness of the HDR image and the pixel values ​​of the HDR image into an image file. Furthermore, during the display of the HDR image, the pixel values ​​in the highlight areas of the HDR image can be increased or maintained, and the pixel values ​​in the shadow areas can be decreased, based on the true brightness and the brightness capabilities of the terminal device. This makes the highlight areas of the HDR image brighter and the shadow areas darker or closer to the original brightness of the shadow areas, increasing the contrast between the highlight and shadow areas, thereby expanding the dynamic range of the HDR image and making the HDR image displayed by the terminal device closer to the real scene.

[0054] The image display method provided in this application can be applied to terminal devices. These terminal devices can be devices equipped with cameras and displays, such as mobile phones, tablets, and laptops. This application does not impose specific limitations on the specific form of the terminal device.

[0055] like Figure 1As shown, the aforementioned terminal device can specifically be a mobile phone 100. Mobile phone 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0056] The aforementioned sensor module 180 may include sensors such as pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, and bone conduction sensors.

[0057] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the mobile phone 100. In other embodiments, the mobile phone 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0058] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0059] The controller can be the nerve center and command center of the mobile phone 100. The controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0060] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0061] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0062] It is understood that the interface connection relationships between the modules illustrated in this embodiment are merely illustrative and do not constitute a structural limitation on the mobile phone 100. In other embodiments, the mobile phone 100 may also adopt different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0063] The charging management module 140 receives charging input from a charger, which can be either a wireless charger or a wired charger. While charging the battery 142, the charging management module 140 can also supply power to the terminal device via the power management module 141.

[0064] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, internal memory 121, external memory, display 194, camera 193, and wireless communication module 160, etc. In some embodiments, the power management module 141 and the charging management module 140 may also be housed in the same device.

[0065] The wireless communication function of mobile phone 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor. In some embodiments, antenna 1 of mobile phone 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling mobile phone 100 to communicate with networks and other devices through wireless communication technology.

[0066] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in mobile phone 100 can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0067] The mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G on the mobile phone 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation.

[0068] The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 can be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be housed in the same device.

[0069] The wireless communication module 160 can provide solutions for wireless communication applications on the mobile phone 100, including WLAN (such as wireless fidelity, Wi-Fi), Bluetooth, global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc.

[0070] The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0071] The mobile phone 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0072] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel.

[0073] Mobile phone 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194, and application processor. The ISP is used to process data fed back by camera 193. Camera 193 is used to capture still images or videos. In some embodiments, mobile phone 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0074] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0075] The internal memory 121 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the mobile phone 100 by running the instructions stored in the internal memory 121. For example, in this embodiment, the processor 110 can execute instructions stored in the internal memory 121, which may include a program storage area and a data storage area.

[0076] The program storage area can store the operating system, at least one application required for a function (such as sound playback, image playback, etc.). The data storage area can store data created during the use of the mobile phone 100 (such as audio data, phonebook, etc.). In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0077] The mobile phone 100 can achieve audio functions such as music playback and recording through the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0078] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch buttons. Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the mobile phone 100. The mobile phone 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0079] This application provides a method for creating / displaying HDR images. The processes of creating backward-compatible HDR images and displaying HDR images will be described below with reference to the accompanying drawings.

[0080] In some embodiments, such as Figure 2A As shown, the process for creating an HDR image may include:

[0081] S201: Acquire multiple frames of images, each corresponding to a different exposure parameter.

[0082] Among them, multiple frames are images taken by the mobile phone in the same scene under different exposure parameters. For example, such as Figure 3As shown, the mobile phone can display a shooting preview interface 301. This shooting preview interface 301 may include a shutter 302. In response to a user's tap / touch of the shutter 302, the mobile phone captures multiple frames of images. In one possible implementation, the mobile phone may include a first camera, a second camera, and a third camera, which can each capture images with different exposure parameters to obtain multiple frames of images. Alternatively, in another possible implementation, the mobile phone can continuously capture multiple frames of images with different exposure parameters, and the capture time of each frame should be as close to synchronous as possible.

[0083] For example, such as Figure 2B As shown, the multi-frame images may include Image 1 and Image 2. Image 1 was captured by the camera based on exposure parameter 1, and Image 2 was captured by the camera based on exposure parameter 2. It can be seen that the EV value corresponding to Image 1 is less than the EV value corresponding to Image 2. Image 1 is underexposed, resulting in relatively clear outlines in the highlight areas, but the shadow areas are blurry due to lack of exposure. Image 2 is overexposed, causing the highlight areas to be distorted due to overexposure, while the shadow areas retain clear details and outlines due to exposure compensation.

[0084] It should be noted that, Figure 2B Taking only image frames under EV- and EV+ conditions as an example, the mobile phone can actually acquire image frames under more exposure conditions, such as image frames under EV0, EV-1, EV-2, EV+1, EV+2, etc., without making specific limitations here.

[0085] S202, tone mapping is performed on multiple frames of images to obtain the first image.

[0086] Mobile phones can fuse multiple frames of images into a single image using tone mapping. Tone mapping can include global tone mapping and local tone mapping. Global tone mapping can use histograms, gamma functions, sigmoid nonlinear functions, etc. Global tone mapping can preserve global contrast well, but it loses some local details. Local tone mapping, on the other hand, first divides the first image into regions, then processes each region in a similar way to global tone mapping, ultimately preserving the relative contrast between adjacent regions and thus preserving local details very well.

[0087] For example, the first image obtained can be as follows: Figure 2B As shown. This first image is relative to a single frame image (e.g., Figure 2B In comparison to images 1 and 2, the first image exhibits more distinct details and contours in both its dark and highlight areas, demonstrating a higher dynamic range. In other words, the first image is an HDR image.

[0088] S203, obtain the pixel value of each pixel in the first image.

[0089] Depending on the image sensor type used to capture the image, the pixel value of each pixel includes, but is not limited to, RGB, RGGB, RGBW, and RYYB values. RGB indicates that the color of each pixel is composed of three components: red (R), green (G), and blue (B). RGGB indicates that the color of each pixel is composed of four components: red, green, green, and blue. RGBW indicates that the color of each pixel is composed of four components: red, green, blue, and white (W). RYYB indicates that the color of each pixel is composed of four components: red, yellow (Y), yellow, and blue. The pixel value of each pixel can be obtained through the image sensor.

[0090] S204, the pixels in the first image are grouped according to a preset pixel grouping strategy, and the brightness information of each group of pixels under the target exposure parameters is obtained.

[0091] The target exposure parameter can be one of multiple exposure parameters, or a combination of at least two exposure parameters, and can be set according to actual shooting needs. For example, if the desired image has clear shadow details, the target exposure parameter can be larger; if the desired image has clear highlights and is not overexposed, the target exposure parameter can be smaller; if the desired image has clear contrast and is not overexposed or underexposed, the target exposure parameter can be moderate. In an optional implementation, the target exposure parameter can be a combination of multiple exposure parameters, i.e., the exposure parameter corresponding to the first image.

[0092] For example, multiple exposure parameters include EV-1, EV0, EV+1, and EV+2. The phone can directly select any one of these parameters as the target exposure parameter. Alternatively, if capturing shadow details in a scene requires an EV between EV+1 and EV+2, the phone can adjust parameters such as exposure time and ISO to achieve the target exposure parameter.

[0093] In another alternative implementation, multiple exposure parameters can be input into a pre-trained neural network model to obtain the target exposure parameters. In summary, the target exposure parameters are values ​​that can be changed according to actual shooting needs, and their setting method can be configured and maintained by the maintainer in the background.

[0094] The image corresponding to the target exposure parameter is the target exposure image. The target exposure image can be understood as the exposure image that the phone should obtain when shooting with the target exposure parameter. If the target exposure parameter is one of multiple exposure parameters, the target exposure image can be the image captured by the phone under the target exposure parameter. If the target exposure parameter is a combination of at least two of the multiple exposure parameters, the target exposure image can be an image obtained by fusing the image frames corresponding to those at least two exposure parameters. For example, the multiple exposure parameters include EV0, EV-, and EV+, and correspond to EV0 image frames, EV- image frames, and EV+ image frames, respectively. If the target exposure parameter is EV0, then the EV0 image frame is used as the target exposure image; if the target exposure parameter is a combination of EV- and EV+, then the image obtained by fusing the EV- and EV+ image frames is used as the target exposure image. As another example, if the target exposure parameter is a combination of the above multiple exposure parameters, then the first image obtained by fusing the multiple frames corresponding to the multiple exposure parameters is used as the target exposure image.

[0095] The brightness information of each group of pixels under the target exposure parameters can be understood as the brightness information of each group of pixels in the target exposure image. For example, a mobile phone captures an EV0 image, an EV- image, and an EV+ image, and obtains a first image based on the EV0 image, EV- image, and EV+ image. If the mobile phone uses EV0 as the target exposure parameter, then the brightness information of a certain group of pixels in the first image under the target exposure parameters is the brightness information of that group of pixels in the EV0 image.

[0096] In addition, the preset pixel grouping strategy can be set according to actual needs. For example, the preset pixel grouping strategy can divide a single pixel into a group, or it can divide a pixel matrix consisting of n×m pixels into a group. For example, it can divide a pixel matrix consisting of 2×2 or 3×2 pixels into a group. No specific restrictions are imposed here.

[0097] If a mobile phone divides a pixel into groups, the brightness information of each group of pixels under the target exposure parameters is the brightness value of that pixel in the target image. If the mobile phone divides a pixel matrix consisting of n×m pixels into groups, the brightness information of that group of pixels under the target exposure parameters can be the average brightness value of all pixels in that group in the target image, or the average brightness value remaining after removing outliers from all pixel brightness values; no specific restriction is imposed here.

[0098] For example, Figure 4 A pixel grouping strategy is shown. Figure 4In this image, the phone treats each pixel as a pixel group. Pixel A1 has a pixel value of R1G1B1, and pixel A2 has a pixel value of R2G2B2. Pixels A1 and A2 have the same coordinates in the image. The brightness information of pixel A2 under the target exposure parameters is the brightness information of pixel A2 in the image under the target exposure parameters, which is equivalent to the brightness information of pixel A1. Therefore, the brightness information L1 of pixel A1 can be calculated using its pixel values ​​R1, G1, and B1, and this value can be used as the brightness information of pixel A2 under the target exposure parameters.

[0099] The brightness value and pixel value can satisfy the formula: L = 0.299R + 0.587G + 0.114B, where L is the brightness value. For example, if the pixel value of pixel A1 is (R100, G100, B100), then the brightness value of pixel A1 can be determined as 0.299R + 0.587G + 0.114B = 29.9 + 58.7 + 11.4 = 100. Therefore, the brightness information of pixel A2 under the target exposure parameters is 100.

[0100] For example, Figure 5 Another pixel grouping strategy is shown. Figure 5 In the image, both the first image and the target exposure image consist of 9×4 pixels. The phone divides these 9×4 pixels into 12 groups, each group containing 3×1 pixels. Pixel group 1 includes pixels M1, M2, and M3. Pixel group 2 includes pixels N1, N2, and N3. Pixels M1 and N1 have the same coordinates in the image, as do M2 and N2, and M3 and N3. The pixel values ​​of M1 are R3, G3, B3, and W3; the pixel values ​​of M2 are R4, G4, B4, and W4; and the pixel values ​​of M3 are R5, G5, B5, and W5. The pixel values ​​of pixel N1 are R6, G6, B6, and W6; the pixel values ​​of pixel N2 are R7, G7, B7, and W7; and the pixel values ​​of pixel N3 are R8, G8, B8, and W8. The brightness information of pixel group 2 under the target exposure parameters is the brightness information of pixel group 2 in the target exposure image, which is the brightness information of pixel group 1. The mobile phone can calculate the brightness value L3 based on the pixel values ​​of pixel M1 (R3, G3, B3, W3), the brightness value L4 based on the pixel values ​​of pixel M2 (R4, G4, B4, W4), and the brightness value L5 based on the pixel values ​​of pixel M3 (R5, G5, B5, W5). Then, the average value L6 of the brightness values ​​L3, L4, and L5 is used as the brightness information of pixel group 2 under the target exposure parameters.

[0101] S205, the brightness information and pixel values ​​are encoded separately to obtain the image file.

[0102] In this embodiment, the mobile phone can encode pixel values ​​using methods such as the Joint Photographic Experts Group (JEPG) standard, high efficiency video coding (HEVC), portable network graphics (PNG), and run-length encoding (RLE) to obtain the first encoded data.

[0103] The format of the first encoded data includes, but is not limited to, JPEG, High Efficiency Image File Format (HEIF), HEIC, PNG, Bitmap (BMP), Graphics Interchange Format (GIF), etc., without specific restrictions.

[0104] In addition, the mobile phone can use linear or non-linear encoding methods to encode the brightness information to obtain the second encoded data. Each encoded brightness information can have a bit width of 8 bits, 10 bits, or higher. Non-linear encoding can include methods such as encoding using gamma functions, and no specific limitations are made here. Furthermore, when encoding the brightness information, the pixel grouping method can also be encoded into the second encoded data.

[0105] The mobile phone can also write the second encoded data into specific fields of the first encoded data to generate an image file. For example, taking PNG as the format of the first encoded data, a standard PNG file includes a PNG file signature field and multiple PNG data blocks. The mobile phone can write the second encoded data into the PNG data blocks to generate an image file.

[0106] It should be noted that, depending on the format of the first encoded data, the fields written for the second encoded data may not be the same. As long as the second encoded data is written into the fields of the first encoded data that do not affect the information carried by the first encoded data itself, no specific restrictions are imposed here.

[0107] In this way, the image file created by the mobile phone includes both the pixel values ​​of the first image and the brightness information before tone mapping. This brightness information can better reflect the actual brightness of the first image, making it easier to adjust the brightness of the first image during subsequent display.

[0108] In some embodiments, such as Figure 6 As shown, the process of displaying an HDR image may include:

[0109] S301, in response to the user's operation of viewing an image file, the mobile phone decodes the image file to obtain brightness information and the pixel values ​​of the first image.

[0110] S302, determine the average brightness value based on the brightness information.

[0111] The average brightness value is the average of all brightness values, reflecting the true brightness level of the first image. For example, if the bit width of the brightness information is 8 bits, the digitized brightness values ​​will range from 0 to 255. The average brightness value is obtained by digitizing each brightness value and averaging the results.

[0112] S303, based on the average brightness, divide the first image into multiple image regions, the multiple image regions including at least the first region and the second region.

[0113] The first region is the dark area in the first image, and the second region is the highlight area in the first image.

[0114] In one alternative implementation, the mobile phone can divide the first image into two image regions, namely the first region and the second region.

[0115] The mobile phone can first determine a first threshold based on the average brightness, and then divide the first image into a first region and a second region based on the first threshold. The first threshold is greater than or equal to the average brightness, and the higher the average brightness, the higher the first threshold. The first threshold can have a linear or non-linear relationship with the average brightness; no specific restriction is placed here. The mobile phone can assign pixels with brightness values ​​below the first threshold to the first region and pixels with brightness values ​​above the second threshold to the second region. This ensures that the first region contains pixels from the darker areas of the first image, and the second region contains pixels from the brighter areas of the first image.

[0116] For example, the first threshold can be 165. If the brightness value of pixel 1 is determined to be 100 based on the brightness information, then since 100 < 165, pixel 1 is assigned to the first region. If the brightness value of pixel 2 is determined to be 188 based on the brightness information, then since 188 > 165, pixel 2 is assigned to the second region.

[0117] In another alternative implementation, the mobile phone can divide the first image into a first region, a second region, and a third region. The third region serves as a transition area between the first and second regions.

[0118] The mobile phone can determine a second threshold and a third threshold based on the average brightness value, and divide the first image into a first region, a second region, and a third region based on these thresholds. The second threshold is less than the average brightness value, and the third threshold is greater than or equal to the average brightness value; the larger the average brightness value, the larger the second and third thresholds. The second and third thresholds can have a linear or non-linear relationship with the average brightness value, without specific restrictions. The mobile phone can assign pixels with brightness values ​​below the second threshold to the first region, pixels with brightness values ​​greater than or equal to the second threshold but less than the third threshold to the third region, and pixels with brightness values ​​greater than or equal to the third threshold to the second region. Thus, the first region is the darker area in the first image, the second region is the brighter area, and the third region is the larger area of ​​the first and second regions, with moderate brightness.

[0119] It should be noted that the above only shows two ways of dividing image regions. In this application embodiment, a more refined partitioning method can also be used to divide the first image into more image regions, and no specific limitation is made here.

[0120] S304, determine the target screen brightness based on the current screen brightness and the maximum screen brightness.

[0121] In this embodiment, to make the first image appear brighter, the screen brightness can be increased. The maximum screen brightness is the maximum brightness supported by the mobile phone screen. In an optional implementation, the mobile phone can first determine a first brightness threshold based on the current screen brightness. For example, the mobile phone can use a preset multiple of the current screen brightness as the first brightness threshold. This preset multiple can be any value greater than 1, such as 3. Then, the mobile phone can compare the first brightness threshold with the maximum screen brightness. If the first brightness threshold is less than or equal to the maximum screen brightness, the target screen brightness is determined to be the first brightness threshold. If the first brightness threshold is greater than the maximum screen brightness, the target screen brightness is determined to be the maximum screen brightness.

[0122] In another alternative implementation, the target screen brightness can be fixed at the maximum screen brightness. That is, regardless of the current screen brightness, the phone always uses the maximum screen brightness as the target screen brightness.

[0123] S305, adjust the pixel values ​​of the pixels in the first region based on the first adjustment coefficient, and adjust the pixel values ​​of the pixels in the second region based on the second adjustment coefficient to obtain an expanded image, wherein the first adjustment coefficient is less than 1 and the second adjustment coefficient is greater than or equal to 1.

[0124] Specifically, the mobile phone can multiply the first adjustment coefficient by the pixel values ​​of all pixels in the first region to obtain the new pixel value for each pixel; and multiply the second adjustment coefficient by the pixel values ​​of all pixels in the second region to obtain the new pixel value for each pixel.

[0125] Specifically, when the phone determines the target screen brightness by comparing a first brightness threshold and the maximum screen brightness, if the first brightness threshold is less than the maximum screen brightness, the target screen brightness is the first brightness threshold, and the first adjustment coefficient is the ratio of the current screen brightness to the first brightness threshold, which is the reciprocal of a preset multiple. If the first brightness threshold is greater than or equal to the maximum screen brightness, the target screen brightness is the maximum screen brightness, and the first adjustment coefficient is the ratio of the current screen brightness to the maximum screen brightness. When the phone consistently uses the maximum screen brightness as the target screen brightness, the first adjustment coefficient is the ratio of the current screen brightness to the maximum screen brightness. Therefore, the first adjustment coefficient is the ratio of the current screen brightness to the target screen brightness.

[0126] Specifically, when the first adjustment coefficient Q1 is the reciprocal of a preset multiple, since the preset multiple is greater than 1, Q1 < 1. Furthermore, when the first adjustment coefficient Q1 is the ratio of the current screen brightness to the maximum screen brightness, with A Nits representing the current screen brightness and B Nits representing the maximum screen brightness, the first adjustment coefficient Q1 can be obtained as A / B.

[0127] In another optional implementation, the second adjustment coefficient is a variable value and is related to the brightness information of each pixel in the second region. Specifically, the second adjustment coefficient of each pixel is related to the brightness information of that pixel. Specifically, the brightness information of each pixel can be numerically converted to a brightness value, and then the second adjustment coefficient of that pixel can be calculated based on the brightness value. If the second region includes N pixels, then the second adjustment coefficient of the i-th pixel satisfies the formula: Q2 i =1+L i / L max Among them, Q2 i L represents the second adjustment factor for the i-th pixel. i L represents the brightness value of the i-th pixel. max This represents the maximum brightness value of a pixel. For example, taking 8-bit brightness information as an example, the brightness information can be digitized from 0 to 255. If the brightness value of a certain pixel is C, then the second adjustment coefficient Q2 corresponding to that pixel is 1 + C / 255.

[0128] This method allows different pixel values ​​to have different second adjustment coefficients, which can preserve the brightness relationship between different pixels in the second region as much as possible.

[0129] In one optional implementation, the second adjustment factor can be 1. When the second adjustment factor is 1, the pixel values ​​of the pixels in the second region remain unchanged.

[0130] Since the first adjustment factor is less than 1, the new pixel value corresponding to each pixel in the first region is less than the original pixel value corresponding to that pixel. Since the second adjustment factor is greater than or equal to 1, the new pixel value corresponding to each pixel in the second region is greater than or equal to the original pixel value corresponding to that pixel.

[0131] S306 adjusts the phone's screen brightness to the target screen brightness and displays an extended image.

[0132] In one optional implementation, the relationship between the observed brightness of a pixel and the screen brightness can satisfy the formula: Lg = Gray * Lp, where Lg is the observed brightness, Lp is the screen brightness, and Gray is the pixel grayscale. The observed brightness of a pixel indicates the brightness of that pixel on the display screen. The pixel grayscale is related to the pixel value, where a larger pixel value results in a larger pixel grayscale. For example, the pixel grayscale and pixel value can satisfy the formula: Gray = 0.299R + 0.587G + 0.114B.

[0133] As can be seen, when displaying an image on a mobile phone screen, both the screen brightness and the pixel value of each pixel affect its apparent brightness. Specifically, with a fixed pixel value, a higher screen brightness results in a higher observed brightness for that pixel, meaning it appears brighter. Conversely, with a fixed screen brightness, a higher pixel value results in a higher observed brightness for that pixel, meaning it appears brighter. Conversely, to maintain a constant observed brightness for a particular pixel while increasing screen brightness, the pixel value of that pixel can be decreased.

[0134] Understandably, the process of displaying the first image in this application is actually as follows: while increasing the screen brightness of the mobile phone, the pixel values ​​of the pixels in the first region of the first image are reduced so that the observed brightness of the pixels in the first region of the first image is close to the observed brightness before the screen brightness was adjusted; at the same time, the pixel values ​​of the pixels in the second region of the first image are maintained or increased so that the observed brightness of the pixels in the second region of the second image is greater than the observed brightness before the screen brightness was adjusted. In other words, the brightness of the first region of the first image remains unchanged, while the brightness of the second region of the first image is increased.

[0135] For example, let's illustrate the process of displaying a first image by adjusting the screen brightness of a mobile phone from the current screen brightness A to the target screen brightness C, with a first adjustment factor of A / C and a second adjustment factor of 1. The mobile phone can first generate an extended image based on the first and second adjustment factors. During the generation of the extended image, the pixel values ​​of pixels in the first region of the first image are reduced by a factor of A / C, resulting in a reduction in the grayscale level of the pixels in the first region by a factor of A / C; the pixel values ​​of pixels in the second region of the first image remain unchanged. When the mobile phone adjusts the screen brightness from the current screen brightness A to the target screen brightness C, the screen brightness of all pixels in the first image is increased by a factor of C / A. Therefore, after displaying the first image, the observed brightness of the pixels in the first region is first reduced by a factor of A / C, then increased by a factor of C / A, becoming the same as their original observed brightness; the observed brightness of the pixels in the second region is increased by a factor of C / A, exceeding their original observed brightness. This increases the contrast between the first and second regions of the first image, expanding the dynamic range of the first image.

[0136] For example, the first image can be as follows: Figure 7 As shown in (a), the first image 701 includes a first region 701a and a second region 701b. If the mobile phone does not obtain an expanded image of the first image beforehand, but directly increases the screen brightness and displays the first image, then the image shown can be obtained as follows: Figure 7 Image 702, shown in (b) of the image, exhibits increased brightness in both the first region 702a and the second region 702b, but its contrast remains unchanged, identical to that of image 701. Furthermore, after obtaining an extended image of the first image, the mobile phone increases the screen brightness and displays this extended image, resulting in an image as shown in the image. Figure 7 Image 703 is shown in (c). It can be seen that the observed brightness of the first region 703a in image 703 is close to that of the first region 701a in the first image 701. The observed brightness of the second region 703b in image 703 is significantly higher than that of the second region 701b in the first image 701. Compared with the first image 701, the first region 703a and the second region 703b in image 703 have a higher contrast between light and dark, and image 703 has a larger dynamic range.

[0137] It should be noted that the first image displayed on the phone can be created by another terminal device and then transmitted to the phone, or it can be created by the phone itself. In other words, the process of creating an HDR image and the process of displaying an HDR image can be completed by the same device or by different devices; no specific restrictions are placed here.

[0138] In summary, this application achieves the effect of increasing the observed brightness of each pixel in the second region without changing it during the display of the first image. This is achieved by simultaneously increasing the observed brightness of pixels in the second region without altering their brightness, and by decreasing the pixel values ​​of pixels in the first region to decrease their observed brightness, and by increasing or maintaining the pixel values ​​of pixels in the second region to increase or maintain their observed brightness. In other words, during the display of an HDR image, bright areas in the HDR image can be made brighter, and dark areas can be made darker or remain unchanged, thereby increasing the contrast and expanding the dynamic range of the HDR image.

[0139] In addition, the HDR image display method provided in this application embodiment can adapt to the brightness capability supported by the screen of the terminal device, and can make full use of the brightness capability of the terminal device.

[0140] This application also provides a chip system, such as... Figure 8 As shown, the chip system includes at least one processor 801 and at least one interface circuit 802. The processor 801 and the interface circuit 802 are interconnected via lines. For example, the interface circuit 802 can be used to receive signals from other devices (e.g., the memory of a terminal device). As another example, the interface circuit 802 can be used to send signals to other devices (e.g., the processor 801).

[0141] For example, interface circuit 802 can read instructions stored in the memory of a terminal device and send those instructions to processor 801. When the instructions are executed by processor 801, they can cause electronic devices (such as...) to... Figure 1 The mobile phone in the embodiment performs the steps described above.

[0142] Of course, the chip system may also include other discrete components, and this application embodiment does not specifically limit this.

[0143] This application also provides a computer-readable storage medium, which includes computer instructions that, when the computer instructions are used in an electronic device (such as...), Figure 1 When the method is run on a mobile phone, it causes the electronic device to perform the various functions or steps performed by the electronic device in the above method embodiments.

[0144] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to perform various functions or steps performed by the electronic device in the above method embodiments.

[0145] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0146] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0147] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0148] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0149] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A terminal device, characterized by comprising: The terminal device comprises a display screen, a memory and one or more processors coupled with the memory; The memory is configured to store computer program code comprising computer instructions; and the one or more processors are configured to execute the computer instructions to perform the following operations: In response to the first operation, a plurality of frames of images are acquired, the plurality of frames of images corresponding to a plurality of exposure parameters one-to-one, the plurality of exposure parameters comprising EV0, EV-1, EV-2, EV+1 and EV+2; The plurality of frames of images are tone-mapped to obtain a first image; Pixel values of each pixel point in the first image are acquired; A target exposure parameter is determined according to the plurality of exposure parameters; The target exposure parameter is determined according to the plurality of exposure parameters, comprising: In the case that dark part details in the first image are clear, a first exposure parameter is determined as the target exposure parameter, the first exposure parameter being one of the plurality of exposure parameters or a parameter obtained by combining at least two parameters in the plurality of exposure parameters; In the case that highlight regions in the first image are clear and there is no overexposure, a second exposure parameter is determined as the target exposure parameter, the second exposure parameter being one of the plurality of exposure parameters or a parameter obtained by combining at least two parameters in the plurality of exposure parameters; In the case that there is clear light-dark contrast in the first image and there is no overexposure or underexposure, a third exposure parameter is determined as the target exposure parameter, the third exposure parameter being one of the plurality of exposure parameters or a parameter obtained by combining at least two parameters in the plurality of exposure parameters; The first exposure parameter is greater than the third exposure parameter, and the third exposure parameter is greater than the second exposure parameter; A target exposure image is determined according to the target exposure parameter; The target exposure image is determined according to the target exposure parameter, comprising: In the case that the target exposure parameter is one of the plurality of exposure parameters, an image captured under the target exposure parameter is determined as the target exposure image; In the case that the target exposure parameter is a parameter obtained by combining K exposure parameters in the plurality of exposure parameters, K target exposure images captured under the K exposure parameters are fused to obtain the target exposure image, wherein K is a positive integer greater than 1; A target pixel grouping strategy is determined; Under the target pixel grouping strategy, luminance information of pixel points in the first image under the target exposure parameter is determined based on the target exposure image; In the case that the target pixel grouping strategy is a first preset pixel grouping strategy, the luminance information of the pixel points in the first image under the target exposure parameter is determined based on the target exposure image, comprising: Luminance information of a first pixel point in the target exposure image is calculated according to a pixel value of the first pixel point, the first pixel point being any one of pixel points in the target exposure image; associate the luminance information of the first pixel point with a second pixel point in the first image, wherein a position of the second pixel point in the first image is same as a position of the first pixel point in the target exposure image; in a case where the target pixel grouping strategy is a second preset pixel grouping strategy, the determining, based on the target exposure image, the luminance information of a pixel point in the first image under the target exposure parameter comprises: determining luminance information of a third pixel point in the first pixel group in the target exposure image according to a pixel value of the third pixel point, wherein the first pixel group is composed of the third pixel point, a fourth pixel point and a fifth pixel point; determining luminance information of the fourth pixel point according to a pixel value of the fourth pixel point; determining luminance information of the fifth pixel point according to a pixel value of the fifth pixel point; determining average luminance information between the luminance information of the third pixel point, the luminance information of the fourth pixel point and the luminance information of the fifth pixel point; associating the average luminance information as luminance information corresponding to a second pixel group in the first image, wherein a position of the second pixel group in the first image is same as a position of the first pixel group in the target exposure image, and the second pixel group is composed of a sixth pixel point, a seventh pixel point and an eighth pixel point; performing an encoding operation on the pixel value of the first image, the associated luminance information and the target pixel grouping strategy to obtain an image file; in response to an operation of a user viewing the image file, performing a decoding operation on the image file to obtain pixel values corresponding to a plurality of pixel points in the first image and a plurality of luminance information; determining the luminance information corresponding to each pixel point in the first image based on the target pixel grouping strategy; wherein, when the target pixel grouping strategy is the second preset pixel grouping strategy, the determining the luminance information corresponding to each pixel point in the first image based on the target pixel grouping strategy comprises: obtaining, from the decoded plurality of luminance information, the luminance information associated with the second pixel group in the first image, wherein the luminance information associated with the second pixel group is the luminance information of the sixth pixel point, the seventh pixel point and the eighth pixel point; determining an average luminance value of the first image according to the luminance information of each pixel point in the first image; determining a first threshold value according to the average luminance value of the first image, wherein the first threshold value is greater than or equal to the average luminance value; dividing the first image into a first region and a second region based on the first threshold value, wherein the luminance value of a pixel point in the first region is less than the first threshold value, and the luminance value of a pixel point in the second region is greater than the first threshold value; determining a target screen brightness based on a current screen brightness and a maximum screen brightness of the display screen, wherein the target screen brightness is greater than the current screen brightness and less than or equal to the maximum screen brightness; adjusting pixel values of the pixel points in the first region based on a first adjustment coefficient and adjusting pixel values of the pixel points in the second region based on a second adjustment coefficient to obtain an expanded image, wherein the first adjustment coefficient is less than 1 and the second adjustment coefficient is greater than or equal to 1; increasing the screen brightness of the display screen to the target screen brightness, and displaying the expanded image.

2. The terminal device according to claim 1, characterized by The bit width of the target brightness value of the pixel points of the first image is 8 bits or 10 bits.

3. The terminal device according to claim 1 or 2, characterized by The decoded pixel points form first data, and the decoded brightness information forms second data, and the resolution of the second data is less than the resolution of the first data.

4. The terminal device according to any one of claims 1-3, characterized by, The brightness value of the first image is calculated based on the formula L = 0.299R + 0.587G + 0.114B, where L is the brightness value, and R, G, and B are three components of the pixel point.

5. The terminal device according to claim 1, characterized by The second region includes M pixel points, and the second adjustment coefficient includes M sub-adjustment coefficients, and the M pixel points correspond to the M sub-adjustment coefficients one by one, and the one or more processors are further configured to perform the following operations: calculating the sub-adjustment coefficient of the jth pixel point according to the target brightness value of the jth pixel point, where j≤M; the sub-adjustment coefficient of the jth pixel point is positively correlated with the target brightness value of the jth pixel point, and the sub-adjustment coefficient of the jth pixel point is greater than 1; adjusting the pixel value of the jth pixel point based on the sub-adjustment coefficient of the jth pixel point.

6. The terminal device of claim 1, wherein, The first image further includes a third region, and the one or more processors are further configured to perform the following operations: calculating the average brightness value of the target brightness value of each pixel point; determining a second threshold and a third threshold according to the average brightness value, the second threshold being less than the average brightness value, the third threshold being greater than the average brightness value, and the second threshold and the third threshold being positively correlated with the average brightness value; if the target brightness value of any one pixel point is less than the second threshold, dividing the any one pixel point to the first region; if the target brightness value of any one pixel point is greater than or equal to the second threshold and less than the third threshold, dividing the any one pixel point to the third region; if the target brightness value of any one pixel point is greater than or equal to the third threshold, dividing the any one pixel point to the second region.

7. The terminal device of claim 1, wherein, The first adjustment coefficient is the ratio of the current screen brightness to the target screen brightness, the current screen brightness is the screen brightness before the display screen adjusts the screen brightness, and the target screen brightness is the screen brightness after the display screen adjusts the screen brightness.

8. A method of displaying an image, characterized by The method is applied to a terminal device, and the method comprises: in response to a first operation, obtaining a plurality of images, the plurality of images corresponding to a plurality of exposure parameters one by one, the plurality of exposure parameters including EV0, EV-1, EV-2, EV+1, and EV+2; performing tone mapping on the plurality of images to obtain a first image; obtaining pixel values of each pixel point in the first image; determining a target exposure parameter according to the plurality of exposure parameters; wherein the determining of the target exposure parameter according to the plurality of exposure parameters comprises: determine the first exposure parameter as the target exposure parameter in a case that dark details of the first image are clear, the first exposure parameter being one of the plurality of exposure parameters or a parameter obtained by combination of at least two of the plurality of exposure parameters; determine the second exposure parameter as the target exposure parameter in a case that highlight regions of the first image are clear and overexposure does not exist, the second exposure parameter being one of the plurality of exposure parameters or a parameter obtained by combination of at least two of the plurality of exposure parameters; determine the third exposure parameter as the target exposure parameter in a case that the first image has clear light-dark contrast and neither overexposure nor underexposure exists, the third exposure parameter being one of the plurality of exposure parameters or a parameter obtained by combination of at least two of the plurality of exposure parameters; wherein the first exposure parameter is greater than the third exposure parameter, and the third exposure parameter is greater than the second exposure parameter; determine a target exposure image according to the target exposure parameter; wherein the determining of the target exposure image according to the target exposure parameter comprises: in a case that the target exposure parameter is one of the plurality of exposure parameters, determine an image captured under the target exposure parameter as the target exposure image; in a case that the target exposure parameter is a parameter obtained by combination of K exposure parameters of the plurality of exposure parameters, fuse K images captured under the K exposure parameters to obtain the target exposure image, where K is a positive integer greater than 1; determine a target pixel grouping strategy; determine, based on the target exposure image, luminance information of a pixel point in the first image under the target exposure parameter according to the target pixel grouping strategy; wherein, in a case that the target pixel grouping strategy is a first preset pixel grouping strategy, the determining of the luminance information of the pixel point in the first image under the target exposure parameter based on the target exposure image comprises: calculate luminance information of a first pixel point in the target exposure image according to a pixel value of the first pixel point, the first pixel point being any one of pixel points in the target exposure image; associate the luminance information of the first pixel point with a second pixel point in the first image, where a position of the second pixel point in the first image is the same as a position of the first pixel point in the target exposure image; wherein, in a case that the target pixel grouping strategy is a second preset pixel grouping strategy, the determining of the luminance information of the pixel point in the first image under the target exposure parameter based on the target exposure image comprises: determine luminance information of a third pixel point in a first pixel group in the target exposure image according to a pixel value of the third pixel point, the first pixel group being composed of the third pixel point, a fourth pixel point and a fifth pixel point; determine luminance information of the fourth pixel point according to a pixel value of the fourth pixel point; determine luminance information of the fifth pixel point according to a pixel value of the fifth pixel point; determine average luminance information between the luminance information of the third pixel point, the luminance information of the fourth pixel point and the luminance information of the fifth pixel point; associate the average luminance information as luminance information corresponding to a second pixel group in the first image, wherein the second pixel group has the same position in the first image as the first pixel group in the target exposure image, and the second pixel group is composed of a sixth pixel point, a seventh pixel point and an eighth pixel point; perform an encoding operation on the pixel value of the first image, the associated luminance information and the target pixel grouping strategy to obtain an image file; in response to a user operation of viewing the image file, perform a decoding operation on the image file to obtain pixel values corresponding to a plurality of pixel points in the first image and a plurality of luminance information; determine the luminance information corresponding to each pixel point in the first image based on the target pixel grouping strategy; wherein when the target pixel grouping strategy is the second preset pixel grouping strategy, the determination of the luminance information corresponding to each pixel point in the first image based on the target pixel grouping strategy comprises: obtaining the luminance information associated with the second pixel group in the first image from the decoded plurality of luminance information, wherein the luminance information associated with the second pixel group is the luminance information of the sixth pixel point, the seventh pixel point and the eighth pixel point; determine an average luminance value of the first image according to the luminance information of each pixel point of the first image; determine a first threshold value according to the average luminance value of the first image, wherein the first threshold value is greater than or equal to the average luminance value; divide the first image into a first region and a second region based on the first threshold value, wherein the luminance value of the pixel point in the first region is less than the first threshold value, and the luminance value of the pixel point in the second region is greater than the first threshold value; determine a target screen brightness based on the current screen brightness and the maximum screen brightness of the display screen, wherein the target screen brightness is greater than the current screen brightness and less than or equal to the maximum screen brightness; adjust the pixel value of the pixel point in the first region based on a first adjustment coefficient and adjust the pixel value of the pixel point in the second region based on a second adjustment coefficient to obtain an expanded image, wherein the first adjustment coefficient is less than 1 and the second adjustment coefficient is greater than or equal to 1; increase the screen brightness of the display screen to the target screen brightness and display the expanded image.

9. The method of claim 8, wherein, The second region includes M pixel points, the second adjustment coefficient includes M sub-adjustment coefficients, the M pixel points correspond to the M sub-adjustment coefficients one by one, and the adjustment of the pixel value of the pixel point in the second region based on the second adjustment coefficient comprises: calculate the sub-adjustment coefficient of the jth pixel point according to the target luminance value of the jth pixel point, wherein j≤M; the sub-adjustment coefficient of the jth pixel point is positively correlated with the target luminance value of the jth pixel point, and the sub-adjustment coefficient of the jth pixel point is greater than 1; adjusting a pixel value of the jth pixel point based on a sub-adjustment coefficient of the jth pixel point.

10. The method of claim 8, wherein, The first image further comprises a third region, and the method further comprises: dividing the first image into a plurality of image regions according to the target brightness value of each pixel point; wherein the first region comprises: calculating a brightness mean value of the target brightness value of each pixel point; determining a second threshold and a third threshold according to the brightness mean value, the second threshold being less than the brightness mean value, the third threshold being greater than the brightness mean value, and the second threshold and the third threshold being positively correlated with the brightness mean value; if the target brightness value of any one pixel point is less than the second threshold, dividing the any one pixel point into the first region; if the target brightness value of any one pixel point is greater than or equal to the second threshold and less than the third threshold, dividing the any one pixel point into the third region; if the target brightness value of any one pixel point is greater than or equal to the third threshold, dividing the any one pixel point into the second region.

11. The method of claim 8, wherein, The first adjustment coefficient is a ratio of a current screen brightness to a target screen brightness, and the current screen brightness is a screen brightness before the display screen adjusts the screen brightness.

12. A computer-readable storage medium, characterized in that, The computer program product comprises computer instructions, and when the computer instructions run on a terminal, the terminal executes the method according to any one of claims 8-11. The computer program product comprises computer instructions, and when the computer instructions run on a terminal, the terminal executes the method according to any one of claims 8-11.

Images