Method for generating HDR image and electronic equipment

Abstract

The embodiment of the invention relates to the technical field of terminals, in particular to a method for generating an HDR image and electronic equipment, and can improve the display effect of the HDR image. The method comprises the following steps: acquiring a first HDR image, and performing tone mapping on the first HDR image to obtain a second HDR image; the first HDR image comprises a first highlight area, and the first highlight area comprises a highlight object; the second HDR image comprises a second highlight area, and the second highlight area corresponds to the first highlight area; correcting the gradient direction of the first pixel in the second highlight area into the gradient direction of the second pixel in the first highlight area to obtain a third HDR image; the second pixel corresponds to the first pixel; wherein the gradient direction of one pixel is positive and negative of the difference value between the brightness value of the next pixel of the pixel on a first direction axis and the brightness value of the pixel, and the first direction axis comprises an x axis and a y axis; wherein the dynamic imaging range of the third HDR image is smaller than that of the first HDR image, and the third HDR image is used for displaying.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of terminals, and in particular, to a method for generating an HDR image and an electronic device. Background Art

[0002] Most terminal devices support high dynamic range (HDR) photography technology. The HDR photography technology can record information of a larger brightness range and display more details of the bright and dark parts in an image. The dynamic imaging range of the HDR image does not match that of the display device, and generally, the dynamic imaging range of the HDR image is higher than that of the display device. Therefore, in order to better display the HDR image, tone mapping is usually used to map the dynamic imaging range of the HDR image to the dynamic imaging range that the display device can display.

[0003] However, the display effect of the tone-mapped HDR image is not good. For example, for an HDR image taken of the sky in a sunset or evening glow scene, bright lines that do not conform to the natural scene, such as bright circles, halos, etc., and dark lines such as dark circles will appear around the sun in the tone-mapped HDR image. Summary of the Invention

[0004] The embodiments of the present application provide a method for generating an HDR image and an electronic device, which can avoid the problem of unnatural brightness transition in the HDR image after tone mapping and improve the display effect of the HDR image.

[0005] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a method for generating an HDR image is provided, which is applied to an electronic device. The method includes: obtaining a first HDR image, and performing tone mapping on the first HDR image to obtain a second HDR image; the first HDR image includes a first highlight area, and the first highlight area includes a highlight object; the second HDR image includes a second highlight area, and the second highlight area corresponds to the first highlight area; then, correcting the gradient direction of a first pixel in the second highlight area to the gradient direction of a second pixel in the first highlight area to obtain a third HDR image; the second pixel corresponds to the first pixel. There are two gradient directions of a pixel, which are respectively the positive or negative of the difference between the brightness value of the next pixel of the pixel on the x-axis and the brightness value of the pixel, and the positive or negative of the difference between the brightness value of the next pixel of the pixel on the y-axis and the brightness value of the pixel. The dynamic imaging range of the third HDR image is smaller than that of the first HDR image, and the third HDR image is used for display. That is, the gradient direction of the corrected first pixel is consistent with the gradient direction of the second pixel on both the x-axis and the y-axis.

[0007] Since the first HDR image is not tone-mapped, the direction of brightness change of the pixels in the first highlight area can better restore the brightness change of the corresponding content in the natural scene. Therefore, if the direction of brightness change of the pixels in the second highlight area after correction in this solution is the same as that of the pixels in the first highlight area, it means that the direction of brightness change of the pixels in the corrected second highlight area can also restore the brightness change of the corresponding content in the natural scene. By adopting this solution, the problem of unnatural brightness transition at the edge of the high-light object caused by inconsistent brightness change directions can be effectively avoided, presenting a more real and natural display effect for the user.

[0008] In another possible implementation of the first aspect, the HDR dynamic imaging range of the display screen of the electronic device is smaller than that of the first HDR image, and the dynamic imaging range of the third HDR image is adapted to the HDR dynamic imaging range of the display screen. In this implementation, although the HDR dynamic imaging range of the display screen is smaller than that of the first HDR image, the dynamic imaging range of the processed third HDR image can be adapted to the HDR dynamic imaging range of the display screen, which can ensure the display effect of the electronic device.

[0009] In another possible implementation of the first aspect, the electronic device can correct the gradient direction of any first pixel in the second highlight area. The any first pixel can be a pixel with a reversed gradient direction or a pixel with a non-reversed gradient direction. In this way, the electronic device can correct all the first pixels in the second highlight area so that the gradient directions of all the first pixels in the corrected second highlight area are the same as those of the corresponding second pixels.

[0010] In another possible implementation of the first aspect, the first pixel is a pixel in the second highlight area whose gradient direction is inconsistent with that of the corresponding pixel in the first highlight area, that is, a pixel with a reversed gradient direction. In this way, the electronic device can correct the first pixels with reversed gradient directions in the second highlight area so that the gradient directions of all the first pixels in the corrected second highlight area are the same as those of the corresponding second pixels.

[0011] In another possible implementation of the first aspect, the electronic device can receive the first HDR image from the second electronic device. Since the HDR dynamic imaging range of the display screen of the electronic device is smaller than that of the first HDR image, the electronic device performs tone mapping on the first HDR image to obtain the second HDR image.

[0012] In another possible implementation of the first aspect, the electronic device may collect multiple frames of images and fuse them to obtain the first HDR image. Specifically, in response to a user's shooting operation in the HDR scenario, the first HDR image is obtained; if the HDR dynamic imaging range of the display screen of the electronic device is smaller than the HDR dynamic imaging range of the first HDR image, then tone mapping is performed on the first HDR image to obtain the second HDR image.

[0013] In another possible implementation of the first aspect, the electronic device corrects the brightness value of the adjacent pixel of the first pixel to correct the brightness change direction of the first pixel. Specifically, the brightness value of the next pixel of the first pixel on the x-axis is corrected so that the gradient direction of the first pixel on the x-axis is consistent with the gradient direction of the second pixel on the x-axis. The brightness value of the next pixel of the first pixel on the y-axis is corrected so that the gradient direction of the first pixel on the y-axis is consistent with the gradient direction of the second pixel on the y-axis. This implementation provides a way to correct the gradient direction of the first pixel.

[0014] In another possible implementation of the first aspect, the electronic device corrects the brightness value of the pixel (i + 1, j) in the second highlighted area so that the gradient direction of the pixel (i, j) in the second highlighted area on the x-axis is consistent with the gradient direction of the pixel (i, j) in the first highlighted area on the x-axis. The brightness value of the pixel (i, j + 1) in the second highlighted area is corrected so that the gradient direction of the pixel (i, j) in the second highlighted area on the y-axis is consistent with the gradient direction of the pixel (i, j) in the first highlighted area on the y-axis. Wherein, the pixel (i, j) in the second highlighted area is the first pixel, and the pixel (i, j) in the first highlighted area is the second pixel; the pixel (i + 1, j) in the second highlighted area is the next pixel of the first pixel on the x-axis, and the pixel (i, j + 1) in the second highlighted area is the next pixel of the first pixel on the y-axis. Wherein, i takes values in sequence in {1, 2,..., m}, j takes values in sequence in {1, 2,..., n}, and the first pixel is the pixel in the second highlighted area whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted area. In this way, the electronic device can correct the first pixel with a reversed gradient among the pixels from (1, 1) to (m, n).

[0015] In another possible implementation of the first aspect, the electronic device corrects the brightness value of the pixel (i + 1, j) in the second highlighted area so that the gradient direction of the pixel (i, j) in the second highlighted area along the x-axis is the same as that of the pixel (i, j) in the first highlighted area along the x-axis. The brightness value of the pixel (i, j + 1) in the second highlighted area is corrected so that the gradient direction of the pixel (i, j) in the second highlighted area along the y-axis is the same as that of the pixel (i, j) in the first highlighted area along the y-axis. Here, the pixel (i, j) in the second highlighted area is the first pixel, and the pixel (i, j) in the first highlighted area is the second pixel; the pixel (i + 1, j) in the second highlighted area is the next pixel of the first pixel along the x-axis, and the pixel (i, j + 1) in the second highlighted area is the next pixel of the first pixel along the y-axis. Here, i takes values in sequence from {1, 2, ……, m}, and j takes values in sequence from {1, 2, ……, n}. In this way, the electronic device can correct each first pixel from pixel (1, 1) to pixel (m, n) in the second highlighted area, and each first pixel includes the first pixel with a reverse gradient and the first pixel with the same gradient direction.

[0016] In another possible implementation of the first aspect, the brightness value of the corrected pixel (i + 1, j) in the second highlighted area is: the product of the gradient of the second pixel along the x-axis and the first correction coefficient plus the brightness value of the pixel (i, j) in the second highlighted area. The brightness value of the corrected pixel (i, j + 1) in the second highlighted area is: the product of the gradient of the second pixel along the y-axis and the second correction coefficient plus the brightness value of the pixel (i, j) in the second highlighted area. The gradient direction of the modified first pixel is the same as that of the second pixel, and the gradient value is a scaled value of the gradient value of the second pixel.

[0017] In another possible implementation of the first aspect, after obtaining the third HDR image, the electronic device uses the first image as a reference frame to perform color correction on the fourth HDR image to obtain a fifth HDR image. Here, the fourth HDR image is the third HDR image. The first HDR image is obtained by fusing multiple frames of images, and the multiple frames of images include the first image and at least one second image; the exposure duration of the second image is greater than that of the first image. Color correction can effectively avoid the problem of excessive image saturation after tone mapping in conventional technologies, making the corrected image better reflect the colors in natural scenes. The electronic device can first perform brightness direction correction on the second HDR image, and then perform color correction on the third HDR image. In this way, the corrected image has a uniform brightness transition and natural and realistic colors.

[0018] In another possible implementation of the first aspect, the electronic device may first determine whether the second highlighted area includes a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted area. If the second highlighted area includes a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted area, the gradient direction of the first pixel in the second highlighted area is corrected. If the second highlighted area does not include a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted area, the gradient direction of the first pixel in the second highlighted area is not corrected.

[0019] In another possible implementation of the first aspect, if the second highlighted area does not include a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted area, the electronic device corrects the color of the second HDR image. Specifically, using the first image as a reference frame, color correction is performed on the fourth HDR image to obtain a sixth HDR image; wherein, the fourth HDR image is the second HDR image; wherein, the first HDR image is obtained by fusing multiple frames of images, and the multiple frames of images include the first image and at least one second image; the exposure duration of the second image is greater than the exposure duration of the first image. Color correction can effectively avoid the problem of excessive image saturation after tone mapping in conventional technologies, making the corrected image better reflect the colors in natural scenes. If the second HDR image does not include a first pixel with a reverse gradient, the electronic device directly performs color correction on the second HDR image. In this way, the corrected image has a uniform brightness transition and natural and true colors.

[0020] In another possible implementation of the first aspect, the color correction method is specifically: obtaining the first chrominance component and the first hue component of the first image; using the first chrominance component to replace the second chrominance component of the fourth HDR image, and using the first hue component to replace the second hue component of the fourth HDR image. This implementation provides a specific implementation of color correction.

[0021] In a second aspect, the present application provides an electronic device, which includes: a memory, a display screen, and one or more processors. The memory and the display screen are coupled to the processor. Among them, the memory is used to store computer program code, and the computer program code includes computer instructions. When the computer instructions are executed by the processor, the electronic device is enabled to execute the method according to the first aspect and any of its possible design manners.

[0022] In a third aspect, the present application provides a chip system, which can be applied to an electronic device including a memory. The chip system includes one or more interface circuits and one or more processors. The interface circuits and the processors are interconnected by lines. The interface circuit is configured to receive a signal from the memory and send the signal to the processor, and the signal includes computer instructions stored in the memory. When the processor executes the computer instructions, the electronic device executes the method according to the first aspect and any possible design thereof.

[0023] In a fourth aspect, the present application provides a computer-readable storage medium, which includes computer instructions. When the computer instructions run on an electronic device, the electronic device is caused to execute the method according to the first aspect and any possible design thereof.

[0024] In a fifth aspect, the present application provides a computer program product, which when running on a computer, causes the computer to execute the method according to the first aspect and any possible design thereof.

[0025] It can be understood that the beneficial effects that can be achieved by the electronic device according to any possible design of the second aspect, the chip system of the third aspect, the computer-readable storage medium of the fourth aspect, and the computer program product of the fifth aspect can refer to the beneficial effects in the first aspect and any possible design thereof, which will not be elaborated here. Description of the Drawings

[0026] Figure 1 Schematic diagram of an HDR image generation process provided by an embodiment of the present application;

[0027] Figure 2 Schematic diagram of different display effects of an untonemapped HDR image and a tonemapped HDR image provided by an embodiment of the present application;

[0028] Figure 3 Schematic diagram of the display effect of a tonemapped HDR image in the conventional technology;

[0029] Figure 4 Schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application;

[0030] Figure 5 Schematic diagram of the software architecture of an electronic device provided by an embodiment of the present application;

[0031] Figure 6 Schematic diagram of the interface display of an electronic device provided by an embodiment of the present application;

[0032] Figure 7 Schematic diagram of the process of a method for generating an HDR image provided by an embodiment of the present application;

[0033] Figure 8 This is a comparison diagram of a first highlighted area and a second highlighted area provided by an embodiment of the present application;

[0034] Figure 9 This is a schematic flowchart of another method for generating an HDR image provided by an embodiment of the present application;

[0035] Figure 10 This is a schematic flowchart of yet another method for generating an HDR image provided by an embodiment of the present application;

[0036] Figure 11 This is a schematic flowchart of a method for generating an HDR image provided by an embodiment of the present application;

[0037] Figure 12 This is a comparison diagram of the display effects of an HDR image before and after correction provided by an embodiment of the present application. Detailed implementation manners

[0038] In digital images, the dynamic imaging range can represent the difference between the darkest and the brightest in the image. For example, the dynamic imaging range can be the ratio of the maximum brightness to the minimum brightness in the image, and its unit can be dB.

[0039] According to different dynamic imaging ranges, images can be classified into standard dynamic range (SDR) images, high dynamic range (HDR) images, etc. The dynamic imaging range of SDR images is between 1 nit and 100 nits. The dynamic imaging range of HDR images is between 0.001 nit and 10,000 nits. Among them, nit is the unit of brightness. It can be seen that compared with ordinary images such as SDR images, HDR images expand the brightness range of the images, and thus can record information of a larger brightness range and display more highlight and shadow details in the images.

[0040] In conventional technologies, the dynamic imaging range that a display device can display is usually lower than the dynamic imaging range of an HDR image. This display device can be a display device that can display SDR images or a display device that can display HDR images. That is to say, the dynamic imaging range of an HDR image does not match the dynamic imaging range that the display device can display, and usually the dynamic imaging range of an HDR image is higher than the dynamic imaging range that the display device can display.

[0041] Therefore, tone mapping is usually performed on HDR images to map the dynamic imaging range of the HDR images to the dynamic imaging range that can be displayed by the display device. Among them, tone mapping refers to mapping the brightness of the HDR image to the brightness range that the target display device can display. For example, an HDR image includes pixels with a brightness value of 4000 nits, while the target device (such as a mobile phone) can only display pixels with a brightness of 0.5 - 500 nits. Since the brightness value exceeds the displayable brightness value of the mobile phone, the mobile phone may display the pixel as white when displaying it. The HDR image includes pixels with a brightness value of 0.05 nits. Due to the displayable brightness value of the mobile phone, the mobile phone may display the pixel as black when displaying it. As Figure 2 shown, when an electronic device such as a mobile phone displays an unmapped HDR image, its display effect is as Figure 2 shown on the left side in Figure 2 . When the mobile phone displays the mapped HDR image, its display effect is as

[0042] shown on the right side in Figure 1 . Usually, a tone mapping algorithm is used to perform tone mapping on the HDR image. The tone mapping algorithm maps the HDR image with a high dynamic imaging range to the low dynamic imaging range that can be displayed by the target device. The tone mapping algorithm is a non-linear mapping, so it can retain the details and local contrast information in the HDR image, that is, the bright and dark regions in the HDR image. Exemplarily, taking a mobile phone as an example, as Figure 1 shown, the mobile phone captures an HDR image corresponding to a natural scene, the mobile phone uses tone mapping to process the HDR image, and finally displays the HDR image with a reduced dynamic imaging range.

[0043] In the conventional technology, when the tone-mapped HDR image is displayed, there will be a problem of unnatural display. Especially when the HDR image includes a high-brightness object, at the transition between the high-brightness object and the surrounding environment, such as the edge of the high-brightness object, there may be a problem of unnatural transition. As Figure 3As shown by a in [description], the highlighted object can be a light source such as the sun. For example, when a user takes a picture of the sky at sunset or during the evening glow, black or white circles usually appear at the edge of the sun. One possible reason is that, in order to highlight the contrast between light and dark in the HDR image, the tone mapping algorithm increases the contrast between the highlighted object and its surrounding environment. For example, the contrast is increased by increasing the brightness value of the edge pixels of the highlighted object. It should be understood that in a natural scene, the brightness of a highlighted object such as the sun decreases from the center to the edge. After increasing the brightness value of the edge pixels of the sun, white circles may appear at the edge. Another possible reason is that the tone mapping algorithm increases the contrast by reducing the brightness value of the edge pixels of the sun. In this way, dark circles may appear at the edge. There is also a possible reason that the parameter settings of the existing tone mapping algorithm are unreasonable and the parameters are not fine enough. The black and white circles mentioned above do not exist in the natural scene and also do not exist in the HDR image before processing. They can be displayed in the processed HDR image after the tone algorithm processing.

[0044] In summary, when the HDR image includes a highlighted object, the display effect of the HDR image processed by the existing tone mapping algorithm is not good. Specifically, bright lines that do not conform to the natural scene, such as bright circles, halos, etc., and dark lines such as dark circles, appear at the edge of the high - light object.

[0045] Therefore, the embodiment of the present application provides a method for generating an HDR image, which can correct the gradient direction of the first pixel in the second highlighted area of the second HDR image after tone mapping to be the same as the gradient direction of the second pixel corresponding to the first pixel in the second highlighted area of the first HDR image before tone mapping. The gradient direction of the first pixel in the corrected second highlighted area is the same as the gradient direction of the corresponding second pixel in the first highlighted area before tone mapping. Among them, the gradient direction of a pixel is the direction of the brightness change of the pixel. The gradient direction of the first pixel includes the gradient direction on the x - axis and the gradient direction on the y - axis. That is to say, the brightness change direction of the corrected first pixel on the x - axis and the brightness change direction on the y - axis are both the same as the brightness change direction of the second pixel on the x - axis and the brightness change direction on the y - axis.

[0046] Since the first HDR image has not undergone tone mapping, the brightness change direction of the pixels in the first highlighted area can better restore the brightness change of the corresponding content in the natural scene. Therefore, if the brightness change direction of the pixels in the corrected second highlighted area is the same as the brightness change direction of the pixels in the first highlighted area, it means that the brightness change direction of the pixels in the corrected second highlighted area can also restore the brightness change of the corresponding content in the natural scene. By adopting this solution, the problem of unnatural brightness transition at the edge of the high - light object caused by inconsistent brightness change directions can be effectively avoided, presenting a more real and natural display effect for the user.

[0047] The method provided by the embodiments of this application can be applied to an electronic device with data processing capabilities and a display screen. The above-mentioned first electronic device may include a server, a mobile phone, a tablet computer, a laptop computer, a personal computer (PC), an ultra-mobile personal computer (UMPC), a handheld computer, a netbook, a smart home device (such as a smart TV, a smart screen, a large screen, a smart speaker, a smart air conditioner, etc.), a personal digital assistant (PDA), a wearable device (such as a smart watch, a smart bracelet, etc.), a vehicle-mounted device, a virtual reality device, etc. The embodiments of this application do not make any restrictions on this. In the embodiments of this application, the above-mentioned electronic device is an electronic device that can run an operating system and install application programs. Optionally, the operating system run by the electronic device may be system, system, system, etc.

[0048] Exemplarily, please refer to Figure 4 , which shows a schematic structural diagram of an electronic device 400. The electronic device 400 may include a processor 410, an external memory interface 420, an internal memory 421, an audio module 430, a speaker 430A, a microphone 430B, a display screen 440, a communication module 450, a power module 460, an input device 470, a sensor module 480, a camera 490, etc. Among them, the sensor module 480 may include a pressure sensor, a touch sensor, etc.

[0049] It can be understood that the structure schematically shown in the embodiments of this application does not constitute a specific limitation on the electronic device 400. In other embodiments of this application, the electronic device 400 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0050] The processor 410 may include one or more processing units. For example, the processor 410 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent components or integrated in one or more processors. In some embodiments, the electronic device 400 may also include one or more processors 410.

[0051] A memory may also be provided in the processor 410 for storing instructions and data. In some embodiments, the memory in the processor 410 is a cache memory. This memory can save the instructions or data that the processor 410 has just used or recycled. If the processor 410 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 410, and thus improves the efficiency of the system.

[0052] The external memory interface 420 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 400. The external memory card communicates with the processor 410 through the external memory interface 420 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0053] The internal memory 421 can be used to store one or more computer programs, and the one or more computer programs include instructions. The processor 410 can execute the application running method provided in some embodiments of the present application, as well as various applications and data management, etc., by running the above instructions stored in the internal memory 421. In some embodiments, the processor 410 can execute the application running method provided in the embodiments of the present application, as well as other applications and data management, by running the instructions stored in the internal memory 421 and / or the instructions stored in the memory provided in the processor 410.

[0054] The electronic device 400 can implement audio functions through the audio module 430, the speaker 430A, the microphone 430B, and the application processor, etc. For example, music playback, recording, etc. The speaker 430A, also known as the "loudspeaker", is used to convert the audio electrical signal into a sound signal.

[0055] The microphone 430B, also known as a "microphone" or "transmitter", is used to convert sound signals into electrical signals. The user can input sound signals into the microphone 430B by speaking close to it with their mouth.

[0056] The communication function of the electronic device 400 can be implemented through the antenna 1, antenna 2, and communication module 450, etc.

[0057] The communication module 450 can provide wireless communication solutions for the electronic device 400, including cellular, Wi-Fi, Bluetooth (BT), and wireless data transmission modules (e.g., 433 MHz, 868 MHz, 915 MHz), etc. The communication module 450 can be one or more devices integrating at least one communication processing module. The communication module 450 receives electromagnetic waves via the antenna 1 or antenna 2, filters and frequency-modulates the electromagnetic wave signals, and sends the processed signals to the processor 410. The communication module 450 can also receive the signals to be sent from the processor 410, frequency-modulate and amplify them, and convert them into electromagnetic waves through the antenna 1 or antenna 2 for radiation.

[0058] The electronic device 400 realizes the display function through the GPU, the display screen 440, and the application processor, etc. The GPU is a microprocessor for image processing, connecting the display screen 440 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 410 may include one or more GPUs, which execute program instructions to generate or change display information.

[0059] The display screen 440 is used to display images, videos, etc. The display screen 440 includes a display panel. In some embodiments, the electronic device 400 may include 1 or N display screens 440, where N is a positive integer greater than 1. In the embodiments of the present application, the display screen 440 can be used to display the UI and receive user operations on the UI.

[0060] The power module 460 can be used to supply power to each component included in the electronic device 400. In some embodiments, the power module 460 can be a battery, such as a rechargeable battery.

[0061] The input device 470 may include a keyboard, a mouse, etc. The keyboard is used to input English letters, numbers, punctuation marks, etc. into the electronic device 400, thereby sending commands to the electronic device 400 and inputting data, etc.

[0062] The electronic device 400 can realize the shooting function through an ISP, a camera 490, a video codec, a GPU, a display screen 440, and an application processor. The ISP is used to process the data fed back by the camera 490. The camera 490 is used to capture static images or videos. In some embodiments, the electronic device 400 includes 1 or N cameras 490, where N is a positive integer greater than 1. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 290 is at a frequency point selection, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc. The video codec is used to compress or decompress digital video.

[0063] Take the above electronic device 400 as a mobile phone. The software system of the electronic device 400 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present invention takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device 400. The software structure is as follows: Figure 5 shown.

[0064] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime), the system library, and the kernel layer.

[0065] The application layer can include a series of application packages.

[0066] like Figure 5 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, social and other applications.

[0067] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0068] like Figure 5 As shown, the application framework layer may include a content provider, a view system, a resource manager, a notification manager, an input system, a brightness correction module, a color correction module, and the like.

[0069] The input system is used to monitor the input module of the mobile phone (such as the touch screen driver), convert the parameters input by the input module into usable events, and transfer them to the relevant modules in the upper layer. For example, the input system is used to monitor the touch screen of the mobile phone through the touch screen driver, convert the touch parameters generated by the touch operation input by the touch screen into usable events, and transfer them to the upper-layer APP.

[0070] The content provider is used to store and obtain data, and make this data accessible to application programs. The data can include videos, images, audio, incoming and outgoing calls, browsing history and bookmarks, phone books, etc.

[0071] The view system includes visible controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build the display interface of an application program.

[0072] The brightness correction module is used to correct the brightness direction of the first pixel in the second highlight area of the second HDR image. The color correction module is used to correct the second HDR image or the third HDR.

[0073] The notification manager enables application programs to display notification information in the status bar, can be used to convey message types of notifications, and can automatically disappear after a short stay without user interaction.

[0074] Android Runtime includes core libraries and a virtual machine. Android Runtime is responsible for the scheduling and management of the Android system. The core libraries contain two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core libraries of Android. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0075] The system library can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (such as OpenGL ES), 2D graphics engines (such as SGL), etc.

[0076] The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple application programs.

[0077] The media library supports the playback and recording of multiple common audio and video formats, as well as static image files, etc. The media library can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0078] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc.

[0079] The 2D graphics engine is a graphics engine for 2D drawing.

[0080] The kernel layer may include a touch screen driver, a display driver, a sensor driver, an audio driver, etc.

[0081] Taking the electronic device as a mobile phone as an example, in combination with the accompanying drawings, a method for generating an HDR image provided by an embodiment of the present application is introduced. This solution is applicable to the scenarios of taking pictures or videos. The first HDR image can be a photo or a video frame. The applicable scenarios of this solution are introduced below in combination with the accompanying drawings.

[0082] The mobile phone can receive the shooting operation of the user to obtain the first HDR image. If the HDR dynamic imaging range of the display screen of the mobile phone is smaller than the HDR dynamic imaging range of the first HDR image, the mobile phone performs tone mapping on the first HDR image to obtain the second HDR image.

[0083] For example, in the scenario of taking pictures, the mobile phone responds to the operation of the user to open the camera application and can display Figure 6 the preview interface 601 for taking pictures shown in (b) of. For example, the above operation of opening the camera application can be a click operation by the user on the Figure 6 application icon of the camera application shown in (a) of. The preview interface 601 for taking pictures may include a preview image 602. The mobile phone responds to the click operation of the user on the "shooting shutter" 603 in the preview interface 601. The mobile phone can execute S701 to obtain the first HDR image, and then execute S702 - S705 to obtain the processed HDR image.

[0084] Further, before executing this solution, the mobile phone can first identify whether the current shooting scene is a shooting scene with a high dynamic imaging range. If the current shooting scene is a shooting scene with a high dynamic imaging range, then in response to the user's click operation on the "Shooting Shutter" 603 in the preview interface 601, S701 is executed. Exemplarily, in response to the user opening the camera APP, the mobile phone collects at least one preview image, and based on the at least one preview image, identifies whether the current shooting scene is a shooting scene with a high dynamic imaging range. Specifically, for the at least one preview image, the mobile phone can calculate the average contrast of the at least one preview image, the average brightness of the dark area in the at least one preview image, or the average brightness of the bright area in the at least one preview image. If the average contrast of the at least one preview image is greater than a preset threshold, or the average brightness of the dark area in the at least one preview image is greater than a preset threshold, or the average brightness of the bright area in the at least one preview image is greater than a preset threshold, then the mobile phone identifies the current shooting scene as a shooting scene with a high dynamic imaging range. As another example, in response to the user opening the camera APP, the mobile phone collects at least one preview image, and the mobile phone identifies whether the preview image includes a preset shooting object. If the preview image includes a preset shooting object, the mobile phone identifies the current shooting scene as a shooting scene with a high dynamic imaging range. The preset shooting object can include preset shooting objects such as the sun, the sky, street lights, lamps, etc. Correspondingly, the shooting scene with a high dynamic imaging range can be a sunset scene, a sunset glow scene, etc.

[0085] Optionally, the mobile phone provides multiple shooting modes, and the multiple shooting modes include the HDR shooting mode. In response to the user selecting the HDR mode for shooting, the mobile phone can execute a method for generating an HDR image provided in an embodiment of the present application during shooting. For example, the camera preview interface of the mobile phone can include an HDR switch 604. In response to the user turning on the HDR switch and clicking the shooting shutter 603, the mobile phone can execute S701 to obtain a first HDR image, and then the mobile phone can execute S702 - S705 to obtain a processed HDR image. The gallery application can display the processed HDR image in a large - image preview mode.

[0086] Optionally, the mobile phone can receive a first HDR image from a second electronic device. If the HDR dynamic imaging range of the display screen of the mobile phone is smaller than the HDR dynamic imaging range of the first HDR image, then the mobile phone performs tone mapping on the first HDR image to obtain a second HDR image.

[0087] That is to say, the first HDR image can be obtained by fusing multiple frames of images collected by the mobile phone, or can be obtained by fusing multiple frames of images collected by a second electronic device. Below, taking the first HDR image obtained by fusing multiple frames of images collected by the mobile phone as an example, this solution is introduced. Figure 7 It is a schematic flowchart of a method for generating an HDR image provided in an embodiment of the present application.

[0088] S701, the mobile phone obtains the first HDR image. Specifically, S701 may include S701a and S701b.

[0089] S701a, the mobile phone captures N frames of images with different exposure values.

[0090] The exposure value can reflect the exposure level of the image. The exposure value is related to exposure parameters, and the exposure parameters include exposure duration, sensitivity, aperture coefficient of the lens, etc. The mobile phone captures N frames of images with N exposure values by adjusting one or more of the parameters of exposure duration, sensitivity, and aperture coefficient of the lens. N is an integer greater than 1. Exemplarily, taking the exposure duration as an example, the longer the exposure duration, the greater the exposure value of the image captured by the mobile phone, and the higher the brightness value of the image, the brighter the image. The shorter the exposure duration, the smaller the exposure value of the image captured by the mobile phone, and the lower the brightness value of the image, the darker the image. Taking the sensitivity as an example, the sensitivity indicates the sensitivity of the image sensor to light. The greater the sensitivity, the higher the brightness of the image captured by the mobile phone. On the contrary, the smaller the sensitivity, the lower the brightness of the image captured by the mobile phone.

[0091] The N frames of images may be RAW images captured by the camera. The N frames of images may be RGB images or YUV images. Among them, the RGB image includes R component, B component, and G component. Each pixel in the RGB image is composed of R component, B component, and G component. The YUV image includes Y component, U component, and V component. The Y component is used to describe the brightness of the pixel, and the U component and V component are used to describe the chromaticity of the pixel. Below, taking the N frames of images as RGB images as an example, this solution will be introduced.

[0092] In this solution, N is an integer greater than 1. That is to say, the mobile phone needs to capture at least two frames of images, such as the first image and at least one second image. The exposure value of the second image is greater than that of the first image. For another example, the mobile phone may capture three frames of images, including the first image with the smallest exposure value, and two second images with exposure values greater than that of the first image. For example, the exposure values of the first image, the second image 1, and the second image 2 may be EV0-1, EV0, and EV+1 in sequence. Exemplarily, the first image is the image captured by the mobile phone using the exposure duration a, the second image 1 is the image captured by the mobile phone using the exposure duration b, and the second image 2 is the image captured by the mobile phone using the exposure duration c. Among them, the exposure duration a is less than the exposure duration b and less than the exposure duration c.

[0093] The N-frame images include a long-exposure frame (second image), a short-exposure frame (first image), and a normal-exposure frame (second image). An image with an exposure value equal to the preset exposure value can be a normal-exposure frame. An image with an exposure value less than the preset exposure value is a short-exposure frame. An image with an exposure value greater than the preset exposure value is a long-exposure frame. Optionally, if the mobile phone captures N-frame images by adjusting the exposure duration. Among them, an image with an exposure duration equal to the first preset exposure duration can be a normal-exposure frame. An image with an exposure duration less than the preset exposure duration can be a short-exposure frame. An image with an exposure duration greater than the preset exposure duration is a long-exposure frame.

[0094] Optionally, if N = 2, the second image with a larger exposure value or a longer exposure duration in the N-frame images is a normal-exposure frame, and the first image with a smaller exposure value or a shorter exposure duration in the N-frame images is a short-exposure frame. If N = 2, the N-frame images can be a long-exposure frame and a short-exposure frame, or a long-exposure frame and a normal-exposure frame, or a short-exposure frame and a normal-exposure frame.

[0095] If N = 3, the N-frame images include a long-exposure frame, a short-exposure frame, and a normal-exposure frame. If N is greater than 3, the N-frame images include a long-exposure frame, a short-exposure frame, and a normal-exposure frame. Further, taking the exposure duration as an example, a long-exposure frame with an exposure duration exceeding the second preset exposure duration in the long-exposure frames can be an ultra-long-exposure frame. A short-exposure frame with an exposure duration less than the third preset exposure duration in the short-exposure frames can be an ultra-short-exposure frame. Among them, the second preset exposure duration is greater than the first preset exposure duration, and the third preset exposure duration is less than the first preset exposure duration.

[0096] S701b, fuse the N-frame images into the first HDR image.

[0097] The mobile phone fuses the N-frame images with N exposure values into the first HDR image. Taking the three-frame images in the previous text as an example, the mobile phone can fuse the long-exposure frame, the medium-exposure frame, and the short-exposure frame into the first HDR image. Exemplarily, the mobile phone uses the following formula to obtain the first HDR image.

[0098] rgb_hdr = rgb_long * weight_long + rgb_normal * weight_normal + rgb_short * weight_short.

[0099] Among them, rgb_hdr is the first HDR image. rgb_long is the long-exposure frame, and weight_long is the weight of the long-exposure frame. rgb_normal is the medium-exposure frame, and weight_normal is the weight of the medium-exposure frame. rgb_short is the short-exposure frame, and weight_short is the weight of the short-exposure frame.

[0100] In this way, the mobile phone can fuse at least two images with different exposure values into a first HDR image based on the multi-frame fusion technology.

[0101] Optionally, the mobile phone may also not fuse N first images. Instead, the mobile phone can use an AI model to adjust the dynamic range of one of the N first images to obtain a first HDR image. Among them, this one image can be any one of the medium exposure frame, short exposure frame, and long exposure frame. In this way, the mobile phone can only collect two first images, including a second image with a large exposure value or a long exposure time, and a first image with a small exposure value or a short exposure time.

[0102] S702, the mobile phone performs tone mapping on the first HDR image to obtain a second HDR image.

[0103] The mobile phone can use a tone mapping algorithm to map the first HDR image to obtain a mapped second HDR image. The tone mapping algorithm can be any tone mapping algorithm, and the embodiments of the present application do not make specific limitations on this. Optionally, the tone mapping algorithm can be any one of global mapping, local mapping, and hybrid mapping.

[0104] Optionally, the mobile phone can also use a tone mapping table to map the first HDR image to obtain a mapped second HDR image.

[0105] The mobile phone can map the first brightness value of each pixel of the first HDR image to a second brightness value through a tone mapping algorithm or a tone mapping table. The numerical range of the second brightness value is smaller than the numerical range of the first brightness value. That is to say, tone mapping compresses the brightness values of the pixels in the first HDR image.

[0106] For example, for pixel a(i,j) in the first HDR, the first luminance Y of this pixel is Y = t1*R + t2*G + t3*B. Exemplarily, t1, t2, and t3 can be 0.299, 0.587, and 0.114 in sequence. If the first HDR image is a YUV image, for each pixel, the luminance value Y of this pixel is the value of the Y component of this pixel. The mobile phone uses a tone mapping algorithm or a tone mapping table to map the luminance value of pixel a(i,j) to Y`. After that, based on the luminance values Y` of each pixel after mapping and the first HDR image, the mobile phone can obtain a second HDR image. Exemplarily, the mobile phone can calculate the first luminance gain of each pixel, and the first luminance gain is equal to the ratio of the luminance value Y` of the first pixel after mapping to the luminance value Y of the first pixel before unmapped. After that, the mobile phone calculates the R`, G`, and B` components of the first pixel after mapping. Specifically, the R` component is equal to the product of the first luminance gain and the R component before unmapped. The G` component is equal to the product of the first luminance gain and the G component before unmapped. The B` component is equal to the product of the first luminance gain and the B component before unmapped. In this way, the mobile phone obtains the R`, G`, and B` components of pixel a`(i,j) after tone mapping. Optionally, R` component = (R component / Y) λ *Y`, G` component = (G component / Y) λ *Y`, B` component = (B component / Y) λ *Y`, where λ is an adjustment coefficient. For each pixel, any of the above methods is executed. In this way, the mobile phone can obtain the second HDR image after mapping, and this second HDR image can be an RGB image.

[0107] Optionally, the mobile phone can also input the first HDR image into a neural network model to obtain the second HDR image after mapping. After that, the mobile phone can calculate the luminance values of each pixel after mapping based on the second HDR image.

[0108] Optionally, the mobile phone can first identify whether the second highlighted area includes a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted area. If it includes, the mobile phone can correct the gradient direction of the first pixel with the reversed gradient direction. Optionally, if it includes, the mobile phone can correct the gradient directions of the first pixel with the reversed gradient direction and the first pixel with the consistent gradient direction. If it does not include, the mobile phone does not correct the luminance direction of the first pixel in the second HDR image. Among them, the first pixel is any pixel in the second highlighted area, and the first pixel can be a pixel with a reversed gradient direction or a pixel with a consistent gradient direction. The second pixel is the pixel corresponding to the first pixel in the first highlighted area. The gradient direction of a pixel is the luminance change direction of the pixel.

[0109] S703, the mobile phone determines whether the second highlighted area includes a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted area.

[0110] The first HDR image includes a first highlighted area, and the second HDR image includes a second highlighted area. The first HDR image and the second HDR image have the same size, and the position of the first highlighted area in the first HDR image is the same as the position of the second highlighted area in the second HDR image, that is, the first highlighted area corresponds to the second highlighted area.

[0111] The first highlighted area includes a highlighted object. Among them, the highlighted object can be composed of pixels with a luminance value greater than the threshold in the first HDR image. For example, the highlighted object can be a light source in the first HDR image, such as the sun, a lamp, etc. The first highlighted area is the area in the first HDR image that includes the highlighted object. As Figure 3 shown, the highlighted object is the sun, and the highlighted area is the area including the sun. The area of the highlighted area is greater than or equal to the area of the highlighted object. The shape of the highlighted object can be circular like the sun, rectangular like a lamp, or other shapes. The shape of the highlighted area can be a square or a circle. Or, the shape of the highlighted area can be the same as the shape corresponding to the highlight. For example, when the highlighted object is the sun, the highlighted area is a circle including the sun. When the highlighted object is a rectangular lamp, the highlighted area is a rectangle including the rectangular lamp. Hereinafter, taking the highlighted object as the sun and the highlighted area as a rectangle including the sun as an example, this solution will be introduced.

[0112] Optionally, before the mobile phone executes S703, it can identify whether the second HDR image includes a highlighted object, or identify whether the first HDR image includes a highlighted object. If the first HDR image does not include a highlighted object, or the second HDR image does not include a highlighted object, the mobile phone can not perform brightness correction on the second HDR image. Hereinafter, taking the mobile phone's identification of whether the first HDR image includes a highlighted object as an example,

[0113] As a possible implementation, the mobile phone can first identify whether the first HDR image includes a highlighted object. If the first HDR image does not include a highlighted object, the mobile phone does not perform a brightness change correction on the first HDR image after tone mapping. If the first HDR image includes a highlighted object, the mobile phone identifies the position of the highlighted object in the first HDR. For example, the position of the highlighted object in the first HDR image can be represented by [x1, y1, w1, h1], where (x1, y1) are the coordinates of the center point of the bounding box enclosing the highlighted object, and (w1, h1) are the width and height of the bounding box enclosing the highlighted object. The position of the first highlighted area in the first HDR image can be represented by [x2, y2, w2, h2], where (x2, y2) are the coordinates of the center point of the first highlighted area, and (w2, h2) are the width and height of the first highlighted area. Among them, the center point of the bounding box enclosing the highlighted object can coincide with the center point of the first highlighted area. The mobile phone increases a preset value on the width and height of the bounding box enclosing the highlighted object to obtain the width and height of the first highlighted area, and further obtains the position of the first highlighted area in the first HDR image. When the mobile phone obtains the position of the first highlighted area in the first HDR image, the mobile phone obtains the position of the second highlighted area in the second HDR image.

[0114] As another possible implementation, the mobile phone identifies whether the first HDR image includes a highlighted object based on the brightness values of the pixels in the first HDR image. For example, the mobile phone identifies the pixels with the maximum brightness value in the first HDR image and the pixels with brightness values exceeding a preset brightness value around the maximum brightness value. The first highlighted area is the area enclosed by the pixels corresponding to the brightness values exceeding the preset brightness value around the maximum brightness value.

[0115] After that, the mobile phone can use the first highlighted area as a reference to identify the brightness change directions of multiple first pixels in the second highlighted area. Taking the upper left corner of the image as the coordinate origin, the coordinate system of the first pixel includes a horizontal direction (x-axis) and a vertical direction (y-axis). The brightness change direction of the first pixel can be increasing along the horizontal direction, or decreasing along the horizontal direction. The brightness change direction of the first pixel can be increasing along the vertical direction, or decreasing along the vertical direction. Exemplarily, in the horizontal direction, the brightness change of the first pixel a(i,j) can be represented by the difference between the next first pixel a(i,j + 1) and a(i,j) in the horizontal direction. If the difference is positive, the brightness change direction of the first pixel a(i,j) is increasing along the horizontal direction. If the difference is negative, the brightness change direction of the first pixel a(i,j) is decreasing along the horizontal direction. Similarly, in the vertical direction, the brightness change of the first pixel a(i,j) can be represented by the difference between the next first pixel a(i + 1,j) and a(i,j) in the vertical direction. If the difference is greater than zero, the brightness change direction of the first pixel a(i,j) is increasing along the vertical direction. If the difference is less than zero, the brightness change direction of the first pixel a(i,j) is decreasing along the vertical direction.

[0116] It should be understood that since the first HDR image is an unprocessed image, the brightness change directions of the respective second pixels in the first highlighted area of the first HDR image can well restore the brightness change of the corresponding content in the natural scene. If the brightness change directions of the respective first pixels in the second highlighted area of the second HDR image are the same as the brightness change directions of the corresponding second pixels in the first highlighted area, it indicates that the brightness of the second HDR image after tone mapping can also well restore the brightness change of the corresponding content in the natural scene and no correction is required. If the brightness change directions of some or all of the first pixels in the highlighted area of the second HDR image are inconsistent with the brightness change directions of the corresponding second pixels in the first highlighted area, then the brightness change directions of some or all of the first pixels in the second HDR image cannot well restore the brightness change of the corresponding content in the natural scene. In this way, display defects such as bright spots, bright lines, halos or dark spots, dark lines may appear in the second HDR image.

[0117] For example, in the vertical direction, assume that the luminance values of the second pixel in the first column of the first HDR image are (10, 9, 8, 7, 6), and its corresponding gradient is (-1, -1, -1, -1, -1). That is, the change direction of the second pixel in the first column of the highlighted area in the first HDR image is decreasing in sequence. The luminance values of the first pixel in the same column of the second HDR image are (5, 4, 6, 2, 1), and its corresponding gradient is (-1, 2, -3, -1, -1). The luminance change between the second first pixel and the third first pixel in the second HDR image is an increase, and the luminance change between the third first pixel and the fourth first pixel is a decrease. That is, the luminance value first decreases, then suddenly increases, and then decreases. Thus, bright spots may be included at the position where the luminance value suddenly increases in the second HDR image, such as a bright spot is displayed at the third first pixel in the first column. Multiple bright spots can form a bright circle or a bright line. Also, for example, the luminance values of the same column in the second luminance map are (5, 1, 4, 3, 1), and the corresponding gradient is (-4, 3, -1, -2, 1). The luminance decrease between the first first pixel and the second first pixel in the second HDR image is too large, resulting in an increase in the luminance change between the second first pixel and the third first pixel. That is, in the first column, the luminance value first decreases, then suddenly decreases, then increases, and then decreases again. Thus, dark spots may be included at the position where the luminance value suddenly decreases in the second HDR image, such as a dark spot is displayed at the second first pixel in the first column, and multiple dark spots can form a dark circle or a dark line.

[0118] Therefore, the mobile phone can calculate the luminance changes of each second pixel in the first highlighted area and calculate the luminance changes of each first pixel in the second highlighted area. Exemplarily, the mobile phone performs image derivation on the first luminance map of the first HDR image and the second luminance map of the second HDR image to obtain the luminance changes of each second pixel in the first highlighted area and the luminance changes of each first pixel in the second highlighted area.

[0119] In a digital image, the first luminance map includes the luminance values Y of each second pixel of the first HDR image. The second luminance map includes the luminance values Y` of each first pixel in the second HDR image. The mobile phone derives the first luminance map to obtain the gradient of each second pixel of the first HDR image. The mobile phone derives the second luminance map to obtain the gradient of each first pixel of the second HDR image.

[0120] Image differentiation, also known as gradient calculation, refers to differentiating the luminance value of each pixel in the first luminance map or the second luminance map in the x and vertical directions respectively. Taking the first luminance map as an example, image differentiation includes calculating the gradient of the first luminance map in the horizontal direction and calculating the gradient of the first luminance map in the vertical direction. Exemplarily, the mobile phone can call the gradient function to perform differentiation on the first luminance map in the horizontal (X) and vertical (Y) directions, and call the gradient function to perform differentiation on the second luminance map in the horizontal (X) and vertical (Y) directions. Among them, the gradient of a pixel can be positive or negative. For example, in the horizontal direction, if the gradient of a pixel is positive, it means that the luminance value of the pixel in the horizontal direction is increasing. If the gradient of a pixel is negative, it means that the luminance value of the pixel in the horizontal direction is decreasing. There are various ways of image differentiation, and the embodiments of this application do not make specific limitations in this regard. A possible implementation manner is given below.

[0121] Exemplarily, in the horizontal direction, the gradient of the i-th row of pixels = (the luminance value of the (i + 1)-th row of pixels - the luminance value of the i-th row of pixels). The gradient of the last row of pixels may not be calculated, or the gradient of the previous row of pixels may be used as the gradient of the last row of pixels. In the vertical direction, the gradient of the j-th column = (the luminance value of the (j + 1)-th column of pixels - the luminance value of the j-th column of pixels). The gradient of the last column of pixels may not be calculated, or the gradient of the previous column of pixels may be used as the gradient of the last column of pixels.

[0122] In this way, the mobile phone can obtain the gradients of each second pixel in the first high-brightness area in the horizontal and vertical directions, and the gradients of each first pixel in the second high-brightness area in the horizontal and vertical directions. The gradient of a pixel reflects the direction and magnitude of the luminance change between the pixel and its adjacent pixels. The gradient of a pixel includes magnitude and direction. Among them, the gradient direction of a pixel is the positive or negative of the difference between the luminance value of the pixel adjacent to the pixel on the x-axis and the luminance value of the pixel. The gradient magnitude of a pixel is the difference between the luminance value of the pixel adjacent to the pixel on the x-axis and the luminance value of the pixel.

[0123] Exemplarily, in the horizontal direction, the gradient of pixel a is -10, indicating that the luminance value of pixel b before pixel a is less than the luminance value of pixel a. That is, the luminance decreases in the horizontal direction from pixel a to pixel b, and the decreasing value is the gradient value of pixel a, which is 10 nits. In the vertical direction, the gradient of pixel c is 30, indicating that the luminance value of pixel d before pixel c is greater than the luminance value of pixel c. That is, the luminance increases in the vertical direction from pixel c to pixel d, and the increasing value is the gradient value of pixel c, which is 30 nits.

[0124] The mobile phone identifies the first pixels with inconsistent brightness changes in the second highlighted area based on the gradients of each second pixel in the first highlighted area in the horizontal and vertical directions and the gradients of each first pixel in the second highlighted area in the horizontal and vertical directions. Specifically, the mobile phone can identify the first pixels with inconsistent horizontal brightness changes in the second highlighted area based on the gradients of each second pixel in the first highlighted area in the horizontal direction and the gradients of each first pixel in the second highlighted area in the horizontal direction. The mobile phone can identify the pixels with inconsistent vertical brightness changes in the second highlighted area based on the gradients of each second pixel in the first highlighted area in the vertical direction and the gradients of each first pixel in the second highlighted area in the vertical direction. If the horizontal brightness change direction of the first pixel is inconsistent with the horizontal brightness change direction of the second pixel, and / or the vertical brightness change direction of the first pixel is inconsistent with the vertical brightness change direction of the second pixel, then the first pixel is a pixel with a gradient direction inconsistent with the gradient direction of the corresponding pixel in the first highlighted area, that is, a pixel with a reverse gradient direction.

[0125] Taking the mobile phone's identification of pixels with inconsistent horizontal brightness changes as an example, for example, the mobile phone can identify one by one whether the gradient directions of each pixel in the horizontal direction in the first highlighted area are consistent with the gradient directions of the corresponding pixels in the horizontal direction in the second highlighted area. Specifically, for example, the mobile phone reads the gradient of the second pixel at the first coordinate position in the first highlighted area, and the mobile phone reads the gradient of the first pixel at the first coordinate position in the second highlighted area, and the mobile phone identifies whether these two gradients are both positive or both negative. Among them, the first coordinate position in the first highlighted area can be the upper left corner position of the first highlighted area. The first coordinate position in the second highlighted area can be the upper left corner position of the second highlighted area. That is, the mobile phone obtains the gradients of the pixels at the same coordinate positions in the first highlighted area and the second highlighted area, and compares whether the gradients of the pixels at the same positions are both positive or both negative. If the brightness values of the two pixels at the same position are both positive, it means that the brightness change directions of these two pixels are the same and are increasing. If the brightness values of the two pixels at the same position are both negative, it means that the brightness change directions of these two pixels are the same and are decreasing. If the brightness values of the two pixels at the same position are both positive or both negative, it means that the brightness change directions of the pixels at the same positions in the first highlighted area of the first HDR image and the second highlighted area of the second HDR image are consistent, and there is no need to correct the brightness change direction of the pixels at this position in the second highlighted area. If the brightness values of the two pixels at the same position are not both positive or not both negative, it means that the brightness change directions of the pixels at the same positions in the first highlighted area of the first HDR image and the second highlighted area of the second HDR image are inconsistent, and it is necessary to correct the brightness change direction of the pixels at this position in the second highlighted area.

[0126] As another possible implementation, the mobile phone can calculate the product of the gradients of two pixels at the same position in the first highlighted area and the second highlighted area one by one. If the product is less than zero, the change directions of the two pixels at the same position are inconsistent. That is, the brightness change direction of this pixel in the second highlighted area is inconsistent with the change direction of the corresponding pixel of this pixel in the first highlighted area. If the product is greater than zero, the brightness change directions of the two pixels at the same position are consistent. That is, the brightness change direction of this pixel in the second highlighted area is consistent with the brightness change direction of the corresponding pixel of this pixel in the first highlighted area.

[0127] S704, if the second highlighted area includes a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted area, the mobile phone corrects the gradient change direction of the first pixel in the second highlighted area.

[0128] The mobile phone corrects the brightness value of the next pixel of the first pixel on the x-axis so that the gradient direction of the first pixel on the x-axis is consistent with the gradient direction of the second pixel on the x-axis. Exemplarily, for the pixel a`(i, j), the mobile phone corrects the brightness value of the pixel a`(i, j + 1) in the second highlighted area so that the gradient direction of the pixel a`(i, j) in the second highlighted area on the x-axis is consistent with the gradient direction of the pixel a(i, j) in the first highlighted area on the x-axis. The corrected brightness value of the pixel a`(i, j + 1) in the second highlighted area is: the product of the gradient of the second pixel corresponding to the pixel a`(i, j), that is, the pixel a(i, j), on the x-axis and the first correction coefficient plus the brightness value of a`(i, j) in the second highlighted area.

[0129] The mobile phone corrects the brightness value of the next pixel of the first pixel on the y-axis so that the gradient direction of the first pixel on the y-axis is consistent with the gradient direction of the second pixel on the y-axis. Exemplarily, for the pixel a`(i, j), the mobile phone corrects the brightness value of a`(i + 1, j) in the second highlighted area so that the gradient direction of the pixel a`(i, j) in the second highlighted area on the y-axis is consistent with the gradient direction of the pixel a(i, j) in the first highlighted area on the y-axis. The corrected brightness value of a`(i + 1, j) in the second highlighted area is: the product of the gradient of the second pixel corresponding to the pixel a`(i, j), that is, the pixel a(i, j), on the y-axis and the second correction coefficient plus the brightness value of the pixel a`(i, j) in the second highlighted area.

[0130] After correction, the gradient direction of each first pixel in the second highlighted area on the x-axis is consistent with the gradient direction of the corresponding second pixel on the x-axis, and the gradient direction of each first pixel on the y-axis is consistent with the gradient direction of the corresponding second pixel on the y-axis. Among them, the pixel a`(i, j) is the pixel after tone mapping of the pixel a(i, j). That is, the first pixel corresponds to the second pixel, and the first pixel is the second pixel after tone mapping.

[0131] Such asFigure 8 As shown, in the horizontal direction, the gradient GX`(i,j) of the first pixel, such as pixel a`(i,j), in the second highlighted area on the x-axis is equal to the difference in the luminance values between pixel a`(i,j+1) and pixel a`(i,j). The gradient GX(i,j) of the second pixel a(i,j) corresponding to the first pixel in the first highlighted area on the x-axis is equal to the difference in the luminance values between pixel a(i,j+1) and pixel a(i,j).

[0132] If the directions of the gradients GX(i,j) and GX`(i,j) are inconsistent, the mobile phone modifies the luminance value of pixel a`(i,j+1). That is, it modifies the luminance value of the adjacent pixel of the first pixel in the direction opposite to the gradient. Specifically, the mobile phone uses the gradient GX(i,j) of the second pixel on the x-axis as the gradient of pixel a`(i,j) on the x-axis. The luminance value of the corrected pixel a`(i,j+1) - the luminance value of pixel a`(i,j) = the gradient GX(i,j) of the second pixel on the x-axis. Then, the luminance value of the corrected pixel a`(i,j+1) = the luminance value of pixel a`(i,j) + the gradient GX(i,j) of the second pixel on the x-axis. Since the dynamic imaging range of the first HDR image is different from that of the second HDR image, directly using the gradient GX(i,j) as the corrected gradient of pixel a`(i,j) may exceed the value range of the luminance value of pixel a`(i,j+1). Therefore, the gradient GX(i,j)*C1 can be used as the gradient of pixel a`(i,j). That is, the luminance value of the corrected pixel a`(i,j+1) = the luminance value of pixel a`(i,j) + the gradient GX(i,j) of the second pixel on the x-axis * C1.

[0133] Optionally, the luminance value of the corrected pixel a`(i,j+1) = the luminance value of pixel a`(i,j) + (gradient GX`(i,j) - GX(i,j))*C1. Among them, the gradient GX`(i,j) - the gradient GX(i,j) is the distance between the two gradients.

[0134] In the vertical direction, the brightness value of the corrected pixel a`(i + 1, j) - the brightness value of the pixel a`(i, j) = the gradient GY(i, j) of the second pixel on the y-axis. Then, the brightness value of the corrected pixel a`(i + 1, j) = the brightness value of the pixel a`(i, j) + the gradient GY(i, j) of the second pixel on the y-axis. Since the dynamic imaging range of the first HDR image is different from that of the second HDR image, directly using the gradient GY(i, j) of the second pixel on the y-axis to correct the pixel a`(i, j) may exceed the value range of the brightness value of a`(i + 1, j). Therefore, GY(i, j)*C1 can be used as the gradient of the pixel a`(i, j). That is, the brightness value of the corrected pixel a`(i + 1, j) = the brightness value of the pixel a`(i, j) + the gradient GY(i, j) of the second pixel on the y-axis * C2.

[0135] Wherein, C1 and C2 are the first correction coefficient and the second correction coefficient. The values of C1 and C2 can be the same or different. The value of this correction coefficient can be between 0 and 0.25. For example, the maximum value of C1 or C2 can be the ratio of the bit width of the brightness value in the first HDR image to the bit width of the brightness value in the second HDR image. Among them, in the first HDR image, the bit width of the brightness value can be 14bit, 16bit or higher. In the second HDR image, the bit width of the brightness value can be 8bit, 10bit, etc. Exemplarily, in the first HDR image, the bit width of the brightness value is 16bit. In the second HDR image, the bit width of the brightness value is 14bit. Then C1 or C2 is the ratio of 16 to 14, and the maximum value of C1 or C2 is 0.25. Optionally, in this solution, both C1 and C2 are 0.18.

[0136] Optionally, the brightness value of the corrected pixel a`(i + 1, j) = the brightness value of the pixel a`(i, j) + (the gradient GY`(i, j) - the gradient GY(i, j)) * C2. Among them, the gradient GY`(i, j) - the gradient GY(i, j) is the distance between the two gradients.

[0137] In this solution, i takes values in sequence in {1, 2,..., m}, and j takes values in sequence in {1, 2,..., n}. m is the row number of the pixels in the second highlighted area, and n is the column number of the pixels in the second highlighted area. In this way, the mobile phone can correct each first pixel from pixel a(1, 1) to a(m, n) in the second highlighted area, and each first pixel includes the first pixel with the opposite gradient direction and the first pixel with the same gradient direction.

[0138] Optionally, the mobile phone can also only correct the first pixel with the reverse gradient direction. That is, i takes values in sequence from {1, 2, ……, m}, j takes values in sequence from {1, 2, ……, n}, and before correction, the mobile phone can identify whether the first pixel is the first pixel with the reverse gradient direction. If the first pixel is the first pixel with the reverse gradient direction, the mobile phone corrects the gradient direction of the first pixel by the method described above. In this way, the mobile phone can correct the first pixels with the reverse gradient direction among pixels a(1, 1) to a(m, n). For example, in the horizontal direction, when the mobile phone corrects the gradient direction of pixel a`(i, j) by correcting the brightness value of pixel a`(i + 1, j), the mobile phone can recalculate the gradient direction of pixel a`(i + 1, j). If the gradient direction of pixel a`(i + 1, j) is consistent with the gradient direction of the corresponding second pixel in the first highlighted area, the mobile phone does not correct the gradient direction of pixel a`(i + 1, j). If the gradient direction of pixel a`(i + 1, j) is inconsistent with the gradient direction of the corresponding pixel in the first highlighted area, the mobile phone corrects the brightness value of a`(i + 2, j) to correct the gradient direction of pixel a`(i + 1, j).

[0139] Optionally, after the mobile phone corrects the gradient direction of the first pixel with the reverse gradient direction in a certain row or a certain column, the mobile phone can directly correct the gradient directions of all the first pixels included in the row or column where the first pixel is located without recalculating the gradients of the remaining pixels.

[0140] Optionally, after the mobile phone corrects the gradient direction of the first pixel in the horizontal direction in the first highlighted area, the mobile phone recalculates the gradient direction of the first pixel in the vertical direction based on the corrected brightness value of the first pixel. The mobile phone can again identify whether the gradient direction of the first pixel in the vertical direction is consistent with the gradient direction of the second pixel in the vertical direction in the first highlighted area. If they are consistent, the mobile phone does not need to correct the gradient direction of the first pixel in the vertical direction. If they are inconsistent, the mobile phone needs to correct the gradient direction of the first pixel in the vertical direction.

[0141] In the example given above, the mobile phone first corrects the gradient direction of the first pixel in the horizontal direction and then corrects the gradient direction of the first pixel in the vertical direction. Optionally, the mobile phone can also first correct the gradient direction of the first pixel in the vertical direction and then correct the gradient direction of the first pixel in the horizontal direction. The embodiments of the present application do not make specific limitations on this.

[0142] S705, the mobile phone generates a third HDR image according to the second HDR image and the third highlighted area.

[0143] The third high-brightness region is the second high-brightness region after correcting the gradient directions of some or all of the first pixels. The mobile phone uses the third brightness region to replace the second high-brightness region in the second HDR image. After that, the mobile phone generates a third HDR image based on the brightness values of the pixels in the third high-brightness region and the brightness values of the pixels in other regions of the second HDR image except the third high-brightness region.

[0144] The third HDR image is used for display. The dynamic imaging range of the third HDR image is smaller than that of the first HDR image. The HDR dynamic imaging range of the display screen of the mobile phone is smaller than the HDR dynamic imaging range of the first HDR image, and the dynamic imaging range of the third HDR image is adapted to the HDR dynamic imaging range of the mobile phone display screen.

[0145] The mobile phone can execute S701 - S705 to correct the brightness values of the pixels in the high-brightness region of the second HDR image, so that the brightness transition in the corrected third HDR is more natural in the high-brightness region.

[0146] Furthermore, in order to avoid the saturation of the second HDR image being too high after tone mapping, the mobile phone can further correct the color of the second HDR image or the third HDR image. As Figure 3 shown in b, the saturation of the sun region is too high, resulting in color distortion.

[0147] Exemplarily as Figure 9 shown. After the mobile phone generates the third HDR image, the mobile phone can execute S901.

[0148] S901, the mobile phone uses the first image as a reference frame to correct the color of the fourth HDR image to obtain a fifth HDR image; wherein, the fourth HDR image is the third HDR image.

[0149] S902, if the second high-brightness region does not include the first pixels whose gradient directions are inconsistent with the gradient directions of the corresponding pixels in the first high-brightness region, using the first image as a reference frame, perform color correction on the fourth HDR image to obtain a sixth HDR image; wherein, the fourth HDR image is the second HDR image.

[0150] That is to say, if the gradient directions of the pixels in the second high-brightness region are consistent with the gradient directions of the pixels in the first high-brightness region, the mobile phone does not need to adjust the gradient directions of the pixels in the second high-brightness region, and the mobile phone can directly perform color correction on the second HDR image. Optionally, if the mobile phone recognizes that the first HDR image does not include the first high-brightness object, or the second HDR image does not include the second high-brightness object, the mobile phone can also directly perform color correction on the second HDR image.

[0151] If the second highlighted region includes a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted region, the mobile phone adjusts the brightness values of the pixels in the second highlighted region to obtain a third HDR image, and then the mobile phone performs color correction on the third HDR image.

[0152] The mobile phone performs color correction on the second HDR image to obtain a sixth HDR image. The mobile phone obtains a fifth HDR image from the third HDR image. The method for the mobile phone to adjust the color of the third HDR image is the same as the method for the mobile phone to adjust the color of the second HDR image. Below, the example of the mobile phone adjusting the color of the third HDR image will be introduced.

[0153] In order not to change the brightness of the corrected third HDR image, the mobile phone can correct the color of the third HDR image by correcting the chromaticity and hue of the third HDR image. Exemplarily, the mobile phone can use the short-exposure frame in the multiple first images obtained in S701a as a reference frame, and correct the color of the third HDR image based on this reference frame. Among them, the short-exposure frame can be the first image with the shortest exposure time or the smallest exposure value among the N frames of images.

[0154] The mobile phone can first convert the short-exposure frame to the LCH color space, then convert the third HDR image to the LCH space, and then use the C component and H component of the short-exposure frame to replace the C component and H component in the third HDR image to obtain a fourth HDR image after color correction.

[0155] Exemplarily, S901 may include S1-S5.

[0156] S1. The mobile phone preprocesses the first image corresponding to the short-exposure frame to obtain a first image in the sRGB format.

[0157] The preprocessing includes but is not limited to automatic white balance (AWB), color correction matrix correction, and gamma correction. Among them, AWB restores the white color imaged under ambient light of different color temperatures to the real white color, that is, the white color observed by the human eye under natural daylight ambient light through an algorithm. The color correction matrix (CCM) is used to correct the ratio of the R, G, and B components of each pixel in the image. Gamma correction can adjust the gray coefficient of the image so that the processed image can look more comfortable when displayed on the display screen and can better express the information and features in the original image. The format of the image after gamma correction can be the sRGB format.

[0158] S2. The mobile phone converts the first image from the sRGB color space to the XYZ color space, then from the XYZ color space to the Lab color space, and from the Lab color space to the LCH space.

[0159] The mobile phone first converts the first image in sRGB format into the first image in XYZ format, that is, converts the first image from the sRGB color space to the XYZ color space. In the XYZ color space, each pixel includes an X component, a Y component, and a Z component. Among them, the X component is the red primary stimulus, the Y component is the green primary stimulus, and the Z component is the blue primary stimulus. Exemplarily, for each pixel in the first image in sRGB format, it can be converted based on the following formula.

[0160] X = 0.412453 * R + 0.357580 * G + 0.180423 * B.

[0161] Y = 0.212671 * R + 0.715160 * G + 0.072169 * B.

[0162] Z = 0.019334 * R + 0.119193 * G + 0.950227 * B.

[0163] Among them, for a pixel, R is the value of the R component of the pixel, G is the value of the G component of the pixel, and B is the value of the B component of the pixel.

[0164] After that, the mobile phone converts the first image in XYZ format into the first image in Lab format. In the Lab color space, each pixel includes an L component, an a component, and a b component. Among them, the L component is the brightness, the a component is the red - green chroma, and the b component is the yellow - blue chroma. Exemplarily, for each pixel in the first image in Lab format, it can be converted based on the following formula.

[0165] First, with the D65 light source standard white point as a reference, the X component, Y component, and Z component of each pixel are normalized to obtain the x component, y component, and z component. Specifically, x = X / Xn1, y = Y / Yn1, z = Z / Zn1. Among them, Xn1 is the normalization parameter of the X component, Yn1 is the normalization parameter of the Y component, and Zn1 is the normalization parameter of the Z component. Exemplarily, the values of Xn1, Yn1, and Zn1 are 0.95047, 1.0, and 1.08883 in sequence.

[0166] After that, a non - linear transformation is performed on the normalized x component, y component, and z component. For example, the following formula is used for non - linear transformation. Substitute the x component, y component, and z component of each pixel into the following formula to calculate F(x), F(y), and F(z) respectively. Exemplarily, if the x component of the pixel is greater than then F(x) is If the x component is less than then F(x) is component.

[0167]

[0168] Finally, the conversion of the Lab color space is completed in the following manner. L = 116 * F(y) - 16, a = 500 * (F(x) - F(y)), b = 200 * (F(y) - F(z)).

[0169] Finally, the mobile phone converts the first image in the Lab format into the first image in the LCH format.

[0170] In the LCH color space, each pixel includes an L component, a C component, and an H component. Among them, the L component is the brightness, the C component is the chroma, and the h component is the hue.

[0171] C = sqrt(0.2989 * a 2 + 0.1368 * b 2 + 0.2330 * (b - a) 2 ). Among them, the sqrt() function is used to return the square root. For example, SQRT(16) = 4.

[0172] H = acos((0.57732 * (b - a) + 1.42879 * (b + a)) / (sqrt(0.2989 * a 2 + 0.1368 * b 2 + 0.2330 * (b -

[0173] a) 2 ))). Among them, the acos() function is used to calculate the arccosine value of a given numerical value. For example, acos(0.5) = 1.047198.

[0174] In this way, the mobile phone completes the conversion of the color space of the first image. The converted first image can be the first LCH image. The first L component, the first C component, and the first H component of each pixel in the first LCH image are obtained by converting the first image in the sRGB format in the manner and formula described above. The mobile phone can obtain the first C component and the first H component of each pixel in the first image. The first image includes a fourth highlight area, and the fourth highlight area corresponds to the first highlight area and also corresponds to the second highlight area. The fourth highlight area includes a third pixel.

[0175] After that, the mobile phone can execute S3.

[0176] S3. The mobile phone sequentially converts the third HDR image from the sRGB color space to the XYZ color space, from the XYZ color space to the Lab color space, and from the Lab color space to the LCH space.

[0177] The mobile phone can perform gamma correction on the third HDR image in RGB format, converting the third HDR image from RGB format to sRGB format. The mobile phone adopts the method shown in S2 to complete the conversion of the color space of the third HDR image. The converted first image can be a second LCH image, and the second L component, second C component, and second H component of each pixel in the second LCH image are obtained by converting the third HDR image in RGB format in the manner and formula described above.

[0178] The mobile phone only performs color correction on the first pixel in the third high-brightness region of the third HDR image. For example, the first image includes a fourth high-brightness region. The position of the fourth high-brightness region in the first image is the same as the position of the third high-brightness region in the third HDR image, and the size of the third HDR image is the same as the size of the first image. In this way, the mobile phone modifies the second C component and second H component of each pixel in the third high-brightness region to the first C component and first H component of the corresponding pixel in the fourth high-brightness region.

[0179] S4, the mobile phone modifies the second C component of the first pixel in the third high-brightness region (fifth high-brightness region) to the first C component of the corresponding third pixel in the fourth high-brightness region, and modifies the second H component of the first pixel in the third HDR image to the first H component of the corresponding third pixel in the fourth high-brightness region, obtaining the corrected fifth HDR image.

[0180] Each pixel in the fifth HDR image after color correction includes a second L component, a first C component, and a first H component.

[0181] Furthermore, the mobile phone can blur the edges of the bright objects in the fifth HDR image. Blurring the edges of the bright objects can effectively remove noise and smooth the edges of the bright objects, making the edges of the highlights softer and more natural. This blurring process can be, for example, a multi-valued blurring process, such as Gaussian blurring.

[0182] Furthermore, in order to display the fifth HDR image, the mobile phone can convert the fifth HDR image from the LCH space back to the sRGB space.

[0183] S5, the mobile phone sequentially converts the fifth HDR image from the LCH color space to the Lab color space, from the Lab color space to the XYZ color space, and from the XYZ color space to the sRGB color space.

[0184] First, the mobile phone first converts the fifth HDR image from the LCH color space to the Lab color space. Specifically, the mobile phone can substitute the L component, C component, and H component of each pixel in the fifth HDR image into the following calculation formula based on the following calculation method to obtain the L component, a component, and b component of each pixel.

[0185] a = (exp 10(L / 100-1) - 1) / 0.0764。

[0186] b = sqrt(C 2 -(1 - sqrt(1 - (a / 298.9) 2 )) * C 2 ) * sign(a). Where sign(a) = {1 if a >= 0, -1 if a < 0}

[0187] Where L is the L component of each pixel in the fifth HDR image, C is the C component of each pixel in the fifth HDR image, and H is the H component of each pixel in the fifth HDR image. a is the a component of each pixel in the fifth HDR image converted to the Lab format. b is the b component of each pixel in the fifth HDR image converted to the Lab format.

[0188] After that, the mobile phone converts the fifth HDR image from the Lab color space to the XYZ color space. Specifically, the mobile phone can substitute the L component, a component, and b component of each pixel in the fifth HDR image into the following calculation formula based on the following calculation method to obtain the X component, Y component, and Z component of each pixel.

[0189] First step: The mobile phone performs a linear transformation on the L component, a component, and b component of each pixel in the fifth HDR image. Specifically, y = (L + 16) / 116; x = a / 500 + y; z = y - b / 200.

[0190] Second step: The mobile phone performs a non - linear transformation on the x, y, and z obtained in the first step.

[0191] Non - linear transformation:

[0192] Exemplarily, if the x component of the pixel is greater than then F(x) is If the x component is less than then F(x) is

[0193] Third step: The mobile phone performs an inverse normalization process on the F(x), F(y), and F(z) obtained in the second step. X = F(x) * Xn2, Y = F(Y) * Yn2, Z = F(Z) * Zn2. Where Xn2 is the inverse normalization parameter of the X component, Yn2 is the inverse normalization parameter of the Y component, and Zn2 is the inverse normalization parameter of the Z component. Exemplarily, the values of Xn2, Yn2, and Zn2 are 0.95047, 1.0, and 1.08883 in sequence.

[0194] After that, the mobile phone converts the fifth HDR image from the XYZ color space to the RGB color space. Specifically, the mobile phone can substitute the X component, Y component, and Z component of each pixel in the fifth HDR image into the following calculation formula based on the following calculation method to obtain the R component, G component, and B component of each pixel.

[0195] [R G B] = [X Y Z](M T RGB2XYZ ) -1 . Among them, M RGB2XYZ is the first transformation matrix for converting the RGB color space to the XYZ color space. M T RGB2XYZ is the transpose matrix of the first transformation matrix. (M T RGB2XYZ ) -1 is the inverse matrix of M T RGB2XYZ .

[0196] Exemplarily,

[0197] after that, the mobile phone converts the fifth HDR image from the RGB color space to the sRGB color space.

[0198] In the first step, the mobile phone first performs gamma correction on the fifth HDR image in RGB format. Specifically, the mobile phone substitutes the R component, G component, and B component of each pixel in the fifth HDR image in RGB format into the following formula to obtain the corrected f(R), f(G), and f(B).

[0199]

[0200] For example, if the R component of a pixel is greater than 0.0031308, then f(R) is 1.055 * R 1 / 2.4 - 0.055. If the x component is less than 0.0031308, then f(R) is 12.92 * R.

[0201] In the second step, clip the f(R), f(G), and f(B) obtained in the first step.

[0202]

[0203] In the third step, perform inverse normalization on the f(R)` , f(G)` , f(B)` obtained in the second step.

[0204] Specifically, r = f(R)` * 255. g = f(G)` * 255. b = f(B)` * 255.

[0205] Optionally, the mobile phone performs color correction on the second HDR, including: obtaining the first chrominance component and the first hue component of the third pixel in the fourth highlight area of the first image. Replacing the second chrominance component of the second pixel in the second highlight area (the fifth highlight area) of the second HDR image with the first chrominance component of the third pixel, and replacing the second hue component of the second pixel in the second highlight area of the second HDR image with the first hue component of the third pixel, to obtain the corrected sixth HDR image.

[0206] Figure 11 It is an example flowchart of a process for generating an HDR image provided by an embodiment of the present application. As Figure 11 shown, the mobile phone performs tone mapping on the first HDR image to obtain a third HDR image. The first HDR image includes a first luminance map, and the first luminance map includes the luminance values of the second pixels in the first HDR image. The second HDR image includes a second luminance map, and the second luminance map includes the luminance values of the respective first pixels in the second HDR image. If the mobile phone recognizes that the first HDR image includes the sun, the mobile phone corrects the gradients of some or all of the first pixels in the sunset area of the second HDR image based on the first luminance map and the second luminance map. Specifically, the mobile phone corrects the luminance values of the pixels adjacent to the first pixel to correct the gradient of the first pixel. In this way, not only the gradient direction of the first pixel is corrected, but also the size of the first pixel is corrected. After correction, the luminance change direction of the first pixel is the same as that of the corresponding second pixel, and the gradient size is scaled with the gradient size of the second pixel. Then, the mobile phone obtains a third luminance map, and the third luminance map includes the luminance values of the respective pixels in the third HDR image. The mobile phone calculates the second luminance gain of the respective pixels in the third HDR image based on the third luminance map and the first luminance map. Wherein, the second luminance gain of the pixel = the luminance value of the pixel in the third luminance map / the luminance value of the pixel in the first luminance map. Then, the mobile phone calculates the R component, G component, and B component of the respective pixels in the third HDR image. Specifically, the R component is equal to the product of the second luminance gain and the R component of the pixel in the first HDR image. The G component is equal to the product of the second luminance gain and the G component of the pixel in the first HDR image. The B component is equal to the product of the second luminance gain and the B component of the pixel in the first HDR image. The mobile phone can save the second luminance gain of each pixel. In this way, the mobile phone can obtain the third HDR image with corrected luminance.

[0207] After that, the mobile phone can perform color correction on the third HDR image. Specifically, the mobile phone first preprocesses the reference frame and converts the reference frame from the sRGB format to the LCH format. The mobile phone performs gamma mapping on the third HDR image to obtain the third HDR image in the sRGB format, and the mobile phone converts the third HDR image from the sRGB format to the LCH format. After that, the mobile phone performs LCH space fusion on the sunset area in the third HDR image in the LCH format. Specifically, for multiple first pixels in the sunset area, the mobile phone uses the C component and the H component of the corresponding multiple third pixels in the sunset area of the reference frame to replace the C component and the H component of the first pixel one by one. In this way, the mobile phone can obtain the fifth HDR image with corrected brightness. As Figure 12 shown, Figure 12 Figure shows a comparison diagram of the HDR image after brightness correction and color correction and the unprocessed first HDR image. It can be seen that the brightness transition in the sunset area of the finally obtained fifth HDR image is natural, and the color saturation is appropriate, presenting a more natural and realistic display effect, realizing the "what you see is what you get" display effect.

[0208] Optionally, the third HDR image after brightness correction, and the fifth HDR image or the sixth HDR image after color correction can be used to train a neural network model, improve the quality of the sample data input into the neural network model, and improve the processing effect of the neural network model. The neural network model has the ability to output the HDR image after tone mapping according to the input HDR image.

[0209] An embodiment of the present application provides an electronic device, which includes: a memory, a display screen, and one or more processors. The display screen is coupled to the processor. The memory is used to store computer program code. The computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device can execute each function or step executed by the mobile phone in the above method embodiment. The structure of the electronic device can refer to Figure 4 the structure of the electronic device 400 shown.

[0210] An embodiment of the present application also provides a computer storage medium, which includes computer instructions. When the computer instructions run on the above electronic device (such as Figure 4 the electronic device 400 shown), the electronic device is enabled to execute each function or step in the above method embodiment.

[0211] An embodiment of the present application also provides a computer program product. When the computer program product runs on a computer, the computer is enabled to execute each function or step in the above method embodiment.

[0212] The embodiments of the present application further provide a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected through a line. For example, the interface circuit can be used to receive signals from other devices (such as the memory of an electronic device). For another example, the interface circuit can be used to send signals to other devices (such as the processor). Exemplarily, the interface circuit can read the instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can be caused to execute the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, and the embodiments of the present application do not make specific limitations thereto.

[0213] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0214] In several embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical or other forms.

[0215] The units described as separate components may or may not be physically separated. The components displayed as units may be one physical unit or multiple physical units, that is, they can be located in one place, or they can be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0216] In addition, each functional unit in the various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0217] When an integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods of the embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0218] The above content is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for generating an HDR image, characterized in that, Applied to an electronic device, the method includes: Obtain a first HDR image, and perform tone mapping on the first HDR image to obtain a second HDR image; the first HDR image includes a first highlight region, and the first highlight region includes a highlight object; the second HDR image includes a second highlight region, and the second highlight region corresponds to the first highlight region; Modify the gradient direction of a first pixel in the second highlight region to the gradient direction of a second pixel in the first highlight region to obtain a third HDR image; the second pixel corresponds to the first pixel; wherein, the gradient direction of a pixel is the sign of the difference between the brightness value of the next pixel of the pixel on the first direction axis and the brightness value of the pixel, and the first direction axis includes the x-axis and the y-axis; Wherein, the dynamic imaging range of the third HDR image is smaller than the dynamic imaging range of the first HDR image, and the third HDR image is used for display.

2. The method according to claim 1, wherein The HDR dynamic imaging range of the display screen of the electronic device is smaller than the HDR dynamic imaging range of the first HDR image, and the dynamic imaging range of the third HDR image is adapted to the HDR dynamic imaging range of the display screen.

3. The method according to claim 2, wherein The first pixel is a pixel in the second highlight region whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlight region.

4. The method according to any one of claims 1 to 3, characterized in that, The obtaining the first HDR image and performing tone mapping on the first HDR image to obtain a second HDR image includes: Receiving the first HDR image from a second electronic device, if the HDR dynamic imaging range of the display screen of the electronic device is smaller than the HDR dynamic imaging range of the first HDR image, then perform tone mapping on the first HDR image to obtain the second HDR image.

5. The method according to any one of claims 1 to 3, characterized in that, The obtaining the first HDR image and performing tone mapping on the first HDR image to obtain a second HDR image includes: In response to a user's shooting operation in an HDR scene, obtain the first HDR image; If the HDR dynamic imaging range of the display screen of the electronic device is smaller than the HDR dynamic imaging range of the first HDR image, then perform tone mapping on the first HDR image to obtain the second HDR image.

6. The method according to any one of claims 1-5, characterized in that, The modifying the gradient direction of a first pixel in the second highlight region to the gradient direction of a second pixel in the first highlight region to obtain a third HDR image includes: Modify the brightness value of the next pixel of the first pixel on the x-axis to make the gradient direction of the first pixel on the x-axis consistent with the gradient direction of the second pixel on the x-axis; Modify the brightness value of the next pixel of the first pixel on the y-axis to make the gradient direction of the first pixel on the y-axis consistent with the gradient direction of the second pixel on the y-axis.

7. The method according to claim 6, wherein The modifying the brightness value of the next pixel of the first pixel on the x-axis to make the gradient direction of the first pixel on the x-axis consistent with the gradient direction of the second pixel on the x-axis includes: Modify the brightness value of the pixel (i, j+1) in the second highlight region to make the gradient direction of the pixel (i, j) in the second highlight region on the x-axis consistent with the gradient direction of the pixel (i, j) in the first highlight region on the x-axis; Among them, correcting the brightness value of the next pixel of the first pixel in the y-axis to make the gradient direction of the first pixel in the y-axis consistent with the gradient direction of the second pixel in the y-axis includes: Correcting the brightness value of the pixel (i + 1, j) in the second highlighted area to make the gradient direction of the pixel (i, j) in the second highlighted area in the y-axis consistent with the gradient direction of the pixel (i, j) in the first highlighted area in the y-axis; Among them, the pixel (i, j) in the second highlighted area is the first pixel, and the pixel (i, j) in the first highlighted area is the second pixel; the pixel (i, j + 1) in the second highlighted area is the next pixel of the first pixel in the x-axis, and the pixel (i + 1, j) in the second highlighted area is the next pixel of the first pixel in the y-axis; Among them, i takes values in sequence from {1, 2,..., m}, j takes values in sequence from {1, 2,..., n}, and the first pixel is the pixel in the second highlighted area whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted area.

8. The method according to claim 7, wherein The brightness value of the pixel (i + 1, j) in the second highlighted area after correction is: the product of the gradient of the second pixel in the x-axis and the first correction coefficient plus the brightness value of the pixel (i, j) in the second highlighted area; The brightness value of the pixel (i, j + 1) in the second highlighted area after correction is: the product of the gradient of the second pixel in the y-axis and the second correction coefficient plus the brightness value of the pixel (i, j) in the second highlighted area.

9. The method according to any one of claims 1-8, characterized in that, After correcting the gradient direction of the first pixel in the second highlighted area to the gradient direction of the second pixel in the first highlighted area to obtain the third HDR image, the method further includes: Using the first image as a reference frame, performing color correction on the fourth HDR image to obtain the fifth HDR image; among them, the fourth HDR image is the third HDR image; Among them, the first HDR image is obtained by fusing multiple frames of images, and the multiple frames of images include the first image and at least one second image; the exposure duration of the second image is greater than the exposure duration of the first image.

10. The method according to any one of claims 1-8, characterized in that, Before correcting the gradient direction of the first pixel in the second highlighted area to the gradient direction of the second pixel in the first highlighted area to obtain the third HDR image, the method further includes: Determining that the second highlighted area includes a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted area.

11. The method according to claim 10, wherein The method further includes: If the second highlighted area does not include the first pixel, using the first image as a reference frame, performing color correction on the fourth HDR image to obtain the sixth HDR image; among them, the fourth HDR image is the second HDR image; Among them, the first HDR image is obtained by fusing multiple frames of images, and the multiple frames of images include the first image and at least one second image; the exposure duration of the second image is greater than the exposure duration of the first image.

12. The method according to claim 9 or 11, characterized in that, Using the first image as a reference frame, performing color correction on the fourth HDR image includes: Obtain the first chrominance component and the first hue component of the third pixel in the fourth highlighted region in the first image; Replace the second chrominance component of the fourth pixel in the fifth highlighted region in the fourth HDR image with the first chrominance component of the third pixel, and replace the second hue component of the fourth pixel in the fifth highlighted region in the fourth HDR image with the first hue component of the third pixel.

13. An electronic device, characterized in that, The electronic device includes: a memory, a display screen, and one or more processors; the memory and the display screen are coupled to the processor; wherein, the memory is used to store computer program code, and the computer program code includes computer instructions; when the computer instructions are executed by the processor, the electronic device is caused to execute the method according to any one of claims 1-12.

14. A computer-readable storage medium, characterized in that, Includes computer instructions, when the computer instructions run on an electronic device, causing the electronic device to execute the method according to any one of claims 1-12.

15. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1-12 are implemented.

Images