Method of generating an hdr image and electronic device

Abstract

Embodiments of the present application relate to the technical field of terminal, and particularly relate to a method for generating an HDR image and an electronic device, which can improve the display effect of the HDR image. The method comprises: obtaining a first HDR image, performing tone mapping on the first HDR image to obtain a second HDR image; the first HDR image comprises a first highlight region, and the first highlight region comprises a highlight object; the second HDR image comprises a second highlight region, and the second highlight region corresponds to the first highlight region; correcting the gradient direction of a first pixel in the second highlight region to the gradient direction of a second pixel in the first highlight region to obtain a third HDR image; the second pixel corresponds to the first pixel; wherein the gradient direction of a pixel is the sign of the difference between the luminance value of the next pixel of the pixel in a first direction axis and the luminance value of the pixel, and the first direction axis comprises an x-axis and a y-axis; wherein the dynamic imaging range of the third HDR image is smaller than the dynamic imaging range of the first HDR image, and the third HDR image is used for display.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a method and electronic device for generating HDR images. Background Technology

[0002] Most terminal devices support High Dynamic Range (HDR) photography. HDR photography can record a wider range of brightness information, revealing more detail in both bright and dark areas of an image. However, the dynamic range of an HDR image does not match the dynamic range that the display device can show; typically, the HDR image's dynamic range is greater than the display device's. Therefore, to better display HDR images, tone mapping is usually used to map the HDR image's dynamic range to the display device's display range.

[0003] However, the display effect of HDR images after tone mapping is not good. For example, in HDR images of the sky taken in sunset or afterglow scenes, the HDR images after tone mapping will show bright lines such as bright rings and halos around the sun that do not match the natural scene, as well as dark lines such as dark rings. Summary of the Invention

[0004] This application provides a method and electronic device for generating HDR images, which can avoid the problem of unnatural brightness transitions in HDR images after tone mapping and improve the display effect of HDR images.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] Firstly, a method for generating HDR images is provided, applicable to electronic devices. The method includes: acquiring a first HDR image; performing tone mapping on the first HDR image to obtain a second HDR image; the first HDR image includes a first highlight region, which includes a highlighted object; the second HDR image includes a second highlight region, which corresponds to the first highlight region; then, the gradient direction of a first pixel in the second highlight region is corrected to the gradient direction of a second pixel in the first highlight region to obtain a third HDR image; the second pixel corresponds to the first pixel. There are two gradient directions for the pixels: the positive or negative value of the difference between the brightness value of the next pixel on the x-axis and the brightness value of the pixel itself, and the positive or negative value of the difference between the brightness value of the next pixel on the y-axis and the brightness value of the pixel itself. The dynamic range of the third HDR image is smaller than that of the first HDR image, and the third HDR image is used for display. That is, the corrected gradient direction of the first pixel is consistent with the gradient directions of the second pixel on both the x-axis and y-axis.

[0007] Since the first HDR image has not undergone tone mapping, the brightness change direction of pixels in the first highlight area can better reproduce the brightness changes of the corresponding content in a natural scene. Therefore, the brightness change direction of pixels in the corrected second highlight area is consistent with that in the first highlight area, meaning that the brightness change direction of pixels in the corrected second highlight area can also reproduce the brightness changes of the corresponding content in a natural scene. This solution effectively avoids the problem of unnatural brightness transitions at the edges of highlight objects caused by inconsistent brightness change directions, presenting users with a more realistic and natural display effect.

[0008] In another possible implementation of the first aspect, the HDR dynamic range of the electronic device's display screen is smaller than the HDR dynamic range of the first HDR image, and the dynamic range of the third HDR image is adapted to the HDR dynamic range of the display screen. In this implementation, although the HDR dynamic range of the display screen is smaller than the HDR dynamic range of the first HDR image, the dynamic range of the processed third HDR image can be adapted to the HDR dynamic range of the display screen, thus ensuring the display effect of the electronic device.

[0009] In another possible implementation of the first aspect, the electronic device can correct the gradient direction of any first pixel within the second highlight region. This arbitrary first pixel can be a pixel with a reverse gradient direction or a pixel with a non-reverse gradient direction. In this way, the electronic device can correct all first pixels within the second highlight region such that the gradient direction of all first pixels in the corrected second highlight region is consistent with the gradient direction of the corresponding second pixel.

[0010] In another possible implementation of the first aspect, the first pixel is a pixel in the second highlight region whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlight region, i.e., a pixel with a reverse gradient direction. In this way, the electronic device can correct the first pixel with a reverse gradient direction in the second highlight region, so that the gradient direction of all first pixels in the corrected second highlight region is consistent with the gradient direction of the corresponding second pixel.

[0011] In another possible implementation of the first aspect, the electronic device can receive a first HDR image from a second electronic device, the HDR dynamic range of the display of the electronic device being smaller than the HDR dynamic range of the first HDR image, so the electronic device performs tone mapping on the first HDR image to obtain a second HDR image.

[0012] In another possible implementation of the first aspect, the electronic device can capture multiple frames of images and fuse them to obtain a first HDR image. Specifically, in response to the user's shooting operation of the HDR scene, a first HDR image is obtained; if the HDR dynamic range of the electronic device's display screen is smaller than the HDR dynamic range of the first HDR image, then tone mapping is performed on the first HDR image to obtain a second HDR image.

[0013] In another possible implementation of the first aspect, the electronic device corrects the direction of brightness change of the first pixel by correcting the brightness values ​​of pixels adjacent to the first pixel. Specifically, it corrects the brightness value of the next pixel on the x-axis so that the gradient direction of the first pixel on the x-axis is consistent with the gradient direction of the second pixel on the x-axis. It also corrects the brightness value of the next pixel on the y-axis so that the gradient direction of the first pixel on the y-axis is consistent with the gradient direction of the second pixel on the y-axis. This implementation provides a way to correct the gradient direction of the first pixel.

[0014] In another possible implementation of the first aspect, the electronic device corrects the brightness value of pixel (i+1,j) in the second highlighted region so that the gradient direction of pixel (i,j) on the x-axis is consistent with the gradient direction of pixel (i,j) in the first highlighted region on the x-axis. The brightness value of pixel (i,j+1) in the second highlighted region is also corrected so that the gradient direction of pixel (i,j) on the y-axis is consistent with the gradient direction of pixel (i,j) in the first highlighted region on the y-axis. Here, pixel (i,j) in the second highlighted region is the first pixel, and pixel (i,j) in the first highlighted region is the second pixel; pixel (i+1,j) in the second highlighted region is the next pixel on the x-axis after the first pixel, and pixel (i,j+1) in the second highlighted region is the next pixel on the y-axis after the first pixel. In this context, i takes values ​​sequentially from {1,2,...,m}, j takes values ​​sequentially from {1,2,...,n}, and the first pixel is the pixel in the second highlight region whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlight region. In this way, the electronic device can correct the first pixel with the inverted gradient from pixel (1,1) to pixel (m,n).

[0015] In another possible implementation of the first aspect, the electronic device corrects the brightness value of pixel (i+1,j) in the second highlighted region so that the gradient direction of pixel (i,j) in the second highlighted region along the x-axis is consistent with the gradient direction of pixel (i,j) in the first highlighted region along the x-axis. The brightness value of pixel (i,j+1) in the second highlighted region is also corrected so that the gradient direction of pixel (i,j) in the second highlighted region along the y-axis is consistent with the gradient direction of pixel (i,j) in the first highlighted region along the y-axis. Here, pixel (i,j) in the second highlighted region is the first pixel, and pixel (i,j) in the first highlighted region is the second pixel; pixel (i+1,j) in the second highlighted region is the next pixel in the x-axis of the first pixel, and pixel (i,j+1) in the second highlighted region is the next pixel in the y-axis of the first pixel. Here, i takes values ​​sequentially in {1,2,...,m}, and j takes values ​​sequentially in {1,2,...,n}. In this way, the electronic device can correct each first pixel from pixel (1,1) to pixel (m,n) in the second highlight area, and each first pixel includes the first pixel with the reverse gradient and the first pixel with the same gradient direction.

[0016] In another possible implementation of the first aspect, the brightness value of pixel (i+1,j) in the corrected second highlight region is: the product of the gradient of the second pixel on the x-axis and the first correction coefficient, plus the brightness value of pixel (i,j) in the second highlight region. The brightness value of pixel (i,j+1) in the corrected second highlight region is: the product of the gradient of the second pixel on the y-axis and the second correction coefficient, plus the brightness value of pixel (i,j) in the second highlight region. The gradient direction of the modified first pixel is consistent with the gradient direction of the second pixel, and the gradient value is a scaling factor of the gradient value of the second pixel.

[0017] In another possible implementation of the first aspect, after obtaining the third HDR image, the electronic device uses the first image as a reference frame to perform color correction on the fourth HDR image to obtain the fifth HDR image. The fourth HDR image is the third HDR image. The first HDR image is obtained by fusing multiple frames, including the first image and at least one second image; the exposure duration of the second image is longer than that of the first image. Color correction effectively avoids the problem of oversaturation after tone mapping in conventional techniques, allowing the corrected image to better reflect the colors of the natural scene. The electronic device can first perform brightness direction correction on the second HDR image, and then perform color correction on the third HDR image. In this way, the corrected image has a uniform brightness transition and natural, realistic colors.

[0018] In another possible implementation of the first aspect, the electronic device may first determine whether the second highlighted region includes a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted region. If the second highlighted region includes a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted region, then the gradient direction of the first pixel in the second highlighted region is corrected. If the second highlighted region does not include a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted region, then the gradient direction of the first pixel in the second highlighted region is not corrected.

[0019] In another possible implementation of the first aspect, if the second highlighted region does not include a first pixel whose gradient direction is inconsistent with that of the corresponding pixel in the first highlighted region, the electronic device corrects the color of the second HDR image. Specifically, using the first image as a reference frame, color correction is performed on the fourth HDR image to obtain the sixth HDR image; wherein the fourth HDR image is the second HDR image; wherein the first HDR image is obtained by fusing multiple frames, including the first image and at least one second image; the exposure duration of the second image is greater than that of the first image. Color correction can effectively avoid the problem of oversaturation after tone mapping in conventional techniques, making the corrected image better reflect the colors in the natural scene. If the second HDR image does not include a first pixel with an inverted gradient, the electronic device directly performs color correction on the second HDR image. In this way, the corrected image has a uniform brightness transition and natural and realistic colors.

[0020] In another possible implementation of the first aspect, the color correction method specifically involves: obtaining a first chromaticity component and a first hue component of a first image; replacing the second chromaticity component of a fourth HDR image with the first chromaticity component, and replacing the second hue component of the fourth HDR image with the first hue component. This implementation provides a specific way to implement color correction.

[0021] Secondly, this application provides an electronic device comprising: a memory, a display screen, and one or more processors. The memory, the display screen, and the processors are coupled together. The memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device performs a method as described in the first aspect and any of its possible design embodiments.

[0022] Thirdly, this application provides a chip system applicable to electronic devices including memory. The chip system includes one or more interface circuits and one or more processors. The interface circuits and processors are interconnected via lines. The interface circuits are used to receive signals from the aforementioned memory and send the signals to the processor, the signals including computer instructions stored in the memory. When the processor executes the computer instructions, the electronic device performs the method as described in the first aspect and any of its possible design embodiments.

[0023] Fourthly, this application provides a computer-readable storage medium including computer instructions. When the computer instructions are executed on an electronic device, they cause the electronic device to perform the method as described in the first aspect and any of its possible design embodiments.

[0024] Fifthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the method as described in the first aspect and any of its possible design methods.

[0025] Understandably, the beneficial effects that can be achieved by the electronic device of any possible design of the second aspect, the chip system of the third aspect, the computer-readable storage medium of the fourth aspect, and the computer program product of the fifth aspect can be referred to as the beneficial effects of the first aspect and any possible design, which will not be repeated here. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of an HDR image generation process provided in an embodiment of this application;

[0027] Figure 2 A schematic diagram illustrating the different display effects of an untone-mapped HDR image and a tone-mapped HDR image, provided as an embodiment of this application;

[0028] Figure 3 This is a schematic diagram illustrating the display effect of an HDR image after tone mapping in a conventional technique.

[0029] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0030] Figure 5 A schematic diagram of the software architecture of an electronic device provided in an embodiment of this application;

[0031] Figure 6 A schematic diagram of the interface display of an electronic device provided in an embodiment of this application;

[0032] Figure 7 This is a flowchart illustrating a method for generating HDR images provided in an embodiment of this application;

[0033] Figure 8 A comparison diagram of a first highlight area and a second highlight area provided for an embodiment of this application;

[0034] Figure 9 A flowchart illustrating another method for generating HDR images provided in an embodiment of this application;

[0035] Figure 10 A flowchart illustrating another method for generating HDR images provided in this application embodiment;

[0036] Figure 11 A schematic diagram illustrating a process for generating an HDR image, provided in an embodiment of this application;

[0037] Figure 12 This is a comparison diagram of the display effects of an HDR image before and after correction, provided as an embodiment of this application. Detailed Implementation

[0038] In digital images, dynamic range can be used to represent the difference between the darkest and brightest parts of an image. For example, dynamic range can be the ratio of the maximum brightness to the minimum brightness in an image, and its unit can be dB.

[0039] Based on different dynamic ranges, images can be categorized into standard dynamic range (SDR) images and high dynamic range (HDR) images. SDR images have a dynamic range between 1 nit and 100 nits. HDR images have a dynamic range between 0.001 nits and 10,000 nits. Here, nits are units of brightness. As can be seen, compared to ordinary images such as SDR images, HDR images expand the brightness range of the image, thus recording a larger range of brightness information and revealing more details in both bright and dark areas.

[0040] In conventional technologies, the dynamic range of a display device is typically lower than that of an HDR image. This display device can be either a device capable of displaying SDR images or a device capable of displaying HDR images. In other words, the dynamic range of an HDR image does not match the dynamic range of the display device, and typically the dynamic range of an HDR image is higher than that of the display device.

[0041] Therefore, HDR images are typically tone-mapped to map the dynamic range of the HDR image to the dynamic range that the display device can display. Tone mapping specifically refers to mapping the brightness of the HDR image to the brightness range that the target display device can display. For example, an HDR image may contain pixels with a brightness value of 4000 nits, while the target device (such as a mobile phone) can only display pixels with a brightness value of 0.5-500 nits. Because this exceeds the brightness value that the mobile phone can display, the phone may display that pixel as white. Similarly, an HDR image may contain pixels with a brightness value of 0.05 nits, which, due to the brightness value that the mobile phone can display, may appear as black. Figure 2 As shown, when an electronic device, such as a mobile phone, displays an unmapped HDR image, the display effect is as follows: Figure 2 As shown on the left. The phone displays the mapped HDR image, and its display effect is as follows. Figure 2 As shown on the right side of the middle section.

[0042] Tone mapping algorithms are typically used to tone-map HDR images. These algorithms map the high dynamic range (HDR) of the image to the low dynamic range (LVR) that the target device can display. Because tone mapping is a non-linear mapping process, it preserves detail and local contrast information in the HDR image, specifically the bright and dark areas. For example, using a mobile phone... Figure 1 As shown, the mobile phone captures an HDR image corresponding to a natural scene, processes the HDR image using tone mapping, and finally displays an HDR image with reduced dynamic range.

[0043] In conventional techniques, HDR images after tone mapping often exhibit unnatural display issues. This is particularly true when HDR images contain highlighted objects; the transition between the highlighted object and its surroundings, such as at the edges of the highlighted object, can appear unnatural. Figure 3As shown in 'a', the highlighted object can be a light source such as the sun. When a user photographs the sky at sunset or afterglow, black or white circles often appear around the sun's edge. One possible reason is that, to enhance the contrast in HDR images, tone mapping algorithms increase the contrast between the highlighted object and its surroundings, for example, by increasing the brightness values ​​of the edge pixels of the highlighted object. It should be understood that in natural scenes, the brightness of a highlighted object like the sun decreases from the center to the edge; increasing the brightness values ​​of the edge pixels of the sun may result in white circles at the edges. Another possible reason is that tone mapping algorithms increase contrast by decreasing the brightness values ​​of the edge pixels of the sun, which may result in dark circles at the edges. Yet another possible reason is that the parameters of existing tone mapping algorithms are not set appropriately, resulting in this issue. The black and white circles mentioned earlier do not exist in natural scenes and are also absent in the unprocessed HDR image; they appear in the processed HDR image after tone mapping.

[0044] In summary, when HDR images include bright objects, the existing tone mapping algorithms do not produce good display effects. Specifically, the edges of bright objects show bright lines such as bright circles or halos that do not match the natural scene, as well as dark lines such as dark circles.

[0045] Therefore, this application provides a method for generating an HDR image, which can correct the gradient direction of the first pixel in the second highlight region of the tone-mapped second HDR image to match the gradient direction of the second pixel corresponding to the first pixel in the second highlight region of the first HDR image before tone mapping. The corrected gradient direction of the first pixel in the second highlight region is consistent with the gradient direction of the corresponding second pixel in the first highlight region before tone mapping. Here, the gradient direction of a pixel is the direction of brightness change of the pixel. The gradient direction of the first pixel includes both the gradient direction along the x-axis and the gradient direction along the y-axis. That is, the corrected brightness change directions of the first pixel along the x-axis and y-axis are consistent with the brightness change directions of the second pixel along the x-axis and y-axis.

[0046] Since the first HDR image has not undergone tone mapping, the brightness change direction of pixels in the first highlight area can better reproduce the brightness changes of the corresponding content in a natural scene. Therefore, the brightness change direction of pixels in the corrected second highlight area is consistent with that in the first highlight area, meaning that the brightness change direction of pixels in the corrected second highlight area can also reproduce the brightness changes of the corresponding content in a natural scene. This solution effectively avoids the problem of unnatural brightness transitions at the edges of highlight objects caused by inconsistent brightness change directions, presenting users with a more realistic and natural display effect.

[0047] The method provided in this application can be applied to electronic devices with data processing capabilities and a display screen. The aforementioned first electronic device may include a server, mobile phone, tablet computer, laptop computer, personal computer (PC), ultra-mobile personal computer (UMPC), handheld computer, netbook, smart home device (e.g., smart TV, smart screen, large screen, smart speaker, smart air conditioner, etc.), personal digital assistant (PDA), wearable device (e.g., smartwatch, smart bracelet, etc.), in-vehicle device, virtual reality device, etc., and this application does not impose any limitations on this. In this application embodiment, the aforementioned electronic device is an electronic device capable of running an operating system and installing applications. Optionally, the operating system running on the electronic device may be... system, system, Systems, etc.

[0048] For example, please refer to Figure 4 The diagram illustrates the structure of an electronic device 400. The electronic device 400 may include a processor 410, an external memory interface 420, an internal memory 421, an audio module 430, a speaker 430A, a microphone 430B, a display screen 440, a communication module 450, a power module 460, an input device 470, a sensor module 480, a camera 490, etc. The sensor module 480 may include a pressure sensor, a touch sensor, etc.

[0049] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 400. In other embodiments of this application, the electronic device 400 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0050] Processor 410 may include one or more processing units. For example, processor 410 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), and / or a neural network processing unit (NPU). Different processing units may be independent components or integrated into one or more processors. In some embodiments, electronic device 400 may also include one or more processors 410.

[0051] The processor 410 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 410 is a cache memory. This memory can store instructions or data that the processor 410 has just used or that are used repeatedly. If the processor 410 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 410, and thus improves the efficiency of the system.

[0052] The external storage interface 420 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 400. The external memory card communicates with the processor 410 through the external storage interface 420 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0053] Internal memory 421 can be used to store one or more computer programs, which include instructions. Processor 410 can execute the instructions stored in internal memory 421, thereby causing electronic device 400 to perform application running methods, as well as various applications and data management, as provided in some embodiments of this application. In some embodiments, processor 410 can execute instructions stored in internal memory 421 and / or instructions stored in memory disposed in processor 410 to cause electronic device 400 to perform application running methods, as well as other applications and data management, as provided in embodiments of this application.

[0054] Electronic device 400 can implement audio functions through audio module 430, speaker 430A, microphone 430B, and application processor, such as music playback and recording. Speaker 430A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals.

[0055] Microphone 430B, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. Users can speak by bringing their mouth close to microphone 430B to input sound signals into it.

[0056] The communication function of electronic device 400 can be realized through antenna 1, antenna 2 and communication module 450, etc.

[0057] Communication module 450 can provide solutions for wireless communication applications on electronic device 400, including cellular, Wi-Fi, Bluetooth, and wireless data transmission modules (e.g., 433MHz, 868MHz, 915MHz). Communication module 450 can be one or more devices integrating at least one communication processing module. Communication module 450 receives electromagnetic waves via antenna 1 or antenna 2, filters and frequency-modulates the electromagnetic wave signals, and sends the processed signal to processor 410. Communication module 450 can also receive signals to be transmitted from processor 410, frequency-modulate and amplify them, and then convert them into electromagnetic waves for radiation via antenna 1 or antenna 2.

[0058] Electronic device 400 implements display functions through a GPU, a display screen 440, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 440 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 410 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0059] Display screen 440 is used to display images, videos, etc. Display screen 440 includes a display panel. In some embodiments, electronic device 400 may include one or N displays screens 440, where N is a positive integer greater than 1. In this embodiment, display screen 440 can be used to display a user interface (UI) and receive user operations on the UI.

[0060] The power module 460 can be used to supply power to the various components included in the electronic device 400. In some embodiments, the power module 460 can be a battery, such as a rechargeable battery.

[0061] Input device 470 may include a keyboard, mouse, etc. The keyboard is used to input English letters, numbers, punctuation marks, etc. into electronic device 400, thereby issuing commands to electronic device 400 and inputting data.

[0062] Electronic device 400 can implement shooting functions through an ISP, camera 490, video codec, GPU, display 440, and application processor. The ISP is used to process data fed back by the camera 490. The camera 490 is used to capture still images or videos. In some embodiments, electronic device 400 may include one or N cameras 490, where N is a positive integer greater than 1. The digital signal processor is used to process digital signals, including digital image signals and other digital signals. For example, when electronic device 400 selects a frequency, the digital signal processor performs Fourier transforms on the frequency energy. The video codec is used to compress or decompress digital video.

[0063] Taking the aforementioned electronic device 400 as an example, which is a mobile phone, the software system of the electronic device 400 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to exemplify the software structure of the electronic device 400. This software structure is as follows... Figure 5 As shown.

[0064] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime, the system libraries, and the kernel layer.

[0065] The application layer can include a series of application packages.

[0066] like Figure 5 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, SMS, and social networking.

[0067] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0068] like Figure 5 As shown, the application framework layer may include a content provider, a view system, a resource manager, a notification manager, an input system, a brightness correction module, a color correction module, etc.

[0069] The input system is used to monitor the phone's input modules (such as touchscreen drivers) and convert the parameters input by the input modules into usable events, which are then passed to the relevant upper-layer modules. For example, the input system is used to monitor the phone's touchscreen through the touchscreen driver and convert the touch parameters generated by the touchscreen input into usable events, which are then passed to the upper-layer APP.

[0070] Content providers store and retrieve data, making that data accessible to applications. This data can include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, and more.

[0071] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build the display interface of an application.

[0072] The brightness correction module is used to correct the brightness direction of the first pixel in the second highlight area of ​​the second HDR image. The color correction module is used to correct the second HDR image or the third HDR image.

[0073] The notification manager allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short time without user interaction.

[0074] The Android Runtime comprises the core libraries and the virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part contains the functionalities that Java calls, and the other part is the core Android library itself. The application layer and application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0075] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0076] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0077] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0078] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0079] A 2D graphics engine is a graphics engine for 2D drawing.

[0080] The kernel layer can include touchscreen drivers, display drivers, sensor drivers, and audio drivers, etc.

[0081] The following uses a mobile phone as an example and, in conjunction with the accompanying drawings, describes a method for generating HDR images provided by an embodiment of this application. This solution is applicable to scenarios involving taking photos or videos, where the first HDR image can be a photograph or a video frame. The applicable scenarios of this solution are described below with reference to the accompanying drawings.

[0082] The mobile phone can receive the user's shooting operation and obtain a first HDR image. If the HDR dynamic range of the mobile phone's display is smaller than the HDR dynamic range of the first HDR image, the mobile phone performs tone mapping on the first HDR image to obtain a second HDR image.

[0083] For example, in a photo-taking scenario, the phone can display [the following text is missing from the original] in response to the user opening the camera app. Figure 6 The preview interface 601 for taking a photo is shown in (b) above. For example, the operation of opening the camera application described above can be performed by the user... Figure 6 The action of clicking the camera app icon shown in (a) is described above. The preview interface 601 for taking a photo may include a preview image 602. In response to the user's click on the "Shoot Shutter" 603 in the preview interface 601, the phone can execute S701 to acquire a first HDR image, and then execute S702-S705 to obtain a processed HDR image.

[0084] Furthermore, before implementing this solution, the mobile phone can first identify whether the current shooting scene is a high dynamic range (HDR) shooting scene. If the current shooting scene is an HDR shooting scene, then in response to the user's click operation on the "Shoot Shutter" 603 in the preview interface 601, S701 is executed. For example, in response to the user opening the camera app, the mobile phone captures at least one preview image, and based on the at least one preview image, identifies whether the current shooting scene is an HDR shooting scene. Specifically, for the at least one preview image, the mobile phone can calculate the average contrast of the at least one preview image, the average brightness of the dark areas in the at least one preview image, or the average brightness of the bright areas in the at least one preview image. If the average contrast of the at least one preview image is greater than a preset threshold, or the average brightness of the dark areas in the at least one preview image is greater than a preset threshold, or the average brightness of the bright areas in the at least one preview image is greater than a preset threshold, then the mobile phone identifies the current shooting scene as an HDR shooting scene. As another example, in response to a user opening the camera app, the phone captures at least one preview image. The phone then identifies whether the preview image includes a preset subject. If the preview image includes a preset subject, the phone recognizes the current shooting scene as a high dynamic range (HDR) scene. Preset subjects can include the sun, sky, streetlights, lamps, etc. Correspondingly, HDR scenes can be sunset scenes, evening glow scenes, etc.

[0085] Optionally, the mobile phone provides multiple shooting modes, including an HDR shooting mode. In response to the user selecting HDR mode, the mobile phone can execute a method for generating an HDR image provided in this application embodiment during shooting. For example, the mobile phone's photo preview interface may include an HDR switch 604. In response to the user turning on the HDR switch and clicking the shutter button 603, the mobile phone can execute S701 to acquire a first HDR image, and then the mobile phone can execute S702-S705 to obtain a processed HDR image. The gallery application can display the processed HDR image in a large image preview mode.

[0086] Optionally, the mobile phone can receive a first HDR image from a second electronic device. If the HDR dynamic range of the mobile phone's display is smaller than the HDR dynamic range of the first HDR image, the mobile phone can perform tone mapping on the first HDR image to obtain a second HDR image.

[0087] In other words, the first HDR image can be obtained by fusing multiple frames captured by a mobile phone, or by fusing multiple frames captured by a second electronic device. The following section uses the example of obtaining the first HDR image by fusing multiple frames captured by a mobile phone to illustrate this solution. Figure 7 This is a flowchart illustrating a method for generating HDR images provided in an embodiment of this application.

[0088] S701, the mobile phone acquires the first HDR image. Specifically, S701 can include S701a and S701b.

[0089] S701a: The mobile phone captures N frames of images, and the exposure values ​​of the N frames are different.

[0090] Exposure value reflects the exposure level of an image. Exposure value is related to exposure parameters, including exposure time, ISO, and lens aperture. A mobile phone captures N frames of images with N exposure values ​​by adjusting one or more of these parameters. N is an integer greater than 1. For example, regarding exposure time, a longer exposure time results in a higher exposure value, meaning a brighter image. A shorter exposure time results in a lower exposure value, meaning a darker image. Similarly, ISO indicates the image sensor's sensitivity to light. Higher ISO indicates higher sensitivity, resulting in a brighter image; conversely, lower ISO indicates lower sensitivity, resulting in a darker image.

[0091] The N frames can be RAW images captured by a camera. These N frames can be RGB or YUV images. RGB images include R, B, and G components. Each pixel in an RGB image is composed of these three components. YUV images include Y, U, and V components. The Y component describes the pixel's luminance, while the U and V components describe its chromaticity. The following section uses an RGB image as an example to illustrate this scheme.

[0092] In this scheme, N is an integer greater than 1. That is, the phone must capture at least two image frames, such as the first image, and at least one second image. The exposure value of the second image is greater than that of the first image. For example, the phone can capture three image frames, including the first image with the smallest exposure value, and two second images with exposure values ​​greater than the first image. For instance, the exposure values ​​of the first image, second image 1, and second image 2 can be EV0-1, EV0, and EV+1, respectively. For example, the first image is captured by the phone using exposure time 'a', second image 1 is captured by the phone using exposure time 'b', and second image 2 is captured by the phone using exposure time 'c'. Here, exposure time 'a' is less than exposure time 'b', which is less than exposure time 'c'.

[0093] The N frames include long-exposure frames (second images), short-exposure frames (first images), and normal-exposure frames (second images). Images with an exposure value equal to a preset exposure value can be normal-exposure frames. Images with an exposure value less than the preset exposure value are short-exposure frames. Images with an exposure value greater than the preset exposure value are long-exposure frames. Optionally, if the mobile phone captures N frames by adjusting the exposure duration, then images with an exposure duration equal to a first preset exposure duration can be normal-exposure frames. Images with an exposure duration less than the preset exposure duration can be short-exposure frames. Images with an exposure duration greater than or equal to the preset exposure duration are long-exposure frames.

[0094] Optionally, if N=2, the second image in the N frames with a large exposure value or a long exposure time is a normal exposure frame, and the first image in the N frames with a small exposure value or a short exposure time is a short exposure frame. If N=2, the N frames can be long exposure frames and short exposure frames, or long exposure frames and normal exposure frames, or short exposure frames and normal exposure frames.

[0095] If N = 3, the N frames include long exposure frames, short exposure frames, and normal exposure frames. If N is greater than 3, the N frames also include long exposure frames, short exposure frames, and normal exposure frames. Furthermore, taking exposure duration as an example, long exposure frames with an exposure duration exceeding a second preset exposure duration can be classified as ultra-long exposure frames. Short exposure frames with an exposure duration less than a third preset exposure duration can be classified as ultra-short exposure frames. The second preset exposure duration is greater than the first preset exposure duration, and the third preset exposure duration is less than the first preset exposure duration.

[0096] S701b merges N frames of images into a first HDR image.

[0097] The mobile phone merges N frames of images with N exposure values ​​into a first HDR image. Taking the three frames mentioned earlier as an example, the mobile phone can merge long exposure frames, medium exposure frames, and short exposure frames into a first HDR image. For example, the mobile phone uses the following formula to obtain the first HDR image.

[0098] rgb_hdr=rgb_long*weight_long+rgb_normal*weight_normal+rgb_short*weight_short.

[0099] Where rgb_hdr is the first HDR image. rgb_long is the long exposure frame, and weight_long is the weight of the long exposure frame. rgb_normal is the medium exposure frame, and weight_normal is the weight of the medium exposure frame. rgb_short is the short exposure frame, and weight_short is the weight of the short exposure frame.

[0100] In this way, the mobile phone can use multi-frame fusion technology to merge at least two images with different exposure values ​​into a first HDR image.

[0101] Optionally, the phone can also choose not to merge the N first images. Instead, the phone can use an AI model to adjust the dynamic range of one frame from the N first images to obtain the first HDR image. This single frame can be any of a medium exposure frame, a short exposure frame, or a long exposure frame. In this way, the phone can capture only two first images: a second image with a large exposure value or long exposure time, and a first image with a small exposure value or short exposure time.

[0102] S702: The phone performs tone mapping on the first HDR image to obtain the second HDR image.

[0103] The mobile phone can use a tone mapping algorithm to map the first HDR image to obtain the mapped second HDR image. The tone mapping algorithm can be any type, and this application embodiment does not impose any specific limitation on it. Optionally, the tone mapping algorithm can be any of global mapping, local mapping, or hybrid mapping.

[0104] Optionally, the phone can also use a tone mapping table to map the first HDR image and obtain the mapped second HDR image.

[0105] The mobile phone can map the first brightness value of each pixel in the first HDR image to a second brightness value using a tone mapping algorithm or tone mapping table. The range of the second brightness value is smaller than the range of the first brightness value. In other words, tone mapping compresses the brightness value of each pixel in the first HDR image.

[0106] For example, in the first HDR, pixel a(i,j) has a first brightness value Y = t1*R + t2*G + t3*14B. For instance, t1, t2, and t3 can be 0.299, 0.587, and 0.114, respectively. If the first HDR image is a YUV image, then for each pixel, the brightness value Y is the value of the pixel's Y component. The phone uses a tone mapping algorithm or tone mapping table to map the brightness value of pixel a(i,j) to Y'. Then, based on the mapped brightness values ​​Y' of each pixel and the first HDR image, the phone can obtain the second HDR image. For instance, the phone can calculate the first brightness gain for each pixel, which is equal to the ratio of the mapped brightness value Y' of the first pixel to the unmapped brightness value Y. Then, the phone calculates the mapped R', G', and B' components of the first pixel. Specifically, the R' component is equal to the product of the first brightness gain and the unmapped R component. The G' component is equal to the product of the first brightness gain and the unmapped G component. The B' component is equal to the product of the first luminance gain and the unmapped B component. Thus, the phone obtains the R', G', and B' components of pixel a'(i,j) after tone mapping. Optionally, R' component = (R component / Y). λ *Y`,G` component=(G component / Y) λ *Y`,B` component = (B component / Y) λ *Y`, where λ is an adjustment coefficient. For each pixel, perform any of the above methods to obtain a mapped second HDR image, which can be an RGB image.

[0107] Optionally, the phone can also input the first HDR image into a neural network model to obtain a mapped second HDR image. Then, based on the second HDR image, the phone can calculate the brightness value of each mapped pixel.

[0108] Optionally, the phone can first identify whether the second highlighted area includes a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted area. If it does, the phone can correct the first pixel with the reversed gradient direction. Optionally, if it does, the phone can correct the gradient direction of both the first pixel with the reversed gradient direction and the first pixel with the same gradient direction. If it does not include the first pixel, the phone does not correct the brightness direction of the first pixel in the second HDR image. Here, the first pixel is any pixel in the second highlighted area; it can be a pixel with the reversed gradient direction or a pixel with the same gradient direction. The second pixel is the pixel in the first highlighted area corresponding to the first pixel. The gradient direction of a pixel is the direction of brightness change of that pixel.

[0109] S703, the mobile phone determines whether the second highlight area includes the first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlight area.

[0110] The first HDR image includes a first highlighted area, and the second HDR image includes a second highlighted area. The first HDR image and the second HDR image have the same size, and the position of the first highlighted area in the first HDR image is the same as the position of the second highlighted area in the second HDR image, that is, the first highlighted area and the second highlighted area correspond.

[0111] The first highlighted region includes a highlighted object. This highlighted object can be composed of pixels in the first HDR image whose brightness values ​​are greater than a threshold. For example, the highlighted object can be a light source in the first HDR image, such as the sun or a lamp. The first highlighted region is the area in the first HDR image that includes the highlighted object. Figure 3 As shown, the highlighted object is the sun, and the highlighted area is the region encompassing the sun. The area of ​​the highlighted area is greater than or equal to the area of ​​the highlighted object. The shape of the highlighted object can be circular like the sun, rectangular like a lamp, or other shapes. The shape of the highlighted area can be square or circular. Alternatively, the shape of the highlighted area can be the same as the shape corresponding to the highlighted object. For example, if the highlighted object is the sun, the highlighted area is a circle encompassing the sun. If the highlighted object is a rectangular lamp, the highlighted area is a rectangle encompassing the rectangular lamp. The following section uses the example of the highlighted object being the sun and the highlighted area being a rectangle encompassing the sun to introduce this solution.

[0112] Optionally, before executing S703, the phone can identify whether the second HDR image includes a highlighted object, or whether the first HDR image includes a highlighted object. If the first HDR image does not include a highlighted object, or the second HDR image does not include a highlighted object, the phone may not perform brightness correction on the second HDR image. The following example uses the phone identifying whether the first HDR image includes a highlighted object.

[0113] As one possible implementation, the phone can first identify whether the first HDR image includes a highlighted object. If the first HDR image does not include a highlighted object, the phone does not perform brightness correction on the tone-mapped first HDR image. If the first HDR image includes a highlighted object, the phone identifies the position of the highlighted object in the first HDR. For example, the position of the highlighted object in the first HDR image can be represented by [x1, y1, w1, h1], where (x1, y1) are the coordinates of the center point of the bounding box surrounding the highlighted object, and (w1, h1) are the width and height of the bounding box surrounding the highlighted object. The position of the first highlighted region in the first HDR image can be represented by [x2, y2, w2, h2], where (x2, y2) are the coordinates of the center point of the first highlighted region, and (w2, h2) are the width and height of the first highlighted region. The center point of the bounding box surrounding the highlighted object can coincide with the center point of the first highlighted region. The phone adds preset values ​​to the width and height of the bounding box surrounding the highlighted object to obtain the width and height of the first highlighted region, and thus obtains the position of the first highlighted region in the first HDR image. The phone obtains the location of the first highlighted area in the first HDR image, and then obtains the location of the second highlighted area in the second HDR image.

[0114] As another possible implementation, the mobile phone identifies whether the first HDR image contains a highlighted object based on the brightness values ​​of the pixels in the first HDR image. For example, the mobile phone identifies the pixel with the maximum brightness value in the first HDR image and the pixels around the maximum brightness value that have brightness values ​​exceeding a preset brightness value. The first highlighted area is the region surrounded by pixels with brightness values ​​exceeding the preset brightness value around the maximum brightness value.

[0115] Then, the phone can use the first highlighted area as a reference to identify the brightness change direction of multiple first pixels in the second highlighted area. With the top left corner of the image as the origin, the coordinate system of the first pixel includes a horizontal direction (x-axis) and a vertical direction (y-axis). The brightness change direction of the first pixel can be increasing or decreasing horizontally. Similarly, the brightness change direction of the first pixel can be increasing or decreasing vertically. For example, in the horizontal direction, the brightness change of the first pixel a(i,j) can be represented by the difference between the next first pixel a(i,j+1) and a(i,j) in the horizontal direction. If the difference is positive, the brightness change direction of the first pixel a(i,j) is increasing horizontally. If the difference is negative, the brightness change direction of the first pixel a(i,j) is decreasing horizontally. Similarly, in the vertical direction, the brightness change of the first pixel a(i,j) can be represented by the difference between the next first pixel a(i+1,j) and a(i,j) in the vertical direction. If the difference is greater than zero, the brightness of the first pixel a(i,j) increases vertically. If the difference is less than zero, the brightness of the first pixel a(i,j) decreases vertically.

[0116] It should be understood that since the first HDR image is an unprocessed image, the brightness change direction of each second pixel in the first highlight area of ​​the first HDR image can well reproduce the brightness change of the corresponding content in the natural scene. If the brightness change direction of each first pixel in the second highlight area of ​​the second HDR image is consistent with the brightness change direction of the corresponding second pixel in the first highlight area, it means that the brightness of the second HDR image after tone mapping can also well reproduce the brightness change of the corresponding content in the natural scene, and no correction is needed. If the brightness change direction of some or all of the first pixels in the highlight area of ​​the second HDR image is inconsistent with the brightness change direction of the corresponding second pixel in the first highlight area, then the brightness change direction of some or all of the first pixels in the second HDR image cannot well reproduce the brightness change of the corresponding content in the natural scene. In this case, the second HDR image may have display defects such as bright spots, bright lines, halos, or dark spots and dark lines.

[0117] For example, in the vertical direction, suppose the brightness value of the second pixel in the first column of the first HDR image is (10, 9, 8, 7, 6), and its corresponding gradient is (-1, -1, -1, -1, -1). That is, the direction of change of the second pixel in the first column of the bright area in the first HDR image is decreasing sequentially. In the second HDR image, the brightness value of the first pixel in the same column is (5, 4, 6, 2, 1), and its corresponding gradient is (-1, 2, -3, -1, -1). The brightness change between the second and third first pixels in the second HDR image is increasing, and the brightness change between the third and fourth first pixels is decreasing. That is, the brightness value first decreases, then suddenly increases, and then decreases again. Thus, the second HDR image may contain bright spots at the locations where the brightness value suddenly increases, such as the bright spot displayed at the third first pixel in the first column. Multiple bright spots can form bright circles or bright lines. For another example, in the second brightness image, the brightness value of the same column is (5, 1, 4, 3, 1), and the corresponding gradient is (-4, 3, -1, -2, 1). The second HDR image experiences an excessive brightness decrease between the first and second pixels, causing the brightness of the second and third pixels to increase. Specifically, in the first column, the brightness value first decreases, then suddenly decreases again, then increases, and then decreases again. Consequently, the second HDR image may contain dark spots at the locations of sudden brightness decreases, such as the dark spot appearing at the second pixel in the first column. Multiple dark spots can form dark circles or lines.

[0118] To this end, the mobile phone can calculate the brightness change of each second pixel in the first bright area and the brightness change of each first pixel in the second bright area. For example, the mobile phone can perform image differentiation on the first brightness map of the first HDR image and the second brightness map of the second HDR image to obtain the brightness changes of each second pixel in the first bright area and the brightness changes of each first pixel in the second bright area.

[0119] In a digital image, a first luminance map includes the luminance value Y of each second pixel in the first HDR image. A second luminance map includes the luminance value Y' of each first pixel in the second HDR image. The phone differentiates the first luminance map to obtain the gradient of each second pixel in the first HDR image. The phone differentiates the second luminance map to obtain the gradient of each first pixel in the second HDR image.

[0120] Image differentiation, also known as gradient calculation, refers to differentiating the brightness value of each pixel in a first or second brightness image in both the x and vertical directions. Taking the first brightness image as an example, image differentiation includes calculating the gradient of the first brightness image in the horizontal direction and calculating the gradient of the first brightness image in the vertical direction. For example, a mobile phone can call the `gradient` function to differentiate the first brightness image in the horizontal (X) and vertical (Y) directions, and call the `gradient` function to differentiate the second brightness image in the horizontal (X) and vertical (Y) directions. The pixel gradient can be positive or negative. For example, a positive gradient in the horizontal direction indicates that the pixel brightness value is increasing in the horizontal direction. A negative gradient indicates that the pixel brightness value is decreasing in the horizontal direction. There are various methods for image differentiation, and this application embodiment does not specifically limit this one. One possible implementation is given below.

[0121] For example, in the horizontal direction, the gradient of the i-th row of pixels = (brightness value of the (i+1)-th row of pixels - brightness value of the i-th row of pixels). The gradient of the last row of pixels may not be calculated, or the gradient of the previous row of pixels may be used as the gradient of the last row of pixels. In the vertical direction, the gradient of the j-th column = (brightness value of the (j+1)-th column of pixels - brightness value of the j-th column of pixels). The gradient of the last column of pixels may not be calculated, or the gradient of the previous column of pixels may be used as the gradient of the last column of pixels.

[0122] In this way, the phone can obtain the horizontal and vertical gradients of each second pixel in the first highlighted area, and the horizontal and vertical gradients of each first pixel in the second highlighted area. The pixel gradient reflects the direction and magnitude of the brightness change between that pixel and its neighboring pixels. The pixel gradient includes both magnitude and direction. Specifically, the gradient direction is the sign of the difference between the brightness value of the pixel and the brightness value of the current pixel on the x-axis. The gradient magnitude is the difference between the brightness value of the pixel and the brightness value of the current pixel on the x-axis.

[0123] For example, in the horizontal direction, the gradient of pixel a is -10, indicating that the brightness value of pixel b, which is located before pixel a, is less than the brightness value of pixel a. That is, the brightness decreases horizontally from pixel a to pixel b, and the decrease is equal to the gradient value of pixel a, which is 10 nits. In the vertical direction, the gradient of pixel c is 30, indicating that the brightness value of pixel d, which is located before pixel c, is greater than the brightness value of pixel c. That is, the brightness increases vertically from pixel c to pixel d, and the increase is equal to the gradient value of pixel c, which is 30 nits.

[0124] The mobile phone identifies first pixels in the second bright area with inconsistent brightness variations based on the horizontal and vertical gradients of each second pixel in the first bright area and the horizontal and vertical gradients of each first pixel in the second bright area. Specifically, the mobile phone can identify first pixels in the second bright area with inconsistent horizontal brightness variations based on the horizontal gradients of each second pixel in the first bright area and the horizontal gradients of each first pixel in the second bright area. The mobile phone can also identify pixels in the second bright area with inconsistent vertical brightness variations based on the vertical gradients of each second pixel in the first bright area and the vertical gradients of each first pixel in the second bright area. If the horizontal brightness variation of the first pixel is inconsistent with the horizontal brightness variation of the second pixel, and / or the vertical brightness variation of the first pixel is inconsistent with the vertical brightness variation of the second pixel, then the first pixel is a pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first bright area, i.e., a pixel with a reverse gradient direction.

[0125] Taking the identification of pixels with inconsistent brightness changes in the horizontal direction by a mobile phone as an example, the phone can, for instance, identify whether the gradient direction of each pixel in the first highlighted area is consistent with the gradient direction of each pixel in the second highlighted area. Specifically, for example, the phone reads the gradient of the second pixel at the first coordinate position in the first highlighted area, and the phone reads the gradient of the first pixel at the first coordinate position in the second highlighted area. The phone then identifies whether these two gradients are both positive or both negative. The first coordinate position in the first highlighted area can be the top-left corner of the first highlighted area. The first coordinate position in the second highlighted area can be the top-left corner of the second highlighted area. That is, the phone obtains the gradients of pixels at the same coordinate positions in the first and second highlighted areas and compares whether the gradients of pixels at the same position are both positive or both negative. If the brightness values ​​of the two pixels at the same position are both positive, it means that the brightness changes in the same direction and are increasing. If the brightness values ​​of the two pixels at the same position are both negative, it means that the brightness changes in the same direction and are decreasing. If the brightness values ​​of two pixels at the same location are both positive or both negative, it indicates that the brightness change direction of the pixels at the same location in the first highlighted area of ​​the first HDR image and the second highlighted area of ​​the second HDR image is consistent, and there is no need to correct the brightness change direction of the pixels at that location in the second highlighted area. If the brightness values ​​of two pixels at the same location are not both positive or both negative, it indicates that the brightness change direction of the pixels at the same location in the first highlighted area of ​​the first HDR image and the second highlighted area of ​​the second HDR image is inconsistent, and it is necessary to correct the brightness change direction of the pixels at that location in the second highlighted area.

[0126] As another possible implementation, the phone can calculate the product of the gradients of two pixels at the same location in the first and second highlighted regions, one by one. If the product is less than zero, the two pixels at the same location have different gradient directions. That is, the brightness change direction of the pixel in the second highlighted region is different from the brightness change direction of the corresponding pixel in the first highlighted region. If the product is greater than zero, the brightness change direction of the two pixels at the same location is the same. That is, the brightness change direction of the pixel in the second highlighted region is the same as the brightness change direction of the corresponding pixel in the first highlighted region.

[0127] S704, if the second highlight area includes a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlight area, the mobile phone corrects the gradient change direction of the first pixel in the second highlight area.

[0128] The phone corrects the brightness value of the next pixel on the x-axis after the first pixel, making the gradient direction of the first pixel on the x-axis consistent with the gradient direction of the second pixel on the x-axis. For example, for pixel a'(i,j), the phone corrects the brightness value of pixel a'(i,j+1) in the second highlight area, making the gradient direction of pixel a'(i,j) in the second highlight area consistent with the gradient direction of pixel a(i,j) in the first highlight area. The corrected brightness value of pixel a'(i,j+1) in the second highlight area is: the product of the gradient of the second pixel corresponding to pixel a'(i,j) on the x-axis and the first correction coefficient, plus the brightness value of a'(i,j) in the second highlight area.

[0129] The phone corrects the brightness value of the next pixel on the y-axis after the first pixel, making the gradient direction of the first pixel on the y-axis consistent with the gradient direction of the second pixel on the y-axis. For example, for pixel a'(i,j), the phone corrects the brightness value of the second highlighted region a'(i+1,j) so that the gradient direction of pixel a'(i,j) on the y-axis of the second highlighted region is consistent with the gradient direction of pixel a(i,j) on the y-axis of the first highlighted region. The corrected brightness value of the second highlighted region a'(i+1,j) is: the product of the gradient of the second pixel corresponding to pixel a'(i,j) on the y-axis and the second correction coefficient, plus the brightness value of pixel a'(i,j) in the second highlighted region.

[0130] After correction, the gradient direction of each first pixel in the second highlight region along the x-axis is consistent with the gradient direction of the corresponding second pixel along the x-axis, and the gradient direction of each first pixel along the y-axis is consistent with the gradient direction of the corresponding second pixel along the y-axis. Here, pixel a`(i, j) is the pixel after tone mapping of pixel a(i, j). That is, the first pixel corresponds to the second pixel, and the first pixel is the tone-mapped second pixel.

[0131] like Figure 8 As shown, in the horizontal direction, the gradient GX`(i,j) of the first pixel, such as pixel a`(i,j), in the second highlighted region on the x-axis is equal to the difference in brightness values ​​between pixel a`(i,j+1) and pixel a`(i,j). Similarly, the gradient GX(i,j) of the second pixel a(i,j) corresponding to the first pixel in the first highlighted region on the x-axis is equal to the difference in brightness values ​​between pixel a(i,j+1) and pixel a(i,j).

[0132] If the directions of gradient GX(i,j) and GX`(i,j) are inconsistent, the phone modifies the brightness value of pixel a`(i,j+1). That is, it modifies the brightness value of the neighboring pixels of the first pixel whose gradient is inversely related. Specifically, the phone uses the x-axis gradient GX(i,j) of the second pixel as the x-axis gradient of pixel a`(i,j). The corrected brightness value of pixel a`(i,j+1) minus the brightness value of pixel a`(i,j) equals the x-axis gradient GX(i,j) of the second pixel. Therefore, the corrected brightness value of pixel a`(i,j+1) equals the brightness value of pixel a`(i,j) plus the x-axis gradient GX(i,j) of the second pixel. Since the dynamic imaging range of the first HDR image differs from that of the second HDR image, directly using gradient GX(i,j) as the corrected gradient of pixel a`(i,j) may exceed the value range of the brightness value of pixel a`(i,j+1). Therefore, the gradient GX(i,j)*C1 can be used as the gradient of pixel a`(i,j). That is, the brightness value of the corrected pixel a`(i,j+1) = the brightness value of pixel a`(i,j) + the gradient GX(i,j)*C1 of the second pixel on the x-axis.

[0133] Optionally, the brightness value of the corrected pixel a`(i,j+1) = the brightness value of pixel a`(i,j) + (gradient GX`(i,j) - GX(i,j)) * C1. Where gradient GX`(i,j) - gradient GX(i,j) is the distance between the two gradients.

[0134] In the vertical direction, the brightness value of the corrected pixel a`(i+1,j) minus the brightness value of pixel a`(i,j) equals the gradient GY(i,j) of the second pixel on the y-axis. Therefore, the brightness value of the corrected pixel a`(i+1,j) equals the brightness value of pixel a`(i,j) plus the gradient GY(i,j) of the second pixel on the y-axis. Since the dynamic imaging range of the first HDR image differs from that of the second HDR image, directly using the gradient GY(i,j) of the second pixel on the y-axis for the correction of pixel a`(i,j) may exceed the range of the brightness value of a`(i+1,j). Therefore, GY(i,j)*C1 can be used as the gradient of pixel a`(i,j). That is, the brightness value of the corrected pixel a`(i+1,j) equals the brightness value of pixel a`(i,j) plus the gradient GY(i,j)*C2 of the second pixel on the y-axis.

[0135] Wherein, C1 and C2 are the first correction coefficient and the second correction coefficient. The values ​​of C1 and C2 can be the same or different. The value of this correction coefficient can be between 0 and 0.25. For example, the maximum value of C1 or C2 can be the ratio of the bit width of the luminance value in the first HDR image to the bit width of the luminance value in the second HDR image. Wherein, in the first HDR image, the bit width of the luminance value can be 14 bits, 16 bits, or higher. In the second HDR image, the bit width of the luminance value can be 8 bits, 10 bits, etc. For example, in the first HDR image, the bit width of the luminance value is 16 bits. In the second HDR image, the bit width of the luminance value is 14 bits. Then C1 or C2 is the ratio of 16 to 14, and the maximum value of C1 or C2 is 0.25. Optionally, in this scheme, both C1 and C2 are 0.18.

[0136] Optionally, the brightness value of the corrected pixel a`(i+1,j) = brightness value of pixel a`(i,j) + (gradient GY`(i,j) - gradient GY(i,j)) * C2. Where gradient GY`(i,j) - gradient GY(i,j) is the distance between the two gradients.

[0137] In this scheme, i takes values ​​sequentially from {1, 2, ..., m}, and j takes values ​​sequentially from {1, 2, ..., n}. m is the row number of the pixel in the second highlighted region, and n is the column number of the pixel in the second highlighted region. In this way, the mobile phone can correct each first pixel in the second highlighted region from pixel a(1, 1) to a(m, n), and each first pixel includes the first pixel with the inverse gradient and the first pixel with the same gradient direction.

[0138] Optionally, the phone can also correct only the first pixel with the inverted gradient. That is, i takes values ​​sequentially in {1,2,...,m}, and j takes values ​​sequentially in {1,2,...,n}. Before correction, the phone can identify whether the first pixel is the first pixel with the inverted gradient direction. If the first pixel is the first pixel with the inverted gradient direction, the phone performs the method described above to correct the gradient direction of the first pixel. In this way, the phone can correct the first pixel with the inverted gradient from pixels a(1,1) to a(m,n). For example, if the phone corrects the gradient direction of pixel a`(i,j) by correcting the brightness value of pixel a`(i+1,j) in the horizontal direction, the phone can recalculate the gradient direction of pixel a`(i+1,j). If the gradient direction of pixel a`(i+1,j) is consistent with the gradient direction of the corresponding second pixel in the first highlighted area, the phone does not correct the gradient direction of pixel a`(i+1,j). If the gradient direction of pixel a`(i+1,j) is inconsistent with the gradient direction of the corresponding pixel in the first bright area, the phone corrects the brightness value of a`(i+2,j) to correct the gradient direction of pixel a`(i+1,j).

[0139] Optionally, after correcting the gradient direction of the first pixel whose gradient direction is reversed in a certain row or column, the phone may not recalculate the gradients of the remaining pixels, but directly correct the gradient directions of all first pixels included in the row or column containing that first pixel.

[0140] Optionally, after correcting the horizontal gradient direction of the first pixel in the first highlighted area, the phone recalculates the vertical gradient direction of the first pixel based on the corrected brightness value. The phone can then re-identify whether the vertical gradient direction of the first pixel is consistent with the vertical gradient direction of the second pixel in the first highlighted area. If they are consistent, the phone does not need to correct the vertical gradient direction of the first pixel. If they are inconsistent, the phone needs to correct the vertical gradient direction of the first pixel.

[0141] In the example given above, the phone first corrects the gradient direction of the first pixel in the horizontal direction, and then corrects the gradient direction of the first pixel in the vertical direction. Optionally, the phone may also first correct the gradient direction of the first pixel in the vertical direction, and then correct the gradient direction of the first pixel in the horizontal direction. This application does not specifically limit this approach.

[0142] The S705 phone generates a third HDR image based on the second HDR image and the third highlight area.

[0143] The third highlight area is the second highlight area after correcting the gradient direction of some or all of the first pixels. The phone uses the third brightness area to replace the second highlight area in the second HDR image. Then, the phone generates a third HDR image based on the brightness values ​​of the pixels in the third highlight area and the brightness values ​​of the pixels in other areas of the second HDR image excluding the third highlight area.

[0144] The third HDR image is used for display. The dynamic range of the third HDR image is smaller than that of the first HDR image. The HDR dynamic range of the phone's display is smaller than that of the first HDR image, and the dynamic range of the third HDR image is adapted to the HDR dynamic range of the phone's display.

[0145] The phone's S701-S705 functions can correct the brightness values ​​of pixels in the bright areas of the second HDR image, making the brightness transition in the bright areas of the corrected third HDR more natural.

[0146] Furthermore, to avoid oversaturation in the second HDR image after tone mapping, the phone can further correct the colors of the second or third HDR image. For example... Figure 3 As shown in b, the solar region has excessive saturation, resulting in color distortion.

[0147] Examples such as Figure 9 As shown. After the phone generates the third HDR image, the phone can execute S901.

[0148] S901, the mobile phone uses the first image as a reference frame to correct the color of the fourth HDR image to obtain the fifth HDR image; wherein, the fourth HDR image is the third HDR image.

[0149] S902, if the second highlight area does not include the first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlight area, the fourth HDR image is color-corrected using the first image as a reference frame to obtain the sixth HDR image; wherein, the fourth HDR image is the second HDR image.

[0150] In other words, if the gradient direction of each pixel in the second highlight area is the same as that of each pixel in the first highlight area, the phone does not need to adjust the gradient direction of the pixels in the second highlight area, and the phone can directly perform color correction on the second HDR image. Optionally, if the phone recognizes that the first HDR image does not include the first highlight object, or the second HDR image does not include the second highlight object, the phone can also directly perform color correction on the second HDR image.

[0151] If the second highlight area includes a first pixel whose gradient direction is inconsistent with that of the corresponding pixel in the first highlight area, the phone adjusts the brightness value of the pixels in the second highlight area to obtain a third HDR image, and then the phone performs color correction on the third HDR image.

[0152] The phone performs color correction on the second HDR image to obtain the sixth HDR image. The phone then applies color correction to the third HDR image to obtain the fifth HDR image. The method for adjusting the colors of the third HDR image is the same as the method for adjusting the colors of the second HDR image; the following explanation will use the adjustment of the third HDR image as an example.

[0153] To avoid altering the brightness of the corrected third HDR image, the phone can correct the color of the third HDR image by correcting its chroma and hue. For example, the phone can use a short-exposure frame from the multiple first images acquired in S701a as a reference frame, and correct the color of the third HDR image based on this reference frame. The short-exposure frame can be the first image among the N frames with the shortest exposure time or the smallest exposure value.

[0154] The phone can first convert the short exposure frame to the LCH color space, then convert the third HDR image to the LCH space, and then use the C and H components of the short exposure frame to replace the C and H components in the third HDR image to obtain the color-corrected fourth HDR image.

[0155] An exemplary S901 may include S1-S5.

[0156] S1, the mobile phone preprocesses the first image corresponding to the short exposure frame to obtain the first image in sRGB format.

[0157] Preprocessing includes, but is not limited to, automatic white balance (AWB), color correction matrix correction, and gamma correction. AWB uses algorithms to restore the white image captured under different color temperatures of ambient light to true white, i.e., the white seen by the human eye under natural sunlight. The color correction matrix (CCM) is used to correct the proportions of the R, G, and B components of each pixel in the image. Gamma correction adjusts the grayscale coefficients of the image, making the processed image more comfortable to view on a screen and better representing the information and features of the original image. The gamma-corrected image can be in sRGB format.

[0158] S2, the phone converts the first image from the sRGB color space to the XYZ color space, then from the XYZ color space to the Lab color space, and finally from the Lab color space to the LCH color space.

[0159] The mobile phone first converts the first image in sRGB format to an XYZ format image. That is, it converts the first image from the sRGB color space to the XYZ color space. In the XYZ color space, each pixel includes an X component, a Y component, and a Z component. The X component is the red primary color stimulus, the Y component is the green primary color stimulus, and the Z component is the blue primary color stimulus. For example, each pixel in the first image in sRGB format can be converted based on the following formula.

[0160] X=0.412453*R+0.357580*G+0.180423*B.

[0161] Y=0.212671*R+0.715160*G+0.072169*B.

[0162] Z=0.019334*R+0.119193*G+0.950227*B.

[0163] For a given pixel, R is the value of the R component of that pixel, G is the value of the G component of that pixel, and B is the value of the B component of that pixel.

[0164] Next, the phone converts the first image in XYZ format to a first image in Lab format. In the Lab color space, each pixel includes an L component, an a component, and a b component. The L component is luminance, the a component is red-green hue, and the b component is yellow-blue hue. For example, each pixel in the Lab format first image can be converted based on the following formula.

[0165] First, using the D65 light source standard white point as a reference, the X, Y, and Z components of each pixel are normalized to obtain the x, y, and z components. Specifically, x = X / Xn1, y = Y / Yn1, and z = Z / Zn1. Where Xn1 is the normalization parameter for the X component, Yn1 is the normalization parameter for the Y component, and Zn1 is the normalization parameter for the Z component. For example, the values ​​of Xn1, Yn1, and Zn1 are 0.95047, 1.0, and 1.08883, respectively.

[0166] Next, a non-linear transformation is applied to the normalized x, y, and z components. For example, the following formula can be used for the non-linear transformation. Substitute the x, y, and z components of each pixel into the following formula to calculate F(x), F(y), and F(z), respectively. For instance, if the x component of a pixel is greater than... Then F(x) is If the x component is less than Then F(x) is Quantity.

[0167]

[0168] Finally, the Lab color space conversion is completed using the following method: L = 116 * F(y) - 16, a = 500 * (F(x) - F(y)), b = 200 * (F(y) - F(z)).

[0169] Finally, the phone converts the first image in Lab format to the first image in LCH format.

[0170] In the LCH color space, each pixel includes an L component, a C component, and an H component. The L component is the luminance, the C component is the chrominance, and the h component is the hue.

[0171] C = sqrt(0.2989 * a) 2 +0.1368*b 2 +0.2330*(ba) 2 The sqrt() function is used to return the square root. For example, SQRT(16) = 4.

[0172] H=acos((0.57732*(ba)+1.42879*(b+a)) / (sqrt(0.2989*a 2 +0.1368*b 2 +0.2330*(b-

[0173] a) 2 The acos() function is used to calculate the arccosine of a given value. For example, acos(0.5) = 1.047198.

[0174] In this way, the phone completes the color space conversion of the first image. The converted first image can be a first LCH image, in which the first L component, first C component, and first H component of each pixel are obtained from the sRGB format first image using the method and formula described above. The phone can then obtain the first C component and first H component of each pixel in the first image. The first image includes a fourth highlight area, which corresponds to both the first and second highlight areas. The fourth highlight area includes the third pixel.

[0175] After that, the phone can execute S3.

[0176] S3, the phone sequentially converts the third HDR image from sRGB color space to XYZ color space, from XYZ color space to Lab color space, and from Lab color space to LCH color space.

[0177] The mobile phone can perform gamma correction on the RGB format third HDR image, converting it from RGB to sRGB format. The phone uses the method shown in S2 to complete the color space conversion of the third HDR image. The converted first image can be a second LCH image, where the second L component, second C component, and second H component of each pixel are obtained from the RGB format third HDR image using the method and formula described above.

[0178] The phone only performs color correction on the first pixel within the third highlighted area of ​​the third HDR image. For example, the first image includes a fourth highlighted area. The position of the fourth highlighted area in the first image is the same as the position of the third highlighted area in the third HDR image, and the size of the third HDR image is the same as the size of the first image. Thus, the phone modifies the second C component and second H component of each pixel in the third highlighted area to the first C component and first H component of the corresponding pixel in the fourth highlighted area.

[0179] S4, the phone modifies the second C component of the first pixel in the third bright area (fifth bright area) to the first C component of the corresponding third pixel in the fourth bright area, and modifies the second H component of the first pixel in the third HDR image to the first H component of the corresponding third pixel in the fourth bright area, thus obtaining the corrected fifth HDR image.

[0180] Each pixel in the color-corrected fifth HDR image includes a second L component, a first C component, and a first H component.

[0181] Furthermore, the phone can blur the edges of bright objects in the fifth HDR image. Blurring the edges of bright objects effectively removes noise, smooths the edges, and makes them more soft and natural. This blurring process can be, for example, multi-valued blurring, such as Gaussian blurring.

[0182] Furthermore, in order to display the fifth HDR image, the phone can convert the fifth HDR image from LCH space back to sRGB space.

[0183] The S5 phone sequentially converts the fifth HDR image from the LCH color space to the Lab color space, from the Lab color space to the XYZ color space, and from the XYZ color space to the sRGB color space.

[0184] First, the phone converts the fifth HDR image from the LCH color space to the Lab color space. Specifically, the phone can use the following calculation method to substitute the L component, C component, and H component of each pixel in the fifth HDR image into the following calculation formula to obtain the L component, a component, and b component of each pixel.

[0185] a=(exp 10(L / 100-1) -1) / 0.0764.

[0186] b = sqrt(C 2 -(1-sqrt(1-(a / 298.9) 2 ))*C 2 )*sign(a). Among them, sign(a)={1if a>=0,-1if a<0}

[0187] Where L represents the L component of each pixel in the fifth HDR image, C represents the C component of each pixel in the fifth HDR image, and H represents the H component of each pixel in the fifth HDR image. a represents the a component of each pixel in the fifth HDR image converted to Lab format. b represents the b component of each pixel in the fifth HDR image converted to Lab format.

[0188] Next, the phone converts the fifth HDR image from the Lab color space to the XYZ color space. Specifically, the phone can use the following calculation method to substitute the L, a, and b components of each pixel in the fifth HDR image into the following calculation formula to obtain the X, Y, and Z components of each pixel.

[0189] Step 1: The phone performs linear transformations on the L, a, and b components of each pixel in the fifth HDR image. Specifically, y = (L + 16) / 116; x = a / 500 + y; z = yb / 200.

[0190] Step 2: The phone performs a nonlinear transformation on the x, y, and z values ​​obtained in Step 1.

[0191] Nonlinear transformation:

[0192] For example, if the x-component of a pixel is greater than Then F(x) is If the x component is less than Then F(x) is Quantity.

[0193] Step 3: The phone performs inverse normalization on F(x), F(y), and F(z) obtained in step 2. X = F(x) * Xn², Y = F(Y) * Yn², Z = F(Z) * Zn². Where Xn² is the inverse normalization parameter for the X component, Yn² is the inverse normalization parameter for the Y component, and Zn² is the inverse normalization parameter for the Z component. For example, the values ​​of Xn², Yn², and Zn² are 0.95047, 1.0, and 1.08883, respectively.

[0194] Next, the phone converts the fifth HDR image from the XYZ color space to the RGB color space. Specifically, the phone can use the following calculation method to substitute the X, Y, and Z components of each pixel in the fifth HDR image into the following calculation formula to obtain the R, G, and B components of each pixel.

[0195] [RGB] = [XYZ](M T RGB2XYZ ) -1 Among them, M RGB2XYZ M is the first transformation matrix for converting RGB color space values ​​to XYZ color space. T RGB2XYZ This is the transpose of the first transformation matrix. (M) T RGB2XYZ ) -1 For M T RGB2XYZ The inverse matrix.

[0196] For example,

[0197] Then, the phone converts the fifth HDR image from the RGB color space to the srgb color space.

[0198] The first step is for the phone to perform gamma correction on the fifth HDR image in RGB format. Specifically, the phone substitutes the R, G, and B components of each pixel in the fifth HDR image in RGB format into the following formula to obtain the corrected f(R), f(G), and f(B).

[0199]

[0200] For example, if the R component of a pixel is greater than 0.0031308, then f(R) is 1.055*R. 1 / 2.4 -0.055. If the x component is less than 0.0031308, then f(R) is 12.92*R.

[0201] The second step is to trim the f(R), f(G), and f(B) obtained in the first step.

[0202]

[0203] The third step is to denormalize f(R)`, f(G)`, and f(B)` obtained in the second step.

[0204] Specifically, r = f(R) * 255. g = f(G) * 255. b = f(B) * 255.

[0205] Optionally, the mobile phone performs color correction on the second HDR image by: obtaining the first chromaticity component and the first hue component of the third pixel in the fourth highlight region of the first image; replacing the second chromaticity component of the second pixel in the second highlight region (fifth highlight region) of the second HDR image with the first chromaticity component of the third pixel; and replacing the second hue component of the second pixel in the second highlight region of the second HDR image with the first hue component of the third pixel, thereby obtaining the corrected sixth HDR image.

[0206] Figure 11 This is a flowchart illustrating a process for generating an HDR image, as provided in an embodiment of this application. Figure 11 As shown, the mobile phone performs tone mapping on the first HDR image to obtain the third HDR image. The first HDR image includes a first luminance map, which includes the luminance values ​​of each second pixel in the first HDR image. The second HDR image includes a second luminance map, which includes the luminance values ​​of each first pixel in the second HDR image. If the mobile phone recognizes that the first HDR image includes the sun, the mobile phone corrects the gradient of some or all of the first pixels in the sunset region of the second HDR image based on the first and second luminance maps. Specifically, the mobile phone corrects the luminance values ​​of pixels adjacent to the first pixel to correct the gradient of the first pixel. In this way, not only the gradient direction of the first pixel is corrected, but also the size of the first pixel is corrected. After correction, the luminance change direction of the first pixel is in the same direction as the luminance change direction of the corresponding second pixel, and the gradient magnitude is scaled relative to the gradient magnitude of the second pixel. Then, the mobile phone obtains the third luminance map, which includes the luminance values ​​of each pixel in the third HDR image. Based on the third and first luminance maps, the mobile phone calculates the second luminance gain of each pixel in the third HDR image. Wherein, the second luminance gain of a pixel = the luminance value of the pixel in the third luminance map / the luminance value of the pixel in the first luminance map. Next, the phone calculates the R, G, and B components of each pixel in the third HDR image. Specifically, the R component is equal to the product of the second luminance gain and the R component of the pixel in the first HDR image. The G component is equal to the product of the second luminance gain and the G component of the pixel in the first HDR image. The B component is equal to the product of the second luminance gain and the B component of the pixel in the first HDR image. The phone can save the second luminance gain of each pixel. In this way, the phone can obtain the luminance-corrected third HDR image.

[0207] Next, the phone can perform color correction on the third HDR image. Specifically, the phone first preprocesses the reference frame, converting it from sRGB format to LCH format. The phone then performs gamma mapping on the third HDR image to obtain an sRGB format third HDR image, which it then converts back to LCH format. Finally, the phone performs LCH spatial fusion on the sunset region in the LCH format third HDR image. Specifically, for multiple first pixels within the sunset region, the phone replaces the C and H components of each first pixel with the corresponding C and H components of multiple third pixels within the sunset region from the reference frame. In this way, the phone can obtain a brightness-corrected fifth HDR image. Figure 12 As shown, Figure 12 The image shows a comparison between the HDR image after brightness and color correction and the unprocessed first HDR image. It can be seen that the final fifth HDR image exhibits a natural brightness transition in the sunset area and appropriate color saturation, presenting a more natural and realistic display effect, achieving a "what you see is what you get" display.

[0208] Optionally, the brightness-corrected third HDR image and the color-corrected fifth or sixth HDR image can be used to train a neural network model, improving the quality of the sample data input to the model and enhancing its processing performance. This neural network model script has the ability to output tone-mapped HDR images based on the input HDR images.

[0209] This application provides an electronic device including a memory, a display screen, and one or more processors. The display screen is coupled to the processors. The memory stores computer program code. The computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device can perform various functions or steps performed by the mobile phone in the above method embodiments. The structure of the electronic device can be referred to... Figure 4 The structure of the electronic device 400 shown.

[0210] This application embodiment also provides a computer storage medium, which includes computer instructions, when the computer instructions are executed in the aforementioned electronic device (such as...). Figure 4 When the electronic device 400 shown is run, it causes the electronic device to perform the various functions or steps in the above method embodiments.

[0211] This application also provides a computer program product that, when run on a computer, causes the computer to perform the various functions or steps described in the above method embodiments.

[0212] This application also provides a chip system including at least one processor and at least one interface circuit. The processor and the interface circuit are interconnected via lines. For example, the interface circuit can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit can be used to send signals to other devices (e.g., the processor). Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, and this application does not specifically limit this.

[0213] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0214] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0215] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0216] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0217] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0218] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for generating HDR images, characterized in that, Applied to electronic devices, the method includes: A first HDR image is acquired, and a second HDR image is obtained by performing tone mapping on the first HDR image; the first HDR image includes a first highlight area, and the first highlight area includes a highlight object; the second HDR image includes a second highlight area, and the second highlight area corresponds to the first highlight area; The gradient direction of the first pixel in the second highlighted region is corrected to the gradient direction of the second pixel in the first highlighted region to obtain the third HDR image; the second pixel corresponds to the first pixel; wherein, the gradient direction of a pixel is the sign of the difference between the brightness value of the next pixel on the first direction axis and the brightness value of the pixel; the first direction axis includes the x-axis and the y-axis. The dynamic imaging range of the third HDR image is smaller than that of the first HDR image, and the third HDR image is used for display; the first pixel is a pixel in the second highlight region whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlight region.

2. The method according to claim 1, characterized in that, The HDR dynamic imaging range of the display screen of the electronic device is smaller than that of the first HDR image, and the dynamic imaging range of the third HDR image is adapted to the HDR dynamic imaging range of the display screen.

3. The method according to claim 1 or 2, characterized in that, The step of obtaining a first HDR image and performing tone mapping on the first HDR image to obtain a second HDR image includes: If a first HDR image is received from a second electronic device, and the HDR dynamic range of the display screen of the electronic device is smaller than the HDR dynamic range of the first HDR image, then tone mapping is performed on the first HDR image to obtain the second HDR image.

4. The method according to claim 1 or 2, characterized in that, The step of obtaining a first HDR image and performing tone mapping on the first HDR image to obtain a second HDR image includes: In response to the user's shooting operation for the HDR scene, the first HDR image is obtained; If the HDR dynamic imaging range of the display screen of the electronic device is smaller than the HDR dynamic imaging range of the first HDR image, then the first HDR image is tone-mapped to obtain the second HDR image.

5. The method according to claim 1 or 2, characterized in that, The step of correcting the gradient direction of the first pixel in the second highlighted region to the gradient direction of the second pixel in the first highlighted region to obtain the third HDR image includes: Correct the brightness value of the next pixel on the x-axis of the first pixel so that the gradient direction of the first pixel on the x-axis is consistent with the gradient direction of the second pixel on the x-axis; Correct the brightness value of the next pixel on the y-axis of the first pixel so that the gradient direction of the first pixel on the y-axis is consistent with the gradient direction of the second pixel on the y-axis.

6. The method according to claim 5, characterized in that, The step of correcting the brightness value of the next pixel on the x-axis of the first pixel, so that the gradient direction of the first pixel on the x-axis is consistent with the gradient direction of the second pixel on the x-axis, includes: The brightness value of the pixel (i,j+1) in the second bright area is corrected so that the gradient direction of the pixel (i,j) in the second bright area on the x-axis is consistent with the gradient direction of the pixel (i,j) in the first bright area on the x-axis. The step of correcting the brightness value of the next pixel on the y-axis of the first pixel so that the gradient direction of the first pixel on the y-axis is consistent with the gradient direction of the second pixel on the y-axis includes: The brightness value of the pixel (i+1,j) in the second bright area is corrected so that the gradient direction of the pixel (i,j) in the second bright area on the y-axis is consistent with the gradient direction of the pixel (i,j) in the first bright area on the y-axis. Wherein, the pixel (i,j) of the second highlighted area is the first pixel, and the pixel (i,j) of the first highlighted area is the second pixel; the pixel (i,j+1) of the second highlighted area is the next pixel of the first pixel on the x-axis, and the pixel (i+1,j) of the second highlighted area is the next pixel of the first pixel on the y-axis; Where i takes values ​​in {1,2,...,m}, and j takes values ​​in {1,2,...,n}.

7. The method according to claim 6, characterized in that, The brightness value of pixel (i+1,j) in the second highlight area after correction is: the product of the gradient of the second pixel on the x-axis and the first correction coefficient plus the brightness value of pixel (i,j) in the second highlight area; The brightness value of pixel (i,j+1) in the second highlighted area after correction is: the product of the gradient of the second pixel on the y-axis and the second correction coefficient plus the brightness value of pixel (i,j) in the second highlighted area.

8. The method according to claim 1 or 2, characterized in that, After correcting the gradient direction of the first pixel in the second highlighted region to the gradient direction of the second pixel in the first highlighted region to obtain the third HDR image, the method further includes: Using the first image as a reference frame, the fourth HDR image is color-corrected to obtain the fifth HDR image; wherein, the fourth HDR image is the third HDR image; The first HDR image is obtained by fusing multiple frames, including the first image and at least one second image; the exposure duration of the second image is greater than that of the first image.

9. The method according to claim 1 or 2, characterized in that, Before correcting the gradient direction of the first pixel in the second highlighted region to the gradient direction of the second pixel in the first highlighted region to obtain the third HDR image, the method further includes: The second highlighted region is defined as a first pixel whose gradient direction is inconsistent with the gradient direction of the corresponding pixel in the first highlighted region.

10. The method according to claim 9, characterized in that, The method further includes: If the second highlighted area does not include the first pixel, the first image is used as a reference frame to perform color correction on the fourth HDR image to obtain the sixth HDR image; wherein, the fourth HDR image is the second HDR image; The first HDR image is obtained by fusing multiple frames, including the first image and at least one second image; the exposure duration of the second image is greater than that of the first image.

11. The method according to claim 10, characterized in that, Using the first image as a reference frame, color correction is performed on the fourth HDR image, including: Obtain the first chromaticity component and the first hue component of the third pixel in the fourth highlighted region of the first image; The second chromaticity component of the fourth pixel in the fifth highlight area of ​​the fourth HDR image is replaced by the first chromaticity component of the third pixel, and the second hue component of the fourth pixel in the fifth highlight area of ​​the fourth HDR image is replaced by the first hue component of the third pixel.

12. An electronic device, characterized in that, The electronic device includes: a memory, a display screen, and one or more processors; the memory, the display screen, and the processors are coupled; wherein the memory is used to store computer program code, the computer program code including computer instructions; when the computer instructions are executed by the processor, the electronic device performs the method as described in any one of claims 1-11.

13. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-11.

14. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1-11.

Images