Image processing method and apparatus, electronic device, and computer program product

Abstract

Embodiments of the present application disclose an image processing method and device, electronic equipment and computer program product. The method comprises: obtaining a first HDR image and a first image set corresponding to a target color space; the first HDR image is obtained by fusing a plurality of original images with different exposure amounts; obtaining a target original image and a second image set corresponding to the target color space; the target original image is one of the plurality of original images with different exposure amounts, and the exposure amount corresponding to the target original image is less than an exposure threshold; determining color gain information according to the first image set and the second image set; performing color adjustment processing on at least one first channel image in the first image set according to the color gain information; and generating a second HDR image according to the first image set after color adjustment. In the embodiments of the present application, the color performance of the HDR image can be improved, and the color presented by the HDR image is closer to the real color.

Description

Technical Field

[0001] This application relates to the field of imaging technology, specifically to an image processing method, apparatus, electronic device, and computer program product. Background Technology

[0002] With the rapid development of electronic technology, using cameras on electronic devices to capture images has become an indispensable function in users' daily lives and work. However, due to limitations in the hardware of electronic devices and some image algorithms (such as white balance algorithms), images captured by cameras on electronic devices currently cannot effectively retain the true colors of the scene being captured. The light and shadow in the images, as well as the colors, deviate significantly from the colors of the real world. Summary of the Invention

[0003] This application discloses an image processing method, apparatus, electronic device, and computer program product that can enhance the color performance of HDR images, make the colors presented in HDR images closer to real colors, and improve the image quality of HDR images.

[0004] This application discloses an image processing method, the method comprising:

[0005] A first image set corresponding to a first high dynamic range (HDR) image and a target color space is obtained. The first image set includes first channel maps corresponding to each channel of the first HDR image and the target color space. The first HDR image is obtained by fusing multiple frames of original images with different exposure levels.

[0006] Obtain a second image set corresponding to the target original image and the target color space. The second image set includes the second channel image corresponding to each channel of the target original image and the target color space. The target original image is one of the original images with different exposure levels among the multiple frames. The exposure level corresponding to the target original image is less than the exposure threshold.

[0007] Based on the first image set and the second image set, determine the color gain information;

[0008] The color adjustment process is performed on at least one first channel image in the first image set according to the color gain information;

[0009] A second HDR image is generated based on the first set of color-adjusted images.

[0010] This application discloses an image processing apparatus, the apparatus comprising:

[0011] The acquisition module is used to acquire a first image set corresponding to a first high dynamic range (HDR) image and a target color space. The first image set includes a first channel image corresponding to each channel of the first HDR image and the target color space. The first HDR image is obtained by fusing multiple frames of images with different exposure levels.

[0012] The acquisition module is further configured to acquire a second image set corresponding to the target original image and the target color space, the second image set including the second channel image corresponding to each channel of the target original image and the target color space; the target original image is one of the multiple frames of images with different exposure levels, and the exposure level corresponding to the target original image is less than the exposure level threshold;

[0013] The gain determination module is used to determine color gain information based on the first image set and the second image set;

[0014] A color adjustment module is used to perform color adjustment processing on at least one first channel image in the first image set according to the color gain information.

[0015] The image generation module is used to generate a second HDR image based on the first set of color-adjusted images.

[0016] This application discloses an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the method described above.

[0017] This application discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described above.

[0018] This application discloses a computer program product, including a computer program, which, when executed by a processor, implements the method described above.

[0019] The image processing method, apparatus, electronic device, and computer program product disclosed in this application discloses the following: A first image set corresponding to a first HDR image and a target color space is obtained. This first image set includes first channel maps corresponding to each channel of the first HDR image and the target color space. A second image set corresponding to a target original image and the target color space is obtained. This second image set includes second channel maps corresponding to each channel of the target original image and the target color space. Color gain information is determined based on the first and second image sets. Color adjustment processing is performed on at least one first channel map in the first image set based on the color gain information. Then, a second HDR image is generated based on the color-adjusted first image set. In this application embodiment, since the color of the target original image with an exposure level less than an exposure threshold is closer to the true color of the shooting scene, the target original image used for fusing the first HDR image can be used as a reference for color adjustment. Color gain information is determined based on the first HDR image and the channel maps of the target original image in the target color space. Color adjustment processing is performed on the channel maps of the first HDR image in the target color space based on the color gain information, thereby obtaining a color-enhanced second HDR image. This enhances the color performance of the HDR image, making the colors presented in the HDR image closer to real colors and improving the image quality of the HDR image. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is an application scenario diagram of an image processing method in one embodiment;

[0022] Figure 2 This is a structural block diagram of an electronic device in one embodiment;

[0023] Figure 3 This is a flowchart of an image processing method in one embodiment;

[0024] Figure 4 A flowchart of an image processing method in another embodiment;

[0025] Figure 5 This is a flowchart of generating the first mask in one embodiment;

[0026] Figure 6 Here is a flowchart of an image processing method in yet another embodiment;

[0027] Figure 7 This is a flowchart for determining tone gain in one embodiment;

[0028] Figure 8 This is a schematic diagram illustrating the process of color adjustment of a first HDR image to obtain a second HDR image in one embodiment;

[0029] Figure 9 This is a block diagram of an image processing apparatus in one embodiment;

[0030] Figure 10 This is a structural block diagram of an electronic device in another embodiment. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0033] It is understood that the terms "first," "second," etc., used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, a first HDR image may be referred to as a second HDR image, and similarly, a second HDR image may be referred to as a first HDR image. Both the first HDR image and the second HDR image are HDR images, but they are not the same HDR image. The term "multiple" as used in this application refers to two or more. The term "and / or" as used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0034] HDR (High-Dynamic Range) imaging is an image technology capable of capturing and presenting a wider range of brightness and color details. Current HDR image capture techniques primarily generate HDR images by fusing multiple frames with different exposure times. During HDR image generation, electronic devices typically use a fused linear graph from multiple frames to recover the colors of highlight areas in the HDR image. The highlights of the final HDR image usually originate from the fused original image. To preserve as many highlight colors as possible from the fused images, an unexposed linear graph is often used for color correction. For example, a CCM (Color Correction Matrix) is used to correct the color of the unexposed linear graph, obtaining the saturated color in the linear domain as a color reference for the highlight areas in the final fused HDR image. This image processing method can generate a high-quality HDR image with more comprehensive information and richer colors, improving the color quality of the HDR image while effectively maintaining contrast.

[0035] In the above image processing methods, the final HDR image may have problems such as color distortion or undersaturation. The main reasons include: 1. When using CCM to perform color correction on an underexposed linear image, if the illumination in the shooting scene is high, it will cause problems such as hue shift and saturation decay in the HDR image; 2. When fusing multiple frames of images with different exposure times into a single frame of HDR image, different curve types of gamma correction will be applied to the multiple frames of images with different exposure times or to the fused HDR image. The non-linear adjustment of the gamma curve will destroy the linear color information acquired in the original RAW domain.

[0036] Therefore, HDR images generated by electronic devices often deviate significantly from real-world colors in terms of lighting and color. For example, HDR images may exhibit reds that appear magenta, yellows that are undersaturated, warm colors that are lost, or colors that are too pale. Improving the color reproduction of HDR images, especially making the colors in highlight and warm areas more closely resemble the true colors perceived by the human eye, is a pressing issue that needs to be addressed.

[0037] This application discloses an image processing method, apparatus, electronic device, and computer program product that can enhance the color performance of HDR images, make the colors presented in HDR images closer to real colors, and improve the image quality of HDR images.

[0038] Figure 1 This is an application scenario diagram of an image processing method in one embodiment. For example... Figure 1As shown, the image processing method can be applied to electronic device 100, which may include, but is not limited to, mobile phones, tablets, wearable devices (such as smartphones, smart glasses, etc.), digital cameras, laptops, vehicle terminals, etc.

[0039] In some embodiments, the electronic device 100 may include one or more cameras for capturing images. These cameras may include, but are not limited to, one or more of a high-resolution camera, a telephoto camera, a wide-angle camera, a macro camera, an infrared camera, and a monochrome camera. Furthermore, the electronic device 100 may include various cameras with different focal lengths and / or different functions. For example, the electronic device 100 may include a telephoto camera and a wide-angle camera, or it may include a high-resolution camera and a telephoto camera, etc., but is not limited to these, thereby improving the image quality and shooting effect obtained by the cameras of the electronic device 100.

[0040] In some embodiments, the electronic device 100 may also be communicatively connected to one or more external cameras, and the images captured by each camera may be transmitted to the electronic device 100 for image processing.

[0041] For example, Figure 2 This is a structural block diagram of an electronic device in one embodiment. For example... Figure 2 As shown, the electronic device may include an imaging device 110, a sensor 120, a memory 130, a processor 140, and a control logic unit 150. Image data captured by the imaging device 110 is first processed by the processor 140, which analyzes the image data to capture image statistics that can be used to determine one or more control parameters of the imaging device 110. The imaging device 110 may include one or more lenses 112 and an image sensor 114. The image sensor 114 may include a color filter array (such as a Bayer filter), and the image sensor 114 can acquire light intensity and wavelength information captured by each imaging pixel, providing a set of raw image data that can be processed by the processor 140.

[0042] Image sensor 114 can transmit raw image data to processor 140 for processing. In some embodiments, processor 140 may include one or more of an ISP (Image Signal Processing) processor, CPU (Central Processing Unit), etc. Image processing functions of electronic devices can be implemented using hardware and / or software components by defining various processing units managed by the ISP.

[0043] Sensor 120 (such as a three-axis gyroscope, Hall sensor, accelerometer, etc.) can provide image processing parameters (such as image stabilization parameters) to processor 140 based on the sensor 120 interface type. The sensor 120 interface can be an SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above interfaces.

[0044] Alternatively, image sensor 114 may also send raw image data to sensor 120, which may provide the raw image data to processor 140 based on the sensor 120 interface type, or sensor 120 may store the raw image data in image memory 130.

[0045] Processor 140 processes raw image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits. Processor 140 may perform one or more image processing operations on the raw image data and collect statistical information about the image data. The image processing operations may be performed with the same or different bit depth precision.

[0046] The processor 140 may also receive image data from the image memory 130. For example, the sensor 120 interface sends raw image data to the image memory 130, and the raw image data in the image memory 130 is then provided to the processor 140 for processing. The image memory 130 may be part of a memory device, a storage device, or a separate dedicated memory within an electronic device, and may include DMA (Direct Memory Access) features.

[0047] Upon receiving raw image data from image sensor 114 interface, sensor 120 interface, or image memory 130, processor 140 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 130 for further processing before display. Processor 140 receives processed data from image memory 130 and performs image data processing in the raw domain (RAW domain), RGB (red-green-blue), and / or YCbCr color space. The processed image data may be output to display 160 for user viewing and / or further processed by a graphics engine or GPU (Graphics Processing Unit). Furthermore, the output of processor 140 may also be sent to image memory 130, and display 160 may read image data from image memory 130. In one embodiment, image memory 130 may be configured to implement one or more frame buffers.

[0048] The statistical data determined by processor 140 can be sent to control logic unit 150. For example, the statistical data may include image sensor 114 statistics such as gyroscope vibration frequency, auto exposure, auto white balance, auto focus, flicker detection, black level compensation, and lens 112 shading correction. Control logic unit 150 may include a processor and / or microcontroller executing one or more routines (such as firmware) that determine control parameters for imaging device 110 and processor 140 based on the received statistical data. For example, control parameters for imaging device 110 may include sensor 120 control parameters (e.g., gain, integral time for exposure control, image stabilization parameters, etc.), camera flash control parameters, camera image stabilization shift parameters, lens 112 control parameters (e.g., focal length for focusing or zooming), or combinations of these parameters. ISP control parameters may include gain levels and color correction matrices for auto white balance and color adjustment (e.g., during RGB processing), and lens 112 shading correction parameters.

[0049] In some embodiments, the processor 140 can acquire multiple frames of images with different exposure levels from the imaging device 110 and fuse these multiple frames of images with different exposure levels to generate a first HDR image. The processor 140 can convert the first HDR image to a target color space to obtain a first image set, which includes the first HDR image and first channel maps corresponding to each channel of the target color space. The processor 140 can convert a target original image to a target color space to obtain a second image set, which includes the target original image and second channel maps corresponding to each channel of the target color space. The target original image is one of the frames used to fuse the first HDR image, and the exposure level corresponding to the target original image is less than an exposure threshold. The processor 140 can determine color gain information based on the first image set and the second image set, perform color adjustment processing on at least one first channel map in the first image set based on the color gain information, and then generate a second HDR image based on the color-adjusted first image set.

[0050] As an optional implementation, the entire image processing can be performed by the ISP processor and the CPU. First, the ISP processor acquires multiple frames of images with different exposure levels from the imaging device 110, and then merges these frames to generate a first HDR image. The ISP processor can then transmit the first HDR image and the target original image to the CPU. The CPU converts the first HDR image and the target original image to the target color space respectively, and performs color adjustment processing based on the converted first image set and second image set, thereby generating a second HDR image.

[0051] As another alternative implementation, the entire image processing process can also be performed in the ISP processor, which is not limited here.

[0052] like Figure 3 As shown, in one embodiment, an image processing method is provided, which can be applied to the above-described electronic device. The method may include the following steps:

[0053] Step 310: Obtain the first image set corresponding to the first HDR image and the target color space. The first image set includes the first channel image corresponding to each channel of the first HDR image and the target color space.

[0054] The first HDR image can be obtained by fusing multiple frames of original images with different exposure levels. Furthermore, the multiple frames of original images with different exposure levels can be original images acquired at the same time using different exposure parameters. These exposure parameters can include, but are not limited to, one or more of aperture, ISO (sensitivity) value, and shutter speed. These exposure parameters can be used to determine the exposure duration of the acquired original images, thereby affecting the exposure level of the original images.

[0055] In some embodiments, the multiple frames of original images with different exposure levels used to fuse and generate the first HDR image may include at least two of short-exposure images, long-exposure images, and normal-exposure images. The short-exposure image may be an original image with an exposure level less than the standard exposure level, the long-exposure image may be an original image with an exposure level greater than the standard exposure level, and the normal-exposure image may be an original image with an exposure level equal to the standard exposure level. For example, the standard exposure level may refer to the exposure level corresponding to an exposure value (EV) of 0. The short-exposure image has an exposure level less than the standard exposure level, therefore it can capture highlight details. The long-exposure image has an exposure level greater than the standard exposure level, therefore it can capture shadow details. By fusing multiple frames of original images with different exposure levels, an HDR image corresponding to a wider brightness range can be obtained.

[0056] The electronic device can acquire a first set of images corresponding to a first HDR image and a target color space, wherein the target color space may refer to a color space capable of color adjustment of the first HDR image.

[0057] In some implementations, color adjustment of the first HDR image may include hue correction and / or saturation enhancement of the first HDR image, thereby making the colors of the resulting second HDR image closer to the true colors observed by the human eye. Therefore, the target color space may include a hue channel and a saturation channel, through which the hue and saturation of the first HDR image are adjusted.

[0058] For example, the target color space may include any one of the HSV (Hue-Saturation-Lightness) color space, HSL (Hue-Saturation-Lightness) color space, and HSI (Hue-Saturation-Intensity) color space.

[0059] The first image set may include a first channel map corresponding to each channel of the first HDR image and the target color space. For example, if the target color space includes a hue channel and a saturation channel, then the first image set may include a first hue map corresponding to the first HDR image and the hue channel, and a first saturation map corresponding to the first HDR image and the saturation channel, so that the first hue map can be hue corrected and / or the first saturation map can be saturated.

[0060] Furthermore, the target color space may also include a luminance channel, and the first image set may also include a first HDR image and a first luminance map corresponding to the luminance channel. It should be noted that the representation of the luminance channel differs in different target color spaces. For example, in the HSV color space, the luminance channel may be the V (Value) channel; in the HSL color space, the luminance channel may be the L (Lightness) channel; and in the HSI color space, the luminance channel may be the I (Intensity) channel.

[0061] In one implementation, the color space corresponding to the first HDR image can be the RGB color space. The electronic device can convert the first HDR image from the RGB color space to the target color space to obtain a first image set. The electronic device can convert the first HDR image from the RGB color space to the target color space to obtain a first HDR image in the target color space, and then decompose the first HDR image in the target color space to obtain a first image set.

[0062] In other embodiments, the electronic device stores a first set of images corresponding to the first HDR image and the target color space in its memory, or the first set of images corresponding to the first HDR image and the target color space can be directly obtained from the memory.

[0063] Step 320: Obtain the second image set corresponding to the target original image and the target color space. The second image set includes the second channel image corresponding to each channel of the target original image and the target color space.

[0064] The target original image can be one of the multiple original images with different exposure levels used to fuse the first HDR image. The exposure level of this target original image is less than the exposure threshold, meaning it can be a short-exposure image. Because the exposure time of the target original image is shorter, its color representation is closer to the true colors of the shooting scene, especially in highlight areas and warm-toned areas. Therefore, in this embodiment, the target original image can be used as a reference for color adjustment of the first HDR image. Color adjustment of the first HDR image is performed based on the target original image, thereby making the colors presented in the HDR image closer to the true colors.

[0065] It should be noted that the exposure threshold mentioned above can be set according to actual needs. For example, the exposure threshold can be less than or equal to the standard exposure value, but it is not limited to this.

[0066] In some embodiments, the target original image may be the original image with the lowest exposure among the multiple frames of original images with different exposure levels used to fuse the first HDR image. The original image with the lowest exposure can be considered as the original image whose colors are closest to the true colors of the shooting scene among the multiple frames of original images with different exposure levels used to fuse the first HDR image, especially the original image whose hue and saturation in the highlight and warm color areas are closest to the true colors of the shooting scene. Therefore, the original image with the lowest exposure can be used as the target original image, and color adjustment of the first HDR image based on the target original image can further improve the color performance of the second HDR image obtained after color adjustment.

[0067] An electronic device can acquire a target original image and a second image set corresponding to a target color space. This second image set includes second channel maps corresponding to each channel of the target original image and the target color space. For example, the target color space may include a hue channel and a saturation channel. Therefore, the second image set may include a second hue map corresponding to the hue channel of the target original image, and a second saturation map corresponding to the saturation channel of the target original image. Since the hue and saturation of the target original image are closer to the true colors of the shooting scene, the second hue map can be used as a reference for hue correction of the first hue map, and the second saturation map can be used as a reference for saturation enhancement of the first saturation map. Further, the target color space may include a luminance channel, and the second image set may also include a second luminance map corresponding to the luminance channel of the target original image.

[0068] In one implementation, the target original image can be a RAW image or an RGB image. The electronic device can convert the target original image from the RAW domain or RGB color space to the target color space to obtain a second image set. Alternatively, the electronic device can convert the target original image from the RAW domain or RGB color space to the target color space to obtain a target original image in the target color space, and then decompose the target original image in the target color space to obtain a second image set.

[0069] Optionally, the target original image can be a RAW image. The electronic device can perform a linear transformation on the target original image to obtain a linear graph, and then convert the linear graph to the target color space to obtain a second image set. Since the colors of the target original image in the RAW domain captured by the camera are closest to the true colors of the shooting scene, the linear graph obtained by linear transformation of the target original image in the RAW domain can be converted to the target color space. For example, the linear graph can be converted from the RGB color space to the target color space to obtain a second image set, which can further improve the color performance of the second HDR image obtained after color adjustment.

[0070] Step 330: Determine color gain information based on the first image set and the second image set.

[0071] The electronic device can determine the color gain information corresponding to a channel based on the first channel map and the second channel map corresponding to the same channel. This color gain information can be used to compensate the first channel map of the channel, thereby enhancing the color performance of the first HDR image in that channel.

[0072] In some embodiments, the color gain information may include the hue gain corresponding to the hue channel and / or the saturation gain corresponding to the saturation channel. The hue gain can be used to perform hue correction on the first hue map corresponding to the first HDR image to reduce hue shift present in the first HDR image. The saturation gain can be used to enhance the saturation of the first saturation map corresponding to the first HDR image, thereby addressing the undersaturation problem present in the first HDR image.

[0073] Step 340: Perform color adjustment processing on at least one first channel image in the first image set according to the color gain information.

[0074] Electronic devices can perform color adjustment processing on the first channel image of a target channel based on the color gain information corresponding to the target channel. The target channel can be one or more channels in a target color space. For example, if the target channel is a hue channel, the first hue image can be hue-corrected based on the hue gain corresponding to the hue channel; or, if the target channel is a saturation channel, the first saturation image can be saturated based on the saturation gain corresponding to the saturation channel.

[0075] In some embodiments, since the first HDR image typically exhibits color distortion in its highlight and / or warm color regions—for example, reds appearing magenta, yellows undersaturated, warm colors lost, or colors too pale—while the target original image's colors in its highlight and / or warm color regions are very close to the true colors of the scene being captured, color adjustment can be performed on the highlight and / or warm color regions of the first HDR image based on the target original image. Here, the highlight region can refer to an image area with a brightness value greater than or equal to a brightness threshold, and the warm color region can refer to an image area with warm colors, such as regions of one or more colors including red, orange, and yellow.

[0076] The electronic device can identify the highlight areas and / or warm areas of the first HDR image, determine the target image area that needs color adjustment, and perform color adjustment processing on the target image area of ​​one or more first channel images according to color gain information, so that the highlight areas and / or warm areas in the second HDR image obtained after color adjustment are closer to the real colors.

[0077] Step 350: Generate a second HDR image based on the first set of color-adjusted images.

[0078] An electronic device can synthesize the various first feature maps contained in a first set of color-adjusted images to generate a second HDR image. Further, based on the first set of color-adjusted images, it can be converted from a target color space to an RGB color space to generate the second HDR image. Alternatively, the various first feature maps contained in the first set of color-adjusted images can be synthesized to obtain a synthesized image in the target color space, and then this synthesized image in the target color space can be converted from the target color space to the RGB color space to obtain a second HDR image in the RGB color space. By converting the first HDR image in the RGB color space and the original target image to the target color space, determining the color gain information using the channel map of the target color space, performing color adjustment processing, and then converting back to the RGB color space, a second HDR image with better color performance is obtained, thus improving the image quality of the HDR image.

[0079] In this embodiment, since the color of the target original image with an exposure level less than the exposure threshold is closer to the true color of the shooting scene, the target original image used to fuse the first HDR image can be used as a reference for color adjustment. Color gain information is determined based on the channel map of the first HDR image and the target original image in the target color space, and color adjustment processing is performed on the channel map of the first HDR image in the target color space based on the color gain information, thereby obtaining a color-enhanced second HDR image. This can increase the color performance of the HDR image, make the colors presented by the HDR image closer to the true colors, and improve the image quality of the HDR image.

[0080] Due to the characteristics of the human eye in its perception of light, the colors of certain objects (such as indoor candles, lanterns, and warm-colored light sources like yellow lights) are perceived as more saturated and warmer by the human eye. However, due to limitations in hardware and some image processing algorithms (such as white balance algorithms), it is currently difficult for images captured by cameras on electronic devices to accurately represent the true colors of these objects as observed by the human eye.

[0081] In some embodiments, the electronic device may perform color adjustment on the highlight areas and / or warm areas of the first HDR image, so that the warm colors in the highlight areas of the adjusted second HDR image appear closer to the true colors observed by the human eye. For example... Figure 4 As shown, an image processing method is provided, which may include the following steps:

[0082] Step 402: Obtain the first image set corresponding to the first HDR image and the target color space. The first image set includes the first tone map corresponding to the first HDR image and the tone channel, and the first luminance map corresponding to the luminance channel.

[0083] Step 404: Obtain the second image set corresponding to the original target image and the target color space.

[0084] The descriptions of steps 402 to 404 can be found in the descriptions of steps 310 to 320 in the above embodiments, and will not be repeated here.

[0085] Step 406: Generate a first mask based on the first tone map, the first brightness map and the first HDR image. The first mask is used to characterize the target image area that needs to be color adjusted.

[0086] The target image region may include the image region corresponding to the first tone range and / or the image region with a brightness value greater than the brightness threshold.

[0087] The first color tone range belongs to the warm color tone range, and the image area corresponding to the first color tone range refers to the warm color area in the image. Further, the first color tone range can be a range of red, orange, and / or yellow tones; for example, the first color tone range includes 310°–360° and / or 0°–69°. The electronic device can extract pixels whose color values ​​belong to the first color tone range from the first color tone map, thereby determining the image area corresponding to the first color tone range.

[0088] Image regions with brightness values ​​greater than a brightness threshold can refer to highlight areas in an image. Brightness information can be analyzed based on the first brightness map and the first HDR image to determine image regions (i.e., highlight areas) with brightness values ​​greater than the brightness threshold.

[0089] like Figure 5 As shown, in some embodiments, the step of generating a first mask based on a first tone map, a first brightness map, and a first HDR image may include steps 502 to 506.

[0090] Step 502: Generate a hue weight map corresponding to the first hue range based on the first hue map.

[0091] A tone weight map can be used to represent the image region corresponding to the first tone interval in the first tone map. An electronic device can extract pixels whose tone values ​​belong to the first tone interval from the first tone map and generate a tone weight map corresponding to the first tone interval.

[0092] In some embodiments, the hue values ​​of pixels belonging to a first hue interval in the first hue map can be smoothed to obtain a hue weight map. Pixels whose hue values ​​belong to the first hue interval in the first hue map can be extracted, and their hue values ​​can be smoothed to obtain a hue weight map.

[0093] Furthermore, based on the exponential function corresponding to the first tone range, the tone values ​​of pixels belonging to the first tone range in the first tone map can be non-linearly smoothed to obtain a tone weight map. For example, the first tone range includes 0°~69° and 310°~360°, and the tone values ​​of pixels belonging to the first tone range can be non-linearly smoothed using the following formulas (1) and (2).

[0094]

[0095] In this process, the hue values ​​in the first hue map belonging to 0° to 69° can be non-linearly smoothed using formula (1), where Huemap1 represents the first processing result obtained after non-linear smoothing of the hue values ​​in the first hue map belonging to 0° to 69°. The hue values ​​in the first hue map belonging to 310° to 360° can be non-linearly smoothed using formula (2), where Huemap2 represents the second processing result obtained after non-linear smoothing of the hue values ​​in the first hue map belonging to 310° to 360°.

[0096] Hue represents the hue value of a pixel in the first hue map. The hue value is non-linearly transformed using formulas (1) and (2), resulting in a smooth transition between 0 and 1 in the output, which changes with the input hue. As the input hue increases, the output gradually rises to near 1; as the input hue decreases, the output gradually decreases to near 0.

[0097] The first processing result, Huemap1, and the second processing result, Huemap2, can be added together to obtain a tone weight map. By smoothing the tone values ​​in the first tone range, the tone mapping can be effectively adjusted, making the resulting tone weight map exhibit a more natural gradient effect and achieving a smooth transition. This results in the final generated second HDR image showing more natural color changes in the warm color region.

[0098] Step 504: Generate a luminance weight map based on the first luminance map and the first HDR image.

[0099] The electronic device can identify the highlight region based on the first brightness map and the first HDR image, and generate a brightness weight map. The brightness weight map can be used to characterize the image region in the first HDR image where the brightness value is greater than the brightness threshold, that is, to characterize the highlight region in the first HDR image.

[0100] In some embodiments, the electronic device can perform grayscale conversion on the first HDR image to obtain a grayscale image, which can be used to better analyze the brightness information of the first HDR image. The first brightness map can be enhanced to adjust its brightness distribution, resulting in an enhanced first brightness map. Similarly, the second brightness map can be enhanced to adjust its brightness distribution, resulting in an enhanced second brightness map. Finally, a brightness weight map is generated based on the grayscale image, the enhanced first brightness map, and the enhanced second brightness map.

[0101] For example, enhancing the first and second brightness images separately may include performing square root processing on each of them. This square root processing makes the brightness distribution of the first and second brightness images more uniform, enhancing the smooth transition of brightness and improving contrast. It should be noted that other methods can also be used to enhance the first and second brightness images separately, such as performing histogram equalization or nonlinear transformation on each, etc., without limitation.

[0102] The electronic device can fuse a grayscale image, an enhanced first luminance map, and an enhanced second luminance map to obtain a luminance weight map. Optionally, the grayscale image, the enhanced first luminance map, and the enhanced second luminance map can be weighted and fused to obtain a luminance weight map; alternatively, the grayscale image, the enhanced first luminance map, and the enhanced second luminance map can be multiplied to obtain a luminance weight map.

[0103] In the above implementation, the first HDR image is first converted to grayscale to obtain a grayscale image, and the first luminance map and the second luminance map are enhanced respectively. Then, a luminance weight map is generated based on the grayscale image, the enhanced first luminance map, and the enhanced second luminance map. This can enhance the smooth transition effect of the luminance of the first luminance map and the second luminance map, so that the generated luminance weight map can represent the highlight area of ​​the smooth transition, making the highlight area of ​​the subsequently generated second HDR image more natural and realistic, and improving the image quality of the subsequently generated second HDR image.

[0104] Step 506: Generate the first mask based on the hue weight map and the luminance weight map.

[0105] The electronic device can fuse the hue weight map and the luminance weight map to obtain a first mask. This first mask can be used to characterize the highlight regions and warm regions with smooth transitions. Furthermore, the first mask can also be used to characterize the warm regions in the highlight regions with smooth transitions.

[0106] Optionally, the hue weight map and the luminance weight map can be weighted and fused to obtain a first mask. The hue weight map can correspond to the first weight, and the luminance weight map can correspond to the second weight. The hue weight map is multiplied by the first weight, and the luminance weight map is multiplied by the second weight. The two multiplication results are then added together to obtain the first mask. The sum of the first weight and the second weight can be equal to 1.

[0107] In some embodiments, since there may be some image regions in the first HDR image that do not require color adjustment, these regions may be areas where the colors will not shift significantly, or areas that are optimized by other dedicated image processing units. For example, these regions may include the portrait area and / or the sky area. For instance, if the first HDR image is a night scene image, the sky area does not require color adjustment, while the portrait area is processed by a dedicated image processing unit for optimizing portrait color representation, or the sky area is also processed by a dedicated image processing unit for optimizing sky color representation. In this case, color adjustment of the portrait area and the sky area does not need to be performed according to the image processing method provided in this embodiment. Therefore, image regions that do not require color adjustment can be eliminated.

[0108] The electronic device can fuse the tone weight map and the brightness weight map to obtain a second mask, and then remove the image region corresponding to the target region mask from the second mask to obtain a first mask. The target region mask can be used to characterize image regions that do not require color adjustment; for example, the target region mask includes a portrait mask and / or a sky mask.

[0109] Optionally, the human figure region in the first HDR image can be identified to obtain a human figure mask, which can be used to represent the human figure region in the first HDR image. Similarly, the sky region in the first HDR image can be identified to obtain a sky mask, which can be used to represent the sky region in the first HDR image. Exemplarily, the human figure region and sky region in the first HDR image can be identified using neural networks or similar methods, or they can be identified using edge detection, feature point detection, or similar methods. This application embodiment does not limit the method used to identify the human figure region and sky region.

[0110] For example, the hue weight map and the luminance weight map can be fused using the following formula (3):

[0111] W_combined=αW_L+βW_H formula (3);

[0112] Where W_L represents the luminance weight map, W_H represents the hue weight map, α represents the second weight, β represents the first weight, and W_combined represents the second mask.

[0113] After generating the second mask, the image region indicated by the target region mask (such as the portrait region and / or sky region) can be removed from the second mask to obtain the first mask. Taking a binary target region mask as an example, a value of 1 in the target region mask indicates that the corresponding pixel belongs to an image region that does not require color adjustment, and a value of 0 in the target region mask indicates that the corresponding pixel does not belong to an image region that does not require color adjustment. Therefore, pixels with a value of 1 in the target region mask can have their values ​​set to 0 in the second mask. That is, pixels with a value of 0 in the second mask have a value of 1 in the target region mask, thus obtaining the first mask. Setting the value of this pixel to 0 in the second mask means that it will not be adjusted during subsequent color adjustments. It should be noted that the target region mask can also be other masks, not limited to binary masks.

[0114] In the above implementation, by removing image areas that do not require color adjustment (such as portrait areas and / or sky areas), a first mask representing the areas that require color adjustment is generated. This avoids the situation where color adjustment of the image area leads to unsatisfactory final colors, further improving the accuracy of subsequent color adjustments and enhancing the image quality of the subsequently generated second HDR image.

[0115] In this embodiment, by fusing the hue weight map and the luminance weight map, a first mask combining the luminance space and hue space can be obtained, which accurately represents the highlight areas and warm areas that need color adjustment. The hue and luminance of the first mask have a smooth transition, thereby improving the warm color performance of the highlight areas of the subsequently generated second HDR image and improving the image quality of the second HDR image.

[0116] Step 408: Determine color gain information based on the first image set and the second image set.

[0117] Step 410: Based on the color gain information and the first mask, perform color adjustment processing on at least one first channel image in the first image set.

[0118] The electronic device can fuse color gain information, a first mask, and at least one first channel image from a first image set to perform color adjustment processing on the target image region in the at least one first channel image that matches the first mask.

[0119] In some embodiments, the color gain information corresponding to the target channel, the first mask, and the first channel image corresponding to the target channel can be fused to obtain the color-adjusted first channel image corresponding to the target channel; the target channel can be one or more channels in the target color space. This can enhance the color of highlight areas and warm areas in the target channel, improving the color performance of the subsequently generated second HDR image.

[0120] Optionally, if the color gain information includes hue gain, then the hue gain, the first mask, and the first tone map can be fused together to perform hue correction on the first tone map, particularly on the highlight and warm areas of the first tone map, to obtain a corrected first tone map. The second HDR image generated from this corrected first tone map can reduce color cast in the highlight and warm areas.

[0121] Optionally, the color gain information includes saturation gain. This saturation gain, the first mask, and the first saturation map can be fused to enhance the saturation of the first saturation map, particularly in the highlight and warm areas of the first tone map, resulting in an enhanced first saturation map. The second HDR image generated from this enhanced first saturation map can resolve the undersaturation problem in warm areas and improve the atmosphere of warm and highlight areas.

[0122] Step 412: Generate a second HDR image based on the first set of color-adjusted images.

[0123] The description of step 412 can be found in the description of step 350 in the above embodiment, and will not be repeated here.

[0124] In this embodiment, by fusing color gain information, the first mask, and at least one first channel image from the first image set, the highlight and warm color areas of the first HDR image can be color-adjusted, so that the warm color of the highlight area in the generated second HDR image is closer to the real color observed by the human eye, thereby improving the image quality of the second HDR image.

[0125] In some embodiments, the electronic device can perform hue correction and saturation enhancement on the highlight and warm areas of the first HDR image, thereby making the colors of the highlight and warm areas of the resulting second HDR image closer to the true colors observed by the human eye. For example... Figure 6 As shown, an image processing method is provided, which may include the following steps:

[0126] Step 602: Obtain the first image set corresponding to the first HDR image and the target color space. The first image set includes the first tone map corresponding to the first HDR image and the tone channel, the first luminance map corresponding to the luminance channel, and the first saturation map corresponding to the saturation channel.

[0127] Step 604: Obtain the second image set corresponding to the target original image and the target color space. The second image set includes the second tone map corresponding to the tone channel of the target original image, the second brightness map corresponding to the brightness channel, and the second saturation map corresponding to the saturation channel.

[0128] Step 606: Generate a first mask based on the first tone map, the first brightness map and the first HDR image. The first mask is used to characterize the target image area that needs to be color adjusted.

[0129] The descriptions of steps 602 to 606 can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0130] Step 608: Determine the hue gain based on the first hue map and the second hue map.

[0131] Since the hues of the original target image in the highlight and warm color areas are very close to the true colors of the shooting scene, the second hue map can be used as a reference to determine the hue gain based on the second hue map and the first hue map.

[0132] As one implementation method, such as Figure 7 As shown, the step of determining the tone gain based on the first tone map and the second tone map may include steps 702 to 704.

[0133] Step 702: Smooth the hue values ​​of pixels belonging to the second hue range in the first hue map to obtain a hue mask.

[0134] The second color grading range can belong to the warm color grading range. The second color grading range can differ from the first color grading range; furthermore, the second color grading range can be a subset of the first color grading range. The second color grading range can be the color grading range with a large color shift in the first HDR image. For example, if the first HDR image has a red bias towards magenta, then the second color grading range can be the color grading range corresponding to red.

[0135] Based on the first tone map, the tone values ​​belonging to the second tone range in the first tone map can be extracted, and the tone values ​​belonging to the second tone range can be smoothed to obtain a tone mask. This tone mask can be used to represent the image area that needs tone correction. Furthermore, this tone mask can be used to represent the image area belonging to the second tone range that needs tone correction.

[0136] Furthermore, the hue values ​​of pixels belonging to the second hue range in the first hue map can be non-linearly smoothed according to the exponential function corresponding to the second hue range to obtain a hue mask. Specifically, the method of non-linearly smoothing the hue values ​​of pixels belonging to the second hue range in the first hue map is similar to the method described in the above embodiment of non-linearly smoothing the hue values ​​of pixels belonging to the first hue range in the first hue map according to the exponential function corresponding to the first hue range, and will not be repeated here.

[0137] By smoothing the hue values ​​of pixels belonging to the second hue range in the first hue map, the resulting hue mask can exhibit a more natural color change effect, thereby improving the effect of subsequent hue correction.

[0138] In some embodiments, before generating a tone mask based on the first tone map, the first tone map can be stretched to adjust its tone range, thereby improving the accuracy of the generated tone mask corresponding to the second tone region. As a specific implementation, tone values ​​in the first tone map greater than or equal to a first tone threshold can be multiplied by a first parameter, and tone values ​​in the first tone map less than the first tone threshold can be multiplied by a second parameter to obtain a stretched first tone map. The first parameter can be greater than the second parameter; for example, tone values ​​greater than or equal to 180° in the first tone map can be multiplied by 1.05, and tone values ​​less than 180° in the first tone map can be multiplied by 0.8. By adjusting the tone distribution of the first tone map, the color contrast of the subsequently generated second HDR image can be improved, making warm colors (e.g., red) in the second HDR image more vivid and lifelike.

[0139] Step 704: Based on the second tone map, perform tone shift processing on the tone mask to obtain tone gain.

[0140] Tone shift processing on a tone mask can refer to shifting the tone mask towards the real color. For example, if the second tone range is the tone range corresponding to red, the pixels in the tone mask belonging to the second tone range can be shifted towards red or orange-red. By performing tone shift processing on the tone mask, the problem of red being magenta in the subsequently generated second HDR image can be reduced, making the red in the highlight area appear as a purer red or closer to the orange-red observed by the human eye.

[0141] An electronic device can perform hue shifting processing on a hue mask based on the difference between a first hue map and a second hue map to obtain hue gain. In some embodiments, the hue difference corresponding to each pixel in a second hue range can be calculated based on the hue mask, the first hue map, and the second hue map. Based on the hue difference corresponding to each pixel in the second hue range and the corresponding second hue value, a reference hue value corresponding to each pixel in the second hue range is determined. Then, based on the reference hue value corresponding to each pixel in the second hue range, the hue mask is hue shifted to obtain hue gain.

[0142] Furthermore, the hue difference corresponding to a pixel can be the difference between the second hue value of the pixel in the second hue image and the first hue value in the first hue image. Based on the hue mask, the hue difference corresponding to each pixel in the second hue range can be obtained by subtracting the first hue value in the first hue image from the second hue value in the second hue image. For example, the hue difference of a pixel is diff = hue2 - hue1, where hue2 represents the second hue value of the pixel in the second hue image, and hue1 represents the first hue value of the pixel in the first hue image.

[0143] Optionally, based on the tonal differences corresponding to each pixel in the second tonal range, target pixels with tonal differences greater than a difference threshold can be identified among the pixels belonging to the second tonal range. If the tonal difference of a target pixel is greater than the difference threshold, it indicates that the target pixel has a significant tonal difference between the first HDR image and the original target image, indicating a tonal shift. Therefore, tonal shift processing is required. For example, pixels with a tonal difference (diff) greater than 180° can be identified as target pixels requiring tonal shift processing. It should be noted that the difference threshold can be set according to actual needs, for example, it could be 175°, 182°, 185°, etc., but is not limited to these.

[0144] The reference hue value corresponding to each target pixel can be determined based on the second hue value of each target pixel in the second hue map. This reference hue value is used as a reference value for hue offset processing. Hue offset processing is then performed on each target pixel in the hue mask based on the reference hue value corresponding to each target pixel.

[0145] For example, if the second hue value of the target pixel in the second hue map is greater than or equal to the second hue threshold, the second hue value can be determined as the reference hue value corresponding to the target pixel. If the second hue value of the target pixel in the second hue map is less than the second hue threshold, the sum of the second hue value and 360° can be determined as the reference hue value.

[0146] By using the reference hue value corresponding to each target pixel to perform hue offset processing on each target pixel, a hue gain is obtained. This makes the hue value mapped by the value of the target pixel after offset processing in the hue gain closer to the reference hue value, thereby improving the accuracy and effect of subsequent hue correction.

[0147] In the above implementation, by smoothing the hue values ​​of pixels belonging to the second hue range in the first hue map, the resulting hue mask can present a more natural color change effect. Then, by using the second hue map to perform hue shift processing on the hue mask, accurate hue gain is obtained, which improves the effect of subsequent hue correction and reduces the color shift in the generated second HDR image.

[0148] Step 610: Perform hue correction on the first hue map based on the hue gain and the first mask.

[0149] The tone gain, the first mask, and the first tone map can be fused to perform tone correction on the first tone map, resulting in a corrected first tone map.

[0150] As one implementation, the tone gain, the first mask, and the first tone map can be multiplied to obtain the corrected first tone map.

[0151] As another implementation, the tone gain can be multiplied by the first tone map to obtain a third tone map. The first mask is used as the weight of the third tone map, and the difference between 1 and the first mask is used as the weight of the first tone map. The third tone map and the first tone map are then weighted and fused to obtain the corrected first tone map. Using this method, tone correction of the second tone region in the highlight area can be achieved, while reducing the color impact on other image areas and improving the color performance of the generated second HDR image.

[0152] For example, when performing tone correction on the first tone map, the value of the tone gain of the pixel can be multiplied by the value of the first mask. The multiplied value can be modulo 360 and then mapped to the range of 0 to 1. The mapped value is then multiplied by the tone value of the pixel in the first tone map, thereby avoiding the situation where the tone-corrected value goes out of bounds and improving the accuracy of tone correction.

[0153] Step 612: Determine the saturation gain based on the first saturation map and the second saturation map.

[0154] In one implementation, the second saturation map can be used as a reference for saturation adjustment, and the saturation gain can be determined based on the saturation difference between the first and second saturation maps. Optionally, the saturation difference between the first and second saturation maps can be the difference between the second saturation value of each pixel in the second saturation map and the first saturation value in the first saturation map.

[0155] Step 614: Enhance the saturation of the first saturation map based on the saturation gain and the first mask.

[0156] The saturation gain, the first mask, and the first saturation map can be fused to enhance the saturation of the first saturation map, resulting in an enhanced first saturation map.

[0157] As one implementation, the saturation gain, the first mask, and the first saturation map can be multiplied to obtain the enhanced first saturation map.

[0158] As another implementation, the saturation gain can be multiplied by the first saturation image to obtain a third saturation image. The first mask is used as the weight of the third saturation image, and the difference between 1 and the first mask is used as the weight of the first saturation image. The third saturation image and the first saturation image are then weighted and fused to obtain the enhanced first saturation image. Using this method, the saturation of the first tone region in the highlight area can be enhanced, while reducing the impact on the color of other image areas, thus improving the color performance of the generated second HDR image.

[0159] Step 616: Generate a second HDR image based on the corrected first tone map, the enhanced first saturation map, and the first luminance map.

[0160] The corrected first tone map, the enhanced first saturation map, and the first luminance map can be synthesized to obtain a composite image in the target color space. Then, the composite image can be converted to the RGB color space to obtain a second HDR image in the RGB color space.

[0161] Figure 8 This is a schematic diagram illustrating the process of color adjustment of a first HDR image to obtain a second HDR image in one embodiment. Figure 8As shown, the first HDR image can be converted from the RGB color space to an HSV color image, yielding a first luminance map corresponding to the V channel, a first saturation map corresponding to the S channel, and a first hue map corresponding to the H channel. The linear graph corresponding to the original target image can be converted from the RGB color space to an HSV color image, yielding a second luminance map corresponding to the V channel, a second saturation map corresponding to the S channel, and a second hue map corresponding to the H channel. A luminance weight map can be generated based on the first luminance map and the first HDR image. Hue regions (such as yellow, orange, and red) can be locked based on the first hue map to generate a hue weight map. After fusing the luminance weight map and the hue weight map, the sky region and the portrait region are removed to obtain a first mask, which represents the image region requiring color adjustment. The saturation gain can be determined based on the first saturation map and the second saturation map, and the hue gain can be determined based on the first hue map and the second hue map. The first mask, saturation gain, and first saturation map are then fused together, and the first mask, hue gain, and first hue map are fused together. Gain map smoothing is then performed on each of these components to obtain a composite image in HSV space. Finally, the composite image is converted from HSV color space to RGB color space to obtain the second HDR image.

[0162] In this embodiment, since the hue and saturation of the target original image in the highlight and warm color regions are closer to the true colors of the shooting scene, by converting the first HDR image and the target original image to the target color space and decomposing them, channel maps of the first HDR image and the target original image in the hue channel, saturation channel and luminance channel are obtained. Then, hue gain is generated based on the hue map of the first HDR image and the target original image in the hue channel, and saturation gain is generated based on the saturation map of the first HDR image and the target original image in the saturation channel. The calculated color gain information has better smoothness and robustness, which can make the warm color of the generated second HDR image in the highlight region closer to the true colors observed by the human eye.

[0163] Furthermore, it can reduce the magenta tint of red in the second HDR image, making the red in the highlight areas appear as a purer red or a more orange-red that is closer to what the human eye perceives. It can also avoid the problem of undersaturation of yellow in the second HDR image, enhance the warm highlight atmosphere, solve problems such as loss of warm colors and pale colors, and improve the color performance of the second HDR image.

[0164] like Figure 9 As shown, in one embodiment, an image processing device 900 is provided, which can be applied to the above-mentioned electronic device. The image processing device 900 may include an acquisition module 910, a gain determination module 920, a color adjustment module 930, and an image generation module 940.

[0165] The acquisition module 910 is used to acquire a first image set corresponding to the first HDR image and the target color space. The first image set includes the first channel map corresponding to each channel of the first HDR image and the target color space. The first HDR image is obtained by fusing multiple frames of images with different exposure levels.

[0166] The acquisition module 910 is also used to acquire a second image set corresponding to the target original image and the target color space. The second image set includes the second channel image corresponding to each channel of the target original image and the target color space. The target original image is one of the multiple frames of images with different exposure levels, and the exposure level corresponding to the target original image is less than the exposure level threshold.

[0167] The gain determination module 920 is used to determine color gain information based on the first image set and the second image set.

[0168] The color adjustment module 930 is used to perform color adjustment processing on at least one first channel image in the first image set according to color gain information.

[0169] Image generation module 940 is used to generate a second HDR image based on the first set of color-adjusted images.

[0170] In one embodiment, the target color space includes a hue channel and a luminance channel, and the first image set includes a first hue map corresponding to the hue channel and a first luminance map corresponding to the luminance channel. The image processing apparatus 900 described above also includes a mask generation module.

[0171] The mask generation module is used to generate a first mask based on the first tone map, the first brightness map, and the first HDR image; wherein, the first mask is used to characterize the target image region that needs to be color adjusted, the target image region includes the image region corresponding to the first tone range and / or the image region with a brightness value greater than the brightness threshold, and the first tone range belongs to the warm tone range.

[0172] The color adjustment module 930 is also used to perform color adjustment processing on at least one first channel image in the first image set based on color gain information and the first mask.

[0173] In one embodiment, the mask generation module includes a hue weight generation unit, a luminance weight generation unit, and a mask generation unit.

[0174] The hue weight generation unit is used to generate a hue weight map corresponding to the first hue range based on the first hue map.

[0175] The brightness weight generation unit is used to generate a brightness weight map based on the first brightness map and the first HDR image; the brightness weight map is used to characterize the image area in the first HDR image where the brightness value is greater than the brightness threshold.

[0176] The mask generation unit is used to generate a first mask based on the hue weight map and the luminance weight map.

[0177] In one embodiment, the tone weight generation unit is further configured to smooth the tone values ​​of pixels belonging to the first tone range in the first tone map to obtain a tone weight map.

[0178] In one embodiment, the first image set includes a first luminance map corresponding to a luminance channel, and the second image set includes a second luminance map corresponding to a luminance channel.

[0179] The brightness weight generation unit is also used to perform grayscale conversion on the first HDR image to obtain a grayscale image; to perform enhancement processing on the first brightness map to adjust the brightness distribution of the first brightness map to obtain an enhanced first brightness map; to perform enhancement processing on the second brightness map to adjust the brightness distribution of the second brightness map to obtain an enhanced second brightness map; and to generate a brightness weight map based on the grayscale image, the enhanced first brightness map, and the enhanced second brightness map.

[0180] In one embodiment, the mask generation unit is further configured to fuse the tone weight map and the brightness weight map to obtain a second mask; and to remove the image region corresponding to the target region mask from the second mask to obtain a first mask, wherein the target region mask is used to characterize the image region that does not need to be color adjusted; wherein the target region mask includes a portrait mask and / or a sky mask.

[0181] In one embodiment, the color adjustment module 930 is further configured to fuse the color gain information corresponding to the target channel, the first mask, and the first channel image corresponding to the target channel to obtain the first channel image after color adjustment corresponding to the target channel; the target channel is one or more channels in the target color space.

[0182] In one embodiment, the target color space includes a tone channel, the color gain information includes tone gain, the first image set includes a first tone map corresponding to the tone channel, and the second image set includes a second tone map corresponding to the tone channel. The gain determination module 920 includes a first gain determination unit, and the color adjustment module 930 includes a tone correction unit.

[0183] The first gain determination unit is used to determine the hue gain based on the first hue map and the second hue map.

[0184] The tone correction unit is used to perform tone correction on the first tone map based on the tone gain.

[0185] In one embodiment, the first gain determination unit is further configured to smooth the hue values ​​of pixels belonging to the second hue range in the first hue map to obtain a hue mask; the second hue range belongs to warm hues; and to perform hue offset processing on the hue mask according to the second hue map to obtain a hue gain.

[0186] In one embodiment, the first gain determination unit is further configured to calculate the hue difference corresponding to each pixel in the second hue interval based on the hue mask, the first hue map, and the second hue map; the hue difference is the difference between the second hue value of the pixel in the second hue map and the first hue value in the first hue map; determine the reference hue value corresponding to each pixel in the second hue interval based on the hue difference and the corresponding second hue value; and perform hue offset processing on the hue mask based on the reference hue value corresponding to each pixel in the second hue interval to obtain the hue gain.

[0187] In one embodiment, the target color space includes a saturation channel, the gain information includes saturation gain, the first image set includes a first saturation map corresponding to the saturation channel, and the second image set includes a second saturation map corresponding to the saturation channel. The gain determination module 920 includes a second gain determination unit, and the color adjustment module 930 includes a saturation enhancement unit.

[0188] The second gain determination unit is used to determine the saturation gain based on the first saturation map and the second saturation map.

[0189] A saturation enhancement unit is used to enhance the saturation of the first saturation map according to the saturation gain.

[0190] In one embodiment, the target original image is the original image with the lowest exposure among the multiple frames of images with different exposure levels.

[0191] The acquisition module 910 is also used to perform a linear transformation on the original target image to obtain a linear graph; and to convert the linear graph to the target color space to obtain a second image set.

[0192] In one embodiment, the acquisition module 910 is further configured to convert the first HDR image from the RGB color space to the target color space to obtain the first image set.

[0193] The image generation module 940 is also used to convert from the target color space to the RGB color space based on the first set of color-adjusted images to generate a second HDR image.

[0194] The target color space includes any one of the HSV color space, HSL color space, and HSI color space.

[0195] In this embodiment, since the color of the target original image with an exposure level less than the exposure threshold is closer to the true color of the shooting scene, the target original image used to fuse the first HDR image can be used as a reference for color adjustment. Color gain information is determined based on the channel map of the first HDR image and the target original image in the target color space, and color adjustment processing is performed on the channel map of the first HDR image in the target color space based on the color gain information, thereby obtaining a color-enhanced second HDR image. This can increase the color performance of the HDR image, make the colors presented by the HDR image closer to the true colors, and improve the image quality of the HDR image.

[0196] Figure 10 This is a structural block diagram of an electronic device in another embodiment. (See diagram below.) Figure 10 As shown, the electronic device 1000 may include one or more of the following components: a processor 1010 and a memory 1020 coupled to the processor 1010, wherein the memory 1020 may store one or more computer programs, which may be configured to implement the methods described in the above embodiments when executed by one or more processors 1010.

[0197] The processor 1010 may include one or more processing cores. The processor 1010 connects to various parts within the electronic device 1000 using various interfaces and lines, and performs various functions and processes data of the electronic device 1000 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1020, and by calling data stored in the memory 1020. Optionally, the processor 1010 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 1010 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 1010 and may be implemented separately using a communication chip.

[0198] The memory 1020 may include random access memory (RAM) or read-only memory (ROM). The memory 1020 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1020 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described above. The data storage area may also store data created by the electronic device 1000 during use.

[0199] Understandably, the electronic device 1000 may include more or fewer structural elements than those shown in the above block diagram, such as radio frequency modules, power modules, physical buttons, WiFi (Wireless Fidelity) modules, speakers, Bluetooth modules, sensors, etc., and may not be limited herein.

[0200] This application discloses a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the methods described in the above embodiments.

[0201] This application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program can be executed by a processor to implement the methods described in the above embodiments.

[0202] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, ROM, etc.

[0203] Any references to memory, storage, databases, or other media used herein may include non-volatile and / or volatile memory. Suitable non-volatile memory may include ROM, Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as an external cache. By way of illustration and not limitation, RAM may take many forms, such as Static RAM (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus DRAM (RDRAM), and Direct Rambus DRAM (DRDRAM).

[0204] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0205] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0206] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0207] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0208] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0209] The foregoing has provided a detailed description of an image processing method, apparatus, electronic device, and computer program product disclosed in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image processing method, characterized in that, The method includes: A first image set corresponding to a first high dynamic range (HDR) image and a target color space is obtained. The first image set includes first channel maps corresponding to each channel of the first HDR image and the target color space. The first HDR image is obtained by fusing multiple frames of original images with different exposure levels. Obtain a second image set corresponding to the target original image and the target color space. The second image set includes the second channel image corresponding to each channel of the target original image and the target color space. The target original image is one of the original images with different exposure levels among the multiple frames. The exposure level corresponding to the target original image is less than the exposure threshold. Based on the first image set and the second image set, determine the color gain information; The color adjustment process is performed on at least one first channel image in the first image set according to the color gain information; A second HDR image is generated based on the first set of color-adjusted images.

2. The method according to claim 1, characterized in that, The target color space includes a hue channel and a luminance channel, and the first image set includes a first hue map corresponding to the hue channel and a first luminance map corresponding to the luminance channel; Before performing color adjustment processing on at least one first channel image in the first image set based on the color gain information, the method further includes: A first mask is generated based on the first tone map, the first brightness map, and the first HDR image; wherein, the first mask is used to characterize the target image region that needs to be color adjusted, the target image region includes the image region corresponding to the first tone interval and / or the image region with a brightness value greater than the brightness threshold, and the first tone interval belongs to the warm tone. The step of performing color adjustment processing on at least one first channel image in the first image set based on the color gain information includes: Based on the color gain information and the first mask, at least one first channel image in the first image set is subjected to color adjustment processing.

3. The method according to claim 2, characterized in that, The step of generating a first mask based on the first tone map, the first brightness map, and the first HDR image includes: Based on the first color tone map, generate a color weight map corresponding to the first color tone range; A brightness weight map is generated based on the first brightness map and the first HDR image; the brightness weight map is used to characterize the image region in the first HDR image where the brightness value is greater than the brightness threshold. A first mask is generated based on the hue weight map and the luminance weight map.

4. The method according to claim 3, characterized in that, The step of generating a hue weight map corresponding to the first hue interval based on the first hue map includes: The hue values ​​of pixels belonging to the first hue range in the first hue map are smoothed to obtain a hue weight map.

5. The method according to claim 3, characterized in that, The first image set includes a first luminance map corresponding to the luminance channel, and the second image set includes a second luminance map corresponding to the luminance channel; The step of generating a luminance weight map based on the first luminance map and the first HDR image includes: The first HDR image is converted to grayscale to obtain a grayscale image; The first luminance map is enhanced to adjust the luminance distribution of the first luminance map, resulting in an enhanced first luminance map. The second luminance map is enhanced to adjust the luminance distribution of the second luminance map, resulting in an enhanced second luminance map; A luminance weight map is generated based on the grayscale image, the enhanced first luminance map, and the enhanced second luminance map.

6. The method according to claim 3, characterized in that, The step of generating a first mask based on the hue weight map and the luminance weight map includes: The hue weight map and the luminance weight map are fused to obtain the second mask; Based on the target region mask, the image region corresponding to the target region mask in the second mask is removed to obtain the first mask, wherein the target region mask is used to characterize the image region that does not need to be color adjusted. The target area mask includes a human face mask and / or a sky mask.

7. The method according to claim 2, characterized in that, The step of performing color adjustment processing on at least one first channel image in the first image set based on the color gain information and the first mask includes: The color gain information corresponding to the target channel, the first mask, and the first channel image corresponding to the target channel are fused together to obtain the first channel image with color adjustment corresponding to the target channel; the target channel is one or more channels of the target color space.

8. The method according to any one of claims 1 to 7, characterized in that, The target color space includes a tone channel, the color gain information includes tone gain, the first image set includes a first tone map corresponding to the tone channel, and the second image set includes a second tone map corresponding to the tone channel. The step of determining color gain information based on the first image set and the second image set includes: Determine the hue gain based on the first hue map and the second hue map; The step of performing color adjustment processing on at least one first channel image in the first image set based on the color gain information includes: The first tone map is hue-corrected based on the hue gain.

9. The method according to claim 8, characterized in that, The step of determining the hue gain based on the first hue map and the second hue map includes: The hue values ​​of pixels belonging to the second hue range in the first hue map are smoothed to obtain a hue mask; the second hue range belongs to the warm hue range. Based on the second tone map, the tone mask is subjected to tone shift processing to obtain tone gain.

10. The method according to claim 9, characterized in that, The step of performing hue shifting processing on the hue mask based on the second hue map to obtain hue gain includes: Based on the tone mask, the first tone map, and the second tone map, the tone difference corresponding to each pixel in the second tone range is calculated; the tone difference is the difference between the second tone value of the pixel in the second tone map and the first tone value in the first tone map. Based on the hue difference corresponding to each pixel in the second hue range and the corresponding second hue value, determine the reference hue value corresponding to each pixel in the second hue range. Based on the reference hue value corresponding to each pixel in the second hue range, the hue mask is subjected to hue offset processing to obtain hue gain.

11. The method according to any one of claims 1 to 7, characterized in that, The target color space includes a saturation channel, the gain information includes saturation gain, the first image set includes a first saturation map corresponding to the saturation channel, and the second image set includes a second saturation map corresponding to the saturation channel. The step of determining color gain information based on the first image set and the second image set includes: Based on the first saturation map and the second saturation map, determine the saturation gain; The step of performing color adjustment processing on at least one first channel image in the first image set based on the color gain information includes: The first saturation map is enhanced with saturation based on the saturation gain.

12. The method according to claim 1, characterized in that, The target original image is the original image with the smallest exposure among the multiple frames of images with different exposure levels; obtaining the second image set corresponding to the target original image and the target color space includes: A linear transformation is performed on the original target image to obtain a linear graph; The linear graph is converted to the target color space to obtain a second set of images.

13. The method according to any one of claims 1 to 7, 12, characterized in that, The step of obtaining the first image set corresponding to the first high dynamic range (HDR) image and the target color space includes: The first HDR image is converted from the RGB color space to the target color space to obtain the first image set; The step of generating a second HDR image based on the first set of color-adjusted images includes: Based on the first set of color-adjusted images, the image is converted from the target color space to the RGB color space to generate a second HDR image; The target color space includes any one of the HSV color space, HSL color space, and HSI color space.

14. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire a first image set corresponding to a first high dynamic range (HDR) image and a target color space. The first image set includes a first channel map corresponding to each channel of the first HDR image and the target color space. The first HDR image is obtained by fusing multiple frames of images with different exposure levels. The acquisition module is further configured to acquire a second image set corresponding to the target original image and the target color space, the second image set including the second channel image corresponding to each channel of the target original image and the target color space; the target original image is one of the multiple frames of images with different exposure levels, and the exposure level corresponding to the target original image is less than the exposure level threshold; The gain determination module is used to determine color gain information based on the first image set and the second image set; A color adjustment module is used to perform color adjustment processing on at least one first channel image in the first image set according to the color gain information. The image generation module is used to generate a second HDR image based on the first set of color-adjusted images.

15. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method as described in any one of claims 1 to 13.

16. A computer program product, characterized in that, The method includes a computer program, and when the computer program is executed by a processor, the processor causes the processor to perform the method as described in any one of claims 1 to 13.

Images