Image processing method, device, electronic device and computer readable storage medium

Abstract

The present application relates to an image processing method, device, electronic device, storage medium and computer program product. The method comprises: acquiring a first image and a second image; the exposure of the first image is less than the exposure of the second image; performing a demosaicing process on the first image to obtain a third image, and performing a demosaicing process on the second image to obtain a fourth image; performing a high dynamic range imaging fusion on the third image and the fourth image to obtain a high dynamic range image; performing image signal processing on the high dynamic range image to obtain a fifth image. The use of this method can improve the clarity of the image.

Description

Technical Field

[0001] The present application relates to imaging technology, and in particular to an image processing method, device, electronic device and computer-readable storage medium. Background Art

[0002] With the development of imaging technology, in order to obtain clearer images, electronic devices take multiple images and fuse them to obtain a new image.

[0003] However, in the process of image processing, the image obtained by fusing multiple images often has the problem of low clarity. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, device, electronic device, computer-readable storage medium, and computer program product, which can improve the clarity of an image.

[0005] In a first aspect, the present application provides an image processing method. The method comprises:

[0006] Acquire a first image and a second image; the exposure amount of the first image is less than the exposure amount of the second image;

[0007] Performing a demosaicing process on the first image to obtain a third image, and performing a demosaicing process on the second image to obtain a fourth image;

[0008] Performing high dynamic range imaging fusion on the third image and the fourth image to obtain a high dynamic range image;

[0009] The high dynamic range image is subjected to image signal processing to obtain a fifth image.

[0010] In a second aspect, the present application also provides an image processing device. The device comprises:

[0011] An acquisition module, used for acquiring a first image and a second image; the exposure amount of the first image is less than the exposure amount of the second image;

[0012] a demosaic processing module, configured to perform a demosaic processing on the first image to obtain a third image, and perform a demosaic processing on the second image to obtain a fourth image;

[0013] A fusion module, used for performing high dynamic range imaging fusion on the third image and the fourth image to obtain a high dynamic range image;

[0014] The image signal processing module is used to perform image signal processing on the high dynamic range image to obtain a fifth image.

[0015] In a third aspect, the present application further provides an electronic device. The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0016] Acquire a first image and a second image; the exposure amount of the first image is less than the exposure amount of the second image;

[0017] Performing a demosaicing process on the first image to obtain a third image, and performing a demosaicing process on the second image to obtain a fourth image;

[0018] Performing high dynamic range imaging fusion on the third image and the fourth image to obtain a high dynamic range image;

[0019] The high dynamic range image is subjected to image signal processing to obtain a fifth image.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0021] Acquire a first image and a second image; the exposure amount of the first image is less than the exposure amount of the second image;

[0022] Performing a demosaicing process on the first image to obtain a third image, and performing a demosaicing process on the second image to obtain a fourth image;

[0023] Performing high dynamic range imaging fusion on the third image and the fourth image to obtain a high dynamic range image;

[0024] The high dynamic range image is subjected to image signal processing to obtain a fifth image.

[0025] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0026] Acquire a first image and a second image; the exposure amount of the first image is less than the exposure amount of the second image;

[0027] Performing a demosaicing process on the first image to obtain a third image, and performing a demosaicing process on the second image to obtain a fourth image;

[0028] Performing high dynamic range imaging fusion on the third image and the fourth image to obtain a high dynamic range image;

[0029] The high dynamic range image is subjected to image signal processing to obtain a fifth image.

[0030] The above-mentioned image processing method, device, electronic device, computer-readable storage medium and computer program product obtain a first image and a second image, and the exposure of the first image is less than the exposure of the second image; the first image and the second image are demosaiced respectively to obtain a third image and a fourth image after the mosaic noise is removed, and the third image and the fourth image are fused by high dynamic range imaging to obtain a high dynamic range image, and then the high dynamic range image is processed by image signal processing. Image signal processing is not performed before demosaicing, which can avoid the problem of low demosaicing accuracy caused by demosaicing after the image is processed by image signal processing, and obtain a fifth image that is clearer and more accurately denoised. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 is a flowchart of an image processing method in one embodiment;

[0033] Figure 2 is a flowchart of image signal processing in one embodiment;

[0034] Figure 3 A flowchart of obtaining a high dynamic range image in one embodiment;

[0035] Figure 4 A flowchart of obtaining an eighth image in a YUV mode in one embodiment;

[0036] Figure 5 is a flowchart of an image processing method in another embodiment;

[0037] Figure 6 is a structural block diagram of an image processing device in one embodiment;

[0038] Figure 7 FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0040] In one embodiment, Figure 1 As shown, an image processing method is provided. This embodiment takes the method applied to an electronic device as an example. The electronic device can be a terminal or a server. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server.

[0041] The terminal may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server may be implemented as an independent server or a server cluster consisting of multiple servers.

[0042] In this embodiment, the image processing method includes steps 102 to 108:

[0043] Step 102, acquiring a first image and a second image; the exposure amount of the first image is less than the exposure amount of the second image.

[0044] Exposure refers to the integral of the light intensity EV received by a certain surface element of the object within the time t. Exposure = illumination × exposure time. Illuminance is determined by the aperture, while exposure time is controlled by the shutter. The aperture size and shutter length determine the amount of exposure. Therefore, the exposure is controlled by the aperture and shutter together.

[0045] The exposure of the first image is less than that of the second image, that is, the first image is a dark frame or an underexposed image relative to the second image. Optionally, the first image is an underexposed image EV-, and the second image is a normal exposure image EV0.

[0046] Step 104: De-mosaic the first image to obtain a third image, and de-mosaic the second image to obtain a fourth image.

[0047] De-mosaicing, also known as color interpolation, restores the real-world colors that match the color display device from the Bayer data obtained from the image sensor, that is, converts the image from RAW mode to RGB mode, in which the color of the image can be adjusted and processed more accurately.

[0048] Optionally, the electronic device uses a demosaicing algorithm to demosaic the first image to obtain a third image, and uses a demosaicing algorithm to demosaic the second image to obtain a fourth image. The demosaicing algorithm includes a traditional mosaic algorithm or an artificial intelligence demosaicing algorithm (AI Demosaic), and the artificial intelligence demosaicing algorithm can more accurately demosaic the image.

[0049] Step 106: Perform high dynamic range imaging fusion on the third image and the fourth image to obtain a high dynamic range image.

[0050] Optionally, the electronic device uses a high dynamic range imaging algorithm (High Dynamic Range Imaging, HDR) to perform high dynamic range imaging fusion on the third image and the fourth image to obtain a high dynamic range image.

[0051] Step 108: Perform image signal processing on the high dynamic range image to obtain a fifth image.

[0052] Optionally, the electronic device inputs the high dynamic range image into an image signal processing processor, and performs image signal processing (ISP, Image Signal Processing) on ​​the high dynamic range image through the image signal processor to obtain a fifth image.

[0053] The image signal processing includes black level reduction processing, lens shading correction compensation (Lens ShadingCorrection, LSC), white balance processing (WB gain, White Balance gain) or color correction matrix processing (Color Correction Matrix, CCM). In other embodiments, the image signal processing also includes other operations, such as deblurring, sharpening, etc., but is not limited thereto.

[0054] Alternatively, if Figure 2 As shown, the electronic device sequentially performs black level reduction processing, lens shading correction compensation, white balance processing, and color correction matrix processing on the 16-bit RGB high dynamic range image to obtain a 16-bit fifth image.

[0055] Optionally, the high dynamic range image is an image in RGB mode, and the following formulas are used to perform black level reduction processing on the R channel, G channel, and B channel of the high dynamic range in the RGB mode respectively:

[0056] Rout[i]=Rin[i]-Blacklevel

[0057] Gout[i]=Gin[i]-Blacklevel

[0058] Bout[i]=Bin[i]-Blacklevel

[0059] Among them, Rout[i] is the pixel value of the i-th R channel of the output, Rin[i] is the pixel value of the i-th R channel of the input, and Blacklevel is the black level value; Gout[i] is the pixel value of the i-th G channel of the output, Gin[i] is the pixel value of the i-th G channel of the input; Bout[i] is the pixel value of the i-th B channel of the output, and Bin[i] is the pixel value of the i-th B channel of the input.

[0060] Furthermore, the high dynamic range image is an image in RGB mode, and the following formulas are used to perform black level reduction processing on the R channel, G channel and B channel of the high dynamic range of the RGB mode respectively:

[0061]

[0062]

[0063]

[0064] Among them, Whitelevel is the maximum value of the image pixels and is a fixed value. For example, the maximum value of a 10-bit image is 1023.

[0065] Optionally, the lens shading correction compensation includes: firstly interpolating the R table, the G table, and the B table of the pre-obtained LSC table to a size the same as the input RGB image by interpolation, and then applying to the input image, and the specific operation is as follows:

[0066]

[0067]

[0068]

[0069] Where Rout[i] is the pixel value of the i-th R channel of the output, Rin[i] is the pixel value of the i-th R channel of the input, and LSC_Rtable[i] is the i-th value in the R table of the LSC table.

[0070] LSC-Gtable[i] is the i-th value in the G table of the LSC table, and LSC-Btable[i] is the i-th value in the B table of the LSC table.

[0071] Optionally, white balance processing is performed on the input RGB image. The specific operations are as follows:

[0072]

[0073]

[0074] Among them, Rout[i] is the pixel value of the i-th R channel output, Rin[i] is the pixel value of the i-th R channel input, R_gain is the white balance gain value of the R channel, and B_gain is the white balance gain value of the B channel.

[0075] It should be noted that, as the human eye is more sensitive to green (G channel), the white balance processing is performed based on green, and the white balance gain value of the G channel G_gain=1.0, which is equivalent to no processing.

[0076] Optionally, the electronic device obtains a color correction matrix, multiplies the color correction matrix by the input RGB image, and outputs the RGB image.

[0077] Optionally, the above-mentioned image signal processing is all performed in the second color depth domain, that is, the input images are all 16-bit RGB images, and the output images are also all 16-bit RGB images.

[0078] The above-mentioned image processing method obtains a first image and a second image, and the exposure of the first image is less than the exposure of the second image; the first image and the second image are demosaiced respectively to obtain a third image and a fourth image after the mosaic noise is removed, the third image and the fourth image are fused by high dynamic range imaging to obtain a high dynamic range image, and then the high dynamic range image is processed by image signal processing. Image signal processing is not performed before demosaicing, which can avoid the problem of low accuracy of demosaicing processing caused by demosaicing processing after image signal processing, and obtain a fifth image that is clearer and more accurately denoised.

[0079] In one embodiment, obtaining a first image and a second image includes: obtaining a first RAW image and multiple frames of second RAW images; the exposure of the first RAW image is less than the exposure of the second RAW image; performing registration processing on the first RAW image and a target RAW image in the multiple frames of second RAW images to obtain the first image, and performing registration processing on the multiple frames of second RAW images to obtain multiple frames of registered second RAW images; performing spatial domain fusion on the multiple frames of registered second RAW images to obtain the second image.

[0080] RAW images are original image data that has not been processed.

[0081] Optionally, the electronic device uses a multi-frame registration algorithm to register the first RAW image and the target RAW image in the multiple frames of second RAW images to obtain a first image, and uses a multi-frame registration algorithm to register the multiple frames of second RAW images to obtain a second RAW image after multi-frame registration.

[0082] The processing of the multi-frame registration algorithm includes: extracting features from at least two images to obtain feature points; searching for matching feature point pairs based on the feature points; obtaining image space coordinate transformation parameters based on the matching feature point pairs; and performing image registration based on the coordinate transformation parameters.

[0083] In an optional embodiment, the electronic device uses a clarity detection algorithm to take the second RAW image with the highest clarity among multiple frames of second RAW images as the target RAW image, and aligns the first RAW image with the target RAW image among the multiple frames of second RAW images to obtain the first image.

[0084] In another optional implementation, the electronic device uses the second RAW image exposed first among the multiple frames of second RAW images as the target RAW image, and performs registration processing on the first RAW image and the target RAW image among the multiple frames of second RAW images to obtain the first image.

[0085] In other implementations, the electronic device may also determine the target RAW image in other ways, and perform registration processing with the first RAW image to obtain the first image.

[0086] Optionally, the electronic device averages the second RAW images after the multi-frame registration processing in the spatial domain to fuse them to obtain the second image. In the spatial domain fusion process, ghost images in the image can be removed, noise can be reduced, and clarity can be improved to obtain a clearer second image.

[0087] In another optional implementation, the electronic device converts the second RAW image after the multi-frame registration process into the frequency domain, performs frequency domain fusion, and then converts the image obtained by the frequency domain fusion into the spatial domain to obtain the second image. The conversion from the spatial domain to the frequency domain is Fourier transform, and the conversion from the frequency domain to the spatial domain is inverse Fourier transform.

[0088] In this embodiment, the electronic device obtains a first RAW image and multiple frames of second RAW images, and performs registration processing on the first RAW image and the multiple frames of second RAW images respectively, so as to obtain the registered first image and the second RAW images after the multiple frames of registration processing, and the images after the registration processing can be used for subsequent more accurate fusion processing, so as to obtain a more accurate and clearer second image. Performing spatial domain fusion of the second RAW images after the multiple frames of registration processing in the RAW domain can ensure the noise originality of the image data of the fused second image, and thus ensure the noise clarity of the fused second image.

[0089] In one embodiment, Figure 3As shown, different exposure parameters are set in the image sensor to obtain a frame of the first RAW image and multiple frames of the second RAW image, and the exposure of the first RAW image is less than that of the second RAW image; the multiple frames of the second RAW image are registered by a multi-frame registration algorithm to obtain the second RAW image after the multi-frame registration, and the second RAW image after the multi-frame registration is spatially fused by a multi-frame fusion algorithm to obtain the second image; the first RAW image and the target RAW image in the multiple frames of the second RAW image are registered by a multi-frame registration algorithm to obtain the first image; the first image is demosaiced to obtain a third image with a color depth of 10 bits, and the second image is demosaiced to obtain a fourth image with a color depth of 10 bits; the third image with a color depth of 10 bits and the fourth image with a color depth of 10 bits are fused by high dynamic range imaging to obtain a high dynamic range image with a color depth of 16 bits.

[0090] In one embodiment, after performing image signal processing on the high dynamic range image to obtain the fifth image, the method further includes: performing tone mapping on the fifth image to obtain the sixth image.

[0091] Tone mapping is a computer graphics technique that approximates the display of high dynamic range images on limited dynamic range media.

[0092] Optionally, the tone mapping includes global tone mapping (Global Tonemapping) or local tone mapping (Local Tonemapping).

[0093] In an optional implementation, the electronic device performs global tone mapping on the fifth image to obtain a tone mapping image; and performs local tone mapping on the tone mapping image to obtain a sixth image. The electronic device first performs global tone mapping on the entire fifth image, and then performs local tone mapping on local pixels of the tone mapping image, which can not only ensure the accuracy of tone mapping of the entire fifth image, but also perform more accurate processing on some pixels in the image to obtain a more accurate sixth image.

[0094] In another optional implementation, the electronic device performs global tone mapping or local tone mapping on the fifth image to obtain a sixth image.

[0095] In another optional implementation, the electronic device performs local tone mapping on the fifth image, and then performs global tone mapping to obtain a sixth image.

[0096] In other implementations, the electronic device may also perform tone mapping in other ways to obtain the sixth image, which is not limited here.

[0097] In this embodiment, the electronic device performs tone mapping on the fifth image after performing image signal processing on the high dynamic range image, which can better debug and output the effect of the tone mapping, avoids the problem that the tone mapping effect is affected by the lens shading correction in the subsequent image signal processing due to the tone mapping being performed before the image signal processing, and can improve the effect of the tone mapping to obtain a more accurate sixth image.

[0098] In one embodiment, the fifth image is in RGB mode; tone mapping is performed on the fifth image to obtain a sixth image, including: converting the fifth image into a grayscale mode, and tone mapping the fifth image in the grayscale mode to obtain a sixth image in the grayscale mode; the above method also includes: dividing the sixth image in the grayscale mode by the fifth image in the grayscale mode to obtain a tone mapping mask image; based on the tone mapping mask image and the fifth image in the RGB mode, a seventh image in the RGB mode is obtained.

[0099] RGB mode is the so-called optical primary colors, R stands for Red, G stands for Green, and B stands for Blue. Grayscale mode uses a single tone to represent an image. The color of a pixel is represented by eight bits, which can represent a total of 256 levels (levels) of gray (including black and white), that is, 256 shades of gray.

[0100] Optionally, the electronic device divides each pixel in the sixth image in grayscale mode by the pixel at the corresponding position in the fifth image in grayscale mode to obtain a tone mapping mask image. The tone mapping mask image is in RGB mode. The tone mapping mask image is a floating point image.

[0101] In an optional implementation, the electronic device multiplies the tone mapping mask image and the fifth image in RGB mode to obtain a seventh image in RGB mode.

[0102] In another optional implementation, the electronic device performs dot multiplication processing on the tone mapping mask image and the R channel, the G channel, and the B channel in the fifth image in RGB mode to obtain a seventh image in RGB mode. In this implementation, the electronic device can process each channel in the fifth image separately to obtain a more accurate seventh image in RGB mode.

[0103] In other implementations, the electronic device may also obtain the seventh image in other ways, which are not limited here.

[0104] In this embodiment, the electronic device converts the fifth image into a grayscale mode, and performs tone mapping on the fifth image in the grayscale mode to obtain a sixth image in the grayscale mode. The sixth image in the grayscale mode is divided by the fifth image in the grayscale mode to accurately obtain a tone mapping mask image, and then based on the tone mapping mask image and the fifth image in the RGB mode, the seventh image in the RGB mode can be obtained.

[0105] In one embodiment, the color depth of the seventh image is a first color depth; the method further includes: compressing the color depth of the seventh image to a second color depth to obtain an eighth image; the second color depth is less than the first color depth.

[0106] Color depth is the number of colors that can be displayed per pixel. The greater the color depth, the more colors are available and the more accurate the color representation of the image.

[0107] Exemplarily, the first color depth is 16 bits, and the second color depth is 10 bits.

[0108] It is understandable that the fifth image in RGB mode, the fifth image in grayscale mode, the sixth image in grayscale mode and the seventh image in RGB mode are all in the first color depth. The first image, the second image and the eighth image are all in the second color depth.

[0109] It can be understood that the intermediate process of image processing is all performed in the first color depth domain, which can ensure the accuracy of the data.

[0110] In this embodiment, the electronic device compresses the color depth of the seventh image to avoid other displays not being able to support the first color depth, and can be compatible with displays of various terminals to display the compressed eighth image.

[0111] Optionally, the eighth image is in RGB mode; after compressing the color depth of the seventh image to the second color depth to obtain the eighth image, the method further includes: converting the eighth image in RGB mode into an eighth image in YUV mode.

[0112] The eighth image in the RGB mode and the eighth image in the YUV mode are of the second color depth.

[0113] In this embodiment, the electronic device converts the eighth image in the RGB mode into the YUV mode, and can obtain the eighth image in the YUV mode that is more suitable for the display process.

[0114] In one embodiment, Figure 4As shown, taking the first color depth of 16 bits and the second color depth of 10 bits as an example, the electronic device processes the high dynamic range image through image signal processing to obtain a fifth image in 16-bit RGB mode, converts the fifth image in 16-bit RGB mode into a grayscale mode to obtain a fifth image in 16-bit grayscale mode, and performs global tone mapping and local tone mapping on the fifth image in 16-bit grayscale mode to obtain a sixth image in 16-bit grayscale mode; divides the sixth image in 16-bit grayscale mode by the fifth image in 16-bit grayscale mode to obtain a 16-bit tone mapping mask image; performs dot multiplication processing on the 16-bit tone mapping mask image and the fifth image in 16-bit RGB mode to obtain a seventh image in 16-bit RGB mode; compresses the color depth of the seventh image in 16-bit RGB mode to 10 bits to obtain an eighth image in 10-bit RGB mode; converts the eighth image in 10-bit RGB mode into a YUV mode to obtain an eighth image in 10-bit YUV mode.

[0115] In one embodiment, Figure 5 As shown, different exposure parameters are set in the image sensor to obtain a frame of the first RAW image and multiple frames of the second RAW image, and the exposure of the first RAW image is less than that of the second RAW image; the multiple frames of the second RAW image are spatially fused by a multi-frame fusion algorithm to obtain the second image, and then the second image is demosaiced to obtain the fourth image in the RGB mode; the first RAW image is demosaiced to obtain the third image in the RGB mode; the third image in the RGB mode and the fourth image in the RGB mode are fused by high dynamic range imaging to obtain the high dynamic range image in the RGB mode; the high dynamic range image in the RGB mode is subjected to image signal processing to obtain the fifth image in the RGB mode; the fifth image in the RGB mode is subjected to tone mapping to obtain the sixth image in the RGB mode; the sixth image in the RGB mode is converted into the YUV mode to obtain the sixth image in the YUV mode.

[0116] The electronic device may also compress the color depth of the sixth image in the RGB mode.

[0117] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0118] Based on the same inventive concept, the embodiment of the present application also provides an image processing device for implementing the above-mentioned image processing method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image processing device embodiments provided below can refer to the limitations on the image processing method above, and will not be repeated here.

[0119] In one embodiment, Figure 6 As shown, an image processing device is provided, including: an acquisition module 602, a demosaic processing module 604, a fusion module 606 and an image signal processing module 608, wherein:

[0120] The acquisition module 602 is used to acquire a first image and a second image; the exposure amount of the first image is less than the exposure amount of the second image.

[0121] The demosaicing processing module 604 is used to perform a demosaicing process on the first image to obtain a third image, and to perform a demosaicing process on the second image to obtain a fourth image.

[0122] The fusion module 606 is used to perform high dynamic range imaging fusion on the third image and the fourth image to obtain a high dynamic range image.

[0123] The image signal processing module 608 is used to perform image signal processing on the high dynamic range image to obtain a fifth image.

[0124] The above-mentioned image processing method obtains a first image and a second image, and the exposure of the first image is less than the exposure of the second image; the first image and the second image are demosaiced respectively to obtain a third image and a fourth image after the mosaic noise is removed, and the third image and the fourth image are fused by high dynamic range imaging to obtain a high dynamic range image, and then the high dynamic range image is processed by image signal processing to avoid the problem of low accuracy of demosaicing processing caused by demosaicing processing after image signal processing, so as to obtain a fifth image that is clearer and has more accurate denoising.

[0125] In one embodiment, the acquisition module 602 is further used to acquire a first RAW image and multiple frames of second RAW images; the exposure of the first RAW image is less than the exposure of the second RAW image; the first RAW image and the target RAW image in the multiple frames of second RAW images are registered to obtain a first image, and the multiple frames of second RAW images are registered to obtain multiple frames of second RAW images after registration; the multiple frames of second RAW images after registration are spatially fused to obtain a second image.

[0126] In one embodiment, the acquisition module 602 is further configured to use the second RAW image with the highest definition among the multiple frames of second RAW images as the target RAW image; or use the second RAW image that is first exposed among the multiple frames of second RAW images as the target RAW image.

[0127] In one embodiment, the above-mentioned device further includes a tone mapping module; the tone mapping module is used to perform tone mapping on the fifth image to obtain a sixth image.

[0128] In one embodiment, the tone mapping module is further used to perform global tone mapping on the fifth image to obtain a tone mapping image; and perform local tone mapping on the tone mapping image to obtain a sixth image.

[0129] In one embodiment, the fifth image is in RGB mode; the above-mentioned device also includes a mode conversion module and a calculation module; the mode conversion module is used to convert the fifth image into a grayscale mode, and the above-mentioned tone mapping module is also used to perform tone mapping on the fifth image in the grayscale mode to obtain a sixth image in the grayscale mode; the above-mentioned calculation module is used to divide the sixth image in the grayscale mode and the fifth image in the grayscale mode to obtain a tone mapping mask image; based on the tone mapping mask image and the fifth image in the RGB mode, a seventh image in the RGB mode is obtained.

[0130] In one embodiment, the calculation module is further used to perform dot multiplication processing on the tone mapping mask image and the R channel, the G channel and the B channel in the fifth image in the RGB mode to obtain the seventh image in the RGB mode.

[0131] In one embodiment, the color depth of the seventh image is a first color depth; the above-mentioned device also includes a compression module; the compression module is used to compress the color depth of the seventh image to a second color depth to obtain an eighth image; the second color depth is less than the first color depth.

[0132] In one embodiment, the eighth image is in RGB mode; the above-mentioned mode conversion module is also used to convert the eighth image in RGB mode into the eighth image in YUV mode.

[0133] Each module in the above-mentioned image processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in an electronic device in the form of hardware, or can be stored in a memory in an electronic device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0134] In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and an external device. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an image processing method is implemented. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the electronic device casing, or an external keyboard, touchpad or mouse.

[0135] Those skilled in the art will understand that Figure 7 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0136] The embodiment of the present application further provides a computer-readable storage medium, one or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, enable the processors to perform the steps of the image processing method.

[0137] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the image processing method.

[0138] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0139] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0140] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0141] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, characterized in that: include: Acquire a first RAW image and multiple frames of second RAW images; The exposure amount of the first RAW image is less than the exposure amount of the second RAW image; Performing registration processing on the first RAW image and a target RAW image in the multiple frames of second RAW images to obtain a first image, and performing registration processing on the multiple frames of second RAW images to obtain a second RAW image after multiple frames of registration processing; Performing spatial domain fusion on the second RAW image after the multi-frame registration processing to obtain a second image; the exposure amount of the first image is less than the exposure amount of the second image; Performing a demosaicing process on the first image to obtain a third image, and performing a demosaicing process on the second image to obtain a fourth image; Performing high dynamic range imaging fusion on the third image and the fourth image to obtain a high dynamic range image; The high dynamic range image is subjected to image signal processing to obtain a fifth image.

2. The method according to claim 1, characterized in that After acquiring the first RAW image and multiple frames of second RAW images, the method further includes: Using the second RAW image with the highest definition among the multiple frames of second RAW images as the target RAW image; or The first exposed second RAW image among the multiple frames of second RAW images is used as the target RAW image.

3. The method according to claim 1, characterized in that After performing image signal processing on the high dynamic range image to obtain a fifth image, the method further includes: Perform tone mapping on the fifth image to obtain a sixth image.

4. The method according to claim 3, characterized in that The step of performing tone mapping on the fifth image to obtain a sixth image comprises: Performing global tone mapping on the fifth image to obtain a tone mapped image; The tone mapping image is subjected to local tone mapping to obtain a sixth image.

5. The method according to claim 3, characterized in that: The fifth image is in RGB mode; and tone mapping the fifth image to obtain a sixth image includes: Converting the fifth image into a grayscale mode, and performing tone mapping on the fifth image in the grayscale mode to obtain a sixth image in the grayscale mode; The method further comprises: Dividing the sixth image in the grayscale mode by the fifth image in the grayscale mode to obtain a tone mapping mask image; A seventh image in RGB mode is obtained based on the tone mapping mask image and the fifth image in RGB mode.

6. The method according to claim 5, characterized in that The step of obtaining a seventh image in RGB mode based on the tone mapping mask image and the fifth image in RGB mode comprises: The tone mapping mask image is subjected to a dot multiplication process with the R channel, the G channel, and the B channel in the fifth image in the RGB mode to obtain a seventh image in the RGB mode.

7. The method according to claim 5 or 6, characterized in that: The color depth of the seventh image is the first color depth; the method further includes: The color depth of the seventh image is compressed to a second color depth to obtain an eighth image; the second color depth is less than the first color depth.

8. The method according to claim 7, characterized in that The eighth image is in RGB mode; after compressing the color depth of the seventh image to the second color depth to obtain the eighth image, the method further includes: The eighth image in the RGB mode is converted into the eighth image in the YUV mode.

9. An image processing device, characterized in that: include: An acquisition module, used for acquiring a first RAW image and multiple frames of second RAW images; The exposure amount of the first RAW image is less than the exposure amount of the second RAW image; Performing registration processing on the first RAW image and a target RAW image in the multiple frames of second RAW images to obtain a first image, and performing registration processing on the multiple frames of second RAW images to obtain a second RAW image after multiple frames of registration processing; Performing spatial domain fusion on the second RAW image after the multi-frame registration process to obtain a second image, wherein the exposure amount of the first image is less than the exposure amount of the second image; a demosaic processing module, configured to perform a demosaic processing on the first image to obtain a third image, and perform a demosaic processing on the second image to obtain a fourth image; A fusion module, used for performing high dynamic range imaging fusion on the third image and the fourth image to obtain a high dynamic range image; The image signal processing module is used to perform image signal processing on the high dynamic range image to obtain a fifth image.

10. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the computer program is executed by the processor, the processor is caused to perform the steps of the image processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images