Image processing method and device, computer readable storage medium and electronic device

Abstract

The disclosure provides an image processing method, an image processing device, a computer readable storage medium and an electronic device, and relates to the technical field of images. The image processing method comprises the following steps: acquiring a plurality of to-be-fused images of a same shooting scene; decomposing each to-be-fused image to obtain a Gaussian pyramid and a Laplacian pyramid of each to-be-fused image; fusing pyramid images located at the same layer in the Gaussian pyramid of each to-be-fused image to obtain a fused Gaussian pyramid; fusing pyramid images located at the same layer in the Laplacian pyramid of each to-be-fused image to obtain a fused Laplacian pyramid; and generating a fused image corresponding to the plurality of to-be-fused images by using the fused Gaussian pyramid and the fused Laplacian pyramid. The disclosure can improve the image quality.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of image technology, and in particular, to an image processing method, an image processing apparatus, a computer readable storage medium and an electronic device. BACKGROUND

[0002] With the development of image technology and the popularization of image equipment, users have higher and higher requirements for image quality, and various image processing algorithms have emerged as the snowballing effect. The HDR (High Dynamic Range Imaging) technology can fuse images with different exposure levels together to determine the highlights and shadow details in a large dynamic range environment.

[0003] Currently, for the image fusion processing process, some schemes may have the problem of poor quality of the fused image. SUMMARY

[0004] The present disclosure provides an image processing method, an image processing apparatus, a computer readable storage medium and an electronic device, thereby at least partially overcoming the problem of poor quality of the fused image.

[0005] According to a first aspect of the present disclosure, an image processing method is provided, comprising: obtaining a plurality of to-be-fused images of a same shooting scene, decomposing each to-be-fused image to obtain a Gaussian pyramid and a Laplacian pyramid of each to-be-fused image; fusing pyramid images at a same layer in the Gaussian pyramids of the to-be-fused images to obtain a fused Gaussian pyramid; fusing pyramid images at a same layer in the Laplacian pyramids of the to-be-fused images to obtain a fused Laplacian pyramid; and generating a fused image corresponding to the plurality of to-be-fused images by using the fused Gaussian pyramid and the fused Laplacian pyramid.

[0006] According to a second aspect of the present disclosure, an image processing method is provided, comprising: obtaining a plurality of to-be-fused images of a same shooting scene, decomposing each to-be-fused image to obtain a Gaussian pyramid of each to-be-fused image; fusing pyramid images at a same layer in the Gaussian pyramids of the to-be-fused images to obtain a fused Gaussian pyramid; determining a Laplacian pyramid of a target to-be-fused image in the plurality of to-be-fused images; and generating a fused image corresponding to the plurality of to-be-fused images by using the fused Gaussian pyramid and the Laplacian pyramid of the target to-be-fused image.

[0007] According to a third aspect of the present disclosure, an image processing method is provided, comprising: obtaining a plurality of to-be-fused images of a same shooting scene, decomposing each to-be-fused image to obtain a Laplacian pyramid of each to-be-fused image; fusing pyramid images at a same layer in the Laplacian pyramids of the to-be-fused images to obtain a fused Laplacian pyramid; determining a Gaussian pyramid of a target to-be-fused image in the plurality of to-be-fused images; and generating a fused image corresponding to the plurality of to-be-fused images by using the Gaussian pyramid of the target to-be-fused image and the fused Laplacian pyramid.

[0008] According to a fourth aspect of the present disclosure, an image processing apparatus is provided, comprising: an image decomposition module configured to obtain a plurality of to-be-fused images of a same shooting scene, and decompose each to-be-fused image to obtain a Gaussian pyramid and a Laplacian pyramid of each to-be-fused image; a first fusion module configured to fuse pyramid images at a same layer in the Gaussian pyramids of the to-be-fused images to obtain a fused Gaussian pyramid; a second fusion module configured to fuse pyramid images at a same layer in the Laplacian pyramids of the to-be-fused images to obtain a fused Laplacian pyramid; and an image generation module configured to generate a fused image corresponding to the plurality of to-be-fused images by using the fused Gaussian pyramid and the fused Laplacian pyramid.

[0009] According to a fifth aspect of the present disclosure, an image processing apparatus is provided, comprising: an image decomposition module configured to obtain a plurality of to-be-fused images of a same shooting scene, and decompose each to-be-fused image to obtain a Gaussian pyramid of each to-be-fused image; a layered fusion module configured to fuse pyramid images at a same layer in the Gaussian pyramids of the to-be-fused images to obtain a fused Gaussian pyramid; a Laplacian pyramid determination module configured to determine a Laplacian pyramid of a target to-be-fused image in the plurality of to-be-fused images; and an image generation module configured to generate a fused image corresponding to the plurality of to-be-fused images by using the fused Gaussian pyramid and the Laplacian pyramid of the target to-be-fused image.

[0010] According to a sixth aspect of the present disclosure, an image processing apparatus is provided, comprising: an image decomposition module configured to obtain a plurality of to-be-fused images of a same shooting scene, and decompose each to-be-fused image to obtain a Laplacian pyramid of each to-be-fused image; a layered fusion module configured to fuse pyramid images at a same layer in the Laplacian pyramids of the to-be-fused images to obtain a fused Laplacian pyramid; a Gaussian pyramid determination module configured to determine a Gaussian pyramid of a target to-be-fused image in the plurality of to-be-fused images; and an image generation module configured to generate a fused image corresponding to the plurality of to-be-fused images by using the Gaussian pyramid of the target to-be-fused image and the fused Laplacian pyramid.

[0011] According to a seventh aspect of the present disclosure, a computer readable storage medium is provided, which stores a computer program, the program being executed by a processor to implement any of the above image processing methods.

[0012] According to an eighth aspect of the present disclosure, an electronic device is provided, which comprises a processor, and a memory for storing one or more programs, the one or more programs being executed by the processor to cause the processor to implement any of the above image processing methods.

[0013] In the technical solutions provided by some embodiments of the present disclosure, the pyramid images at the same layer in the Gaussian pyramid of the images to be fused are fused and / or the pyramid images at the same layer in the Laplacian pyramid of the images to be fused are fused, and a processed image is generated in combination with the fused pyramid. Through the layer-by-layer fusion of the pyramid images, the image information of the same scale can be better merged, the detail performance of the image is enhanced, the fusion is more delicate, and the image edge transition is smooth, and the image fusion scheme of the present disclosure can effectively improve the image quality.

[0014] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure. It is clear that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained from these drawings by those of ordinary skill in the art without creative labor. In the drawings:

[0016] Figure 1 a schematic diagram of an image processing stage of an embodiment of the present disclosure is shown;

[0017] Figure 2 a process schematic diagram of pyramid-based image decomposition and reconstruction of an embodiment of the present disclosure is shown;

[0018] Figure 3 a schematic diagram of a Gaussian pyramid of an embodiment of the present disclosure is shown;

[0019] Figure 4 a generation schematic diagram of a Laplacian pyramid of an embodiment of the present disclosure is shown;

[0020] Figure 5 a flowchart of an image processing method according to an embodiment of the present disclosure is shown schematically;

[0021] Figure 6 a schematic diagram of a fusion weight according to an embodiment of the present disclosure is shown schematically;

[0022] Figure 7 A flowchart schematically illustrating an overall process of an image processing method according to an embodiment of the present disclosure is shown;

[0023] Figure 8 A flowchart schematically illustrating an image processing method according to another embodiment of the present disclosure is shown;

[0024] Figure 9 A flowchart schematically illustrating an image processing method according to yet another embodiment of the present disclosure is shown;

[0025] Figure 10 A block diagram schematically illustrating an image processing apparatus according to an embodiment of the present disclosure is shown;

[0026] Figure 11 A block diagram schematically illustrating an image processing apparatus according to another embodiment of the present disclosure is shown;

[0027] Figure 12 A block diagram schematically illustrating an image processing apparatus according to yet another embodiment of the present disclosure is shown;

[0028] Figure 13 A block diagram schematically illustrating an electronic device according to an example embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0029] Example embodiments now will be described more fully hereinafter with reference to the accompanying drawings. Example embodiments, may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the example embodiments to those skilled in the art. The features, structures, or characteristics described in connection with the embodiments disclosed herein can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the present disclosure. One skilled in the relevant art will recognize, however, that the techniques disclosed herein can be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. Other implements, apparatuses, methods, steps, etc. can be utilized without departing from the scope of the present disclosure.

[0030] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0031] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all steps. For example, some steps may be decomposed, while others may be combined or partially combined. Therefore, the actual order of execution may vary depending on the actual situation. In addition, all terms such as "first," "second," "third," and "fourth" below are used for distinction purposes only and should not be construed as limitations of this disclosure.

[0032] When electronic devices capture images, they may be underexposed or overexposed. For example, if the scene is a sunset, the probability of underexposed or overexposed single-frame shots is very high. In this case, bracketing can be used for shooting, and after HDR synthesis, an image with details in both bright and dark areas can be obtained. However, how to select appropriate information from multiple frames of images and organize and fuse them reasonably is crucial to the presentation of the final image. The disclosed solution focuses on the fusion process of multiple frames of images and proposes a new image processing solution to improve the image quality of the fused image.

[0033] The image processing scheme of the embodiments of the present disclosure can be implemented by an electronic device. That is, the electronic device can perform the various steps of the following image processing method, and the following image processing device can be configured within the electronic device. For example, the image processing scheme of the present disclosure can be implemented by an image signal processor equipped in the electronic device. In addition, the present disclosure does not limit the type of electronic device, which may include but is not limited to smartphones, tablet computers, smart wearable devices, personal computers, servers, etc.

[0034] Figure 1 Schematic diagram showing the stages of application of the image processing solution of the embodiment of the present disclosure. Figure 1 The input images of the embodiments of the present disclosure are multiple images with different exposure levels, the number of input images is two or more, and the input images are images generated for the same shooting scene. The input images can be images captured by a camera module equipped with an electronic device, or images obtained by the electronic device from the outside (i.e., other devices). The present disclosure does not impose any restrictions on the image source, image content, image size, bit width of the image data, etc. of the input images.

[0035] The image processing process disclosed herein can be used to process the input image to obtain a fused image. The electronic device can then transmit the fused image back to the ISP pipeline (image signal processing pipeline) to continue processing such as brightness enhancement, tone mapping, denoising, and recognition. This disclosure does not limit the subsequent processing of the fused image.

[0036] The image processing solution of the disclosed embodiments can be applied to scenarios such as camera preview and video stream processing. For example, in scenarios where a highly exposed image is fused with a lightly exposed image, the dynamic range of the highly exposed image can be supplemented, given that the lightly exposed image is typically darker.

[0037] In order to better illustrate the image processing scheme of the embodiment of the present disclosure, the following reference Figures 2 to 4 The following describes the pyramid processing method in the image field.

[0038] refer to Figure 2 For an original image, we can use a Gaussian kernel to perform Gaussian blur processing on the original image, and then downsample the Gaussian blurred image to obtain a scaled image. Repeating the above process can obtain multiple images with successively reduced scales, and these images construct a Gaussian pyramid. Figure 3 A schematic diagram of a Gaussian pyramid is shown, as Figure 3 As shown in the figure, after multiple levels of Gaussian blurring and downsampling, images such as Level 0, Level 1, Level 2, Level 3, and Level 4 can be obtained. These images of different scales constitute a Gaussian pyramid.

[0039] The present disclosure does not limit the downsampling method, such as the downsampling method of removing odd rows and odd columns (or removing even rows and even columns), or the random downsampling method, or the pooled downsampling method, etc.

[0040] Each layer of the Gaussian pyramid, except for the top layer, is subtracted from the upsampled image of the layer above it (the layer with a smaller scale) to obtain a Laplacian pyramid. The present disclosure does not limit the upsampling method, and upsampling can be achieved, for example, by padding rows and columns. Furthermore, it is understood that Gaussian blurring is also performed after upsampling.

[0041] refer to Figure 4 , if the original image is recorded as G0, G0 is downsampled by 2 times to obtain G1, G1 is downsampled by 2 times to obtain G2, and G2 is downsampled by 2 times to obtain G3. G0 to G3 correspond to the Gaussian pyramid.

[0042] G3 is LP3. G3 is interpolated and enlarged to obtain G*3. G*3 is subtracted from G2 to obtain LP2. G2 is interpolated and enlarged to obtain G*2. G*2 is subtracted from G1 to obtain LP1. G1 is interpolated and enlarged to obtain G*1. G*1 is subtracted from G0 to obtain LP0. LP0 to LP3 correspond to a Laplacian pyramid. The Laplacian pyramid can be understood as a pyramid composed of residual images obtained by subtraction from above.

[0043] Figure 4 An example is shown in a 4-level pyramid structure, i.e., 4 images of different scales. However, it should be understood that the number of layers of the pyramid is not limited in the present disclosure, and the number of layers of the pyramid can be determined by comprehensively considering device processing efficiency, task requirement accuracy, and other factors.

[0044] After the Gaussian pyramid and the Laplacian pyramid of the original image are determined, the processed image can be generated by combining the two. Specifically, the Gaussian pyramid and the Laplacian pyramid can be fused in order from small to large scale, undergo upsampling and convolution operation of the Gaussian kernel, and finally generate the processed image.

[0045] The image processing method of the embodiment of the present disclosure will be described below by taking an electronic device performing an image processing process as an example. It should be noted that the image processing method of the embodiment of the present disclosure is applicable to raw data. Compared with data in RGB or other formats, raw data has no loss, and performing a processing process based on raw data can preserve more image details.

[0046] Figure 5 A flowchart of the image processing method of the example embodiment of the present disclosure is schematically shown. Referring to Figure 5 , the image processing method can include the following steps:

[0047] S52. Obtain a plurality of to-be-fused images of the same shooting scene, and decompose each to-be-fused image to obtain a Gaussian pyramid and a Laplacian pyramid of each to-be-fused image.

[0048] In the example embodiment of the present disclosure, the to-be-fused image is an image to be decomposed by a pyramid. That is, the object of the image decomposition of the present disclosure is the to-be-fused image.

[0049] The plurality of to-be-fused images of the same shooting scene can be images obtained by performing brightness alignment on a plurality of original images captured under the same scene. The present disclosure does not have any requirements on the shooting scene, and the scheme can be applied to any shooting scene.

[0050] First, the electronic device can obtain a plurality of original images captured based on different exposure levels under the same shooting scene.

[0051] According to some embodiments of the present disclosure, the different exposure degrees can correspond to at least two of overexposed exposure values, normally exposed exposure values, and underexposed exposure values. That is, the plurality of acquired original images can include at least two types of images of overexposed images, normally exposed images, and underexposed images.

[0052] Since the overexposed images have good dark details and signal-to-noise ratios, and the underexposed images have good bright details, combining the overexposed original images and the underexposed original images for image fusion helps to obtain a high-quality HDR image subsequently.

[0053] On one hand, the present disclosure does not limit the degree of overexposure and underexposure, that is, it does not limit the specific exposure values used for different exposure values. On the other hand, the number of original images in the embodiments of the present disclosure is more than two, and the specific number is also not limited.

[0054] It should be noted that the plurality of original images can include overexposed images and not underexposed images, or can include underexposed images and not overexposed images.

[0055] Next, the electronic device can perform luminance alignment on the original images by using the proportional relationship of the exposure degrees between the original images, to obtain a plurality of to-be-fused images of the same shooting scene.

[0056] It can be understood that for any original image, there is an exposure time and a gain corresponding to the original image, and in the case of using the exposure time and the gain to represent the exposure degree, the exposure degree can be defined as the product of the exposure time and the gain. In this case, the proportional relationship of the exposure degrees between two original images can be the ratio of the exposure degrees of the two original images.

[0057] Taking two original images as an example, the ratio of the exposure degrees between the original image with a large exposure degree and the original image with a small exposure degree is denoted as Ratio, and then luminance alignment can be performed by using Formula 1:

[0058] L = S * Ratio (Formula 1)

[0059] Wherein, S is the original image with a small exposure degree, and L is the image obtained after luminance alignment of S, that is, the to-be-fused image corresponding to S.

[0060] For the embodiment of the two original images, L and the original image with a large exposure degree are to-be-fused images of the present disclosure, and the bit width of L is greater than the bit width of the original image with a large exposure degree.

[0061] For the case where there are more than two original images, in one embodiment, the original image with the largest exposure degree can be taken as the reference image for luminance alignment, and the to-be-fused images corresponding to each original image can be determined respectively in combination with Formula 1 above.

[0062] In another embodiment, the original image with the best image quality among the multiple original images can also be taken as the reference image for luminance alignment, and the to-be-fused images corresponding to each original image can be determined respectively in combination with Formula 1 above. The image quality can be determined by one or more evaluation indexes such as signal-to-noise ratio, underexposure degree, overexposure degree, and contrast, and the process of determining the image quality is not limited in the present disclosure.

[0063] In addition, when the difference between the exposure degrees of the original images is less than a difference threshold, the electronic device can directly take the original images as the to-be-fused images.

[0064] Specifically, the electronic device can determine the difference between the exposure degrees of the original images, and compare the difference with the difference threshold. If the determined difference is less than the difference threshold, in order to reduce the algorithm calculation cost, the electronic device can directly take the original images as the to-be-fused images. If the determined difference is greater than or equal to the difference threshold, the above-mentioned luminance alignment process can be performed to obtain multiple to-be-fused images.

[0065] After obtaining the multiple to-be-fused images of the same shooting scene, the electronic device can decompose each to-be-fused image to obtain the Gaussian pyramid and the Laplacian pyramid of each to-be-fused image.

[0066] Specifically, the Gaussian pyramid of the to-be-fused image can be determined by Gaussian blur and down-sampling, and the Laplacian pyramid of the to-be-fused image can be generated by subtracting the up-sampled image of the upper layer from each layer of the Gaussian pyramid. The specific process is shown in Figures 2 to 4 , which will not be described here again.

[0067] It should be noted that the present disclosure does not limit the Gaussian kernel used for Gaussian blur, the manner of down-sampling and up-sampling, the number of layers of the pyramid, etc.

[0068] After decomposing the to-be-fused images, the number of layers of the Gaussian pyramid of each to-be-fused image is the same, and the number of layers of the Laplacian pyramid of each to-be-fused image is the same.

[0069] S54. The pyramid images at the same layer in the Gaussian pyramids of the to-be-fused images are fused to obtain a fused Gaussian pyramid.

[0070] The Gaussian pyramid is a pyramid composed of multiple pyramid images with scales from large to small. For the Gaussian pyramid of each image to be fused, the electronic device can fuse the pyramid images of each layer layer by layer to obtain a fused Gaussian pyramid.

[0071] For example, the multiple images to be fused include image A and image B. Taking a four-layer pyramid as an example, the Gaussian pyramid of image A can include first-layer pyramid image a11, second-layer pyramid image a12, third-layer pyramid image a13, and fourth-layer pyramid image a14, and the Gaussian pyramid of image B can include first-layer pyramid image b11, second-layer pyramid image b12, third-layer pyramid image b13, and fourth-layer pyramid image b14. The fusion of the pyramid images at the same layer in the Gaussian pyramid in the present disclosure refers to the fusion of a11 and b11, the fusion of a12 and b12, the fusion of a13 and b13, and the fusion of a14 and b14, respectively.

[0072] The fusion of the pyramid images at the same layer in the Gaussian pyramid is described below by taking the fusion of two pyramid images as an example. It can be understood that the fusion process of the two pyramid images described below can be extended to the fusion process between multiple pyramid images. That is, after determining the fusion manner of the two pyramid images, the same processing manner can be used to perform the fusion process between other pyramid images, and then the fusion result of multiple pyramid images is obtained.

[0073] The multiple images to be fused obtained in step S52 can include a first image to be fused and a second image to be fused, and the exposure degree of the original image corresponding to the first image to be fused is greater than the exposure degree of the original image corresponding to the second image to be fused. The Gaussian pyramid of the first image to be fused includes a first pyramid image, and the image at the same layer as the first pyramid image in the Gaussian pyramid of the second image to be fused is a second pyramid image. In this case, the process of fusing the first pyramid image and the second pyramid image by the electronic device can include determining the fusion weight of the first pyramid image and the second pyramid image, and fusing the first pyramid image and the second pyramid image by using the fusion weight.

[0074] The process of determining the fusion weight of the first pyramid image and the second pyramid image by the present disclosure is described exemplarily below.

[0075] The first embodiment of the present disclosure for determining the fusion weight is as follows:

[0076] The electronic device can determine the difference between the first pyramid image and the second pyramid image, and determine the fusion weight of the first pyramid image and the second pyramid image according to the difference.

[0077] By the difference between the first pyramid image and the second pyramid image, it can be determined whether there is a moving object in the image, and then the weight calculation based on the moving object is realized. Then, the image fusion is realized by using the weight, which can reflect the motion information of the image, and then the performance of the motion details of the fused image is improved, and the image quality of the fused image is improved.

[0078] The electronic device can determine the difference between the pixel values of each pixel point in the first pyramid image and the pixel point corresponding in position in the second pyramid image to generate a first difference weight map.

[0079] For example, for the pixel point e at (i, j) in the first pyramid image, the pixel point f at (i, j) in the second pyramid image is also determined, and the difference between the pixel point e and the pixel point f is calculated. Wherein (i, j) is the coordinate of the pixel point in the pyramid image, and the difference between the pyramid images is calculated by traversing all the pixel points in the pyramid image, and the first difference weight map can be generated.

[0080] It can be understood that the difference calculated by the present disclosure includes the operation of taking the absolute value, and the difference is a value greater than or equal to 0,

[0081] For the process of determining the difference between the pixel values of each pixel point in the first pyramid image and the pixel point corresponding in position in the second pyramid image, in some embodiments of the present disclosure, the electronic device can directly subtract the two pixel values and take the absolute value to obtain the difference between the pixel values of the pixel points at the same position.

[0082] In addition, the direct subtraction of the pixel values may cause the calculation result to be too discrete, causing the problem of unsmooth transition of the image region. In order to solve this problem, in some other embodiments of the present disclosure, the determination of the difference can be realized in combination with the pixel region to which the pixel point belongs.

[0083] First, the electronic device can determine the adjacent pixel points of the first pixel point in the first pyramid image, and the first pixel point and the adjacent pixel points of the first pixel point form a first pixel region. For example, the first pixel region is a region composed of the first pixel point and the surrounding 8 adjacent pixel points.

[0084] Next, the electronic device can determine the second pixel point corresponding to the first pixel point in position in the second pyramid image, for example, both of them are at (i, j) in the pyramid image. And determine the adjacent pixel points of the second pixel point, and the second pixel point and the adjacent pixel points of the second pixel point form a second pixel region. For example, similarly, the second pixel region is a region composed of the second pixel point and the surrounding 8 adjacent pixel points.

[0085] Subsequently, the electronic device can determine the difference in pixel values between the first pixel region and the second pixel region as the difference in pixel values between the first pixel point and the second pixel point.

[0086] For the process of determining the difference in pixel values between the first pixel region and the second pixel region, the electronic device can traverse each pixel point in the first pixel region to determine the difference in pixel values between the pixel point in the first pixel region and the pixel point in the second pixel region corresponding in position. For example, for a pixel region composed of 9 pixel points, the first pixel point in the first pixel region and the second pixel point in the second pixel region correspond in position, the pixel point at the upper left corner of the first pixel point in the first pixel region and the pixel point at the upper left corner of the second pixel point in the second pixel region correspond in position, and the pixel point at the lower right corner of the first pixel point in the first pixel region and the pixel point at the lower right corner of the second pixel point in the second pixel region correspond in position.

[0087] The electronic device can determine the difference in pixel values between the first pixel region and the second pixel region according to the difference in pixel values between the pixel point in the first pixel region and the pixel point in the second pixel region corresponding in position.

[0088] Specifically, the difference in pixel values between the pixel point in the first pixel region and the pixel point in the second pixel region corresponding in position can be accumulated, and the accumulated result is determined as the difference in pixel values between the first pixel region and the second pixel region. Alternatively, the difference in pixel values between the pixel point in the first pixel region and the pixel point in the second pixel region corresponding in position can be averaged, and the average value is determined as the difference in pixel values between the first pixel region and the second pixel region. In addition, the difference in pixel values between the first pixel region and the second pixel region can also be represented by a statistical value such as variance, and the present disclosure does not limit this.

[0089] After one pixel point is calculated, the next pixel point calculation will overlap the pixel region, which helps to obtain a transition smooth image.

[0090] After determining the difference in pixel values between each pixel point in the first pyramid image and the pixel point in the second pyramid image corresponding in position, a first difference weight map can be constructed from the difference in each pixel point. The first difference weight map is consistent in size with the first pyramid image, and each value on the first difference weight map is a pixel point weight value, which is the difference in pixel values of the pixel point corresponding in position.

[0091] After generating the first difference weight map, the electronic device can determine the fusion weight of the first pyramid image and the second pyramid image using the first difference weight map.

[0092] The electronic device can map the pixel point weight value in the first difference weight map to a predetermined bit width range to obtain a second difference weight map. Specifically, the electronic device can determine whether the pixel point weight value in the first difference weight map is consistent with the predetermined bit width range. If consistent, no processing is performed, and the second difference weight map is the first difference weight map. If inconsistent, the pixel point weight value is mapped to the predetermined bit width range to obtain the second difference weight map. For example, the predetermined bit width range is 8 bits (i.e., 0-255), and if the pixel point weight value is 10 bits (0-1023), the electronic device can map the 10-bit data to 8 bits using a mapping curve. The mapping curve is a curve with 10 bits and 8 bits as the horizontal and vertical coordinates, respectively, and the present disclosure does not limit the construction process of the mapping curve or the specific form of the curve. In addition, the mapping curve mentioned here is for the dimension of the difference weight map, and the different dimensions result in different mapping curves in the embodiments of the present disclosure.

[0093] To facilitate subsequent weighted calculation, next, the electronic device can perform normalization processing on the second difference weight map, and determine the result of the normalization processing as the fusion weight of the first pyramid image and the second pyramid image.

[0094] The second embodiment of the present disclosure for determining the fusion weight is as follows:

[0095] The electronic device can adjust the pixel data of the second pyramid image to have a bit width consistent with that of the first pyramid image to generate a third pyramid image. In this way, the third pyramid image is aligned with the first pyramid image in terms of brightness information and ignores the overexposed area.

[0096] Since the exposure degree of the original image corresponding to the first image to be fused is greater than the exposure degree of the original image corresponding to the second image to be fused, after the above brightness alignment processing, the bit width of the pixel data of the second pyramid image is greater than the bit width of the pixel data of the first pyramid image. In this case, the electronic device can perform a clipping operation on the pixel data of the second pyramid image to adjust the bit width to be consistent with that of the first pyramid image. The clipping operation can be referred to as a Clip operation, that is, setting 0 when less than 0 and setting Max when greater than Max. Max is the bit width of the first pyramid image, such as 1024. In addition, considering the black level, Max can also be, for example, 1024-64=960.

[0097] After generating the third pyramid image, the electronic device can determine the fusion weight of the first pyramid image and the second pyramid image according to the third pyramid image.

[0098] First, the electronic device can perform mean filtering (the size of the filter kernel is, for example, 5x5) on the third pyramid image to obtain a fourth pyramid image.

[0099] Next, the electronic device can determine the difference between the bit width of the first pyramid image and the pixel value of each pixel point in the fourth pyramid image to obtain a first luminance weight map. That is, the result of subtracting the fourth pyramid image from MAX is the first luminance weight map. Wherein, MAX is a matrix with the same size as the fourth pyramid image and each value is the above Max (such as 1024).

[0100] Then, the electronic device can determine the fusion weight of the first pyramid image and the second pyramid image by using the first luminance weight map.

[0101] The electronic device can map the pixel point weight value in the first luminance weight map to a predetermined bit width range to obtain a second luminance weight map. Specifically, the electronic device can determine whether the pixel point weight value in the first luminance weight map is consistent with the predetermined bit width range. If it is consistent, no processing is performed, and the second luminance weight map is the first luminance weight map at this time; if it is not consistent, the pixel point weight value is mapped to the predetermined bit width range to obtain the second luminance weight map. For example, the predetermined bit width range is 8 bits (i.e. 0-255), and if the pixel point weight value is 10 bits (0-1023), the electronic device can map the 10-bit data to 8 bits using a mapping curve. Wherein, the mapping curve is a curve with 10 bits and 8 bits as the horizontal and vertical coordinates respectively, and the present disclosure does not limit the construction process of the mapping curve and the specific form of the curve. In addition, the mapping curve mentioned here is for the dimension of the luminance weight map, and the different dimensions result in different mapping curves mentioned in the embodiments of the present disclosure.

[0102] In order to facilitate subsequent weighted calculation, next, the electronic device can perform normalization processing on the second luminance weight map, and determine the result of the normalization processing as the fusion weight of the first pyramid image and the second pyramid image.

[0103] The third embodiment of the present disclosure for determining the fusion weight is as follows:

[0104] The electronic device can adjust the pixel data of the second pyramid image to have a bit width consistent with that of the first pyramid image to generate a third pyramid image. This process is the same as that in the second embodiment, and will not be repeated here.

[0105] After generating the third pyramid image, the electronic device can determine the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs, and generate a first local maximum weight map from the maximum value corresponding to each pixel point. That is, the maximum value of the pixel value in the Bayer unit to which the pixel point belongs can be used to replace the pixel value of the pixel point, and all pixel points are traversed to obtain the first local maximum weight map.

[0106] Next, the electronic device can determine the fusion weight of the first pyramid image and the second pyramid image by using the first local maximum weight map.

[0107] The electronic device can map the pixel weight value in the first local maximum weight map to a predetermined bit width range to obtain a second local maximum weight map. Specifically, the electronic device can determine whether the pixel weight value in the first local maximum weight map is consistent with the predetermined bit width range. If consistent, no processing is performed, and the second local maximum weight map is the first local maximum weight map. If inconsistent, the pixel weight value is mapped to the predetermined bit width range to obtain the second local maximum weight map. For example, the predetermined bit width range is 8 bits (i.e., 0-255), and if the pixel weight value is 10 bits (0-1023), the electronic device can map the 10-bit data to 8 bits by using a mapping curve. The mapping curve is a curve with 10 bits and 8 bits as the horizontal and vertical coordinates, respectively, and the present disclosure does not limit the construction process of the mapping curve or the specific form of the curve. In addition, the mapping curve mentioned here is for the dimension of the local maximum weight map, and different dimensions result in different mapping curves in the embodiments of the present disclosure.

[0108] To facilitate subsequent weighted calculation, the electronic device can perform normalization processing on the second local maximum weight map, and determine the result of the normalization processing as the fusion weight of the first pyramid image and the second pyramid image.

[0109] A fourth embodiment of the present disclosure for determining the fusion weight is as follows:

[0110] The electronic device can adjust the pixel data of the second pyramid image to have a bit width consistent with that of the first pyramid image to generate a third pyramid image. This process is the same as that in the second embodiment, and will not be repeated here.

[0111] On the one hand, the electronic device can perform mean filtering on the third pyramid image to obtain a fourth pyramid image, and determine the difference between the bit width of the first pyramid image and the pixel value of each pixel point in the fourth pyramid image to obtain a first brightness weight map. The pixel weight in the first brightness weight map is mapped to a predetermined bit width range to obtain a second brightness weight map. This process is the same as that in the second embodiment, and will not be repeated here.

[0112] On the other hand, the electronic device can determine the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs, and generate a first local maximum weight map from the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs. Map the weight value of the pixel point in the first local maximum weight map to the predetermined bit width range to obtain a second local maximum weight map. This process is the same as that in the third embodiment, and will not be described again.

[0113] Then, the electronic device can multiply the second brightness weight map and the second local maximum weight map, normalize the result of the multiplication, and determine the normalized result as the fusion weight of the first pyramid image and the second pyramid image.

[0114] The fifth embodiment of the present disclosure determines the fusion weight:

[0115] The electronic device can adjust the pixel data of the second pyramid image to have a bit width consistent with that of the first pyramid image to generate a third pyramid image. This process is the same as that in the second embodiment, and will not be described again.

[0116] After generating the third pyramid image, the electronic device can determine the fusion weight of the first pyramid image and the second pyramid image according to the first pyramid image, the second pyramid image, and the third pyramid image.

[0117] Specifically, the electronic device can determine the difference between the pixel values of each pixel point in the first pyramid image and the pixel point corresponding in position in the second pyramid image to generate a first difference weight map. This process is the same as that in the first embodiment, and will not be described again.

[0118] The electronic device can determine the fusion weight of the first pyramid image and the second pyramid image according to the first difference weight map and the third pyramid image.

[0119] On the one hand, the electronic device can perform mean filtering on the third pyramid image to obtain a fourth pyramid image, and determine the difference between the bit width of the first pyramid image and the pixel value of each pixel point in the fourth pyramid image to obtain a first brightness weight map. Map the weight of the pixel point in the first brightness weight map to the predetermined bit width range to obtain a second brightness weight map. This process is the same as that in the second embodiment, and will not be described again.

[0120] On the other hand, the electronic device can map the weight value of the pixel point in the first difference weight map to the predetermined bit width range to obtain a second difference weight map. This process is the same as that in the first embodiment, and will not be described again.

[0121] Then, the electronic device can multiply the second local maximum weight map and the second difference weight map, normalize the multiplication result, and determine the normalized result as the fusion weight of the first pyramid image and the second pyramid image.

[0122] The present disclosure determines a sixth embodiment of the fusion weight:

[0123] The electronic device can adjust the pixel data of the second pyramid image to a bit width consistent with the bit width of the first pyramid image to generate a third pyramid image. This process is the same as that in the second embodiment, and will not be repeated here.

[0124] After generating the third pyramid image, the electronic device can determine the fusion weight of the first pyramid image and the second pyramid image according to the first pyramid image, the second pyramid image, and the third pyramid image.

[0125] Specifically, the electronic device can determine the difference between the pixel values of each pixel point in the first pyramid image and the pixel point in the second pyramid image corresponding in position to generate a first difference weight map. This process is the same as that in the first embodiment, and will not be repeated here.

[0126] The electronic device can determine the fusion weight of the first pyramid image and the second pyramid image according to the first difference weight map and the third pyramid image.

[0127] On the one hand, the electronic device can determine the maximum value of the pixel values in the Bayer cell to which each pixel point in the third pyramid image belongs, and generate a first local maximum weight map from the maximum value of the pixel values in the Bayer cell to which each pixel point in the third pyramid image belongs. Map the weight values of the pixel points in the first local maximum weight map to the predetermined bit width range to obtain a second local maximum weight map. This process is the same as that in the third embodiment, and will not be repeated here.

[0128] On the other hand, the electronic device can map the weight values of the pixel points in the first difference weight map to the predetermined bit width range to obtain a second difference weight map. This process is the same as that in the first embodiment, and will not be repeated here.

[0129] Then, the electronic device can multiply the second local maximum weight map and the second difference weight map, normalize the multiplication result, and determine the normalized result as the fusion weight of the first pyramid image and the second pyramid image.

[0130] The present disclosure determines a seventh embodiment of the fusion weight:

[0131] The electronic device can adjust the pixel data of the second pyramid image to a bit width consistent with the bit width of the first pyramid image to generate a third pyramid image. This process is the same as that in the second embodiment, and will not be repeated here.

[0132] After generating the third pyramid image, the electronic device can determine the fusion weight of the first pyramid image and the second pyramid image according to the first pyramid image, the second pyramid image and the third pyramid image.

[0133] Specifically, the electronic device can determine the difference between the pixel values of each pixel point in the first pyramid image and the pixel point corresponding in position in the second pyramid image to generate a first difference weight map. This process is the same as that in the first embodiment and will not be repeated here.

[0134] The electronic device can determine the fusion weight of the first pyramid image and the second pyramid image according to the first difference weight map and the third pyramid image.

[0135] On the one hand, the electronic device can perform mean filtering on the third pyramid image to obtain a fourth pyramid image, and determine the difference between the bit width of the first pyramid image and the pixel value of each pixel point in the fourth pyramid image to obtain a first brightness weight map. Map the pixel point weight in the first brightness weight map to a predetermined bit width range to obtain a second brightness weight map. This process is the same as that in the second embodiment and will not be repeated here.

[0136] On the other hand, the electronic device can determine the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs, and generate a first local maximum value weight map from the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs. Map the pixel point weight value in the first local maximum value weight map to the predetermined bit width range to obtain a second local maximum value weight map. This process is the same as that in the third embodiment and will not be repeated here.

[0137] On the other hand, the electronic device can determine the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs, and generate a first local maximum value weight map from the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs. Map the pixel point weight value in the first local maximum value weight map to the predetermined bit width range to obtain a second local maximum value weight map. This process is the same as that in the third embodiment and will not be repeated here.

[0138] Then, the electronic device can multiply the second difference weight map, the second brightness weight map and the second local maximum value weight map, normalize the result of the multiplication, and determine the normalized result as the fusion weight of the first pyramid image and the second pyramid image.

[0139] Reference Figure 6 In the process of determining the fusion weight of the first pyramid image and the second pyramid image, the pixel difference weight, the brightness weight and the local maximum value weight can be combined, multiplied to determine an intermediate weight, and then normalized to obtain the final fusion weight.

[0140] It should be understood that one of the above seven embodiments of determining the fusion weight can be implemented to determine the fusion weight, and the fusion weight determined by the present disclosure is the fusion weight of the first pyramid image, denoted as w1. The fusion weight of the second pyramid image is denoted as w2, and w2 = 1-w1. If the first pyramid image is denoted as L1 and the second pyramid image is denoted as L2, then the fusion result L' of the first pyramid image and the second pyramid image can be expressed as formula 2:

[0141] L' = L1*w1 + L2*w2 (formula 2)

[0142] The fusion process of the first pyramid image and the second pyramid image described above is performed for each pyramid image of each layer of the Gaussian pyramid, and thus the fused Gaussian pyramid can be obtained.

[0143] S56. The pyramid images at the same layer in the Laplacian pyramid of each image to be fused are fused to obtain a fused Laplacian pyramid.

[0144] For the Laplacian pyramid of each image to be fused, the electronic device can fuse the pyramid images of each layer layer by layer to obtain a fused Laplacian pyramid.

[0145] For example, the Laplacian pyramid of the image to be fused A can include a first layer pyramid image a21, a second layer pyramid image a22, a third layer pyramid image a23 and a fourth layer pyramid image a24, and the Laplacian pyramid of the image to be fused B can include a first layer pyramid image b21, a second layer pyramid image b22, a third layer pyramid image b23 and a fourth layer pyramid image b24. The present disclosure refers to the fusion of the pyramid images at the same layer in the Laplacian pyramid as the fusion of a21 and b21, the fusion of a22 and b22, the fusion of a23 and b23, and the fusion of a24 and b24, respectively.

[0146] The fusion process of the pyramid images of the Laplacian pyramid is similar to the fusion process of the first pyramid image and the second pyramid image described in step S54. That is, in the fusion process of the pyramid images of the Laplacian pyramid, the fusion weight can also be determined by one of the above seven embodiments of determining the fusion weight, and the fusion of the pyramid images of the Laplacian pyramid is realized in combination with the fusion weight. Herein, no further description is given.

[0147] In addition, it can be understood that the present disclosure does not limit the execution order of step S54 and step S56. That is, step S56 can also be executed before step S54, or the two steps can be executed simultaneously.

[0148] S58. Utilizing the fused Gaussian pyramid and the fused Laplacian pyramid, a fused image corresponding to the plurality of images to be fused is generated.

[0149] After the fused Gaussian pyramid and the fused Laplacian pyramid are determined, the electronic device can fuse the fused Gaussian pyramid and the fused Laplacian pyramid, specifically as Figure 2 shown, the fused image corresponding to the plurality of images to be fused is generated, i.e., the image corresponding to the plurality of original images with improved quality is obtained, through upsampling and calculation of the Gaussian kernel.

[0150] The entire process of the image processing method of the embodiment of the present disclosure will be described below with reference to Figure 7 two input images as an example.

[0151] In step S702, the electronic device acquires the long-exposure raw image and the short-exposure raw image, and performs brightness alignment on the long-exposure raw image and the short-exposure raw image.

[0152] In step S704, the electronic device can perform pyramid decomposition on the brightness-aligned image to obtain a Gaussian pyramid and a Laplacian pyramid.

[0153] In step S706, for the brightness-aligned pyramid image corresponding to the short-exposure raw image, the electronic device can perform clip processing to generate a short-frame brightness image with a bit width consistent with the long-exposure raw image.

[0154] In step S708, the electronic device can calculate the fusion weight in combination with the short-frame brightness image. For example, the dimensions of the fusion weight include the pixel difference weight, the brightness weight, and the local maximum weight described above, which are multiplied and normalized to obtain the fusion weight.

[0155] In step S710, the electronic device can perform pyramid image fusion layer by layer according to the fusion weight corresponding to the different layer pyramid images.

[0156] In step S712, the electronic device can perform image reconstruction using the Gaussian pyramid and the Laplacian pyramid generated by layer-by-layer fusion to generate a fused image corresponding to the long-exposure raw image and the short-exposure raw image acquired in step S710.

[0157] In the image processing method, the layer-by-layer fusion is respectively performed on the Gaussian pyramid and the Laplacian pyramid, on one hand, a fusion result with good detail performance and smooth transition can be obtained, and ghost is eliminated while obtaining high dynamic range; on the other hand, the motion object can be effectively detected and the brightness effect can be balanced through the fusion weight construction strategy combining the pixel difference, the brightness and the local maximum, and the image quality of the fusion image is further improved.

[0158] In addition, although the image processing process is mainly for raw domain images, it should be noted that the pyramid layer-by-layer fusion concept and the fusion weight determination concept of the present disclosure can also be applied to RGB, RYB images, and the present disclosure does not limit this.

[0159] In the image processing process, the layer-by-layer fusion is performed on both the Gaussian pyramid and the Laplacian pyramid. However, in some other embodiments of the present disclosure, the layer-by-layer fusion can also be performed on one of the Gaussian pyramid and the Laplacian pyramid.

[0160] Further, the present disclosure also provides another image processing method.

[0161] Figure 8 The flow chart of the image processing method of another embodiment of the present disclosure is schematically shown. Referring to Figure 8 The image processing method can include the following steps:

[0162] S82. Obtain a plurality of to-be-fused images of the same shooting scene, and decompose each to-be-fused image to obtain a Gaussian pyramid of each to-be-fused image.

[0163] The process of obtaining the Gaussian pyramid of each to-be-fused image is the same as the content in step S52, and will not be repeated here.

[0164] S84. Fuse the pyramid images at the same layer in the Gaussian pyramid of each to-be-fused image to obtain a fused Gaussian pyramid.

[0165] The process of layer-by-layer fusion of the Gaussian pyramid is the same as the process of step S54, and will not be repeated here.

[0166] S86. Determine a Laplacian pyramid of a target to-be-fused image in the plurality of to-be-fused images.

[0167] In the exemplary embodiments of the present disclosure, the target image to be fused can be any one of the images to be fused, or can be an image corresponding to the largest, smallest or medium exposure degree among the images to be fused, or can be an image with the highest image quality among the images to be fused, which can be determined by one or more evaluation indexes such as signal-to-noise ratio, underexposure degree, overexposure degree, contrast, etc. The present disclosure does not limit the manner of determining the target image to be fused.

[0168] In the case where the Gaussian pyramid of the target image to be fused is determined in step S82, the Laplacian pyramid of the target image to be fused can be obtained by using the Gaussian pyramid of the target image to be fused. Details of this process are not repeated here.

[0169] S88. Using the fused Gaussian pyramid and the Laplacian pyramid of the target image to be fused, a fused image corresponding to the plurality of images to be fused is generated.

[0170] In the case where the fused Gaussian pyramid and the Laplacian pyramid of the target image to be fused are determined, the electronic device can generate a fused image corresponding to the plurality of images to be fused by using the two.

[0171] In the image processing method of steps S82 to S86, by layer-by-layer fusion of the Gaussian pyramid, in addition to obtaining a fused image with better detail performance and smooth transition, low-frequency noise can also be effectively reduced, and the image quality is improved.

[0172] Further, the present disclosure also provides another image processing method.

[0173] Figure 9 A flowchart schematically showing an image processing method of another embodiment of the present disclosure is shown in FIG. 9. Referring to FIG. 9, the image processing method can include the following steps: Figure 9

[0174] S92. Obtain a plurality of images to be fused of the same shooting scene, and decompose each image to be fused to obtain a Laplacian pyramid of each image to be fused.

[0175] For each image to be fused and the process of obtaining the Laplacian pyramid of each image to be fused, the same as the content in step S52 described above, details are not repeated here.

[0176] S94. Fuse the pyramid images at the same layer in the Laplacian pyramid of each image to be fused to obtain a fused Laplacian pyramid.

[0177] ​The process of layer-by-layer fusion of the Laplacian pyramid here is the same as the process of step S56 above, i.e. similar to the process of fusing the first pyramid image and the second pyramid image in step S54 above, which will not be repeated here.

[0178] S96. Determine the Gaussian pyramid of a target image to be fused in the plurality of images to be fused.

[0179] In the exemplary embodiments of the present disclosure, the target image to be fused can be any one of the images to be fused; can also be the image to be fused with the largest, smallest or medium exposure degree corresponding to the original exposure; can also be the image to be fused with the highest image quality, which can be determined by one or more evaluation indexes such as signal-to-noise ratio, underexposure degree, overexposure degree, contrast, etc. The present disclosure does not limit the way of determining the target image to be fused.

[0180] The process of determining the Gaussian pyramid of the target image to be fused is similar to the relevant description in step S52, which will not be repeated here.

[0181] S98. Generate the fused image corresponding to the plurality of images to be fused by using the Gaussian pyramid of the target image to be fused and the fused Laplacian pyramid.

[0182] After the fused Laplacian pyramid and the Gaussian pyramid of the target image to be fused are determined, the electronic device can generate the fused image corresponding to the plurality of images to be fused by using the two.

[0183] In the image processing method of steps S92 to S96, by layer-by-layer fusion of the Laplacian pyramid, in addition to obtaining a fused image with better detail performance and smooth transition, the high-frequency noise can also be effectively reduced, and the image quality is improved.

[0184] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. In addition or alternatively, some steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of steps, etc.

[0185] Further, the present example embodiment also provides an image processing device.

[0186] Figure 10 The block diagram of the image processing device of the exemplary embodiments of the present disclosure is schematically shown. Referring to Figure 10According to an exemplary embodiment of the present disclosure, the image processing apparatus 10 can include an image decomposition module 101, a first fusion module 103, a second fusion module 105, and an image generation module 107.

[0187] Specifically, the image decomposition module 101 can be configured to acquire a plurality of to-be-fused images of a same shooting scene, decompose each to-be-fused image to obtain a Gaussian pyramid and a Laplacian pyramid of each to-be-fused image; the first fusion module 103 can be configured to fuse pyramid images at a same layer in the Gaussian pyramids of the to-be-fused images to obtain a fused Gaussian pyramid; the second fusion module 105 can be configured to fuse pyramid images at a same layer in the Laplacian pyramids of the to-be-fused images to obtain a fused Laplacian pyramid; and the image generation module 107 can be configured to generate a fused image corresponding to the plurality of to-be-fused images by using the fused Gaussian pyramid and the fused Laplacian pyramid.

[0188] According to an exemplary embodiment of the present disclosure, the image decomposition module 101 can be configured to acquire a plurality of to-be-fused images of a same shooting scene by performing the following: acquiring a plurality of original images captured based on different exposure levels under the same shooting scene; and performing brightness alignment on the plurality of original images based on a proportional relationship between exposure levels of the original images to obtain the plurality of to-be-fused images of the same shooting scene.

[0189] According to an exemplary embodiment of the present disclosure, the plurality of to-be-fused images include a first to-be-fused image and a second to-be-fused image, the exposure level of the original image corresponding to the first to-be-fused image is greater than the exposure level of the original image corresponding to the second to-be-fused image, the Gaussian pyramid of the first to-be-fused image includes a first pyramid image, and the image at a same layer as the first pyramid image in the Gaussian pyramid of the second to-be-fused image is a second pyramid image. In this case, the first fusion module 103 can be configured to fuse the first pyramid image and the second pyramid image by performing the following: determining a fusion weight of the first pyramid image and the second pyramid image; and fusing the first pyramid image and the second pyramid image by using the fusion weight.

[0190] According to an exemplary embodiment of the present disclosure, the first fusion module 103 can be configured to determine the fusion weight of the first pyramid image and the second pyramid image by performing the following: determining a difference between the first pyramid image and the second pyramid image, and determining the fusion weight of the first pyramid image and the second pyramid image based on the difference.

[0191] According to an example embodiment of the present disclosure, the first fusion module 103 can be configured to perform: determining a difference in pixel values between each pixel point in the first pyramid image and a pixel point corresponding in position in the second pyramid image to generate a first difference weight map; and determining a fusion weight of the first pyramid image and the second pyramid image using the first difference weight map.

[0192] According to an example embodiment of the present disclosure, the process of the first fusion module 103 determining a fusion weight of the first pyramid image and the second pyramid image using the first difference weight map can be configured to perform: mapping a weight value of a pixel point in the first difference weight map to a predetermined bit width range to obtain a second difference weight map; and performing normalization processing on the second difference weight map, and determining a result of the normalization processing as the fusion weight of the first pyramid image and the second pyramid image.

[0193] According to an example embodiment of the present disclosure, the process of the first fusion module 103 determining a fusion weight of the first pyramid image and the second pyramid image can be configured to perform: adjusting pixel data of the second pyramid image to a bit width consistent with a bit width of the first pyramid image to generate a third pyramid image; and determining the fusion weight of the first pyramid image and the second pyramid image according to the third pyramid image.

[0194] According to an example embodiment of the present disclosure, the process of the first fusion module 103 determining a fusion weight of the first pyramid image and the second pyramid image according to the third pyramid image can be configured to perform: performing mean filtering on the third pyramid image to obtain a fourth pyramid image; and determining the fusion weight of the first pyramid image and the second pyramid image according to the fourth pyramid image.

[0195] According to an example embodiment of the present disclosure, the process of the first fusion module 103 determining a fusion weight of the first pyramid image and the second pyramid image according to the fourth pyramid image can be configured to perform: determining a difference value of a bit width of the first pyramid image and a pixel value of each pixel point in the fourth pyramid image to obtain a first brightness weight map; and determining the fusion weight of the first pyramid image and the second pyramid image using the first brightness weight map.

[0196] According to an example embodiment of the present disclosure, the process of the first fusion module 103 determining a fusion weight of the first pyramid image and the second pyramid image using the first brightness weight map can be configured to perform: mapping a weight value of a pixel point in the first brightness weight map to a predetermined bit width range to obtain a second brightness weight map; and performing normalization processing on the second brightness weight map, and determining a result of the normalization processing as the fusion weight of the first pyramid image and the second pyramid image.

[0197] According to an example embodiment of the present disclosure, the process in which the first fusion module 103 determines the fusion weight of the first pyramid image and the second pyramid image according to the third pyramid image can be configured to perform: determining the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs, and generating a first local maximum weight map from the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs; and determining the fusion weight of the first pyramid image and the second pyramid image by using the first local maximum weight map.

[0198] According to an example embodiment of the present disclosure, the process in which the first fusion module 103 determines the fusion weight of the first pyramid image and the second pyramid image by using the first local maximum weight map can be configured to perform: mapping the weight value of the pixel point in the first local maximum weight map to a predetermined bit width range to obtain a second local maximum weight map; performing normalization processing on the second local maximum weight map, and determining the result of the normalization processing as the fusion weight of the first pyramid image and the second pyramid image.

[0199] According to an example embodiment of the present disclosure, the process in which the first fusion module 103 determines the fusion weight of the first pyramid image and the second pyramid image according to the third pyramid image can be configured to perform: performing mean filtering on the third pyramid image to obtain a fourth pyramid image; determining the difference value between the bit width of the first pyramid image and the pixel value of each pixel point in the fourth pyramid image to obtain a first brightness weight map; determining the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs, and generating a first local maximum weight map from the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs; and determining the fusion weight of the first pyramid image and the second pyramid image by using the first brightness weight map and the first local maximum weight map.

[0200] According to an example embodiment of the present disclosure, the process in which the first fusion module 103 determines the fusion weight of the first pyramid image and the second pyramid image by using the first brightness weight map and the first local maximum weight map can be configured to perform: mapping the weight value of the pixel point in the first brightness weight map to a predetermined bit width range to obtain a second brightness weight map; mapping the weight value of the pixel point in the first local maximum weight map to a predetermined bit width range to obtain a second local maximum weight map; multiplying the second brightness weight map and the second local maximum weight map, performing normalization processing on the multiplied result, and determining the result of the normalization processing as the fusion weight of the first pyramid image and the second pyramid image.

[0201] According to an exemplary embodiment of the present disclosure, the process of determining the fusion weight of the first pyramid image and the second pyramid image by the first fusion module 103 may be configured to perform the following steps: adjusting the pixel data of the second pyramid image to have a bit width consistent with that of the first pyramid image to generate a third pyramid image; and determining the fusion weight of the first pyramid image and the second pyramid image based on the first pyramid image, the second pyramid image, and the third pyramid image.

[0202] According to an exemplary embodiment of the present disclosure, the process of the first fusion module 103 determining the fusion weight of the first pyramid image and the second pyramid image based on the first pyramid image, the second pyramid image, and the third pyramid image can be configured to perform: determining the difference in pixel value between each pixel point in the first pyramid image and the corresponding pixel point in the second pyramid image to generate a first difference weight map; and determining the fusion weight of the first pyramid image and the second pyramid image based on the first difference weight map and the third pyramid image.

[0203] According to an exemplary embodiment of the present disclosure, the process in which the first fusion module 103 determines the difference in pixel value between each pixel in the first pyramid image and the corresponding pixel in the second pyramid image can be configured to perform the following steps: determining neighboring pixels of a first pixel in the first pyramid image, where the first pixel and its neighboring pixels constitute a first pixel region; determining a second pixel in the second pyramid image corresponding to the first pixel, and determining neighboring pixels of the second pixel, where the second pixel and its neighboring pixels constitute a second pixel region; and determining a difference in pixel value between the first pixel region and the second pixel region, which is used as the difference in pixel value between the first pixel and the second pixel.

[0204] According to an exemplary embodiment of the present disclosure, the process of the first fusion module 103 determining the difference in pixel values ​​between the first pixel area and the second pixel area can be configured to perform: traversing each pixel point in the first pixel area, determining the difference in pixel values ​​between the pixel point in the first pixel area and the pixel point corresponding to the position in the second pixel area; determining the difference in pixel values ​​between the first pixel area and the second pixel area based on the difference in pixel values ​​between the pixel point in the first pixel area and the pixel point corresponding to the position in the second pixel area.

[0205] According to an example embodiment of the present disclosure, the process in which the first fusion module 103 determines the fusion weight of the first pyramid image and the second pyramid image according to the first difference weight map and the third pyramid image can be configured to perform: mean filtering the third pyramid image to obtain a fourth pyramid image; determining the difference between the bit width of the first pyramid image and the pixel value of each pixel point in the fourth pyramid image to obtain a first brightness weight map; and determining the fusion weight of the first pyramid image and the second pyramid image using the first brightness weight map and the first difference weight map.

[0206] According to an example embodiment of the present disclosure, the process in which the first fusion module 103 determines the fusion weight of the first pyramid image and the second pyramid image using the first brightness weight map and the first difference weight map can be configured to perform: mapping the weight value of each pixel point in the first brightness weight map to a predetermined bit width range to obtain a second brightness weight map; mapping the weight value of each pixel point in the first difference weight map to a predetermined bit width range to obtain a second difference weight map; multiplying the second brightness weight map and the second difference weight map, normalizing the multiplication result, and determining the normalized result as the fusion weight of the first pyramid image and the second pyramid image.

[0207] According to an example embodiment of the present disclosure, the process in which the first fusion module 103 determines the fusion weight of the first pyramid image and the second pyramid image according to the first difference weight map and the third pyramid image can be configured to perform: determining the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs, and generating a first local maximum weight map from the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs; and determining the fusion weight of the first pyramid image and the second pyramid image using the first local maximum weight map and the first difference weight map.

[0208] According to an example embodiment of the present disclosure, the process in which the first fusion module 103 determines the fusion weight of the first pyramid image and the second pyramid image using the first local maximum weight map and the first difference weight map can be configured to perform: mapping the weight value of each pixel point in the first local maximum weight map to a predetermined bit width range to obtain a second local maximum weight map; mapping the weight value of each pixel point in the first difference weight map to a predetermined bit width range to obtain a second difference weight map; multiplying the second local maximum weight map and the second difference weight map, normalizing the multiplication result, and determining the normalized result as the fusion weight of the first pyramid image and the second pyramid image.

[0209] According to the exemplary embodiments of the present disclosure, the process that the first fusion module 103 determines the fusion weight of the first pyramid image and the second pyramid image according to the first difference weight map and the third pyramid image can be configured to perform: mean filtering the third pyramid image to obtain a fourth pyramid image; determining the difference between the bit width of the first pyramid image and the pixel value of each pixel point in the fourth pyramid image to obtain a first brightness weight map; determining the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs, and generating a first local maximum weight map from the maximum value of the pixel value in the Bayer unit to which each pixel point in the third pyramid image belongs; and determining the fusion weight of the first pyramid image and the second pyramid image by using the first difference weight map, the first brightness weight map and the first local maximum weight map.

[0210] According to the exemplary embodiments of the present disclosure, the process that the first fusion module 103 determines the fusion weight of the first pyramid image and the second pyramid image by using the first difference weight map, the first brightness weight map and the first local maximum weight map can be configured to perform: mapping the weight value of each pixel point in the first difference weight map to a predetermined bit width range to obtain a second difference weight map; mapping the weight value of each pixel point in the first brightness weight map to a predetermined bit width range to obtain a second brightness weight map; mapping the weight value of each pixel point in the first local maximum weight map to a predetermined bit width range to obtain a second local maximum weight map; multiplying the second difference weight map, the second brightness weight map and the second local maximum weight map, normalizing the multiplication result, and determining the normalized result as the fusion weight of the first pyramid image and the second pyramid image.

[0211] Further, another image processing device is also provided in the exemplary embodiments of the present disclosure.

[0212] Figure 11 The block diagram of the image processing device of another exemplary embodiment of the present disclosure is schematically shown. Referring to Figure 11 According to the exemplary embodiments of the present disclosure, the image processing device 11 can include an image decomposition module 111, a hierarchical fusion module 113, a Laplacian pyramid determination module 115 and an image generation module 117.

[0213] Specifically, the image decomposition module 111 can be configured to acquire a plurality of to-be-fused images of a same shooting scene, decompose each to-be-fused image to obtain a Gaussian pyramid of each to-be-fused image; the hierarchical fusion module 113 can be configured to fuse pyramid images located at a same layer in the Gaussian pyramids of the to-be-fused images to obtain a fused Gaussian pyramid; the Laplacian pyramid determination module 115 can be configured to determine a Laplacian pyramid of a target to-be-fused image in the plurality of to-be-fused images; and the image generation module 117 can be configured to generate a fused image corresponding to the plurality of to-be-fused images by using the fused Gaussian pyramid and the Laplacian pyramid of the target to-be-fused image.

[0214] The hierarchical fusion module 113 performs the process of layer-by-layer pyramid image fusion similar to the process of fusing the first pyramid image and the second pyramid image by the first fusion module 103, and thus details are not repeated here.

[0215] Further, the present disclosure also provides another image processing device in an exemplary embodiment.

[0216] Figure 12 An exemplary block diagram of an image processing device according to another exemplary embodiment of the present disclosure is shown schematically. Referring to Figure 12 The image processing device 12 according to an exemplary embodiment of the present disclosure can include an image decomposition module 121, a hierarchical fusion module 123, a Gaussian pyramid determination module 125, and an image generation module 127.

[0217] Specifically, the image decomposition module 121 can be configured to acquire a plurality of to-be-fused images of a same shooting scene, decompose each to-be-fused image to obtain a Laplacian pyramid of each to-be-fused image; the hierarchical fusion module 123 can be configured to fuse pyramid images located at a same layer in the Laplacian pyramids of the to-be-fused images to obtain a fused Laplacian pyramid; the Gaussian pyramid determination module 125 can be configured to determine a Gaussian pyramid of a target to-be-fused image in the plurality of to-be-fused images; and the image generation module 127 can be configured to generate a fused image corresponding to the plurality of to-be-fused images by using the Gaussian pyramid of the target to-be-fused image and the fused Laplacian pyramid.

[0218] The hierarchical fusion module 123 performs the process of layer-by-layer pyramid image fusion similar to the process of fusing the first pyramid image and the second pyramid image by the first fusion module 103, and thus details are not repeated here.

[0219] Since the functions of each functional module of the image processing device according to the present embodiment are the same as those in the above method embodiment, details are not repeated here.

[0220] Figure 13Schematic diagram of an electronic device suitable for implementing the exemplary embodiment of the present disclosure is shown. Figure 13 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0221] The electronic device of the present disclosure includes at least a processor and a memory, wherein the memory is used to store one or more programs. When the one or more programs are executed by the processor, the processor can implement the image processing method of the exemplary embodiment of the present disclosure.

[0222] Specifically, such as Figure 13 As shown, the electronic device 130 may include: a processor 1310, an internal memory 1321, an external memory interface 1322, a Universal Serial Bus (USB) interface 1330, a charging management module 1340, a power management module 1341, a battery 1342, antenna 1, antenna 2, a mobile communication module 1350, a wireless communication module 1360, an audio module 1370, a sensor module 1380, a display 1390, a camera module 1391, an indicator 1392, a motor 1393, a button 1394, and a Subscriber Identification Module (SIM) card interface 1395. The sensor module 1380 may include a depth sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, and a bone conduction sensor.

[0223] It should be understood that the structures illustrated in the embodiments of the present disclosure do not constitute a specific limitation on the electronic device 130. In other embodiments of the present disclosure, the electronic device 130 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0224] The processor 1310 can include one or more processing units, for example: the processor 1310 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors. In addition, the processor 1310 can also be provided with a memory for storing instructions and data.

[0225] The electronic device 130 can realize the photographing function through the ISP, the camera module 1391, the video codec, the GPU, the display screen 1390, and the application processor, etc. In some embodiments, the electronic device 130 can include 1 or N camera modules 1391, N is a positive integer greater than 1, if the electronic device 130 includes N cameras, one of the N cameras is a main camera.

[0226] The internal memory 1321 can be used to store computer executable program codes, the executable program codes including instructions. The internal memory 1321 can include a program storage area and a data storage area. The external memory interface 1322 can be used to connect an external memory card, such as a Micro SD card, to realize the expansion of the storage capacity of the electronic device 130.

[0227] The present disclosure also provides a computer readable storage medium, which can be included in the electronic device described in the above embodiments; or can exist independently without being assembled into the electronic device.

[0228] The computer readable storage medium may, for example, be tangible or intransitory and may include an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a computer readable storage medium can include, but are not limited to, the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0229] A computer readable storage medium can send, propagate or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0230] A computer readable storage medium can send, propagate or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0231] The flow diagrams and the block diagrams in the drawings are illustrations of possible architectures, functions, and operations for systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0232] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware, or be implemented by a combination of software and hardware. The units described may, in some cases, not be implemented by a single software code or hardware component, but can be implemented by a combination of software codes or hardware components.

[0233] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0234] Furthermore, the above-mentioned figures are merely illustrative of the processes included in the methods according to exemplary embodiments of the present disclosure and are not intended to be limiting. It is readily understood that the processes illustrated in the above-mentioned figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0235] It should be noted that although several modules or units of the device for action execution are mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0236] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing what is disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

[0237] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized by, The method comprises the following steps: obtaining a plurality of to-be-fused images of the same shooting scene, decomposing each of the to-be-fused images to obtain a Gaussian pyramid and a Laplacian pyramid of each of the to-be-fused images; fusing the pyramid images at the same layer in the Gaussian pyramid of each of the to-be-fused images to obtain a fused Gaussian pyramid; fusing the pyramid images at the same layer in the Laplacian pyramid of each of the to-be-fused images to obtain a fused Laplacian pyramid; generating a fused image corresponding to the plurality of to-be-fused images by using the fused Gaussian pyramid and the fused Laplacian pyramid.

2. The image processing method of claim 1, wherein, The method comprises the following steps: obtaining a plurality of original images collected based on different exposure degrees under the same shooting scene; aligning the brightness of the plurality of original images by using the proportional relationship of the exposure degrees between the original images to obtain a plurality of to-be-fused images of the same shooting scene.

3. The image processing method of claim 2, wherein, The plurality of to-be-fused images comprise a first to-be-fused image and a second to-be-fused image, the exposure degree of the original image corresponding to the first to-be-fused image is greater than the exposure degree of the original image corresponding to the second to-be-fused image, the Gaussian pyramid of the first to-be-fused image comprises a first pyramid image, and the image at the same layer in the Gaussian pyramid of the second to-be-fused image as the first pyramid image is a second pyramid image; wherein fusing the first pyramid image and the second pyramid image comprises: determining the fusion weight of the first pyramid image and the second pyramid image; fusing the first pyramid image and the second pyramid image by using the fusion weight.

4. The image processing method of claim 3, wherein, The method comprises the following steps: determining the difference between the first pyramid image and the second pyramid image, and determining the fusion weight of the first pyramid image and the second pyramid image according to the difference.

5. The image processing method of claim 4, wherein, The method comprises the following steps: determining the difference between the first pyramid image and the second pyramid image, and determining the fusion weight of the first pyramid image and the second pyramid image according to the difference. The method comprises the following steps:

6. The image processing method of claim 5, wherein, determining the difference between each pixel point in the first pyramid image and the pixel point at the corresponding position in the second pyramid image to generate a first difference weight map; determining the fusion weight of the first pyramid image and the second pyramid image by using the first difference weight map. The method comprises the following steps:

7. The image processing method of claim 3, wherein, mapping the pixel weight value in the first difference weight map to a predetermined bit width range to obtain a second difference weight map; normalizing the second difference weight map, and determining the result of the normalization as the fusion weight of the first pyramid image and the second pyramid image. The method comprises the following steps: adjusting the pixel data of the second pyramid image to be consistent with the bit width of the first pyramid image to generate a third pyramid image; determining fusion weights of the first pyramid image and the second pyramid image according to the third pyramid image.

8. The image processing method according to claim 7, wherein: determining fusion weights of the first pyramid image and the second pyramid image according to the third pyramid image comprises: performing mean filtering on the third pyramid image to obtain a fourth pyramid image; determining fusion weights of the first pyramid image and the second pyramid image according to the fourth pyramid image.

9. The image processing method of claim 8, wherein, determining fusion weights of the first pyramid image and the second pyramid image according to the fourth pyramid image comprises: determining a difference between a bit width of the first pyramid image and pixel values of each pixel point in the fourth pyramid image to obtain a first brightness weight map; determining fusion weights of the first pyramid image and the second pyramid image by using the first brightness weight map.

10. The image processing method of claim 9, wherein, determining fusion weights of the first pyramid image and the second pyramid image by using the first brightness weight map comprises: mapping pixel weight values in the first brightness weight map to a predetermined bit width range to obtain a second brightness weight map; performing normalization processing on the second brightness weight map, and determining a result of the normalization processing as the fusion weights of the first pyramid image and the second pyramid image.

11. The image processing method of claim 7, wherein, determining fusion weights of the first pyramid image and the second pyramid image according to the third pyramid image comprises: determining a maximum value of pixel values in a Bayer cell to which each pixel point in the third pyramid image belongs, and generating a first local maximum weight map from the maximum value of pixel values in the Bayer cell to which each pixel point in the third pyramid image belongs; determining fusion weights of the first pyramid image and the second pyramid image by using the first local maximum weight map.

12. The image processing method of claim 11, wherein, determining fusion weights of the first pyramid image and the second pyramid image by using the first local maximum weight map comprises: mapping pixel weight values in the first local maximum weight map to a predetermined bit width range to obtain a second local maximum weight map; performing normalization processing on the second local maximum weight map, and determining a result of the normalization processing as the fusion weights of the first pyramid image and the second pyramid image.

13. The image processing method of claim 7, wherein, determining fusion weights of the first pyramid image and the second pyramid image according to the third pyramid image comprises: performing mean filtering on the third pyramid image to obtain a fourth pyramid image; determining a difference between a bit width of the first pyramid image and pixel values of each pixel point in the fourth pyramid image to obtain a first brightness weight map; determining a maximum value of pixel values in a Bayer cell to which each pixel point in the third pyramid image belongs, and generating a first local maximum weight map from the maximum value of pixel values in the Bayer cell to which each pixel point in the third pyramid image belongs; determining fusion weights of the first pyramid image and the second pyramid image by using the first brightness weight map and the first local maximum weight map.

14. The image processing method of claim 13, wherein, Determining the fusion weight of the first pyramid image and the second pyramid image according to the first pyramid image, the second pyramid image and the third pyramid image comprises: mapping the pixel point weight value in the first luminance weight graph to a predetermined bit width range to obtain a second luminance weight graph; mapping the pixel point weight value in the first local maximum weight graph to the predetermined bit width range to obtain a second local maximum weight graph; multiplying the second luminance weight graph and the second local maximum weight graph, normalizing the multiplication result, and determining the normalized result as the fusion weight of the first pyramid image and the second pyramid image.

15. The image processing method of claim 3, wherein, Determining the fusion weight of the first pyramid image and the second pyramid image comprises: adjusting the pixel data of the second pyramid image to a bit width consistent with the bit width of the first pyramid image to generate a third pyramid image; determining the fusion weight of the first pyramid image and the second pyramid image according to the first pyramid image, the second pyramid image and the third pyramid image.

16. The image processing method of claim 15, wherein, Determining the fusion weight of the first pyramid image and the second pyramid image according to the first pyramid image, the second pyramid image and the third pyramid image comprises: determining the difference of pixel values between each pixel point in the first pyramid image and the pixel point corresponding in position in the second pyramid image to generate a first difference weight graph; determining the fusion weight of the first pyramid image and the second pyramid image according to the first difference weight graph and the third pyramid image.

17. The image processing method of claim 5 or 16, wherein, Determining the difference of pixel values between each pixel point in the first pyramid image and the pixel point corresponding in position in the second pyramid image comprises: determining the adjacent pixel points of a first pixel point in the first pyramid image, the first pixel point and the adjacent pixel points of the first pixel point forming a first pixel region; determining a second pixel point corresponding in position to the first pixel point in the second pyramid image, and determining the adjacent pixel points of the second pixel point, the second pixel point and the adjacent pixel points of the second pixel point forming a second pixel region; determining the difference of pixel values between the first pixel region and the second pixel region as the difference of pixel values between the first pixel point and the second pixel point.

18. The image processing method of claim 17, wherein, Determining the difference of pixel values between the first pixel region and the second pixel region comprises: traversing each pixel point in the first pixel region to determine the difference of pixel values between the pixel points in the first pixel region and the pixel points corresponding in position in the second pixel region; determining the difference of pixel values between the first pixel region and the second pixel region according to the difference of pixel values between the pixel points in the first pixel region and the pixel points corresponding in position in the second pixel region.

19. The image processing method of claim 16, wherein, Determining the fusion weight of the first pyramid image and the second pyramid image according to the first difference weight graph and the third pyramid image comprises: performing mean filtering on the third pyramid image to obtain a fourth pyramid image; determining a difference between a bit width of the first pyramid image and pixel values of each pixel point in the fourth pyramid image to obtain a first brightness weight map; determining a fusion weight of the first pyramid image and the second pyramid image according to the first difference weight map and the third pyramid image includes:

20. The image processing method of claim 19, wherein, mapping a weight value of each pixel point in the first difference weight map to a predetermined bit width range to obtain a second difference weight map; multiplying the second local maximum weight map and the second difference weight map, performing normalization processing on a multiplication result, and determining a normalization processing result as the fusion weight of the first pyramid image and the second pyramid image. determining a difference between a bit width of the first pyramid image and pixel values of each pixel point in the fourth pyramid image to obtain a first brightness weight map; determining a fusion weight of the first pyramid image and the second pyramid image according to the first difference weight map and the third pyramid image includes:

21. The image processing method of claim 16, wherein, determining a maximum value of pixel values in a Bayer unit to which each pixel point in the third pyramid image belongs, and generating a first local maximum weight map from the maximum value of pixel values in the Bayer unit to which each pixel point in the third pyramid image belongs; determining a fusion weight of the first pyramid image and the second pyramid image according to the first difference weight map and the third pyramid image includes: mapping a weight value of each pixel point in the first difference weight map to a predetermined bit width range to obtain a second difference weight map; 22. The image processing method of claim 21, wherein, multiplying the second local maximum weight map and the second difference weight map, performing normalization processing on a multiplication result, and determining a normalization processing result as the fusion weight of the first pyramid image and the second pyramid image. determining a difference between a bit width of the first pyramid image and pixel values of each pixel point in the fourth pyramid image to obtain a first brightness weight map; determining a fusion weight of the first pyramid image and the second pyramid image according to the first difference weight map and the third pyramid image includes: performing mean filtering on the third pyramid image to obtain a fourth pyramid image; 23. The image processing method of claim 16, wherein, determining a difference between a bit width of the first pyramid image and pixel values of each pixel point in the fourth pyramid image to obtain a first brightness weight map; determining a maximum value of pixel values in a Bayer unit to which each pixel point in the third pyramid image belongs, and generating a first local maximum weight map from the maximum value of pixel values in the Bayer unit to which each pixel point in the third pyramid image belongs; determining a fusion weight of the first pyramid image and the second pyramid image according to the first difference weight map and the third pyramid image includes: mapping a weight value of each pixel point in the first difference weight map to a predetermined bit width range to obtain a second difference weight map; multiplying the second local maximum weight map and the second difference weight map, performing normalization processing on a multiplication result, and determining a normalization processing result as the fusion weight of the first pyramid image and the second pyramid image.

24. The image processing method of claim 23, wherein, Determining the fusion weight of the first pyramid image and the second pyramid image by using the first difference weight map, the first brightness weight map, and the first local maximum weight map includes: Mapping the pixel weight values ​​in the first difference weight map to a predetermined bit width range to obtain a second difference weight map; Mapping the pixel weight values ​​in the first luminance weight map to the predetermined bit width range to obtain a second luminance weight map; Mapping the pixel weight values ​​in the first local maximum weight map to the predetermined bit width range to obtain a second local maximum weight map; The second difference weight map, the second brightness weight map, and the second local maximum weight map are multiplied, a multiplication result is normalized, and the normalization result is determined as a fusion weight of the first pyramid image and the second pyramid image.

25. An image processing method, characterized by, include: Acquire multiple images to be fused of the same shooting scene, and decompose each of the images to be fused to obtain a Gaussian pyramid of each of the images to be fused; fusing the pyramid images at the same layer in the Gaussian pyramids of the images to be fused to obtain a fused Gaussian pyramid; Determining a Laplacian pyramid of a target image to be fused among the multiple images to be fused; A fused image corresponding to the multiple images to be fused is generated by using the fused Gaussian pyramid and the Laplacian pyramid of the target image to be fused.

26. An image processing method, characterized by, include: Acquire multiple images to be fused of the same shooting scene, and decompose each of the images to be fused to obtain a Laplacian pyramid of each of the images to be fused; Fusing the pyramid images located at the same layer in the Laplacian pyramids of the images to be fused to obtain a fused Laplacian pyramid; Determining a Gaussian pyramid of a target image to be fused among the multiple images to be fused; A fused image corresponding to the multiple images to be fused is generated by using the Gaussian pyramid of the target image to be fused and the fused Laplacian pyramid.

27. An image processing apparatus characterized by comprising: include: An image decomposition module is used to obtain multiple images to be fused of the same shooting scene, and decompose each of the images to be fused to obtain a Gaussian pyramid and a Laplacian pyramid of each of the images to be fused; A first fusion module is used to fuse the pyramid images located at the same layer in the Gaussian pyramids of the images to be fused to obtain a fused Gaussian pyramid; A second fusion module is used to fuse the pyramid images located at the same layer in the Laplacian pyramid of the images to be fused to obtain a fused Laplacian pyramid; An image generation module is configured to generate a fused image corresponding to the plurality of images to be fused by using the fused Gaussian pyramid and the fused Laplacian pyramid.

28. An image processing apparatus characterized by comprising: include: An image decomposition module is used to obtain multiple images to be fused of the same shooting scene, and decompose each of the images to be fused to obtain a Gaussian pyramid of each of the images to be fused; a layered fusion module, configured to fuse the pyramid images at the same layer in the Gaussian pyramids of the images to be fused to obtain a fused Gaussian pyramid; a Laplacian pyramid determination module configured to determine a Laplacian pyramid of a target image to be fused from the plurality of images to be fused; an image generation module configured to generate a fused image corresponding to the plurality of images to be fused by using the fused Gaussian pyramid and the Laplacian pyramid of the target image to be fused.

29. An image processing apparatus characterized by comprising: The method comprises: an image decomposition module configured to obtain a plurality of images to be fused of a same shooting scene, and decompose each of the images to be fused to obtain a Laplacian pyramid of each of the images to be fused; a hierarchical fusion module configured to fuse pyramid images at a same layer of the Laplacian pyramids of the images to be fused to obtain a fused Laplacian pyramid; a Gaussian pyramid determination module configured to determine a Gaussian pyramid of a target image to be fused from the plurality of images to be fused; an image generation module configured to generate a fused image corresponding to the plurality of images to be fused by using the Gaussian pyramid of the target image to be fused and the fused Laplacian pyramid.

30. A computer readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the image processing method according to any one of claims 1 to 26.

31. An electronic device, comprising: The method comprises: a processor; a memory configured to store one or more programs, which, when executed by the processor, cause the processor to implement the image processing method according to any one of claims 1 to 26.

Images