Shooting method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to a shooting method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: obtaining a first image and a second image collected under different exposure durations in response to a detected shooting instruction; wherein the exposure duration of the first image is greater than the exposure duration of the second image; performing brightness alignment based on the first image and the second image to obtain a second image after brightness alignment; wherein the difference between the brightness distribution of the second image after brightness alignment and the brightness distribution of the first image is smaller than a preset brightness difference threshold value; performing image registration on the first image by taking the second image after brightness alignment as a reference to obtain a registered first image; and based on the registered first image and the second image after brightness alignment, carrying out fusion processing to obtain a fused target image and outputting the fused target image. Through the method, the quality of the target image output after fusion can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image shooting, and in particular to a shooting method and device, electronic device, storage medium and program product. BACKGROUND

[0002] High Dynamic Range Imaging (HDR) technology provides a more realistic and rich visual experience by showing a wider range of brightness and higher contrast. At present, this technology has been widely used in television, film, vehicle-mounted camera, network video, photography, and other fields, and also shows great potential in emerging fields such as virtual reality and augmented reality. Due to hardware cost constraints, the HDR in image shooting currently mostly adopts a multi-exposure image fusion method, but the quality of the fused image is poor, for example, artifacts and smearing are prone to occur. SUMMARY

[0003] To overcome the problems in the related art, the present disclosure provides a shooting method and device, electronic device, storage medium and program product, which can improve the quality of the shot image.

[0004] According to a first aspect of an embodiment of the present disclosure, a shooting method is provided, comprising:

[0005] In response to detecting a shooting instruction, a first image and a second image collected under different exposure times are obtained; wherein the exposure time of the first image is greater than the exposure time of the second image;

[0006] Performing brightness alignment based on the first image and the second image to obtain a second image after brightness alignment; wherein the difference between the brightness distribution of the second image after brightness alignment and the brightness distribution of the first image is less than a preset brightness difference threshold;

[0007] Performing image registration on the first image with the second image after brightness alignment as a reference to obtain a first image after registration;

[0008] Performing fusion processing based on the first image after registration and the second image after brightness alignment to obtain a target image after fusion and output.

[0009] In some embodiments, the fusion processing based on the first image after registration and the second image after brightness alignment to obtain a target image after fusion and output comprises:

[0010] Determining image blocks that match in the first image after registration and the second image after brightness alignment;

[0011] fuse the registered first image and the brightness-aligned second image based on each group of matched image blocks to obtain the target image and output the target image.

[0012] In some embodiments, the fusing the registered first image and the brightness-aligned second image based on each group of matched image blocks to obtain the target image and output the target image comprises:

[0013] For each group of matched image blocks, determining a fusion weight of the matched image blocks based on a comparison result of brightness of the image block belonging to the registered first image and a preset brightness threshold, and / or based on a motion displacement detection result between the matched image blocks;

[0014] fusing the registered first image and the brightness-aligned second image based on each group of matched image blocks and the corresponding fusion weight to obtain the target image and output the target image.

[0015] In some embodiments, the determining the fusion weight of the matched image blocks based on the comparison result of brightness of the image block belonging to the registered first image and the preset brightness threshold, and based on the motion displacement detection result between the matched image blocks comprises:

[0016] in response to the brightness of the image block belonging to the registered first image being greater than the preset brightness threshold, determining that the weight of the image block in the brightness-aligned second image is greater than the weight of the image block in the registered first image;

[0017] in response to the brightness of the image block belonging to the registered first image being less than or equal to the preset brightness threshold, determining the fusion weight of the matched image blocks based on the motion displacement detection result between the matched image blocks.

[0018] In some embodiments, the determining the fusion weight of the image block based on the motion displacement detection result between the matched image blocks comprises:

[0019] in response to the existence of the motion displacement between the matched image blocks, determining that the weight of the image block in the brightness-aligned second image is greater than the weight of the image block in the registered first image;

[0020] in response to the non-existence of the motion displacement between the matched image blocks, determining that the weight of the image block in the brightness-aligned second image is less than the weight of the image block in the registered first image.

[0021] In some embodiments, the method further comprises:

[0022] determining a difference between the matched image blocks;

[0023] in response to the difference being greater than a preset difference threshold, determining that there is a motion displacement between the matched image blocks;

[0024] in response to the difference being less than or equal to a preset difference threshold, determining that there is no motion displacement between the matched image blocks.

[0025] In some embodiments, the determining the matched image blocks in the registered first image and the brightness-aligned second image comprises:

[0026] determining, in the registered first image, reference image blocks;

[0027] for each reference image block, performing a similarity comparison between the reference image block and each search image block in a preset search region of the brightness-aligned second image, and determining a search target image block satisfying a preset similarity condition;

[0028] determining each reference image block and the search target image block corresponding to the reference image block as a set of matched image blocks.

[0029] In some embodiments, the brightness alignment based on the first image and the second image comprises:

[0030] determining a brightness adjustment coefficient based on a first exposure parameter of the first image and a second exposure parameter of the second image;

[0031] performing brightness alignment on the second image based on the brightness adjustment coefficient to obtain the brightness-aligned second image.

[0032] In some embodiments, the brightness alignment based on the first image and the second image comprises:

[0033] performing bad pixel correction on the first image and the second image respectively, and performing black level correction on the bad pixel corrected images;

[0034] performing brightness alignment based on the black level corrected first image and the black level corrected second image to obtain the brightness-aligned second image.

[0035] According to a second aspect of the embodiments of the present disclosure, a photographing device is provided, comprising:

[0036] an acquisition module configured to, in response to detecting a photographing instruction, acquire a first image and a second image collected under different exposure durations; wherein an exposure duration of the first image is greater than an exposure duration of the second image;

[0037] a brightness alignment module configured to perform brightness alignment based on the first image and the second image to obtain a brightness-aligned second image, wherein a difference between a brightness distribution of the brightness-aligned second image and a brightness distribution of the first image is less than a preset brightness difference threshold;

[0038] a registration module configured to perform image registration on the first image based on the brightness-aligned second image to obtain a registered first image;

[0039] a fusion module configured to perform fusion processing based on the registered first image and the brightness-aligned second image to obtain a fused target image and output the fused target image.

[0040] In some embodiments, the fusion module is further configured to determine matching image blocks in the registered first image and the brightness-aligned second image, and perform fusion processing on the registered first image and the brightness-aligned second image based on each group of matching image blocks to obtain and output the target image.

[0041] In some embodiments, the fusion module is further configured to, for each group of matching image blocks, determine a fusion weight of the matching image blocks based on a comparison result of brightness of image blocks belonging to the registered first image and a preset brightness threshold, and / or based on a motion displacement detection result between the matching image blocks, and perform fusion processing on the registered first image and the brightness-aligned second image based on each group of matching image blocks and the corresponding fusion weights to obtain and output the target image.

[0042] In some embodiments, the fusion module is further configured to, in response to brightness of image blocks belonging to the registered first image being greater than the preset brightness threshold, determine a weight of image blocks in the brightness-aligned second image to be greater than a weight of image blocks in the registered first image, and in response to brightness of image blocks belonging to the registered first image being less than or equal to the preset brightness threshold, determine a fusion weight of the matching image blocks based on a motion displacement detection result between the matching image blocks.

[0043] In some embodiments, the fusion module is further configured to, in response to there being a motion displacement between the matching image blocks, determine a weight of image blocks in the brightness-aligned second image to be greater than a weight of image blocks in the registered first image, and in response to there being no motion displacement between the matching image blocks, determine a weight of image blocks in the brightness-aligned second image to be less than a weight of image blocks in the registered first image.

[0044] In some embodiments, the apparatus further comprises:

[0045] a motion detection module, configured to determine a difference between the matched image blocks; in response to the difference being greater than a preset difference threshold, determine that there is a motion displacement between the matched image blocks; in response to the difference being less than or equal to the preset difference threshold, determine that there is no motion displacement between the matched image blocks.

[0046] In some embodiments, the fusion module is further configured to determine, in the registered first image, reference image blocks; for each reference image block, perform a similarity comparison between the reference image block and each search image block in a preset search region in the brightness-aligned second image, and determine a search target image block that satisfies a preset similarity condition; and determine each reference image block and the search target image block corresponding to the reference image block as a set of matched image blocks.

[0047] In some embodiments, the brightness alignment module is further configured to determine a brightness adjustment coefficient based on a first exposure parameter of the first image and a second exposure parameter of the second image; and perform brightness alignment on the second image based on the brightness adjustment coefficient to obtain the brightness-aligned second image.

[0048] In some embodiments, the brightness alignment module is further configured to perform bad pixel correction on the first image and the second image respectively, and perform black level correction on the bad pixel corrected images; and perform brightness alignment based on the black level corrected first image and the black level corrected second image to obtain the brightness-aligned second image.

[0049] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, comprising:

[0050] a processor;

[0051] a memory for storing computer programs or instructions;

[0052] The processor executes the computer programs or instructions to implement the steps of the method according to the first aspect.

[0053] According to a fourth aspect of the embodiments of the present disclosure, a non-transitory computer readable storage medium is provided, which stores computer programs or instructions, and when the computer programs or instructions in the storage medium are executed by a processor, the steps of the method according to the first aspect are implemented.

[0054] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, which comprises computer programs or instructions, and when the computer programs or instructions are executed by a processor, the steps of the method according to the first aspect are implemented.

[0055] The technical scheme provided by the embodiments of the present disclosure can include the following beneficial effects:

[0056] In the embodiments of the present disclosure, when the electronic device fuses images with different exposure times to obtain a target image, the brightness of a second image with a shorter exposure time is first aligned to the brightness of a first image with a longer exposure time, so that subsequent image fusion is more natural, and the occurrence of brightness unevenness and discontinuity is reduced, and more dark details can also be retained. In addition, after the brightness alignment, the first image is registered with the second image after the brightness alignment, and the registration is followed by fusion, which can improve the quality of the target image output after fusion. In particular, for spatial position differences caused by motion, registration based on the second image after brightness alignment can also reduce registration errors caused by motion.

[0057] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0058] The accompanying drawings, which are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0059] Figure 1 is an example of a fused image.

[0060] Figure 2 is a flowchart of a photographing method according to an exemplary embodiment.

[0061] Figure 3 is a principle diagram of a motion detection process in the embodiments of the present disclosure.

[0062] Figure 4 is a search example diagram of a matching image block in the embodiments of the present disclosure.

[0063] Figure 5 is a principle diagram of processing a raw image collected by an image sensor in the embodiments of the present disclosure.

[0064] Figure 6 is a block diagram of a photographing apparatus according to an exemplary embodiment.

[0065] Figure 7 is a structural block diagram of an apparatus according to an exemplary embodiment. DETAILED DESCRIPTION

[0066] The exemplary embodiments will be described in detail below with reference to the accompanying drawings. The following description is presented in connection with the accompanying drawings in which the same numbers are used in different drawings to refer to the same or similar elements. The implementations described in the following exemplary embodiments are not meant to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0067] In the related art, when multi-exposure image fusion is performed using an HDR technology, due to handheld shooting device shaking or object motion, ghosting may exist in the fused image. Figure 1 is an example of a fused image, as Figure 1 shown, due to shaking when taking a picture with a mobile phone, the object edge shown by 101 has obvious artifacts.

[0068] To this end, the embodiments of the present disclosure disclose a photographing method, Figure 2 is a flowchart of a photographing method according to an exemplary embodiment, as Figure 2 shown, the method mainly includes the following steps:

[0069] S201, in response to detecting a photographing instruction, acquiring a first image and a second image collected under different exposure times; wherein the exposure time of the first image is greater than the exposure time of the second image;

[0070] S202, performing brightness alignment based on the first image and the second image to obtain a brightness-aligned second image; wherein the difference between the brightness distribution of the brightness-aligned second image and the brightness distribution of the first image is less than a preset brightness difference threshold;

[0071] S203, performing image registration on the first image based on the brightness-aligned second image as a reference to obtain a registered first image;

[0072] S204, performing fusion processing based on the registered first image and the brightness-aligned second image to obtain a fused target image and output.

[0073] The photographing method provided by the embodiments of the present disclosure can be an electronic device including a camera, wherein the electronic device can be a mobile phone, a camera, a tablet computer, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the photographing method can be realized by a processor calling computer readable instructions stored in a memory.

[0074] In the embodiments of the present disclosure, the electronic device can include a front camera and / or a rear camera, and the photographing method of the embodiments of the present disclosure can be applied to photographing for any camera in the electronic device. The camera includes a lens and an image sensor, external light is transmitted to the image sensor through the lens, so as to convert the light signal into an electrical signal, and then the image sensor transmits the electrical signal to an image signal processor (ISP) for processing, and converts it into a visible image. The ISP can perform algorithm optimization on the noise, brightness, etc. of the image. In some embodiments, the ISP can also belong to part of the image sensor.

[0075] In step S201, the electronic device can detect a photographing instruction based on manual or voice operation of the user, or the execution of an application in the electronic device triggers the start of the camera, and the corresponding electronic device can detect the photographing instruction based on the execution of the application. After detecting the photographing instruction, the electronic device acquires a first image and a second image collected by the camera under different exposure times. The first image and the second image can be image frames with close intervals in a video recorded in a video recording scene, or image frames with close intervals in a photographing scene. The close intervals can mean adjacent.

[0076] In the embodiments of the present disclosure, the exposure time of the first image is greater than the exposure time of the second image. The first image and the image processed based on the first image can be referred to as a long-exposure image, and the second image and the image processed based on the second image can be referred to as a short-exposure image. Generally, a longer exposure time can retain details in dark areas, but may cause overexposure in highlight areas; a shorter exposure time can retain details in highlight areas, but may cause overdarkness in dark areas. In addition, in the case of motion, for example, in the case of motion of the photographed object and / or shaking of the electronic device when taking a photo, the long-exposure image is more likely to produce smearing than the short-exposure image.

[0077] In step S202, the electronic device performs brightness alignment based on the first image and the second image, so that the difference between the brightness distribution of the brightness-aligned second image and the brightness distribution of the first image is less than a preset brightness difference threshold, for example, the brightness distribution of the second image is adjusted to be consistent with the brightness distribution of the first image. In the embodiments of the present disclosure, the brightness of the second image can be aligned according to the exposure parameters of the first image and the second image; or the brightness of the second image can be aligned with the brightness of the first image by histogram matching, for example, according to the pixel values of each pixel point in the first image and the second image, and the present disclosure does not limit this.

[0078] It should be noted that, since the long-exposure image (the first image) can retain more dark details, aligning the brightness of the second image with the brightness of the first image can retain the dark details in the first image without loss.

[0079] In step S203, the electronic device performs image registration on the first image based on the second image after brightness alignment to find a spatial transformation to map the first image so that the mapped first image and the second image after brightness alignment can correspond to each other for points at the same position in space.

[0080] In the embodiments of the present disclosure, image registration can be performed based on a spatial domain or a transform domain. For example, in the spatial domain, features in the second image after brightness alignment and the first image are extracted respectively, and then image registration is implemented through feature matching and transform model estimation. The features can be local features of the image, such as SIFT features, etc. Feature matching establishes a corresponding relationship between the extracted features through feature descriptors and similarity measures. Transform model estimation estimates the model according to the corresponding relationship between the features, for example, an affine transformation, a perspective transformation, a polynomial transformation, etc. based on the corresponding relationship between the features. Exemplarily, the transform model can be shown in the following formula (1):

[0081]

[0082] wherein x, y are pixel point coordinates before registration of the first image, f(x, y), g(x, y) are conversion functions, and x', y' are pixel point coordinates after registration of the first image.

[0083] In the embodiments of the present disclosure, since the first image is more likely to produce smearing relative to the second image, the second image has better stability, and an image with better stability can better maintain the consistency of feature points in the registration process. Therefore, the embodiments of the present disclosure perform image registration on the first image based on the second image after brightness alignment, which can reduce registration errors caused by motion.

[0084] In step S204, the electronic device performs fusion processing based on the registered first image and the second image after brightness alignment to obtain a fused target image and output, for example, directly fusing the first image and the second image in the whole image, or fusing the first image and the second image in image blocks. The embodiments of the present disclosure do not limit this.

[0085] It should be noted that in the embodiments of the present disclosure, the electronic device can fuse according to a preset weight during whole image fusion or fusion in image blocks, and can also detect whether there is a motion displacement and / or detect brightness, and assign a fusion weight based on the motion displacement and / or brightness. The embodiments of the present disclosure do not limit this.

[0086] In addition, in the embodiment of the present disclosure, the first image and the second image can be RGB images after ISP processing. After the first image and the second image based on the RGB domain are fused by the shooting method of the embodiment of the present disclosure, the target image for output can be directly obtained; in addition, the first image and the second image can also be RAW images or YUV images in the process of performing ISP processing. The first image and the second image are fused by the shooting method of the embodiment of the present disclosure, and then the ISP processing process is completed to obtain the target image for output.

[0087] It is understandable that in the embodiment of the present disclosure, when the electronic device fuses images with different exposure times to obtain the target image, it first aligns the brightness of the second image with a shorter exposure time to the brightness of the first image with a longer exposure time, so that subsequent image fusion is more natural, reduces the occurrence of uneven and discontinuous brightness, and can also retain more dark details; in addition, after brightness alignment, the embodiment of the present disclosure uses the brightness-aligned second image as a reference to perform image registration on the first image. Since images under different exposure conditions may have differences in spatial position, distortion, etc., and image registration can alleviate these differences so that the images to be fused have consistent corresponding points at the same pixel position, the embodiment of the present disclosure can improve the quality of the target image output after fusion by performing registration and then fusion. In particular, for spatial position differences caused by motion, registration based on the brightness-aligned second image can also reduce registration errors caused by motion.

[0088] In some embodiments, performing fusion processing based on the registered first image and the brightness aligned second image to obtain and output a fused target image includes:

[0089] Determining matching image blocks in the registered first image and the brightness aligned second image;

[0090] The registered first image and the brightness-aligned second image are fused based on the groups of matched image blocks to obtain and output the target image.

[0091] In the embodiments of the present disclosure, the electronic device fuses the registered first image and the brightness-aligned second image based on the matched image blocks, wherein the matched image blocks can include one or a pair of groups. In some embodiments, when determining the matched image blocks in the registered first image and the brightness-aligned second image, since the first image and the brightness-aligned second image have been registered, the registered first image and the brightness-aligned second image can be directly divided into regions, and the image blocks at the coordinate positions are determined as the matched image blocks. In other embodiments, the electronic device can further determine the matched image blocks in the registered first image and the second image based on a matching algorithm, so as to improve the matching degree between the image blocks, thereby performing local matching on the basis of the whole-image registration to improve the accuracy of the fusion.

[0092] In some embodiments, the matched image blocks can cover the entire registered first image and the brightness-aligned second image, and the electronic device can fuse the target image based on each group of matched image blocks. In other embodiments, the matched image blocks are only part of the registered first image and the brightness-aligned second image, and the electronic device can fuse the target image based on each group of matched image blocks and in combination with the part of the registered first image or the brightness-aligned second image that is outside the matched image blocks; for example, based on the fusion result of each group of matched image blocks, the part of the registered first image or the brightness-aligned second image that is outside the matched image blocks is spliced to obtain the target image.

[0093] In the embodiments of the present disclosure, when the electronic device performs the fusion processing based on each matched image block, in some embodiments, the weight of the matched image block can be assigned according to the position of the image block in the image, for example, the weight of the matched image block at the central position is greater than the weight of the matched image block at the edge position; in other embodiments, the fusion weight of the matched image block can also be assigned according to the brightness of the matched image block or the difference between the matched image blocks. The method of the present disclosure for performing the fusion processing based on each matched image block is not limited.

[0094] It can be understood that, in the embodiments of the present disclosure, the electronic device fuses the registered first image and the brightness-aligned second image based on each group of matched image blocks, which can make the image fusion more detailed, and thus help to improve the quality of the target image after the fusion.

[0095] In some embodiments, the fusion processing based on each group of matched image blocks on the registered first image and the brightness-aligned second image to obtain the target image and output, includes:

[0096] For each matched image block, it is determined whether the brightness belonging to the image block in the registered first image is greater than a preset brightness threshold, and / or whether there is a motion displacement between the matched image blocks;

[0097] Based on the comparison result of the brightness belonging to the image block in the registered first image and the preset brightness threshold, and / or based on the motion displacement detection result between the matched image blocks, the fusion weight of the matched image block is determined.

[0098] Based on each group of matched image blocks and the corresponding fusion weight, the registered first image and the brightness-aligned second image are fused to obtain the target image and output.

[0099] As described above, the electronic device can determine the fusion weight of each matched image block in the registered first image and the brightness-aligned second image. In this embodiment, the electronic device assigns the weight based on the brightness of the image block in the registered first image and / or the motion displacement detection result between the matched image blocks. Wherein, when comparing the brightness of the image block in the registered first image with the preset brightness threshold, the electronic device can calculate the mean value or median value of each pixel point in the image block, for example, the image in the RGB domain is grayed, and the mean value of each pixel point grayscale is compared with the preset brightness threshold. In addition, when determining the motion displacement detection result between the matched image blocks, for example, the matched image blocks can be subtracted, and whether there is a motion displacement is detected by the inter-frame difference method; for another example, whether there is a motion displacement can also be detected by the background difference method, the optical flow method, etc., and the present embodiment does not limit this.

[0100] In the embodiment of the present disclosure, the electronic device determines the fusion weight based on the motion displacement detection result between the matched image blocks. Since the image has been registered before detecting the motion displacement, the motion detection is performed based on the registered image between the matched image blocks, which can improve the accuracy of motion displacement detection, thereby helping to improve the accuracy of assigning the weight based on the motion displacement detection result, so as to improve the quality of image fusion. In addition, since the long-exposure image and the short-exposure image have their own characteristics, for example, the long-exposure image can retain dark details, but the highlight area may be overexposed, and the ghosting phenomenon caused by motion in the short-exposure image is relatively weak, therefore, the fusion weight is assigned based on the brightness and / or motion displacement detection result of the matched image blocks, which facilitates to make the best of the characteristics of different exposure time images and avoid their shortcomings, thereby helping to improve the image fusion quality.

[0101] In some embodiments, the determining the fusion weight of the matched image block based on the comparison result of the brightness of the image block in the registered first image with the preset brightness threshold and the motion displacement detection result between the matched image blocks comprises:

[0102] In response to the brightness of the image block in the registered first image being greater than the preset brightness threshold, determining that the weight of the image block in the brightness-aligned second image is greater than the weight of the image block in the registered first image;

[0103] In response to the brightness of the image block in the registered first image being less than or equal to the preset brightness threshold, determining the fusion weight of the matched image block based on the motion displacement detection result between the matched image blocks.

[0104] In the embodiments of the present disclosure, as described above, the long-exposure image can retain dark details, but overexposure may exist in the highlight area, while the short-exposure image can retain highlight area details. Therefore, in the embodiments of the present disclosure, in the case that the brightness of the image block in the registered first image is greater than the preset brightness threshold, the weight of the image block in the brightness-aligned second image is set to be greater than the weight of the image block in the registered first image, so as to retain the details of the highlight area. The preset brightness threshold can be a brightness value that causes the human eye to be unable to see the details of the image, and the weight of the image block in the brightness-aligned second image being greater than the weight of the image block in the registered first image can mean that the weight of the image block in the brightness-aligned second image is 1, while the weight of the image block in the registered first image is 0. Of course, the embodiments of the present disclosure are not limited to specific weight values.

[0105] In addition, considering that the long-exposure image is more sensitive to motion, in the embodiments of the present disclosure, in the case that the brightness of the image block in the registered first image is less than or equal to the preset brightness threshold, the fusion weight of the matched image block is further determined in combination with the motion displacement detection result.

[0106] In some embodiments, the determining the fusion weight of the image block based on the motion displacement detection result between the matched image blocks comprises:

[0107] In response to the existence of the motion displacement between the matched image blocks, determining that the weight of the image block in the brightness-aligned second image is greater than the weight of the image block in the registered first image;

[0108] In response to the non-existence of the motion displacement between the matched image blocks, determining that the weight of the image block in the brightness-aligned second image is less than the weight of the image block in the registered first image.

[0109] In the embodiment of the present disclosure, since long-exposure images are more likely to produce ghosting than short-exposure images, when the electronic device detects motion displacement, it determines that the weight of the image block in the second image after brightness alignment is greater than the weight of the image block in the first image after alignment; otherwise, it determines that the weight of the image block in the second image after brightness alignment is less than the weight of the image block in the first image after alignment.

[0110] It can be understood that the embodiment of the present disclosure combines the brightness and motion detection results of matching image blocks to allocate fusion weights through the above-mentioned weight allocation method, and fully utilizes the characteristics of images with different exposure times to maximize their strengths and minimize their weaknesses, which helps to improve the image fusion quality.

[0111] In some embodiments, the method further comprises:

[0112] Determining differences between matching image patches;

[0113] In response to the difference being greater than a preset difference threshold, determining that there is motion displacement between the matching image blocks;

[0114] In response to the difference being less than or equal to a preset difference threshold, it is determined that there is no motion displacement between the matching image blocks.

[0115] As previously mentioned, the presence of motion displacement can be detected by the inter-frame difference method. In this embodiment, the electronic device compares the differences between the matching image blocks based on the inter-frame difference method. For example, the matching image blocks are subjected to a difference operation to obtain a differential image block, and each pixel in the differential image block represents the pixel difference between the matching image blocks. In some embodiments, the pixel mean or median of all pixels in the differential image block can be compared with a preset difference threshold to determine whether motion displacement exists. In this case, the preset difference threshold can be a threshold that characterizes the difference in pixel values. In other embodiments, the differential image block can be binarized, and then the number of foreground pixels in the binarized image can be compared with a preset difference threshold to determine whether motion displacement exists. In this case, the preset difference threshold can be a threshold that characterizes the number of pixel differences.

[0116] Figure 3 This is a schematic diagram of a motion detection process according to an embodiment of the present disclosure. Figure 3 As shown, the long-exposure image shown at 301 is the first image, and the short-exposure image shown at 302 is the second image after brightness alignment. After feature point detection is performed on the first and second images at 303, the feature points of the two images are matched to obtain a transformation matrix 304. Transformation matrix 304 is used to align the long-exposure image 301. After alignment, the electronic device performs block matching at 305 to determine matching image blocks, and then calculates differences between the matching image blocks to perform motion detection at 306.

[0117] It can be understood that, in the embodiments of the present disclosure, detecting motion based on the registered images can improve the accuracy of motion detection, and in addition, detecting whether there is a motion displacement based on the difference between the matched image blocks is simple and can save power consumption.

[0118] In some embodiments, the determining the matched image blocks in the registered first image and the brightness-aligned second image comprises:

[0119] In the registered first image, a reference image block is determined.

[0120] For each reference image block, the reference image block is compared with each search image block in a preset search region in the brightness-aligned second image in terms of similarity, and a search target image block satisfying a preset similarity condition is determined.

[0121] Each reference image block and the search target image block corresponding to the reference image block are determined as a group of matched image blocks.

[0122] In the embodiments of the present disclosure, the electronic device further determines the matched image blocks in the registered first image and the second image based on a matching algorithm. Specifically, the electronic device determines a reference image block in the registered first image, which can be one or multiple. The reference image block can be a preset image block of a local region, for example, an image block of a central region and / or a region close to the central region, or an image block of a region of interest determined based on a saliency detection algorithm. The electronic device searches for a matched image block in the brightness-aligned second image based on the reference image block in the registered first image. It should be noted that, in the embodiments of the present disclosure, when there are multiple reference image blocks, the reference image blocks can be non-overlapping or partially overlapping, which is not limited in the embodiments of the present disclosure.

[0123] After determining the reference image block, the electronic device compares each reference image block with each search image block in a preset search region in the brightness-aligned second image in terms of similarity, and determines a search target image block satisfying a preset similarity condition. The preset search region can be a region determined based on the position of the reference image block, for example, a small region containing the coordinates of the reference image block. In addition, it should be noted that the search image block in the brightness-aligned second image is an image block of the same size as the reference image block, and the electronic device can move the window according to the size of the reference image block to determine each search image block in the preset search region. The moving step can be preset, for example, 2 pixels each time, 0.5 times the width or height of the reference image block each time, and the like, which is not limited in the embodiments of the present disclosure.

[0124] Figure 4 This is an example diagram of a search for a matching image block in an embodiment of the present disclosure, such as Figure 4 As shown, 401 identifies the first image after registration, wherein 401b identifies a reference image block with a size of n*m, and the preset search area is an area that is moved up, down, left, and right by a range of D based on the position of 401b, where d is the moving step size; 402 identifies the second image after brightness alignment, and the electronic device moves the window in the preset search area of ​​image 402 with a step size of d until the similarity comparison of the search image blocks in the entire area is completed, wherein 402a, 402b, and 402c are examples of search image blocks at different positions. In the embodiment of the present disclosure, when the electronic device performs a similarity comparison between the reference image block and the search image block, for example, the absolute value of the corresponding pixel difference (SAD), the sum of the squares of the corresponding pixel difference (SSD), or the correlation of the images (NCC) is calculated, and the embodiment of the present disclosure is not limited to this.

[0125] Taking the calculation of the SAD between the search image block and the reference image block as an example, the following formula (2) is the calculation formula for SAD:

[0126]

[0127] Among them, I se (x, y) is the pixel value of the pixel point of the reference image block in the second image after brightness alignment, I ne (x, y) is the pixel value of the corresponding pixel point of the search image block in the first image after registration, and SAD is the sum of pixel differences.

[0128] In the embodiment of the present disclosure, when the electronic device determines the search target image block whose similarity satisfies a preset similarity condition, for example, the electronic device may determine the search image block with the smallest SAD value as the search target image block. Subsequently, the electronic device determines the reference image block and the corresponding search target image block as a set of matching image blocks.

[0129] It should be noted that the electronic device may also determine a reference image block in the second image after brightness alignment, and search within a preset search area of ​​the registered first image to determine a search target image block, which is not limited in this embodiment of the present disclosure.

[0130] It can be understood that in the embodiment of the present disclosure, the electronic device uses a region-based local matching method to determine matching image blocks using similarity, which is a simple and time-saving solution.

[0131] In some embodiments, the brightness alignment based on the first image and the second image comprises:

[0132] determining a brightness adjustment coefficient based on a first exposure parameter of the first image and a second exposure parameter of the second image;

[0133] performing brightness alignment on the second image based on the brightness adjustment coefficient to obtain the brightness-aligned second image.

[0134] In the embodiments of the present disclosure, the first exposure parameter and the second exposure parameter can both be exposure time, and can also be exposure time and exposure gain. The electronic device can process the first exposure parameter and the second exposure parameter by using a preset calculation rule to obtain the brightness adjustment coefficient, so as to perform brightness alignment on the second image. The preset calculation rule can be a calculation formula obtained by fitting, and can also be a model obtained by training based on a deep learning network. For example, a sample image pair can be collected based on a large number of different exposure parameters, a short-exposure sample image in the sample image pair is adjusted by using a preset brightness adjustment coefficient, and user satisfaction is collected, and the brightness adjustment coefficient that meets the user's visual demand is recorded to determine the calculation rule together with the exposure parameter.

[0135] For example, in the embodiments of the present disclosure, the brightness adjustment coefficient can be determined based on the following formula (3):

[0136]

[0137] wherein Shutter se , Shutter ne respectively represent preset exposure times of the second image and the first image; Gain se , Gain ne respectively represent preset exposure gains of the second image and the first image, and ratio is the brightness adjustment coefficient. In the embodiments of the present disclosure, after the electronic device determines the brightness adjustment coefficient based on the above formula (3), for example, the pixel value of each pixel point in the second image is multiplied by the above ratio, and the brightness-aligned second image is obtained. Generally, the value of ratio is between 1 and 6.

[0138] It can be understood that, in the embodiments of the present disclosure, the brightness adjustment coefficient is determined based on the first exposure parameter of the first image and the second exposure parameter of the second image, and the brightness alignment is performed on the second image. Since the first exposure parameter and the second exposure parameter are both known, and both directly affect the brightness of the image, the brightness alignment on the second image by this method can be simple and convenient, and the accuracy of the brightness alignment is high.

[0139] In some embodiments, performing brightness alignment based on the first image and the second image to obtain a brightness-aligned second image includes:

[0140] performing bad pixel correction on the first image and the second image respectively, and performing black level correction on the images after the bad pixel correction;

[0141] Brightness alignment is performed based on the first image after black level correction and the second image after black level correction to obtain the second image after brightness alignment.

[0142] As mentioned above, both the first image and the second image can be the original RAW images captured by the image sensor. Usually, the RAW images need to be ISP processed so that better quality images can be output. In the embodiment of the present disclosure, the electronic device performs bad pixel correction on the first image and the second image in the RAW domain respectively, and then performs black level correction on the bad pixel corrected image. Among them, bad pixels refer to pixels in the pixel array that show obvious differences in changes from the surrounding pixels. The embodiment of the present disclosure can correct bad pixels by filtering. In addition, due to the influence of dark current, the actual raw data from the image sensor is not the black balance we need (the data is not 0), so black level correction is required. For example, the first few rows of the pixel area can be used as the insensitive area, and the average value of the pixels in the first few rows is calculated as the correction value. Then, the pixels in the subsequent rows are subtracted from this correction value to achieve black level correction. Of course, the embodiment of the present disclosure is not limited to the above-mentioned bad pixel correction and black level correction methods.

[0143] In embodiments of the present disclosure, after the electronic device obtains the first and second images after black level correction during ISP processing, brightness alignment is performed based on the first and second images after black level correction. In some embodiments, the brightness of the second image after black level correction can be aligned with the brightness of the first image after black level correction using the aforementioned histogram matching method based on the first and second images after black level correction. In other embodiments, the brightness-aligned second image can be obtained by multiplying the pixel value of each pixel in the second image after black level correction by the aforementioned ratio based on the aforementioned brightness adjustment coefficient. The brightness adjustment coefficient can be determined based on the exposure parameters corresponding to the first and second images after black level correction, respectively, namely, the aforementioned first and second exposure parameters.

[0144] It can be understood that the embodiment of the present disclosure performs brightness alignment after black level correction, which can improve the quality of the second image after brightness alignment, thereby facilitating subsequent image processing.

[0145] It should be noted that in the embodiments of the present disclosure, for the first image and the second image in the RAW domain, the electronic device performs image registration on the first image based on the second image after brightness alignment, that is, performs image registration on the first image after black level correction based on the second image after brightness alignment to obtain the registered first image. In addition, after the fusion processing based on the registered first image and the second image after brightness alignment, the electronic device still needs to perform subsequent lens shading correction and other ISP processing procedures on the fused image to obtain the processed target image and output.

[0146] Figure 5 is a schematic diagram of processing an original image collected by an image sensor in the embodiments of the present disclosure, as shown in Figure 5 The electronic device performs the bad pixel correction steps 501a and 502a on the long-exposure RAW image (first image) 501 and the short-exposure RAW image (second image) 502 respectively, and then further performs the black level correction 501b and 502b on the bad pixel corrected images. Then, the electronic device performs the brightness alignment 502c on the black level corrected short-exposure image based on the long-exposure image after black point correction, and performs the image alignment 501c on the long-exposure image after black level correction based on the output of 502c, that is, image registration. After brightness alignment and registration, the electronic device performs motion detection 503 on the brightness aligned short-exposure image and the registered long-exposure image, and then performs fusion 504, for example, can determine matching image blocks based on the foregoing method, perform motion detection to assign weights, and then fuse the matching image blocks. After image fusion, the electronic device performs a step of digital gain processing 505 to improve the brightness of the fused image, and performs subsequent lens shading correction 506, RGB conversion 507, and denoising processing 508 to obtain the target image for output.

[0147] It can be understood that since the data amount of the RAW domain is relatively small compared with the RGB domain, the embodiments of the present disclosure perform HDR image fusion processing in the RAW domain, so that the target image with better image quality can be quickly and efficiently obtained and output.

[0148] Figure 6 is a block diagram of a photographing device according to an exemplary embodiment. As shown in Figure 6 The device 600 mainly includes:

[0149] The acquisition module 601 is configured to acquire a first image and a second image collected under different exposure times in response to detecting a photographing instruction; wherein the exposure time of the first image is greater than the exposure time of the second image.

[0150] The brightness alignment module 602 is configured to perform brightness alignment based on the first image and the second image, to obtain a brightness-aligned second image; wherein a difference between a brightness distribution of the brightness-aligned second image and a brightness distribution of the first image is less than a preset brightness difference threshold.

[0151] The registration module 603 is configured to perform image registration on the first image based on the brightness-aligned second image, to obtain a registered first image.

[0152] The fusion module 604 is configured to perform fusion processing based on the registered first image and the brightness-aligned second image, to obtain a fused target image and output the target image.

[0153] In some embodiments, the fusion module 604 is further configured to determine matched image blocks in the registered first image and the brightness-aligned second image; perform fusion processing on the registered first image and the brightness-aligned second image based on each group of matched image blocks, to obtain the target image and output the target image.

[0154] In some embodiments, the fusion module 604 is further configured to, for each matched image block, determine whether a brightness of an image block belonging to the registered first image is greater than a preset brightness threshold, and / or determine whether there is a motion displacement between the matched image blocks; determine a fusion weight of the matched image blocks based on a comparison result of the brightness of the image block belonging to the registered first image and the preset brightness threshold, and / or based on a motion displacement detection result between the matched image blocks; perform fusion processing on the registered first image and the brightness-aligned second image based on each group of matched image blocks and the corresponding fusion weights, to obtain the target image and output the target image.

[0155] In some embodiments, the fusion module 604 is further configured to, in response to the brightness of the image block belonging to the registered first image being greater than the preset brightness threshold, determine that a weight of an image block in the brightness-aligned second image is greater than a weight of an image block in the registered first image; and in response to the brightness of the image block belonging to the registered first image being less than or equal to the preset brightness threshold, determine the fusion weight of the matched image blocks based on the motion displacement detection result between the matched image blocks.

[0156] In some embodiments, the fusion module 604 is further configured to determine, in response to the existence of the motion displacement between the matched image blocks, a weight of the image block in the second image after the brightness alignment to be greater than a weight of the image block in the first image after the registration; and determine, in response to the non-existence of the motion displacement between the matched image blocks, the weight of the image block in the second image after the brightness alignment to be less than the weight of the image block in the first image after the registration.

[0157] In some embodiments, the apparatus further comprises:

[0158] a motion detection module configured to determine a difference between the matched image blocks; determine, in response to the difference being greater than a preset difference threshold, the existence of the motion displacement between the matched image blocks; and determine, in response to the difference being less than or equal to the preset difference threshold, the non-existence of the motion displacement between the matched image blocks.

[0159] In some embodiments, the fusion module 604 is further configured to determine, in the first image after the registration, a reference image block; for each reference image block, perform a similarity comparison between the reference image block and each search image block in a preset search region of the second image after the brightness alignment, and determine a search target image block satisfying a preset similarity condition; and determine each reference image block and the search target image block corresponding to the reference image block as a group of matched image blocks.

[0160] In some embodiments, the brightness alignment module 602 is further configured to determine a brightness adjustment coefficient based on a first exposure parameter of the first image and a second exposure parameter of the second image; and perform brightness alignment on the second image based on the brightness adjustment coefficient to obtain the second image after the brightness alignment.

[0161] In some embodiments, the brightness alignment module 602 is further configured to perform bad pixel correction on the first image and the second image respectively, and perform black level correction on the images after the bad pixel correction; and perform brightness alignment based on the first image after the black level correction and the second image after the black level correction to obtain the second image after the brightness alignment.

[0162] As to the apparatus in the above embodiments, the specific manners in which the modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0163] Figure 7 FIG. 7 is a structural block diagram of an apparatus 700 according to an exemplary embodiment. For example, the apparatus 700 can be a mobile phone, a tablet computer, a wearable device, or a vehicle-mounted device, etc.

[0164] Referring to Figure 7The device 700 can include one or more of the following components: a processing component 702, a memory 704, a power supply component 706, a multimedia component 708, an audio component 710, an input / output (I / O) interface 712, a sensor component 714, and a communication component 716.

[0165] The processing component 702 typically controls overall operations of the device 700, such as operations associated with display, telephony calls, data communications, camera operations, and recording operations. The processing component 702 can include one or more processors 720 to execute instructions to complete all or part of steps of the above methods. In addition, the processing component 702 can include one or more modules to facilitate

[0166] The memory 704 is configured to store various types of data to support operations of the device 700. Examples of these data include at least one of the following: instructions for any application or method operating on the device 700, contact data, phonebook data, messages, pictures, and videos. The memory 704 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0167] The power supply component 706 supplies electrical power for the various components of the device 700. The power supply component 706 can include at least one of the following: a power supply management system, one or more power supplies, and other components associated with generating, managing, and distributing electrical power for the device 700.

[0168] The multimedia component 708 includes a screen providing an output interface between the device 700 and a user. In some embodiments, the screen includes a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swiping, and gestures on the touch panel. The touch sensor can not only sense a boundary of a touching or swiping action, but also detect duration and pressure related to the touching or swiping action. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the device 700 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zooming capability.

[0169] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC) that is configured to receive external audio signals when the device 700 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 704 or transmitted via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0170] The I / O interface 712 provides an interface between the processing component 702 and peripheral interface modules, such as a keyboard, a click wheel, and buttons. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0171] The sensor component 714 includes one or more sensors for providing status assessments for various aspects of the device 700. For example, the sensor component 714 can detect an open / closed position of the device 700, relative positioning of components, such as a display and keypad of the device 700, changes in position of the device 700 or a component of the device 700, presence or absence of user contact with the device 700, orientation or acceleration / deceleration of the device 700, and temperature changes of the device 700. The sensor component 714 can include proximity sensor(s) configured to detect presence of a proximity object without any physical touch. The sensor component 714 can also include a light sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, utilized in imaging applications. In some embodiments, the sensor component 714 can further include at least one of an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, and a temperature sensor, among others.

[0172] The communication component 716 is configured to facilitate wired or wireless communication between the device 700 and another device. The device 700 can access a wireless network based on a communication standard, such as Wi-Fi, 4G, 5G, or a combination thereof. In an example embodiment, the communication component 716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 716 can further include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) techniques, Infrared Data Association (IrDA) techniques, Ultra-WideBand (UWB) techniques, Bluetooth (BT) techniques, and other techniques.

[0173] In exemplary embodiments, the apparatus 700 can be implemented using one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic devices.

[0174] In exemplary embodiments, a non-transitory computer readable storage medium including instructions, such as the memory 704 including executable instructions or a computer program, which can be executed by the processor 720 of the apparatus 700 to implement the above-described methods, is also provided. For example, the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk and an optical data storage device, etc.

[0175] A non-transitory computer readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform any of the photographing methods according to the embodiments of the present disclosure.

[0176] The embodiments of the present disclosure provide a computer program product, which includes a computer program or executable instructions stored in a computer readable storage medium. A processor of a computer device reads the computer program or executable instructions from the computer readable storage medium, and the processor executes the computer program or executable instructions, so that the computer device performs any of the photographing methods according to the embodiments of the present disclosure.

[0177] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure following, in general, the principles of the present disclosure and including such

[0178] It should be understood that the present disclosure is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the present disclosure. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A photographing method characterized by comprising: The method comprises: in response to detecting a shooting instruction, acquiring a first image and a second image captured under different exposure durations; wherein the exposure duration of the first image is greater than the exposure duration of the second image; performing brightness alignment based on the first image and the second image to obtain a brightness-aligned second image; wherein the difference between the brightness distribution of the brightness-aligned second image and the brightness distribution of the first image is less than a preset brightness difference threshold; performing image registration on the first image based on the brightness-aligned second image to obtain a registered first image; performing fusion processing on the registered first image and the brightness-aligned second image to obtain a fused target image and output.

2. The method of claim 1, wherein, The fusion processing on the registered first image and the brightness-aligned second image to obtain the fused target image and output comprises: determining matching image blocks in the registered first image and the brightness-aligned second image; performing fusion processing on the registered first image and the brightness-aligned second image based on each group of matching image blocks to obtain the target image and output.

3. The method of claim 2, wherein, The fusion processing on the registered first image and the brightness-aligned second image based on each group of matching image blocks to obtain the target image and output comprises: for each group of matching image blocks, determining a fusion weight of the matching image blocks based on a comparison result of the brightness of the image blocks in the registered first image with a preset brightness threshold, and / or based on a motion displacement detection result between the matching image blocks; performing fusion processing on the registered first image and the brightness-aligned second image based on each group of matching image blocks and the corresponding fusion weights to obtain the target image and output.

4. The method of claim 3, wherein, The determination of the fusion weight of the matching image blocks based on the comparison result of the brightness of the image blocks in the registered first image with the preset brightness threshold and based on the motion displacement detection result between the matching image blocks comprises: in response to the brightness of the image blocks in the registered first image being greater than the preset brightness threshold, determining that the weight of the image blocks in the brightness-aligned second image is greater than the weight of the image blocks in the registered first image; in response to the brightness of the image blocks in the registered first image being less than or equal to the preset brightness threshold, determining the fusion weight of the matching image blocks based on the motion displacement detection result between the matching image blocks.

5. The method of claim 3, wherein, The determination of the fusion weight of the image blocks based on the motion displacement detection result between the matching image blocks comprises: in response to there being motion displacement between the matching image blocks, determining that the weight of the image blocks in the brightness-aligned second image is greater than the weight of the image blocks in the registered first image; in response to there being no motion displacement between the matching image blocks, determining that the weight of the image blocks in the brightness-aligned second image is less than the weight of the image blocks in the registered first image.

6. The method according to claim 3 or 5, characterized in that, The method further comprises: determining the difference between the matching image blocks; in response to the difference being greater than a preset difference threshold, determining that there is a motion displacement between the matched image blocks; in response to the difference being less than or equal to the preset difference threshold, determining that there is no motion displacement between the matched image blocks.

7. The method of claim 2, wherein, The determining of the matched image blocks in the registered first image and the brightness-aligned second image comprises: determining, in the registered first image, a reference image block; for each reference image block, performing a similarity comparison between the reference image block and each search image block in a preset search region of the brightness-aligned second image, and determining a search target image block that satisfies a preset similarity condition; determining each reference image block and the search target image block corresponding to the reference image block as a group of matched image blocks.

8. The method of claim 1, wherein, The brightness alignment based on the first image and the second image to obtain the brightness-aligned second image comprises: determining a brightness adjustment coefficient based on a first exposure parameter of the first image and a second exposure parameter of the second image; performing brightness alignment on the second image based on the brightness adjustment coefficient to obtain the brightness-aligned second image.

9. The method according to claim 1 or 8, characterized in that, The brightness alignment based on the first image and the second image to obtain the brightness-aligned second image comprises: respectively performing bad pixel correction on the first image and the second image, and performing black level correction on the bad pixel corrected images; performing brightness alignment based on the black level corrected first image and the black level corrected second image to obtain the brightness-aligned second image.

10. An imaging device, characterized by comprising: comprises: an acquisition module configured to, in response to detecting a shooting instruction, acquire a first image and a second image collected under different exposure durations; wherein an exposure duration of the first image is greater than an exposure duration of the second image; a brightness alignment module configured to perform brightness alignment based on the first image and the second image to obtain a brightness-aligned second image; wherein a difference between a brightness distribution of the brightness-aligned second image and a brightness distribution of the first image is less than a preset brightness difference threshold; a registration module configured to perform image registration on the first image based on the brightness-aligned second image to obtain a registered first image; a fusion module configured to perform fusion processing based on the registered first image and the brightness-aligned second image to obtain a fused target image and output the fused target image.

11. An electronic device, comprising: comprises: a processor; a memory for storing computer programs or instructions; wherein the processor executes the computer programs or instructions to implement the steps of the method of any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing a computer program or instructions, wherein, When the computer programs or instructions in the storage medium are executed by the processor, the steps of the method of any one of claims 1 to 9 are implemented.

13. A computer program product comprising computer programs or instructions, characterized in that, When the computer programs or instructions are executed by the processor, the steps of the method of any one of claims 1 to 9 are implemented.

Images