Image processing method and device, electronic equipment and computer readable storage medium

Abstract

This application relates to an image processing method, apparatus, electronic device, storage medium, and computer program product. The method includes: acquiring multiple frames of images of the same scene; selecting a reference image from the multiple frames according to preset image quality screening conditions; registering other images with the reference image as a reference to obtain displacement change maps corresponding to the other images; wherein the other images are images other than the reference image in the multiple frames; the displacement change maps are used to characterize the displacement change between the registered images; for each frame of other images, determining a first fusion weight based on the corresponding displacement change map; obtaining a second fusion weight corresponding to the reference image; and fusing the other images and their corresponding first fusion weights with the reference image and its corresponding second fusion weights to obtain a target image. This method can effectively reduce artificial artifacts in moving areas and also achieve noise reduction.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the development of electronic device technology, users are increasingly using electronic devices to take photos or videos. Taking photos as an example, when users take photos in environments with strong or insufficient light, they typically use multi-frame high dynamic range (HDR) imaging or noise reduction techniques to enhance the image quality. Multi-frame noise reduction (MFNR) technology is widely used in the industry because it offers better noise reduction and better preservation of detail and texture compared to single-frame technology. However, in actual shooting scenarios, when the camera is capturing moving scenes or experiencing slight camera shake, artifacts can easily appear in the moving areas of the captured image.

[0003] Traditional techniques typically use threshold control or frame clipping to suppress artificial artifacts in moving areas, but the results are not satisfactory. Summary of the Invention

[0004] This application provides an image processing method, apparatus, electronic device, and computer-readable storage medium that can effectively suppress artificial artifacts in captured images.

[0005] Firstly, this application provides an image processing method. The method includes:

[0006] Acquire multiple frames of images of the same scene;

[0007] A reference image is selected from the multi-frame images according to preset image quality filtering conditions;

[0008] Using the reference image as a reference, other images are registered with the reference image to obtain displacement change maps corresponding to the other images; the other images are images other than the reference image in the multi-frame images; the displacement change maps are used to characterize the amount of displacement change between the registered images;

[0009] For each frame of the other images, a first fusion weight is determined based on the corresponding displacement change map; a second fusion weight is obtained corresponding to the reference image.

[0010] The other images and their corresponding first fusion weights are fused with the reference image and its corresponding second fusion weights to obtain the target image.

[0011] Secondly, this application also provides an image processing apparatus. The apparatus includes:

[0012] The image acquisition module is used to acquire multiple frames of images of the same scene;

[0013] The image filtering module is used to select reference images from the multi-frame images according to preset image quality filtering conditions;

[0014] An image registration module is used to register other images with the reference image as a reference to obtain a displacement change map corresponding to the other images; the other images are images other than the reference image in the multi-frame images; the displacement change map is used to characterize the displacement change between the registered images;

[0015] The weight determination module is used to determine the first fusion weight of the other images in each frame based on the corresponding displacement change map; and to obtain the second fusion weight corresponding to the reference image.

[0016] The image fusion module is used to fuse the other images and their corresponding first fusion weights with the reference image and its corresponding second fusion weights to obtain the target image.

[0017] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0018] Acquire multiple frames of images of the same scene;

[0019] A reference image is selected from the multi-frame images according to preset image quality filtering conditions;

[0020] Using the reference image as a reference, other images are registered with the reference image to obtain displacement change maps corresponding to the other images; the other images are images other than the reference image in the multi-frame images; the displacement change maps are used to characterize the amount of displacement change between the registered images;

[0021] For each frame of the other images, a first fusion weight is determined based on the corresponding displacement change map; a second fusion weight is obtained corresponding to the reference image.

[0022] The other images and their corresponding first fusion weights are fused with the reference image and its corresponding second fusion weights to obtain the target image.

[0023] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0024] Acquire multiple frames of images of the same scene;

[0025] A reference image is selected from the multi-frame images according to preset image quality filtering conditions;

[0026] Using the reference image as a reference, other images are registered with the reference image to obtain displacement change maps corresponding to the other images; the other images are images other than the reference image in the multi-frame images; the displacement change maps are used to characterize the amount of displacement change between the registered images;

[0027] For each frame of the other images, a first fusion weight is determined based on the corresponding displacement change map; a second fusion weight is obtained corresponding to the reference image.

[0028] The other images and their corresponding first fusion weights are fused with the reference image and its corresponding second fusion weights to obtain the target image.

[0029] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0030] Acquire multiple frames of images of the same scene;

[0031] A reference image is selected from the multi-frame images according to preset image quality filtering conditions;

[0032] Using the reference image as a reference, other images are registered with the reference image to obtain displacement change maps corresponding to the other images; the other images are images other than the reference image in the multi-frame images; the displacement change maps are used to characterize the amount of displacement change between the registered images;

[0033] For each frame of the other images, a first fusion weight is determined based on the corresponding displacement change map; a second fusion weight is obtained corresponding to the reference image.

[0034] The other images and their corresponding first fusion weights are fused with the reference image and its corresponding second fusion weights to obtain the target image.

[0035] The aforementioned image processing method, apparatus, electronic device, and computer-readable storage medium can select a reference image from multiple frames of images by using preset image quality screening conditions. Using the reference image as a reference, other images are registered with the reference image to obtain displacement change maps corresponding to the other images. For each frame of other images, a first fusion weight of the other images is determined based on the corresponding displacement change map, and a second fusion weight corresponding to the reference image is obtained. The other images and their corresponding first fusion weights, and the reference image and its corresponding second fusion weights are fused to obtain the target image. Information from the moving parts can be incorporated into the fusion weights to regulate the fusion weights, thereby effectively reducing artificial artifacts in the moving areas and achieving noise reduction. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart of an image processing method in one embodiment;

[0038] Figure 2 One is a pixel distribution map in a frame of an image in the embodiment;

[0039] Figure 3 Here is a flowchart of step 106 in one embodiment;

[0040] Figure 4 Here is a flowchart of step 110 in one embodiment;

[0041] Figure 5 One is a flowchart of step 308 in the embodiment;

[0042] Figure 6A Other images in one embodiment are schematic diagrams;

[0043] Figure 6B One of the embodiments Figure 6A Schematic diagram of the corresponding displacement change;

[0044] Figure 7 Here is a flowchart of step 108 in one embodiment;

[0045] Figure 8 This is a schematic diagram of the distribution of raw format image data in one embodiment;

[0046] Figure 9 One is a flowchart of step 404 in the embodiment;

[0047] Figure 10 A flowchart of an image processing method in another embodiment;

[0048] Figure 11A This is a schematic diagram of a target image obtained by conventional weighted fusion in one embodiment;

[0049] Figure 11B This is a schematic diagram of a target image obtained by fusing after adjusting the fusion weights by adjusting the displacement changes of the moving region in one embodiment.

[0050] Figure 12 This is a structural block diagram of an image processing device in one embodiment;

[0051] Figure 13 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] Multi-frame fusion noise reduction (MFNR) achieves noise reduction by weighting and fusing image frames based on their similarity. The more similar the frames, the greater the weight given to the fusion, thus reducing noise through multi-frame image fusion. Typically, 3 to 10 frames are used for fusion. In image signal processing (ISP) systems, fusing consecutive multiple frames can effectively filter out Gaussian noise in the temporal domain, improving image quality and enhancing its expressiveness. However, during multi-frame fusion noise reduction, the fusion effect is significantly reduced when the camera is capturing moving scenes or experiences slight camera shake. Artifacts are more likely to appear in the moving areas of the acquired image. The greater the motion or shaking, the more pronounced the artifacts in the moving areas of the acquired image. Artifacts can be understood as unnatural, abnormal, or obviously manipulated traces, areas, or flaws in the synthesized image.

[0054] Currently, threshold control is generally used to distinguish moving regions and reduce their fusion weight, thereby suppressing artifacts. Another approach is frame truncating, which fuses only a few frames that are close to the reference frame to achieve noise reduction and suppress artifacts. However, threshold control is difficult to adjust; different scenes require different thresholds, resulting in a large amount of debugging work. It also generates strong edge noise, leading to uneven noise reduction. Furthermore, when the target object moves in a region with complex texture, it is difficult to accurately judge using thresholds, making it difficult to avoid artifacts. Frame truncating can easily lead to excessive truncating, resulting in insufficient noise reduction and noticeable noise in the acquired image. Moreover, in cases of large motion (such as a fast-moving car or a pedestrian at close range), even between adjacent frames of the reference image, the differences between the two frames can be significant, easily producing artifacts.

[0055] Based on this, this application proposes an image processing method that involves acquiring multiple frames of images of the same scene; selecting a reference image from the multiple frames according to preset image quality screening conditions; registering other images with the reference image to obtain displacement change maps corresponding to the other images; the other images are images other than the reference image in the multiple frames; the displacement change maps are used to characterize the displacement changes between the registered images; for each frame of other images, a first fusion weight is determined based on the corresponding displacement change map; a second fusion weight corresponding to the reference image is obtained; and the other images and their corresponding first fusion weights, and the reference image and its corresponding second fusion weights are fused to obtain the target image. Information from the moving parts can be incorporated into the fusion weights to regulate them, thereby effectively reducing artificial artifacts in moving areas and achieving noise reduction.

[0056] The image processing method provided in this application is described using an electronic device as an example. The electronic device can be, but is not limited to, various personal computers, cameras, scanners, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc.

[0057] In one embodiment, such as Figure 1 As shown, an image processing method is provided, including the following steps 102 to 110.

[0058] Step 102: Obtain multiple frames of images of the same scene.

[0059] Among them, multi-frame images are images taken continuously with a short interval between frames. The interval can vary depending on the scene. For example, the time interval for multi-frame images taken during the day is 5ms, and the time interval for images taken at night is 20ms. A multi-frame image consists of at least 2 frames and usually includes 3-10 frames. The specific number can be selected according to the actual scene.

[0060] Multi-frame images can be raw image data directly acquired by CMOS (Complementary Metal-Oxide-Semiconductor) or CCD (charge-coupled device) image sensors, or they can be images that have undergone image signal processing. That is, the image format of multi-frame images can be raw, RGB (Red, Green, Blue), HSV (Hue, Saturation, Value), or YUV, etc. In YUV format images, the Y component represents luminance (Luminance or Luma), while the U and V components represent chrominance (Chrominance or Chroma).

[0061] Optionally, the electronic device can access the camera and control the camera to continuously capture images of the same scene, obtaining multiple frames of images at preset intervals.

[0062] Step 104: Select a reference image from multiple frames of images according to preset image quality filtering conditions.

[0063] Electronic devices select a reference image from multiple frames based on preset image quality filtering conditions. The reference image can be a single frame or multiple frames. Image quality filtering conditions refer to filtering criteria that characterize image quality. These can be based on at least one setting such as image sharpness, brightness, signal-to-noise ratio, or distortion, or other image quality filtering conditions specific to the application scenario. Image quality filtering conditions are not limited to a single condition; they can also consist of multiple conditions, such as satisfying both image sharpness and brightness filtering conditions. Preset image quality filtering conditions can be, for example, the highest possible image sharpness or meeting a preset brightness filtering condition.

[0064] Step 106: Using the reference image as a reference, register other images with the reference image to obtain displacement change maps corresponding to the other images; wherein, the other images are images other than the reference image in the multi-frame images; the displacement change map is used to characterize the displacement change between the registered images.

[0065] Using a reference image as a baseline, registering other images with the reference image represents the process of finding features in the reference image that have a preset correspondence with features in other images. The preset correspondence can refer to the feature with the highest similarity, or a feature selected from features with similarity greater than a similarity threshold. Here, "other images" refers to images in multiple frames other than the reference image. Optionally, the registration can be performed entirely between the other images and the reference image, or the other images and the reference image can be divided into multiple image blocks using the same method, and then the image blocks in the other images are sequentially registered with the image blocks in the reference image. When an image block in another image is registered with an image block in the reference image, it means that the two registered image blocks satisfy a preset correspondence; for example, it is the image block in the reference image with the highest similarity to image blocks in other images.

[0066] A displacement map is used to characterize the displacement changes between registered image features. For each frame of other images, the corresponding displacement changes are determined based on the positions between the registered image features, and the displacement map for each other image is obtained based on these displacement changes. After dividing multiple frames of images into multiple image blocks, image blocks in other images are registered with image blocks in the reference image. The corresponding displacement changes are determined based on the positions between the two registered image blocks, and the corresponding displacement map is obtained based on the displacement changes between the registered image blocks in other images.

[0067] Step 108: For each frame of other images, determine the first fusion weight of other images based on the corresponding displacement change map; obtain the second fusion weight corresponding to the reference image.

[0068] The electronic device determines the first fusion weight of other images based on the displacement change map corresponding to other images in each frame. There can be multiple first fusion weights. When dividing each frame of other images into multiple image blocks, the first fusion weight is the fusion weight corresponding to each image block in the other images. The second fusion weight corresponding to the reference image can be directly obtained from the input port of the electronic device or a database; that is, it can be input from the terminal or obtained from the stored database. There can also be multiple second fusion weights; when dividing the reference image into multiple image blocks, the second fusion weight is the fusion weight corresponding to each image block in the reference image. It should be noted that the fusion weights corresponding to each image block in the other images or the reference image can be the same or different.

[0069] Optionally, the second fusion weight for the reference image can also be determined based on the displacement change map corresponding to the reference image. In this case, the reference image is used as a reference, and the reference image is registered with itself to obtain the displacement change map corresponding to the reference image. Since the displacement change map represents the displacement change between the registered image features, the displacement change in the displacement change map corresponding to the reference image is always zero. It can be understood that determining the second fusion weight for the reference image based on the displacement change map is the same as determining the first fusion weight based on the displacement change maps corresponding to other images; the only difference is that when determining the second fusion weight, the portion corresponding to the displacement change map becomes zero.

[0070] In an optional embodiment, the first fusion weight corresponding to other images can be determined based on the reference image, the displacement change maps corresponding to other images, and the displacement change maps corresponding to other images. For example, the first fusion weight corresponding to other images can be determined based on the pixel difference between other images and the reference image and the displacement change maps corresponding to other images. When dividing each frame of other images and the reference image into multiple image blocks, for each first image block in each frame of other images, a second image block in the reference image that is registered with the first image block is found, the pixel difference between the first image block and the second image block is determined, and the fusion weight corresponding to the first image block is determined based on the displacement change between the first image block and the second image block, the pixel difference between the first image block and the displacement change between the first image block and the second image block, and the fusion weight corresponding to the other images in the frame of other images is obtained based on the fusion weight corresponding to each first image block in the frame of other images.

[0071] Step 110: Fuse the other images and their corresponding first fusion weights with the reference image and its corresponding second fusion weights to obtain the target image.

[0072] In this embodiment, the fusion can be performed by one other image and one reference image, or by multiple other images and multiple reference images, or by one other image and multiple reference images, or by multiple other images and one reference image. The specific method can be selected according to the requirements of the actual application scenario.

[0073] In some optional embodiments, the other images and their corresponding first fusion weights, and the reference image and its corresponding second fusion weights are weighted and fused to obtain the target image. Optionally, the other images and their corresponding first fusion weights, and the reference image and its corresponding second fusion weights are weighted and averaged to obtain the target image. That is, the other images and their corresponding first fusion weights, and the reference image and its corresponding second fusion weights are weighted and summed, and then divided by the sum of the first and second fusion weights to obtain the target image.

[0074] In some optional embodiments, a target image is obtained by fusing one frame of other images and their corresponding first fusion weights with one frame of reference images and their corresponding second fusion weights. This can improve image processing speed, reduce system operating pressure, and lower the processing requirements of computer hardware.

[0075] In some optional embodiments, the target image is obtained by fusing one frame of other images and their corresponding first fusion weights with multiple reference images and their corresponding second fusion weights. This can preserve the original information of the moving regions, suppress artificial artifacts, achieve better noise reduction, and improve the quality of the target image.

[0076] In some optional embodiments, multiple frames of other images and their corresponding first fusion weights, and a reference image and its corresponding second fusion weights are fused to obtain the target image. Information from the moving parts can be fully incorporated into the fusion weights to regulate them, thereby effectively reducing artifacts in moving regions.

[0077] In some optional embodiments, multiple frames of other images and their corresponding first fusion weights, and multiple frames of reference images and their corresponding second fusion weights are fused to obtain the target image. Optionally, multiple frames of other images can be fused with each frame of reference image separately to obtain target sub-images, and then multiple target sub-images can be fused to obtain the target image. This can effectively reduce artificial artifacts in moving areas and achieve better noise reduction.

[0078] In some optional embodiments, when dividing the reference image and other images into multiple image blocks, fusion is performed on an image block-by-image block basis. For each second image block in the reference image, the corresponding first image block in the other images is fused, that is, the second image block and its corresponding fusion weight are fused with the first image block and its corresponding fusion weight to obtain the target image block in the target image, and the target image is obtained based on the target image block.

[0079] In the above image processing method, multiple frames of images of the same scene are acquired, and a reference image is selected from the multiple frames according to preset image quality screening conditions. Using the reference image as a reference, other images are registered with the reference image to obtain the displacement change map corresponding to the other images. For each frame of other images, the first fusion weight of the other images is determined according to the corresponding displacement change map, and the second fusion weight corresponding to the reference image is obtained. The other images and their corresponding first fusion weights and the reference image and their corresponding second fusion weights are fused to obtain the target image. The information of the moving parts can be included in the fusion weights to adjust the fusion weights, thereby effectively reducing the artificial artifact phenomenon in the moving areas and also achieving noise reduction.

[0080] In some embodiments, step 104, which selects a reference image from multiple frames of images according to preset image quality screening criteria, includes:

[0081] A reference image is selected from multiple frames based on at least one of image sharpness and image brightness.

[0082] Optionally, a reference image can be selected from multiple frames based on image sharpness. For example, selecting at least one reference image from multiple frames based on image sharpness can be achieved by selecting the frame with the highest sharpness among the multiple frames, or by selecting an image with a sharpness threshold as the reference frame image. Sharpness can be represented using any of the following methods: Brenner gradient function, Tenengrad gradient function, Laplacian gradient function, gray-level variance, gray-level variance product, variance function, energy gradient function, volatil function, entropy function, etc. Optionally, image sharpness can be represented by the sum of the absolute differences between pixels in each frame of the multiple frames. For example, for each frame, the sharpness of the corresponding frame image can be represented by calculating the absolute differences between the top and bottom pixels, the left and right pixels, and the diagonal pixels. In a specific example, such as... Figure 2 The image frame shown contains 9 pixels. The sum of the absolute values ​​of the differences in this frame, SAD, is:

[0083]

[0084] The SAD values ​​for other frames follow the same pattern. Optionally, the smaller the sum of the absolute values ​​of the differences between the images, the sharper the image in that frame.

[0085] Optionally, a reference image can be selected from multiple frames based on image brightness. For example, to select at least one reference image from multiple frames based on image brightness, the image whose brightness is closest to the average brightness of the multiple frames can be selected as the reference image, or an image that meets the preset image brightness range can be selected as the reference frame image. In one example, the multiple frames consist of 8 frames, with average image brightness values ​​of 82, 89, 90, 85, 95, 91, 98, and 90 respectively, and an average brightness of 90. In this case, the frame image corresponding to the brightness value closest to the average brightness value is selected as the reference frame image.

[0086] Optionally, a reference image can be selected from multiple frames based on image sharpness and image brightness. The reference image must be at least one frame, and the selected reference image must simultaneously meet preset image sharpness and preset image brightness requirements. The preset image sharpness or preset image brightness requirements can be set with reference to the above embodiments, and will not be repeated here.

[0087] In this embodiment, a reference image is selected from multiple frames based on at least one of image sharpness and image brightness. This allows for convenient and accurate selection of a reference image suitable for the application scenario, which is beneficial for fusing a target image of better quality.

[0088] In one embodiment, such as Figure 3 As shown, step 106, which involves registering other images with the reference image to obtain the displacement change map corresponding to the other images, includes steps 302 to 308.

[0089] Step 302: Divide each multi-frame image into multiple image blocks.

[0090] Following the same partitioning method, each multi-frame image is divided into multiple image blocks. Each image block includes at least one pixel; for example, an image block can be 1×1 pixel, 4×4 pixel, 8×8 pixel, or 16×16 pixel in size.

[0091] Step 304: For each first image patch in other images, search for the second image patch in the reference image that has the highest similarity to the first image patch.

[0092] For each first image patch in each frame of other images, a second image patch with the highest similarity to the first image patch is sequentially searched in the reference image. Optionally, the search can be performed globally in the reference image, or within a preset range in the reference image, the second image patch with the highest similarity to the first image patch can be searched. Optionally, "highest similarity" can refer to the minimum sum of the absolute values ​​of the pixel value differences between corresponding pixels in the matching first and second image patches.

[0093] Step 306: Based on the first position of the first image block and the second position of the second image block, obtain the displacement change corresponding to the first image block.

[0094] The electronic device obtains the displacement change of the first image block based on the first position of the first image block and the second position of the second image block. Optionally, the displacement change of the first image block relative to the second image block can be obtained based on the first coordinates corresponding to the first image block and the second coordinates corresponding to the second image block. Specifically, the vector between the first and second coordinates can be used as the displacement change of the first image block relative to the second image block. Alternatively, the sum of the absolute values ​​of the differences between the horizontal and vertical coordinates corresponding to the first and second coordinates can be used as the displacement change of the first image block relative to the second image block. The coordinates of the image block can be represented using any pixel coordinate within the image block.

[0095] In an optional example, the first coordinate of the first image patch is (x1, y1), and the second coordinate of the second image patch is (x2, y2). Then, the displacement L of the first image patch relative to the corresponding second image patch is:

[0096] L=|x1-x2|+|y1-y2| Formula (2)

[0097] That is, the amount of displacement change of the first image block relative to the second image block can be characterized by a numerical value.

[0098] It should be noted that in image processing, each frame of an image corresponds to a coordinate system, and each pixel in each frame corresponds to corresponding coordinates. When acquiring an image, the electronic device can identify the coordinates of each pixel in the image. In this embodiment, the reference image and other images have the same coordinate system, and pixels at the same position also have the same coordinates. Therefore, the displacement change of the first image block can be obtained based on the first coordinates of the first image block and the second coordinates of the second image.

[0099] Step 308: Based on the displacement change amount corresponding to each first image block in the other images, obtain the displacement change map corresponding to the other images.

[0100] Based on the displacement change of each first image patch in other images, a displacement change map of other images in the frame relative to the reference image is obtained. Optionally, the displacement change of each first image patch is directly placed at the position of the corresponding first image patch to form a displacement change map of other images in the frame. Optionally, a change amount mapping value corresponding to the displacement change of the first image patch can be obtained by setting a change amount threshold and based on the relationship between the displacement change of the first image patch and the change amount threshold; alternatively, a function can be used to perform the mapping to obtain the change amount mapping value. The change amount mapping value can be directly used to characterize the grayscale information of the image; the change amount mapping value corresponding to each first image patch yields the displacement change map of other images in the frame.

[0101] In this embodiment, by dividing each frame of multi-frame images into multiple image blocks, registering the first image block in other images with the second image block in the reference image, and obtaining the displacement change amount corresponding to the first image block based on the position between the registered image blocks, and obtaining the displacement change map corresponding to the other images based on the displacement change amount corresponding to each first image block, that is, by registering on an image block basis, the registration accuracy between other images and the reference image can be improved, thereby improving the quality of the fused target image and effectively suppressing the generation of artificial artifacts.

[0102] Corresponding to the above embodiments, when dividing each multi-frame image into multiple image blocks, such as Figure 4As shown in the illustration, further exemplarily, step 110, which involves fusing other images and their corresponding first fusion weights with a reference image and its corresponding second fusion weights to obtain a target image, includes steps 402 to 406.

[0103] Step 402: Using each second image patch as a reference, search for a first image patch in other images that is registered with the second image patch.

[0104] Using each second image patch in the reference image as a reference, search for first image patches in other images of each frame that are registered with the second image patch. When there are multiple frames in the other images, there are multiple first image patches that are registered with the second image patch.

[0105] Step 404: Weight the second image block and its corresponding second weight, and the first image block and its corresponding first weight, to obtain the target image block in the target image.

[0106] Optionally, a weighted average is performed on the second image block and its corresponding second weight, and on all the first image blocks and their corresponding first weights, to obtain the target image block in the target image. That is, the sum of the product of the second image block and its corresponding second weight, and the product of the first image block and its corresponding first weight, is divided by the sum of the first weight and the second weight to obtain the target image block in the target image.

[0107] In an optional embodiment, the second pixel and its corresponding second weight in the second image block, and the first pixel and its corresponding first weight in the first image block are weighted together to obtain the pixel value of the pixel at the position corresponding to the second pixel in the target image. Optionally, the second pixel and its corresponding second weight in the second image block, and the first pixel and its corresponding first weight in the first image block are weighted and averaged to obtain the pixel value of the pixel at the position corresponding to the second pixel in the target image.

[0108] Optionally, for the first second pixel in a second image block of the reference image, first image blocks registered with the second image block are searched in other images of each frame. The pixel value and corresponding first weight of the first first pixel at the corresponding position in each first image block are obtained. These are then weighted and summed with the pixel value and corresponding second weight of the first second pixel in the second image block. Finally, the sum is divided by the sum of the first weight of the first first pixel in each first image block and the second weight of the first second pixel in the second image block to obtain the pixel value of the first pixel in the target image block at the corresponding position of the second image block. The pixel values ​​of other pixels in the target image block are obtained in the same way. It should be noted that pixels belonging to the same image block have the same weight, which is equal to the weight corresponding to the corresponding image block. For example, the weight of all pixels in the first image block is the same as the first weight corresponding to the first image block, and the weight of all pixels in the second image block is the same as the second weight corresponding to the second image block.

[0109] For example, the second weight corresponding to each second image block in the reference image is a fixed value, that is, the weight of each second pixel in the second image block is a fixed value, for example, 1. An electronic device acquires 6 frames of images for the same scene, selects 1 frame as the reference image, and the remaining 5 frames as other images. That is, the 5 other images are fused with the 1 reference image. Specifically: the first second pixel of the second image block in the reference image is p′1, where the second image block can be any image block in the reference image, and the first pixel can be, for example, the pixel in the upper left corner of the second image block; in the 5 other images, first image blocks registered with the second image blocks are found respectively, resulting in 5 first image blocks. The pixel values ​​of the first pixel in the 5 first image blocks are p1, p2, p3, p4, and p5 respectively; the weights corresponding to the first pixel in the 5 other images are w1, w2, w3, w4, and w5 respectively; in the reference image, the pixel value of the first pixel in the second image block is p′1, then the pixel value P1 of the first pixel in the target image at the position corresponding to the second image block is:

[0110]

[0111] Step 406: Obtain the target image based on each target image block.

[0112] In this embodiment, a target image block can be obtained based on the pixel values ​​corresponding to each pixel in the target image block; and a target image can be obtained based on each target image block.

[0113] In the above embodiments, the second image block is used as a reference, and the second image block and its corresponding second weight, as well as the first image block and its corresponding first weight, are weighted to obtain the target image block in the target image. The target image is obtained based on the target image block. The image block is fused, which can fully integrate the information of the reference image and other images in the target image. The fusion ratio is adjusted according to the corresponding weight, so that the target image can better optimize the information of the moving area, thereby reducing the generation of artificial artifacts and effectively suppressing noise.

[0114] In one embodiment, such as Figure 5 As shown, step 308, which obtains the displacement change map corresponding to each first image block in other images based on the displacement change amount, includes steps 502 to 504.

[0115] Step 502: Based on the relationship between the displacement change amount corresponding to the first image block and the change amount threshold, obtain the change amount mapping value corresponding to the displacement change amount corresponding to the first image block.

[0116] By setting a change threshold, the change mapping value corresponding to the displacement change of the first image block can be obtained based on the relationship between the displacement change of the first image block and the change threshold. Optionally, if the displacement change of the first image block is less than the change threshold, the corresponding change mapping value can be 0; if the displacement change of the first image block is greater than or equal to the change threshold, the corresponding change mapping value can be obtained according to a specific ratio.

[0117] In an optional example, since the search can be conducted within a search range corresponding to a specific search radius in the reference image to find the second image patch that matches the first image patch, assuming the maximum value of the displacement change corresponding to the first image patch is 20 and the change threshold can be set to 4, then: when the displacement change is less than 4, the change mapping value corresponding to the first image patch is 0; when the displacement change is between 4 and 8, the change mapping value is 0.2; when the displacement change is between 8 and 12, the change mapping value is 0.5; when the displacement change is between 12 and 16, the change mapping value is 0.8; and when the displacement change is between 16 and 20, the change mapping value is 1.0. The change mapping values ​​of 0, 0.2, 0.5, 0.8, and 1.0 can directly represent the grayscale values ​​of the corresponding displacement change maps.

[0118] Step 504: Obtain the displacement change map corresponding to each first image block in the other images based on the change mapping value.

[0119] Since the change mapping value can directly represent the grayscale information of the corresponding displacement change map, the displacement change map corresponding to the other images in that frame can be obtained based on the change mapping value corresponding to each first image block in the other images of that frame. The displacement change map can represent the motion region in other images; see reference for details. Figure 6A Other images shown and Figure 6B The diagram shown is a schematic representation of the displacement change.

[0120] In this embodiment, based on the relationship between the displacement change amount corresponding to the first image block and the change amount threshold, the change amount mapping value corresponding to the displacement change amount of the first image block is obtained. Based on the change amount mapping value corresponding to the displacement change amount of each first image block, the displacement change map corresponding to other images of the corresponding frame is obtained. The displacement change map can more prominently display the features of the moving regions in other images, which is conducive to adding the features of the moving regions to the fusion weight and adjusting the corresponding fusion weight to achieve more accurate image fusion and reduce the generation of artificial artifacts.

[0121] In one embodiment, when dividing each multi-frame image into multiple image blocks, such as Figure 7 As shown, in step 108, for each frame of other images, the first fusion weight of other images is determined according to the corresponding displacement change map, which may include the following steps 702 to 706.

[0122] Step 702: For each other image in each frame, calculate the difference between the first image block and the second image block.

[0123] For each frame of other images, the pixel difference between each first image block and its corresponding second image block is calculated. That is, different first image blocks correspond to different second image blocks. The pixel difference between the first and second image blocks can be the sum of the absolute values ​​of the differences in pixel values ​​at corresponding positions in the first and second image blocks.

[0124] Step 704: Determine the first sub-weight of the first image block based on the difference and the change mapping value corresponding to the first image block.

[0125] The first sub-weight of the first image block is determined based on the pixel difference between the first and second image blocks and the change mapping value corresponding to the first image block. The first sub-weight of the first image block is negatively correlated with the change mapping value corresponding to the first image block.

[0126] In one possible implementation, the ratio of the product of the pixel difference between the first image block and the second image block and the corresponding displacement change in the target displacement change map to the sum of the pixel difference and an empirical constant is determined, and the difference between 1 and this ratio is used as the first sub-weight of the first image block. For example, the weight wi of the i-th first image block is:

[0127]

[0128] Where diffi represents the pixel difference between the i-th first image block and the registered second image block, Mi represents the change mapping value corresponding to the i-th first image block, N represents an empirical constant, and wi≥0.

[0129] The first sub-weight of the first image patch is negatively correlated with the corresponding change mapping value. A negative correlation means that the larger the change mapping value, the smaller the first sub-weight. When the change mapping value corresponding to the first image patch is larger, the first sub-weight of the first image patch is smaller. In this case, the first image patch tends to be in a moving region. When the first sub-weight of the first image patch is relatively small, the weight of the second image patch is relatively large, and the fused target image retains more information from the reference image. Conversely, when the displacement change corresponding to the first image patch is smaller, the first sub-weight of the first image patch is larger, and the first image patch tends to be in a non-moving region. When the first sub-weight of the first image patch is relatively large, the second sub-weight of the second image patch in the reference image is relatively small, and the fused target image retains more information from other images.

[0130] In other words, in moving regions, more information from the reference image is fused, while in non-moving regions, more information from other images is fused. For each first image patch in each frame of other images, the first sub-weights are adjusted using the change mapping value corresponding to each first image patch to preserve the superior image features corresponding to each first image patch.

[0131] Step 706: Obtain the first fusion weights of other images based on the first sub-weights of each first image block.

[0132] For each frame of other images, different first image blocks correspond to different first sub-weights. The first fusion weights corresponding to other images in that frame are obtained from each first sub-weight.

[0133] In this embodiment, the first sub-weight of the first image block is determined based on the difference between the first image block and the second image block and the change mapping value corresponding to the first image block. The first fusion weight of other images in the corresponding frame is obtained based on each first sub-weight. The image is divided into multiple image blocks, and the corresponding sub-weights are calculated. This allows the images to be fused in smaller units, improving the accuracy of image fusion. At the same time, the sub-weights corresponding to each image block can be adjusted according to the change mapping value, retaining the better image features in each image block. This ensures that the fused target image retains the high-quality information in the image to the greatest extent, thereby reducing the generation of artificial artifacts.

[0134] In some embodiments, the format of the multi-frame images is raw format. Step 108, which involves determining the first fusion weight of each other image based on its corresponding displacement transformation map and obtaining the second fusion weight corresponding to the reference image, includes:

[0135] For each frame of other images, the first sub-weights corresponding to each channel component in each first image block are determined according to the corresponding displacement change map; and the second sub-weights corresponding to each channel component in each second image block are obtained.

[0136] RAW format images refer to the raw data from which the sensor converts the captured light source signal into a digital signal. Each channel component contains only one color information from RGB. The pixel composition format can be GRBG, RGGB, BGGR, GBRG, RGBW, etc., meaning that four channel components can constitute a corresponding pixel D in RGB. In an example, the data distribution of RAW format images is as follows: Figure 8 As shown, the BGGR format is used as an example for explanation.

[0137] When dividing a multi-frame image in raw format into multiple image blocks, the size of the image blocks must be an even number in order to preserve complete color information, such as dividing them into image blocks of 2×2, 4×4, 6×6 or 8×8 size.

[0138] Optionally, for each first image block of other images in each frame, the first sub-weights corresponding to each channel component in the first image block are determined according to the change mapping value corresponding to the first image block in the corresponding displacement change map; and the second sub-weights corresponding to each channel component in each second image block are obtained. The second sub-weights corresponding to each channel component in each second image block can be obtained directly from the port or database, or they can be obtained by determining the first sub-weights.

[0139] Optionally, the first sub-weight corresponding to the corresponding channel component in the first image block is determined based on the difference between the corresponding channel components of the first image block and the second image block registered with it, and the change mapping value corresponding to the first image block. For example, the first sub-weight corresponding to the R component in the first image block can be determined based on the difference between the R component in the first image block and the second image block, and the change mapping value corresponding to the first image block. The first sub-weights corresponding to different channel components are generally different. The difference between the corresponding channel components between the first image block and the second image block, for the same channel component, can be the sum of the absolute values ​​of the differences between the corresponding channel components at the same position.

[0140] The above embodiments illustrate the determination of the first sub-weights corresponding to each channel component in the first image block and the acquisition of the second sub-weights corresponding to each channel component in the second image block. Next, it is explained how to perform weighted fusion of each channel component in the first image block and each channel component in the second image block when the multi-frame image format is raw.

[0141] like Figure 9 As shown, step 404, which involves weighting the second image block and its corresponding second weight, and the first image block and its corresponding first weight to obtain the target image block in the target image, includes steps 902 to 904.

[0142] Step 902: Weight each channel component and its corresponding first sub-weight in each first image block, and each channel component and its corresponding second sub-weight in the second image block, to obtain each channel component in the fused target image block.

[0143] Optionally, a weighted average is taken of each channel component and its corresponding first sub-weight in each first image block, and each channel component and its corresponding second sub-weight in each second image block, to obtain each channel component in the fused target image block. Alternatively, a weighted average is taken of the channel components and their corresponding first sub-weights in the first image block, and the corresponding channel components and their corresponding second sub-weights in the second image block, to obtain the corresponding channel component in the fused target image block. For example, for a first channel component in a first image block, the first channel component and its corresponding first sub-weight in the first image block are weighted and summed with the first channel component and its corresponding second sub-weight in the second image block, and then divided by the sum of the first and second sub-weights to obtain the first channel component in the fused target image block. Other channel components in the target image block are obtained in the same way.

[0144] Step 904: Obtain the target image based on the channel components in the target image block.

[0145] The target image is obtained based on the channel components in all target image blocks.

[0146] For RAW format images, by processing the raw image data acquired by the image sensor, as much of the original image information as possible can be preserved. At the same time, each channel component is processed separately, that is, each channel component is assigned a different sub-weight, and different channel components are fused separately. This allows for targeted processing of noise in each channel component, thereby achieving a large degree of noise reduction while preserving the original image information and avoiding artificial artifacts.

[0147] In an optional embodiment, for a multi-frame image in raw format, each frame image can be divided into different sub-images according to the channel components, that is, a sub-image of a frame includes only one channel component, and then the sub-images of each channel component are processed separately to obtain the corresponding target sub-images. The target sub-images correspond to each channel component in the target image, and the target image is obtained based on the target sub-images.

[0148] In one embodiment, such as Figure 10 As shown, the image processing method includes steps 1002 to 1012.

[0149] Step 1002: Acquire multiple frames of images of the same scene.

[0150] Step 1004: Select a reference image from multiple frames based on image sharpness.

[0151] Step 1006: Divide each frame of multi-frame images into multiple image blocks; for each first image block of other images, search for the second image block in the reference image that has the highest similarity to the first image block.

[0152] Step 1008: Based on the first position of the first image block and the second position of the second image block, obtain the displacement change amount corresponding to the first image block; based on the displacement change amount corresponding to each first image block in the other images, obtain the displacement change map corresponding to the other images.

[0153] Step 1010: Using each second image patch as a reference, search for a first image patch in other images that is registered with the second image patch.

[0154] Step 1012: Weight the second image block and its corresponding second weight, and the first image block and its corresponding first weight, to obtain the target image block in the target image; and obtain the target image based on the target image block.

[0155] The above embodiments adjust the fusion weights of corresponding image blocks by varying the displacement of the image blocks. This allows for more precise and effective adjustment of moving areas in the image, while remaining unaffected by static or strong edge regions. This reduces artifacts in moving areas and effectively suppresses noise. Actual processing results can be found in [reference needed]. Figure 11A The diagram shown illustrates the traditional weighted fusion method and Figure 11B The diagram shown illustrates the fusion process performed by adjusting the fusion weights using the method described in the above embodiments. Figure 11B The motion area displayed within the box is relatively clear, without any artificial artifacts. Figure 11A There are obvious artifacts in the motion area shown in the box.

[0156] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0157] Based on the same inventive concept, this application also provides an image processing apparatus for implementing the image processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more image processing apparatus embodiments provided below can be found in the limitations of the image processing method described above, and will not be repeated here.

[0158] In one embodiment, such as Figure 12 As shown, an image processing apparatus is provided, including: an image acquisition module 1202, an image filtering module 1204, an image registration module 1206, a weight determination module 1208, and an image fusion module 1210, wherein:

[0159] Image acquisition module 1202 is used to acquire multiple frames of images of the same scene;

[0160] Image filtering module 1204 is used to select reference images from the multi-frame images according to preset image quality filtering conditions;

[0161] The image registration module 1206 is used to register other images with the reference image as a reference to obtain a displacement change map corresponding to the other images; the other images are images other than the reference image in the multi-frame images; the displacement change map is used to characterize the displacement change between the registered images;

[0162] The weight determination module 1208 is used to determine the first fusion weight of the other images in each frame based on the corresponding displacement change map; and to obtain the second fusion weight corresponding to the reference image.

[0163] The image fusion module 1210 is used to fuse the other images and their corresponding first fusion weights and the reference image and its corresponding second fusion weights to obtain the target image.

[0164] In one embodiment, the image filtering module 1204 is further configured to: select the reference image from the multi-frame images based on at least one of image sharpness and image brightness.

[0165] In one embodiment, the image registration module 1206 is further configured to: divide each frame of the multi-frame image into multiple image blocks; for each first image block of the other images, search in the reference image for a second image block with the highest similarity to the first image block; obtain the displacement change amount corresponding to the first image block based on the first position of the first image block and the second position of the second image block; and obtain the displacement change map corresponding to the other images based on the displacement change amount corresponding to each first image block in the other images.

[0166] In some embodiments, the image fusion module 1210 is further configured to: search for a first image block in other images that is registered with the second image block, based on each second image block; weight the second image block and the second weight corresponding to the second image block, and the first image block and the first weight corresponding to the first image block, to obtain a target image block in the target image; and obtain the target image based on each of the target image blocks.

[0167] In one embodiment, the image registration module 1206 is further configured to: obtain a change mapping value corresponding to the displacement change of the first image block based on the relationship between the displacement change amount corresponding to the first image block and the change threshold; and obtain a displacement change map corresponding to the other images based on the change mapping value corresponding to each first image block in the other images.

[0168] In one embodiment, the weight determination module 1208 is further configured to: calculate the difference between the first image block and the second image block for each frame of the other images; determine the first sub-weight of the first image block based on the difference and the change mapping value corresponding to the first image block; and obtain the first fusion weight of the other images based on the first sub-weight of each first image block.

[0169] In one embodiment, the format of the multi-frame images is raw format, and the weight determination module 1208 is further configured to: for each frame of the other images, determine the first sub-weights corresponding to each channel component in each first image block according to the corresponding displacement change map; and obtain the second sub-weights corresponding to each channel component in each second image block.

[0170] The image fusion module 1210 is further configured to: weight each channel component and its corresponding first sub-weight in each of the first image blocks, and each channel component and its corresponding second sub-weight in the second image blocks, to obtain each channel component in the fused target image block; and obtain the target image based on each channel component in the target image block.

[0171] Each module in the aforementioned image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0172] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 13 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores displacement maps or second fusion weight data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements an image processing method.

[0173] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0174] This application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, cause the processors to perform the steps of an image processing method.

[0175] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform an image processing method.

[0176] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0177] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0178] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0179] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image processing method, characterized by, include: Acquire multiple frames of images of the same scene; A reference image is selected from the multi-frame images based on at least one of image sharpness and image brightness; Using the reference image as a reference, image blocks in other images are registered with the image blocks in the reference image to obtain displacement change maps corresponding to the other images; the other images are images other than the reference image in the multi-frame images; the displacement change map is used to characterize the amount of displacement change between the registered images; For each frame of the other images, a first fusion weight for the other images is determined based on the corresponding displacement change map; Obtain the second fusion weight corresponding to the reference image; The other images and their corresponding first fusion weights are fused with the reference image and its corresponding second fusion weights to obtain the target image.

2. The method of claim 1, wherein, The step of registering image blocks in other images with the image blocks in the reference image to obtain displacement change maps corresponding to the other images includes: Each frame of the multi-frame image is divided into multiple image blocks; For each first image patch in the other images, a second image patch with the highest similarity to the first image patch is searched in the reference image; Based on the first position of the first image block and the second position of the second image block, the displacement change corresponding to the first image block is obtained; Based on the displacement change amount corresponding to each first image block in the other images, a displacement change map corresponding to the other images is obtained; The step of fusing the other images and their corresponding first fusion weights with the reference image and its corresponding second fusion weights to obtain the target image includes: Using each second image patch as a reference, search for a first image patch in the other images that is registered with the second image patch; The second image block and its corresponding second weight, and the first image block and its corresponding first weight are weighted together to obtain the target image block in the target image; The target image is obtained based on each of the target image blocks.

3. The method of claim 2, wherein, The step of obtaining the displacement change map corresponding to the other images based on the displacement change amount corresponding to each first image block in the other images includes: Based on the relationship between the displacement change amount corresponding to the first image block and the change amount threshold, the change amount mapping value corresponding to the displacement change amount corresponding to the first image block is obtained. Based on the change mapping value corresponding to each first image block in the other images, the displacement change map corresponding to the other images is obtained.

4. The method of claim 3, wherein, The step of determining the first fusion weight of the other images for each frame based on the corresponding displacement change map includes: For each of the other images in a frame, calculate the difference between the first image block and the second image block; The first sub-weight of the first image block is determined based on the difference and the change mapping value corresponding to the first image block; The first fusion weights of the other images are obtained based on the first sub-weights of each of the first image blocks.

5. The method of claim 2, wherein, The format of the multi-frame images is raw format. For each frame of the other images, determining the first fusion weight of the other images based on the corresponding displacement transformation map, and obtaining the second fusion weight corresponding to the reference image, includes: For each frame of other images, based on the corresponding displacement change map, the first sub-weights corresponding to each channel component in each first image block are determined; and the second sub-weights corresponding to each channel component in each second image block are obtained. The step of weighting the second image block and its corresponding second weight, and the first image block and its corresponding first weight, to obtain the target image block in the target image includes: The channel components and their corresponding first sub-weights in each of the first image blocks, and the channel components and their corresponding second sub-weights in each of the second image blocks are weighted to obtain the channel components in the fused target image block. The target image is obtained based on the channel components in the target image block.

6. The method according to claim 1, characterized in that, The step of fusing the other images and their corresponding first fusion weights with the reference image and its corresponding second fusion weights to obtain the target image includes: The other images and their corresponding first fusion weights, and the reference image and its corresponding second fusion weights are weighted and fused to obtain the target image.

7. The method according to claim 1, characterized in that, The step of fusing the other images and their corresponding first fusion weights with the reference image and its corresponding second fusion weights to obtain the target image includes: The target image is obtained by fusing one frame of the other images and their corresponding first fusion weights with one frame of the reference image and their corresponding second fusion weights; or, The target image is obtained by fusing one frame of the other images and their corresponding first fusion weights with multiple frames of the reference images and their corresponding second fusion weights; or, The target image is obtained by fusing multiple frames of the other images and their corresponding first fusion weights with one frame of the reference image and its corresponding second fusion weights; or, The target image is obtained by fusing the other images and their corresponding first fusion weights from multiple frames, and the reference images and their corresponding second fusion weights from multiple frames.

8. An image processing apparatus, characterized in that, include: The image acquisition module is used to acquire multiple frames of images of the same scene; An image filtering module is used to select a reference image from the multi-frame images based on at least one of image sharpness and image brightness; An image registration module is used to register image blocks in other images with the image blocks in the reference image, using the reference image as a reference, to obtain a displacement change map corresponding to the other images; the other images are images other than the reference image in the multi-frame images; the displacement change map is used to characterize the amount of displacement change between the registered images; The weight determination module is used to determine the first fusion weight of the other images for each frame based on the corresponding displacement change map. Obtain the second fusion weight corresponding to the reference image; The image fusion module is used to fuse the other images and their corresponding first fusion weights with the reference image and its corresponding second fusion weights to obtain the target image.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, the processor performs the steps of the image processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images