Image fusion method, storage medium and terminal device

Abstract

The present application discloses an image fusion method, storage medium and terminal device, wherein the method divides a plurality of acquired image frames into a first image group and a second image group based on exposure; respectively determines a first weight map corresponding to the first image group and a second weight map corresponding to the second image group; and finally fuses the plurality of image frames based on the determined first weight map and second weight map to obtain an output image. In this way, the first weight map corresponding to the first image group and the second weight map corresponding to the second image group are determined, and the plurality of image frames are fused according to the fusion weight map corresponding to the first weight map and the second weight map, so that the image frames with different exposures correspond to different weight maps, and the weight maps corresponding to the image frames with different exposures are used to improve their respective effects, thereby improving the image effect of the fused output image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image fusion method, a storage medium and a terminal device. Background Art

[0002] At present, in order to obtain clear and natural images in both highlights and shadows, multi-exposure fusion technology is usually used to determine the image. Multi-exposure fusion technology usually obtains multiple frames of multi-exposure images and then fuses the multiple frames of multi-exposure images to obtain an output image. However, the existing multi-exposure fusion technology has the problem that the image effect of the fused output image is poor. Summary of the invention

[0003] The technical problem to be solved by the present application is to provide an image fusion method, a storage medium and a terminal device in view of the deficiencies of the prior art.

[0004] In order to solve the above technical problems, the first aspect of the embodiment of the present application provides an image fusion method, the method comprising:

[0005] Acquire several image frames;

[0006] dividing the plurality of image frames into a first image group and a second image group based on exposure;

[0007] Determining a first weight map corresponding to the first image group and a second weight map corresponding to the second image group;

[0008] Based on the determined first weight map and the second weight map, a plurality of image frames are fused to obtain an output image.

[0009] The image fusion method, wherein the similarity of the image content of each image frame in the plurality of image frames satisfies a preset condition.

[0010] The image fusion method, wherein the several image frames are all original image data, and for each first image in the first image group, the exposure corresponding to the first image is less than or equal to the preset exposure; for each second image in the second image group, the exposure corresponding to the second image is greater than the preset exposure.

[0011] The image fusion method, wherein determining the first weight map corresponding to the first image group specifically includes:

[0012] For each first image in the first image group, determine a first grayscale image corresponding to the first image, and determine first pixel weights corresponding to each pixel in the first image based on the first grayscale image to obtain a first weight matrix corresponding to the first image;

[0013] Based on first weight matrices respectively corresponding to all first images in the first image group, a first weight map corresponding to the first image group is determined.

[0014] The image fusion method, wherein for each first image in the first image group, determining a first grayscale image corresponding to the first image, and determining first pixel weights corresponding to each pixel in the first image based on the first grayscale image to obtain a first weight matrix corresponding to the first image specifically includes:

[0015] For each first image in the first image group, determining a first grayscale image corresponding to the first image;

[0016] Based on a preset first Gaussian curve, determining a first distance between each pixel point in the first grayscale image and a first preset expected value;

[0017] The determined first distances corresponding to the respective pixels are used as the first pixel weights corresponding to the respective pixels, so as to obtain a first weight matrix corresponding to the first image.

[0018] The image fusion method, wherein, for each first image in the first image group, after determining the first grayscale image corresponding to the first image, the method further comprises:

[0019] The first grayscale image is normalized, and the normalized first grayscale image is used as the first grayscale image corresponding to the first image.

[0020] The image fusion method, wherein the determining the second weight map corresponding to the second image group specifically comprises:

[0021] For each second image in the second image group, determine a second grayscale image corresponding to the second image; determine a mean matrix corresponding to the second image based on the second grayscale image; and determine a second weight matrix corresponding to the second image based on the mean matrix and the second grayscale image;

[0022] A second weight map corresponding to the second image group is determined based on the second weight matrices respectively corresponding to all the second images in the second image group.

[0023] The image fusion method, wherein the step of determining the second weight matrix corresponding to the second image based on the mean matrix and the second grayscale image specifically includes:

[0024] Based on a preset second Gaussian curve, determine the second distance between each pixel point in the mean matrix and the second expected value, and use the determined second distances as the second pixel point weights corresponding to each pixel point, so as to obtain a first temporary weight matrix;

[0025] Determine a second temporary weight matrix corresponding to the second image based on a preset third Gaussian curve and the second grayscale image;

[0026] A second weight matrix corresponding to the second image is determined based on the first temporary weight matrix and the second temporary weight matrix.

[0027] The image fusion method, wherein the step of determining the second temporary weight matrix corresponding to the second image based on the preset third Gaussian curve and the second grayscale image specifically includes:

[0028] Determine a guided filter map corresponding to the second image based on the second grayscale map;

[0029] Based on a preset third Gaussian curve, a third distance between each pixel point in the guided filter image and a third expected value is determined, and each determined third distance is used as a third pixel point weight corresponding to each pixel point to obtain a second temporary weight matrix.

[0030] In the image fusion method, the plurality of image frames are all original image data; and obtaining the plurality of image frames specifically comprises:

[0031] Acquire several image frames;

[0032] For each image frame, the channel of the image frame is separated according to the color order corresponding to the image frame to obtain a multi-channel image;

[0033] The obtained multi-channel images are used as a number of image frames.

[0034] The image fusion method, wherein the step of fusing a plurality of image frames based on the determined first weight map and the second weight map to obtain an output image specifically comprises:

[0035] Based on the determined first weight map and the second weight map, each color channel of each image frame is fused to obtain a fused image;

[0036] The fused image is converted into a single-channel image in color order, and the single-channel image is used as an output image.

[0037] The image fusion method, wherein the step of fusing the color channels of the image frames to obtain the fused image based on the determined first weight map and the second weight map specifically includes:

[0038] Selecting a target image frame from the plurality of image frames, and using the other image frames except the target image frame from the plurality of image frames as reference image frames;

[0039] For each color channel in the target image frame, respectively determine a reference color channel in each reference image frame corresponding to the color channel, wherein the channel number of each reference color channel in the corresponding reference image frame is the same as the channel number of the color channel in the target image frame;

[0040] Based on the first weight map and the second weight map, weighting the color channel and each reference color channel to obtain a fusion channel corresponding to the color channel;

[0041] Based on all the determined fusion channels, fusion images corresponding to the plurality of image frames are determined.

[0042] A second aspect of an embodiment of the present application provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in any of the image fusion methods described above.

[0043] A second aspect of an embodiment of the present application provides a terminal device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program that can be executed by the processor;

[0044] The communication bus realizes the connection and communication between the processor and the memory;

[0045] When the processor executes the computer-readable program, the steps in any of the above-mentioned image fusion methods are implemented.

[0046] Beneficial effects: Compared with the prior art, the present application provides an image fusion method, storage medium and terminal device, wherein the method divides a plurality of acquired image frames into a first image group and a second image group based on the exposure; respectively determines a first weight map corresponding to the first image group and a second weight map corresponding to the second image group; and finally, based on the determined first weight map and second weight map, fuses the plurality of image frames to obtain an output image. In this way, the first weight map corresponding to the first image group and the second weight map corresponding to the second image group are determined, and the plurality of image frames are fused according to the fusion weight map corresponding to the first weight map and the second weight map, so that the image frames with different exposures correspond to different weight maps, and the weight maps corresponding to the image frames with different exposures are used to improve their respective effects, thereby improving the image effect of the fused output image. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without inventive work.

[0048] Figure 1 This is a flowchart of the image fusion method provided in this application.

[0049] Figure 2 This is an example of an image captured directly by an image acquisition device.

[0050] Figure 3 This is an example diagram of an output image obtained by fusion using the image fusion method provided in this application.

[0051] Figure 4 This is a schematic diagram of the structure of the terminal device provided in this application. DETAILED DESCRIPTION

[0052] The present application provides an image fusion method, a storage medium, and a terminal device. To make the purpose, technical solution, and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0054] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0055] After research, the inventor found that in a night scene environment, in order to obtain an image that is clear in both highlights and shadows, multiple frames of multi-exposure images are usually acquired, and then processed through fusion technology to obtain an image with clear highlights and shadows. This is because the image produced by a single exposure cannot suppress the highlight area and brighten the shadow area, while the multi-exposure image contains image data of the same scene with different exposure amounts. The long exposure image can be used to brighten the shadow area, while the short exposure image can be used to suppress the highlight area. By fusing the multi-exposure images, an image with clear highlights and shadows can be obtained.

[0056] However, when using multi-exposure fusion technology to determine an image, it is necessary to determine the fusion weight map corresponding to multiple image frames, wherein the fusion weight map is generally obtained based on the quality assessment of local image features, such as the most common calculation of the fusion weight map by the contrast, saturation and exposure of the image, and the extraction of the brightness component weights of different image layers after decomposing the image in detail, etc. The weight matrices corresponding to each image frame in the fusion weight map are all determined by the same method, which cannot reflect the suppression effect of the short-exposure image on the highlight area and the brightening effect of the long-exposure image on the shadow area, resulting in poor image effect of the fused output image.

[0057] In order to solve the above problem, in an embodiment of the present application, the acquired image frames are divided into a first image group and a second image group based on the exposure; the first weight map corresponding to the first image group and the second weight map corresponding to the second image group are determined respectively; finally, based on the determined first weight map and the second weight map, the image frames are fused to obtain an output image. In this way, the first weight map corresponding to the first image group and the second weight map corresponding to the second image group are determined, and the image frames are fused according to the fusion weight map corresponding to the first weight map and the second weight map, so that the image frames with different exposures correspond to different weight maps, and the weight maps corresponding to the image frames with different exposures are used to improve their respective effects, thereby improving the image effect of the fused output image.

[0058] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0059] This implementation provides an image fusion method, such as Figure 1 As shown, the method includes:

[0060] S10. Acquire several image frames.

[0061] Specifically, the several image frames may be images acquired by an electronic device (e.g., a smart phone, etc.) using the image fusion method, and the several image frames may be original images acquired by an image acquisition device, wherein the image acquisition device may be configured on an electronic device running the image fusion method, or may be configured on other external devices, and the acquired original images are sent to the electronic device running the image fusion method through the external device. In a possible implementation of this embodiment, the image acquisition device is configured on an electronic device running the image fusion method, so that after the electronic device acquires the several image frames, the several image frames can be directly fused, and after acquiring the several image frames, the electronic device can directly output the fused images corresponding to the several image frames, thereby improving the real-time performance of the fusion of the several image frames.

[0062] Further, each of the plurality of image frames corresponds to each other, wherein each of the image frames corresponds to each other in the same image size, and any two of the plurality of image frames are recorded as the first image frame and the second image frame, respectively, and the first image frame and the second image frame correspond to the same image scene. The first image frame and the second image frame correspond to the same image scene, which means that the similarity between the image content carried by the first image frame and the image content carried by the second image frame meets a preset condition. It can be understood that the image size of the first image frame is the same as the image size of the second image frame, and when the first image frame and the second image frame overlap, the coverage rate of the object carried by the first image frame to the object corresponding to it in the second image frame meets a preset condition. Wherein, the preset condition is a preset threshold, the similarity meeting the preset condition means that the similarity reaches the preset threshold, and the coverage rate meeting the preset condition means that the coverage rate reaches the preset threshold, and the preset threshold is preset, for example, 99%.

[0063] Further, in an implementation of the present embodiment, the plurality of image frames include at least two image frames, which are respectively recorded as image frame A and image frame B, and the exposure corresponding to image frame A is different from the exposure corresponding to image frame B. For example, the exposure corresponding to image frame A is EV-8, and the exposure corresponding to image frame B is EV+12, wherein EV (Exposure Values) is used to reflect the exposure, and the exposure is used to reflect the amount of light entering the aperture when shooting an image, that is, the amount of light entering the aperture during the process from shutter opening to closing; and when the sensitivity is ISO 100, the aperture factor is F1, and the exposure time is 1 second, the exposure is defined as 0. In addition, EV-N indicates that the exposure is reduced by N levels (for example, the shutter time is reduced to 1 / N or the aperture is reduced to 1 / N), EV+N; the exposure is increased by N levels (the shutter time is increased by N times or the aperture is increased by N levels), wherein N is a positive integer. In a possible implementation of this embodiment, the exposure parameters (such as aperture coefficient and sensitivity, etc.) corresponding to several image frames are the same, and image frames with different exposure amounts in the several image frames are generated by adjusting the exposure time corresponding to the image frames. For example, for image frame A with an exposure amount of EV-8 and image frame B with an exposure amount of EV+12, the exposure time corresponding to image frame A is less than the exposure time corresponding to image frame B, wherein the exposure time refers to the time interval from the opening to the closing of the shutter, and the aperture blades of the camera device lens can leave the impact of the object on the film to leave an image within the time interval, and when the exposure time of the camera device is long, more light enters the aperture; when the exposure time of the camera device is short, less light enters the aperture.

[0064] Based on this, in one implementation of this embodiment, a plurality of image frames may be obtained by taking multiple shots of the same shooting scene by an imaging device configured with the same exposure parameters, and the exposure of some of the multiple image frames with different exposures is less than the preset exposure, which is recorded as under-exposed image frames, and the exposure of some of the image frames is greater than the preset exposure, which is recorded as over-exposed image frames, wherein the preset exposure may be 0, which is the exposure when the sensitivity is ISO 100, the aperture factor is F1, and the exposure time is 1 second. Of course, among the multiple image frames with different exposures, there may be some image frames whose exposure is equal to the preset exposure, which is recorded as normal exposure image frames.

[0065] For example: a number of image frames are image frames acquired by an image acquisition device through different exposure times, and a number of image frames are 5 image frames with different exposure values, and the 5 image frames include a normal exposure image frame, an underexposed image frame, and an overexposed image frame. The specific exposure values ​​of the 5 image frames can be: EV-24, EV-16, EV-8, EV0, and EV+12, that is, the 5 image frames include 3 underexposed image frames, 1 normal exposure image frame, and 1 overexposed image frame.

[0066] Furthermore, in order to improve the fusion speed of several image frames, in one implementation of this embodiment, several image frames are raw image data RAW, wherein the raw image data can be the raw data of the light source signal captured by the CMOS (Complementary Metal-Oxide-Semiconductor) or CCD (Charge Coupled Device) image sensor converted into a digital signal, and the raw image data is the image data that has not been processed and compressed. In this embodiment, by avoiding the discarding of image details caused by the image processing and compression process, each image frame contains each image detail during fusion, thereby improving the image quality of the fused output. In addition, by using the raw image data RAW as the image frame, there is no need to decode the image frame during the fusion process, which can reduce the time required for the fusion of several image frames, thereby improving the real-time performance of image fusion.

[0067] S20 , dividing a plurality of image frames into a first image group and a second image group based on exposure.

[0068] Specifically, the exposure value is used to reflect the amount of light entering the aperture when capturing an image, that is, the amount of light entering the aperture during the process from the shutter opening to closing. The time taken during the process from the shutter opening to closing is the exposure time, and when the exposure parameters are fixed, the longer the exposure time, the more light enters the aperture, that is, the greater the exposure; conversely, the shorter the exposure time, the less light enters the aperture, that is, the smaller the exposure.

[0069] Furthermore, the first image group includes at least one image frame, and the second image group includes at least one image frame, where the image frames in the first image group are recorded as first images, and the image frames in the second image group are recorded as second images. Each first image in the first image group is an image frame among a number of image frames, and each second image in the second image group is an image frame among a number of image frames, and for any first image in the first image group, there is no second image in the second image group that is identical to the first image; similarly, for any second image in the second image group, there is no first image in the first image group that is identical to the second image. It can be understood that the union of the first image group and the second image group is the number of image frames, and the intersection of the first image group and the second image group is an empty set.

[0070] Further, in an implementation of the present embodiment, the first image group and the second image group are obtained by division based on an exposure threshold. Accordingly, the division of the plurality of image frames into the first image group and the second image group based on the exposure may specifically include: for each image frame in the plurality of image frames, comparing the exposure corresponding to the image frame with a preset exposure threshold; if the exposure corresponding to the image frame is greater than the preset exposure, adding the image frame to the second image group; if the exposure corresponding to the image frame is less than or equal to the preset exposure, adding the image frame to the first image group, thereby obtaining the first image group and the second image group. In addition, it can be known from the division process of the first image group and the second image group that for each first image in the first image group, the exposure corresponding to the first image is less than or equal to the preset exposure; for each second image in the second image, the exposure corresponding to the second image is greater than the preset exposure.

[0071] Furthermore, the preset exposure can be 0, and the preset exposure is the exposure when the sensitivity is ISO 100, the aperture factor is F1, and the exposure time is 1 second, which is referred to as normal exposure; there is at least one first image in the first image group, and the exposure corresponding to the first image is less than the preset exposure. The first image is referred to as an under-exposed image, and the highlight area can be suppressed by the under-exposed image. In addition, the exposure of the second image in the second image group is greater than the preset exposure. The second image is referred to as an over-exposed image, and the over-exposed image can be used to brighten the shadow area. In this way, by fusing several image frames including the under-exposed image and the over-exposed image, both the highlight area and the shadow area can be clear, thereby improving the image effect of the fused output image.

[0072] S30: Determine a first weight map corresponding to the first image group and a second weight map corresponding to the second image group.

[0073] Specifically, the first weight map is used to reflect the weight of the fusion process of several image frames of each first image in the first image group; the second weight map is used to reflect the weight of the fusion process of several image frames of each second image in the second image group. The first weight image includes several image channels, the number of channels of the several image channels is the same as the number of first images included in the first image group, and the several image channels correspond one-to-one to the several first images in the first image group. For example, the first image group includes 3 first images, namely, first image A, first image B and first image C, then the first weight image is a 3-channel image, namely, image channel 1, image channel 2 and image channel 3; then the first image A corresponds to image channel 1, the first image B corresponds to image channel 2, and the first image C corresponds to image channel 3.

[0074] For each first image, the image channel corresponding to the first image is used to reflect the weight of the first image in the fusion process of several image frames. It can be understood that the image size of the first image is the same as the image size corresponding to the image channel, and for each pixel in the image channel, the pixel value is the weight value corresponding to the target pixel in the first image, and the pixel position of the target pixel in the first image is the same as the pixel position of the pixel in the image channel. For example, for the first image A, the image channel corresponding to it in the first weight image is image channel 1, and the pixel value corresponding to the pixel point (10,20) in image channel 1 is the weight corresponding to the pixel point (10,20) in the first image A.

[0075] Furthermore, the second weighted image includes a plurality of image channels, the number of channels of the plurality of image channels is the same as the number of second images included in the second image group, and the plurality of image channels correspond one-to-one to the plurality of second images in the second image group. For example, the second image group includes three second images, namely, second image A, second image B, and second image C, then the second weighted image is a three-channel image, namely, image channel 1, image channel 2, and image channel 3; then the second image A corresponds to image channel 1, the second image B corresponds to image channel 2, and the second image C corresponds to image channel 3.

[0076] Further, for each second image, the image channel corresponding to the second image is used to reflect the weight of the second image in the fusion process of several image frames. It can be understood that the image size of the second image is the same as the image size corresponding to the image channel, and for each pixel in the image channel, the pixel value is the weight value corresponding to the target pixel in the second image, and the pixel position of the target pixel in the second image is the same as the pixel position of the pixel in the image channel. For example, for the second image A, the image channel corresponding to it in the second weight image is image channel 1, and the pixel value corresponding to the pixel point (30,20) in image channel 1 is the weight corresponding to the pixel point (30,20) in the second image A.

[0077] Further, in an implementation of this embodiment, determining the first weight map corresponding to the first image group specifically includes:

[0078] For each first image in the first image group, determine a first grayscale image corresponding to the first image, and determine first pixel weights corresponding to each pixel in the first image based on the first grayscale image to obtain a first weight matrix corresponding to the first image;

[0079] Based on first weight matrices respectively corresponding to all first images in the first image group, a first weight map corresponding to the first image group is determined.

[0080] Specifically, a grayscale image (Gray Scale Image or Grey Scale Image) is also called a grayscale image, and is an image represented by grayscale. The grayscale is formed by dividing white and black into several levels (such as 256, etc.) according to a logarithmic relationship, wherein each level in the several levels is a grayscale. In addition, since the original image data is single-channel data, before determining the first grayscale image corresponding to the first image, the original image data needs to be separated into color channels to obtain multi-channel original image data, wherein the color of each pixel in each channel in the multi-channel original image data is the same.

[0081] Based on this, in an implementation of this embodiment, acquiring a plurality of image frames specifically includes:

[0082] Acquire several image frames;

[0083] For each image frame, the channel of the image frame is separated according to the color order corresponding to the image frame to obtain a multi-channel image;

[0084] The obtained multi-channel images are used as a number of image frames.

[0085] Specifically, the color sequence is the color arrangement sequence of each pixel in the image frame, and performing channel separation on the image frame according to the color sequence corresponding to the image frame refers to performing color channel separation on the image frame data according to the color sequence of each pixel in the image frame. Figure 2 As shown, the image frame is the original image data of H*W*1, and the color order of the image frame is RGBG, where H represents the height of the first image, W represents the width of the first image, and 1 represents the color channel data of the first image. Then the image frame can generate a multi-channel image of H / 2*W / 2*4 after color channel separation, where H / 2 represents the height of the first image after color channel separation, W / 2 represents the width of the first image after color channel separation, and 4 represents the color channel data of the first image after color channel separation. Here, the four color channels are respectively recorded as the first color channel 1, the second color channel 2, the third color channel 3, and the fourth color channel 4, where the first color channel stores R pixels, the second color channel stores G pixels, the third color channel stores B pixels, and the fourth color channel stores G pixels; R pixels represent red pixels, G pixels represent green pixels, and B pixels represent blue pixels. In addition, after each image frame is divided into its corresponding multi-channel images, several multi-channel images are used as several image frames, so that each image frame is a four-channel image, and the color of the pixels of each channel in the four-channel image is the same.

[0086] Further, after acquiring the four-channel image frame, for each image frame, the RGBG data of the four channels of the image frame are averaged to generate a grayscale image corresponding to the image frame. The process of averaging the RGBG data of the four channels of the image frame can be: for each pixel point in the R channel, the G channel, the B channel and the reference pixel point corresponding to the pixel point in the G channel are respectively calculated, and the average value of the pixel value of each target pixel point and the pixel value of the pixel point is calculated, and the average value is used as the pixel value of the target pixel point in the grayscale image. The pixel position of each reference pixel point in its corresponding channel is the same as the pixel position of the pixel point in the R channel, and the pixel position of the target pixel point in the grayscale image is the same as the pixel position of the pixel point in the R channel. For example, the first color channel is the R channel, the second color channel is the G channel, the third color channel is the B channel, and the fourth color channel is the G channel. The red pixel value of the pixel point (10, 15) in the first color channel is 60; the green pixel value of the pixel point (10, 15) in the second color channel is 60, the blue pixel value of the pixel point (10, 15) in the third color channel is 55, and the green pixel value of the pixel point (10, 15) in the fourth color channel is 65. Then the grayscale value of the pixel point (10, 15) in the grayscale image is (60+60+55+65) / 4=60.

[0087] In an implementation of this embodiment, after obtaining the first grayscale image corresponding to each first image, the first grayscale image may be normalized. Accordingly, for each first image in the first image group, after determining the first grayscale image corresponding to the first image, the method further includes: normalizing the first grayscale image, and using the normalized first grayscale image as the first grayscale image corresponding to the first image. The normalization process is to divide the original image by the maximum value of the stored image. For example, if the maximum number of bits of the device to store images is 10 bits, then the maximum value of the stored image is 1023.

[0088] Further, after obtaining the first grayscale images corresponding to each first image, a mean filter (for example, a filter kernel of 9*9, etc.) may be used to process the first grayscale images corresponding to each first image to obtain each filtered grayscale image. After obtaining each filtered grayscale image, for each filtered first grayscale image, a first weight matrix corresponding to the grayscale image may be determined based on the first grayscale image. Accordingly, in an implementation of the embodiment, for each first image in the first image group, determining the first grayscale image corresponding to the first image, and determining the first pixel weights corresponding to each pixel in the first image based on the first grayscale image to obtain the first weight matrix corresponding to the first image specifically includes:

[0089] For each first image in the first image group, determining a first grayscale image corresponding to the first image;

[0090] Based on a preset first Gaussian curve, determining a first distance between each pixel point in the first grayscale image and a first preset expected value;

[0091] The determined first distances corresponding to the respective pixels are used as the first pixel weights corresponding to the respective pixels, so as to obtain a first weight matrix corresponding to the first image.

[0092] Specifically, the first Gaussian curve is preset and used to determine the Gaussian distance of each pixel in the first grayscale image. The first preset expected value is preset and used to represent the expected Gaussian distance corresponding to each pixel. For each pixel, the first distance corresponding to the pixel is the distance between the Gaussian distance corresponding to the pixel and the first preset expected value. The calculation formula of the Gaussian distance can be:

[0093]

[0094] Among them, W k is the kth weight matrix, k = 1, 2, ..., M, M is the number of first images in the first image group; P ij,kis the pixel value of the pixel point with coordinates i, j in the kth grayscale image; σ is the standard deviation, for example, 0.18, etc.; u is the first expected value; wherein the first expected value can be determined according to the exposure amount of the first image corresponding to the first grayscale image.

[0095] Further, after determining the first distance corresponding to each pixel, each first distance is used as the first pixel weight corresponding to each pixel, so as to obtain the first pixel weight value corresponding to each pixel in the first grayscale image. In addition, after obtaining the first pixel weight value corresponding to each pixel in the first grayscale image, each first pixel weight value is used as the element value of the target element corresponding to the pixel in the first weight matrix, wherein the position of the target element in the first weight matrix corresponds to the position of the pixel in the grayscale image (for example, the position of the target element in the first weight matrix is ​​(35,45), and the position of the pixel in the grayscale image is (35,45)); finally, the first weight matrix is ​​used as the first weight matrix corresponding to the first image.

[0096] For example: the first image group includes 4 first images, namely, first image A, first image B, first image C and first image D, among which, the exposure of first image A is EV-24, the exposure of first image B is EV-16, the exposure of first image C is EV-8 and the exposure of first image D is EV-0, then the expected value corresponding to first image A is 0.6, the expected value corresponding to first image B is 0.45, the expected value corresponding to first image C is 0.3 and the expected value corresponding to first image D is 0.15.

[0097] Further, the determination process of the first weight map may be to arrange each first weight matrix in the channel direction to obtain the first weight map. It is understandable that the first weight map is a multi-channel image, the number of channels of the first weight map is the same as the number of images of the first image in the first image group, and each image channel in the first weight map corresponds to one weight matrix in several first weight matrices, and the first weight matrices corresponding to each image channel are different from each other. In one implementation of this embodiment, in order to quickly determine the weight matrix corresponding to each first image, when generating the first weight map, each weight matrix may be sorted according to the shooting time of the first image corresponding to it, for example, in ascending order of shooting time, or in descending order of shooting time, etc. Here, the sorting is explained in ascending order of shooting time, then for any two adjacent channels in the first weight image, the channel A and the channel B, if the channel number of channel A is less than the channel number of channel B, then the shooting time of the first image corresponding to channel A is earlier than the shooting time of the first image corresponding to channel B, and the first image A is adjacent to the first image B.

[0098] For example: the first image group includes 3 image frames, namely the first image frame, the second image frame and the third image frame. The shooting time corresponding to the first image frame is 11:55:25, the shooting time corresponding to the second image frame is 11:55:26, and the shooting time corresponding to the third image frame is 11:55:28. Then the channel number of the weight matrix corresponding to the first image frame in the first weight map is 0, the channel number of the second image frame in the corresponding weight matrix in the first weight map is 1, and the channel number of the third image frame in the corresponding weight matrix in the first weight map is 1.

[0099] Further, in an implementation of this embodiment, determining the second weight map corresponding to the second image group specifically includes:

[0100] For each second image in the second image group, determine a second grayscale image corresponding to the second image; determine a mean matrix corresponding to the second image based on the second grayscale image; and determine a second weight matrix corresponding to the second image based on the mean matrix and the second grayscale image;

[0101] A second weight map corresponding to the second image group is determined based on the second weight matrices respectively corresponding to all the second images in the second image group.

[0102] Specifically, the grayscale image (Gray Scale Image or Grey Scale Image) is also called a grayscale image, and is an image represented by grayscale. The grayscale is formed by dividing white and black into several levels (such as 256, etc.) according to a logarithmic relationship, wherein each level in the several levels is a grayscale. In addition, since the original image data is single-channel data, before determining the grayscale image corresponding to the first image, the original image data needs to be separated into color channels to obtain multi-channel original image data, wherein the color of each pixel in each channel of the multi-channel original image data is the same. In addition, the determination process of the second grayscale image corresponding to each second image in the second image group is the same as the determination process of the first grayscale image corresponding to each first image in the first image group, which will not be described one by one here, and the specific process can refer to the determination process of the first grayscale image corresponding to the first image. Of course, it is worth explaining that before determining the second grayscale image corresponding to each second image, it is also necessary to perform color channel separation on each second image to convert each second image into a multi-channel image, wherein the color channel separation process for the second image is the same as the color channel separation process for the first image, which will not be explained one by one. For details, please refer to the color channel separation process for the first image.

[0103] Further, the values ​​of each element in the mean matrix are equal and are equal to the average value of the pixel values ​​of all pixels in the grayscale image. Wherein, the determination process of the mean matrix can be: first, create a two-dimensional matrix, the matrix dimension of the two-dimensional matrix is ​​the same as the image size of the grayscale image, and each element in the two-dimensional matrix corresponds to a target pixel in the grayscale image, and each element in the two-dimensional matrix is ​​used to reflect the pixel weight value of the corresponding target pixel; wherein, the position of the target pixel in the grayscale corresponds to the position of the element in the two-dimensional matrix, for example, the position of the element in the two-dimensional matrix is ​​(30,40), then the pixel position of the target pixel corresponding to the element in the grayscale image is (30,40); secondly, after determining the two-dimensional matrix, set each element value in the two-dimensional matrix to a preset value (for example, 1, etc.), and calculate the mean of the pixel values ​​of all pixels in the grayscale image; finally, replace the element value of each target element in the two-dimensional matrix with the calculated mean to obtain the mean matrix. For example, if the element value of each element in the two-dimensional matrix is ​​1, and the average pixel value of all pixels in the grayscale image is 5, then the two-dimensional matrix can be multiplied by the average pixel value to make the element value of each element in the two-dimensional matrix 5. Of course, when the element value in the two-dimensional matrix is ​​a non-1 value (such as 3, etc.), the element value of each element in the two-dimensional matrix can be replaced by the average pixel value (such as 5, etc.) to make the element value of each element in the two-dimensional matrix the average pixel value (such as 5, etc.).

[0104] Further, in an implementation of this embodiment, determining a second weight matrix corresponding to the second image based on the mean matrix and the second grayscale image specifically includes:

[0105] Based on a preset second Gaussian curve, determine the second distance between each pixel point in the mean matrix and the second expected value, and use the determined second distances as the second pixel point weights corresponding to each pixel point, so as to obtain a first temporary weight matrix;

[0106] Determine a second temporary weight matrix corresponding to the second image based on a preset third Gaussian curve and the second grayscale image;

[0107] A second weight matrix corresponding to the second image is determined based on the first temporary weight matrix and the second temporary weight matrix.

[0108] Specifically, the second Gaussian curve is preset and used to determine the Gaussian distance of each pixel in the mean matrix. The second preset expected value is preset and used to represent the expected Gaussian distance corresponding to each pixel. For each pixel, the second distance corresponding to the pixel is the distance between the Gaussian distance corresponding to the pixel and the first preset expected value. The calculation formula of the Gaussian distance can be:

[0109]

[0110] Among them, W k is the k-th transformed mean matrix, k = 1, 2, ..., N, N is the number of second images in the second image group; P ij,k is the pixel value of the pixel point with coordinates i, j in the mean matrix before the kth transformation; σ is the standard deviation, for example, 018, etc.; u is the second expected Gaussian distance; wherein the second expected Gaussian distance can be determined according to the exposure of the second image corresponding to the mean matrix. For example, if the exposure of the second image A is EV+12, then the expected value corresponding to the second image A is 0.05. Of course, it is worth noting that the element values ​​of each element in the mean matrix corresponding to each second image are equal. Then, when the element values ​​in the mean matrix are converted to weight values ​​by Gaussian distance, a target element can be selected in the mean matrix, and the weight value corresponding to the target element can be calculated by the Gaussian distance formula. Then, the original values ​​of all elements in the mean matrix are set to the weight value to obtain the converted mean matrix. In this way, the calculation steps of the mean matrix conversion can be reduced, thereby increasing the acquisition speed of the converted mean matrix, thereby increasing the speed of image fusion.

[0111] Further, in an implementation of this embodiment, determining the second temporary weight matrix corresponding to the second image based on the preset third Gaussian curve and the second grayscale image specifically includes:

[0112] Determine a guided filter map corresponding to the second image based on the second grayscale map;

[0113] Based on a preset third Gaussian curve, a third distance between each pixel point in the guided filter image and a third expected value is determined, and each determined third distance is used as a third pixel point weight corresponding to each pixel point to obtain a second temporary weight matrix.

[0114] Specifically, the guided filter image is obtained by performing guided filtering on the second grayscale image, wherein the guided filter can be a guided filter with a half-price of 21 and a regularization parameter of 0.12. The third Gaussian curve is preset, the third expected value is preset, and based on the preset third Gaussian curve, the third distance between each pixel point and the third expected value in the guided filter image is determined in the same manner as the process of determining the first distance based on the first Gaussian curve, which will not be described one by one here. In addition, after obtaining the second grayscale image corresponding to each second image, the second grayscale image can be normalized. Correspondingly, for each second image in the second image group, after determining the second grayscale image corresponding to the second image, the method further includes: normalizing the second grayscale image, and using the normalized second grayscale image as the second grayscale image corresponding to the second image. Wherein, the normalization process is to divide the original image by the maximum value of the stored image. For example, if the maximum number of bits of the device to store images is 10 bits, then the maximum value of the stored image is 1023.

[0115] Furthermore, after obtaining the first temporary weight matrix and the second temporary weight matrix, the matrix dimension of the first temporary weight matrix is ​​the same as the matrix dimension of the second temporary weight matrix, and both are equal to the image size of the second image, where the matrix dimension refers to the number of rows and columns of the matrix. For example, if the image size of the second image is 224*224, then the matrix dimension of the first temporary weight matrix is ​​224 rows and 224 columns, and the matrix dimension of the second temporary weight matrix is ​​224 rows and 224 columns. Based on this, after obtaining the first temporary weight matrix and the second temporary weight matrix, the second weight matrix corresponding to the grayscale image can be determined by the matrix dot product operation of the first temporary weight matrix and the second temporary weight matrix, that is, the second weight matrix of the second image corresponding to the second grayscale image is obtained.

[0116] Further, after obtaining the second weight matrices corresponding to each second grayscale image, each second weight matrix is ​​arranged in the channel direction to obtain a second weight map. It can be understood that the second weight map is a multi-channel image, and the number of channels of the second weight map is the same as the number of images of the second image in the second image group. In one implementation of this embodiment, in order to quickly determine the second weight matrix corresponding to each second image, when generating the second weight map, each second weight matrix can be sorted according to the shooting time of the corresponding first image, for example, in ascending order of shooting time, or in descending order of shooting time, etc. Here, the sorting is explained in ascending order of shooting time. Then for any two adjacent channels in the second weight image, channel A and channel B, if the channel number of channel A is less than the channel number of channel B, the shooting time of the second image A corresponding to channel A is earlier than the shooting time of the second image B corresponding to channel B, and the second image A is adjacent to the second image B.

[0117] S40: Based on the determined first weight map and the second weight map, fuse several image frames to obtain an output image.

[0118] Specifically, the output image is a fusion of several image frames to obtain an image frame, wherein the output image can be used as a captured image output by an image acquisition device. After obtaining the first weight map and the second weight map, the first weight map includes a weight matrix corresponding to each first image, and the second weight map includes a weight matrix corresponding to each second image. Thus, the first weight map and the second weight map can be spliced ​​in the channel direction to obtain a weight map, wherein the weight map includes a weight matrix corresponding to each image frame in the several image frames. In addition, when splicing the first weight map and the second weight map, the first weight map can be located before the second weight map or after the second weight map in the channel direction.

[0119] For example: the first weight map is a 4-channel image with channel numbers 0, 1, 2 and 3 respectively, and the second weight image is a single-channel image with channel number 0; then the weight map obtained by splicing the first weight map and the second weight map is a 5-channel image; when the second weight map is located after the first weight map in the channel direction, the channel with channel number 0 in the second weight image becomes the channel with channel number 4 in the weight map; when the second weight map is located before the first weight map in the channel direction, the channel with channel number 0 in the first weight map becomes the channel with channel number 1 in the weight map; the channel with channel number 1 in the first weight map becomes the channel with channel number 2 in the weight map; the channel with channel number 2 in the first weight map becomes the channel with channel number 3 in the weight map; the channel with channel number 3 in the first weight map becomes the channel with channel number 4 in the weight map.

[0120] Based on this, in an implementation of this embodiment, the step of fusing the color channels of the image frames to obtain the fused image based on the determined first weight map and the second weight map specifically includes:

[0121] Selecting a target image frame from the plurality of image frames, and using the other image frames except the target image frame from the plurality of image frames as reference image frames;

[0122] For each color channel in the target image frame, respectively determine a reference color channel in each reference image frame corresponding to the color channel, wherein the channel number of each reference color channel in the corresponding reference image frame is the same as the channel number of the color channel in the target image frame;

[0123] Based on the first weight map and the second weight map, weighting the color channel and each reference color channel to obtain a fusion channel corresponding to the color channel;

[0124] Based on all the determined fusion channels, fusion images corresponding to the plurality of image frames are determined.

[0125] Specifically, the target image frame is one of the plurality of image frames, and the reference image frame is all the image frames except the target image frame in the plurality of image frames, wherein the target image frame may be any image frame in the plurality of image frames, or may be the image frame that is located at the front in the shooting order in the plurality of image frames, or may be the image frame that is located at the back in the shooting order in the plurality of image frames, etc. For example, the plurality of image frames include image frame A, image frame B, and image frame C, and image frame A is taken as the target image frame, and image frame B and image frame C are reference image frames.

[0126] Further, the color channels included in the target image frame correspond to the color channels included in each reference image frame, so that for each color channel in the target image frame, there is a reference color channel in each reference image frame corresponding to the color channel, wherein the correspondence means that for each reference color channel, the channel number of the reference color channel in the reference image frame to which it belongs is the same as the channel number of the color channel in the target image frame. For example, a number of image frames include image frame A, image frame B, and image frame C, image frame A is the target image frame, image frame B and image frame C are reference image frames, image frame A includes four color channels, image frame B and image frame C both include four color channels, and the channel numbers of the four color channels in image frame A are 0, 1, 2, and 3, respectively; and the channel numbers of the four color channels in image frame B and image frame C are also 0, 1, 2, and 3, respectively. Then, the reference color channel A with channel number 0 in image frame B and the reference color channel B with channel number 0 in image frame C are both reference color channels corresponding to the color channel A with channel number 0 in image frame A.

[0127] Furthermore, before weighted processing is performed on the color channel and each reference color channel based on the first weight map and the second weight map, the first weight map and the second weight map need to be spliced ​​in the channel direction to obtain a weight map, wherein the weight map includes a plurality of image channels, the number of the plurality of image channels is the same as the number of the plurality of image frames, and the plurality of channels correspond to the plurality of image frames one by one, and each channel is used to reflect the weight coefficient of the corresponding image frame in the fusion process of the plurality of image frames. For example, if the plurality of image frames include 5 image frames, then the weight map is a 5-channel image, and the 5 channels correspond to the 5 image frames one by one.

[0128] Furthermore, after obtaining the weight map, each pixel of the weight map is normalized, and the normalized weight map is used as the weight map, wherein the normalization operation formula is as follows:

[0129]

[0130] in, is the pixel value with coordinates i,j in the kth weight map after normalization, W ij,k is the pixel value with coordinates i, j in the kth weight map, M is the number of first images in the first image group, N is the number of second images in the second image group, and M+N is the number of images in the plurality of image frames.

[0131] Furthermore, after the normalized weight map, each image frame is multiplied with its corresponding normalized weight matrix (image channel) to update the pixel value of each pixel in each image frame. For example, image frame A includes pixel (0,0), pixel (0,1), pixel (1,1) and pixel (1,0), and the weight matrix corresponding to image frame A is 00 , a 01 , a 11 and a 10 , pixel (0,0) and a 00 , the updated pixel value corresponding to the pixel point (0,0) is the pixel value before the update corresponding to the pixel point (0,0) and a 00 For example, the pixel value before updating corresponding to the pixel point (0,0) is 50, a 00 is 0.5, then the updated pixel value corresponding to the pixel point (0,0) is 0.5*50=25; similarly, the updated pixel value corresponding to the pixel point (0,1) is the pixel value before the update corresponding to the pixel point (0,1) and a 01 The updated pixel value corresponding to the pixel point (1,1) is the product of the pixel value before the update corresponding to the pixel point (1,1) and a 11 The updated pixel value corresponding to the pixel point (1,0) is the product of the pixel value before the update corresponding to the pixel point (1,0) and a 10 The product of .

[0132] Further, after obtaining each updated image frame, the pixel values ​​of the corresponding pixel positions in each updated image frame are added, and the pixel value obtained by the addition is used as the pixel value of the pixel position in the output image to obtain the output. For example, the plurality of image frames include image frame A and image frame B, and the image scale of the output image obtained by fusing image frame A and image frame B is the same as the image scale of image frame A, wherein the image scale of image frame A is the same as the image scale of image frame B; and the pixel value of any pixel point (i, j) in the output image is equal to the pixel value of the pixel point (i, j) of image frame A and the pixel value of the pixel point (i, j) of image frame B.

[0133] Further, in an implementation of this embodiment, after acquiring the plurality of image frames, each of the plurality of image frames is subjected to color channel separation. Thus, when the output image is acquired, the output image is the original image data after the color channel separation. Therefore, when the plurality of image frames are fused, the channel images of the color channels in each image frame are fused. Accordingly, the fusion of the plurality of image frames to obtain the output image based on the determined first weight map and the second weight map specifically includes:

[0134] Based on the determined first weight map and the second weight map, each color channel of each image frame is fused to obtain a fused image;

[0135] The fused image is converted into a single-channel image in color order, and the single-channel image is used as an output image.

[0136] Specifically, after obtaining the first weight map and the second weight map, the weight map for fusion is determined based on the first weight map and the second weight map. The process of obtaining the weight map has been described in the above implementation method and will not be repeated here. After obtaining the weight map, each weight matrix A (image channel) in the weight map corresponds to an image frame A, and each color channel in the image frame A corresponds to the weight map. It can be understood that the weight matrices corresponding to the color channels of image frame A are all weight matrix A. For example, image frame A includes a first color channel (R channel), a second color channel (G channel), a third color channel (B channel) and a fourth color channel (G channel). Then the first color channel (R channel), the second color channel (G channel), the third color channel (B channel) and the fourth color channel (G channel) all correspond to weight matrix A.

[0137] Based on this, fusing the color channels of each image frame to obtain a fused image can be fusing the color channels with channel number 0 of each image frame, fusing the color channels with channel number 1 of each image frame, fusing the color channels with channel number 2 of each image frame, fusing the color channels with channel number 3 of each image frame, and using the fused color channels corresponding to each channel number as the color channels of the fused image to obtain the fused image. In addition, after obtaining the fused image, the channels of the fused image are rearranged according to the color order of the image frames to obtain the output image. For example, the fused images are R channel, G channel 1, B channel and G channel 2 in the order of channel numbers, and the image size of the fused image is a multi-channel image of H / 2*W / 2*4, where H / 2 represents the height of the fused image, W / 2 represents the width of the fused image, and 4 represents the fused image; then create a H*W*1 reference image, where H represents the height of the reference image, W represents the width of the reference image, and 1 represents the number of color channels of the reference image; add the pixels of each color channel in the fused image to the reference image in the order of channel numbers, namely, pixels in R channel, pixels in G channel 1, pixels in B channel, and pixels in G channel 2, to obtain the output image.

[0138] For example: the fused images are R channel, G channel 1, B channel and G channel 2 in the order of channel numbers, where the R channel includes the pixel point (0,0) R 、(0,1) R 、(1,0) R and (1,1) R , G channel 1 includes pixel (0,0) G1 、(0,1) G1 、(1,0) G1 and (1,1) G1 , the B channel includes the pixel (0,0) B 、(0,1) B 、(1,0) B and (1,1) B , G channel 2 includes pixel (0,0) G2 、(0,1) G2 、(1,0) G2 and (1,1) G2 , then the pixel arrangement of the first row of the output image is: (0,0) R 、(0,0) G1 、(0,0) B 、(0,0) G2 ; The second line is (0,1) G1 、(0,1) R 、(0,1) B 、(0,1) G2; The third line is (1,0) B 、(1,0) G2 、(1,0) R 、(1,0) G1 , the fourth line is (1,1) G2 、(1,1) R 、(1,1) G1 、(1,1) B .

[0139] In summary, the present application provides an image fusion method, storage medium and terminal device, wherein the method divides a plurality of acquired image frames into a first image group and a second image group based on exposure; respectively determines a first weight map corresponding to the first image group and a second weight map corresponding to the second image group; and finally, based on the determined first weight map and second weight map, fuses the plurality of image frames to obtain an output image. In this way, the first weight map corresponding to the first image group and the second weight map corresponding to the second image group are determined, and the plurality of image frames are fused according to the fusion weight map corresponding to the first weight map and the second weight map. This can improve the correspondence between image frames with different exposure amounts and different weight maps, and improve the respective effects through the weight maps corresponding to image frames with different exposure amounts, thereby improving the image effect of the fused output image. For example, Figure 2 is an image of the shooting scene directly captured by an imaging device, such as Figure 3 is an image obtained by fusing a plurality of image frames corresponding to the shooting scene according to the image fusion method of this embodiment, Figure 2 and Figure 3 It can be seen that Figure 3 The clarity of the text in Figure 2 The clarity of the text.

[0140] Based on the above-mentioned image fusion method, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the image fusion method described in the above-mentioned embodiment.

[0141] Based on the above image fusion method, the present application also provides a terminal device, such as Figure 4As shown, it includes at least one processor (processor) 20; display screen 21; and memory (memory) 22, and may also include a communications interface (Communications Interface) 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0142] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0143] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0144] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0145] In addition, the specific process of loading and executing the multiple instruction processors in the above-mentioned storage medium and the terminal device has been described in detail in the above-mentioned method, and will not be described one by one here.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image fusion method, characterized in that: The method comprises: Acquire several image frames; dividing the plurality of image frames into a first image group and a second image group based on exposure; Determining a first weight map corresponding to the first image group and a second weight map corresponding to the second image group; the first weight map is determined according to pixel values ​​of a grayscale image corresponding to the first image group; Based on the determined first weight map and the second weight map, a plurality of image frames are fused to obtain an output image; Determining the second weight map corresponding to the second image group specifically includes: Determining a first temporary weight matrix based on a preset second Gaussian curve and a second grayscale image corresponding to each second image in the second image group; Determine a guided filtering image corresponding to each second image based on a second grayscale image corresponding to each second image in the second image group; Based on a preset third Gaussian curve, determine a third distance between each pixel point in the guided filter image and a third expected value, and use each determined third distance as a third pixel point weight corresponding to each pixel point, so as to obtain a second temporary weight matrix corresponding to each second image; Determine a second weight matrix corresponding to the second image based on the first temporary weight matrix and the second temporary weight matrix; Determining a second weight map corresponding to the second image group based on second weight matrices respectively corresponding to all second images in the second image group; Determining the first weight map corresponding to the first image group specifically includes: For each first image in the first image group, determining a first grayscale image corresponding to the first image; Based on a preset first Gaussian curve, determining a first distance between each pixel point in the first grayscale image and a first preset expected value; The determined first distances corresponding to the respective pixels are used as the first pixel weights corresponding to the respective pixels, so as to obtain a first weight matrix corresponding to the first image; Based on the first weight matrices respectively corresponding to all the first images in the first image group, a first weight map corresponding to the first image group is determined.

2. The image fusion method according to claim 1, characterized in that: The similarity of the image content of each image frame in the plurality of image frames meets a preset condition.

3. The image fusion method according to claim 1, characterized in that: The several image frames are all original image data, and for each first image in the first image group, the exposure corresponding to the first image is less than or equal to the preset exposure; for each second image in the second image group, the exposure corresponding to the second image is greater than the preset exposure.

4. The image fusion method according to claim 1, characterized in that: After determining the first grayscale image corresponding to each first image in the first image group, the method further includes: The first grayscale image is normalized, and the normalized first grayscale image is used as the first grayscale image corresponding to the first image.

5. The image fusion method according to any one of claims 1 to 3, characterized in that: The plurality of image frames are all original image data; the obtaining of the plurality of image frames specifically includes: Acquire several image frames; For each image frame, the channel of the image frame is separated according to the color order corresponding to the image frame to obtain a multi-channel image; The obtained multi-channel images are used as a number of image frames.

6. The image fusion method according to claim 5, characterized in that: The step of fusing a plurality of image frames based on the determined first weight map and the second weight map to obtain an output image specifically includes: Based on the determined first weight map and the second weight map, each color channel of each image frame is fused to obtain a fused image; The fused image is converted into a single-channel image in color order, and the single-channel image is used as an output image.

7. The image fusion method according to claim 6, characterized in that: The step of fusing the color channels of the image frames to obtain the fused image based on the determined first weight map and the second weight map specifically includes: Selecting a target image frame from the plurality of image frames, and using the other image frames except the target image frame from the plurality of image frames as reference image frames; For each color channel in the target image frame, respectively determine a reference color channel in each reference image frame corresponding to the color channel, wherein the channel number of each reference color channel in the corresponding reference image frame is the same as the channel number of the color channel in the target image frame; Based on the first weight map and the second weight map, weighting the color channel and each reference color channel to obtain a fusion channel corresponding to the color channel; Based on all the determined fusion channels, fusion images corresponding to the plurality of image frames are determined.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the image fusion method according to any one of claims 1 to 7.

9. A terminal device, characterized in that: include: Processor, memory and communication bus; The memory stores a computer-readable program executable by the processor; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, the steps in the image fusion method according to any one of claims 1 to 7 are implemented.

Images