Image Fusion Method, Apparatus, Electronic Device, and Storage Medium

Abstract

The present application discloses an image fusion method, apparatus, electronic device, and storage medium, belonging to the field of shooting technology. The image fusion method includes: collecting N frames of first images, each frame of the first images including M pixels, where both N and M are positive integers; dividing a first target image into P first pixel groups, the first target image being any one of the N frames of first images, and P being a positive integer; obtaining the optical parameters of a target pixel group among the P first pixel groups; determining at least one second target image among the N frames of first images according to the optical parameters of the target pixel group, the second target image including a second pixel group that matches the optical parameters of the target pixel group, the second target image being an image among the N frames of first images, the target pixel group including Q pixels, and Q < M; and performing a fusion process on the target pixel group and the second pixel group.

Description

Technical Field

[0001] This application belongs to the field of shooting technology, and particularly relates to an image fusion method, apparatus, electronic device, and storage medium. Background Art

[0002] In the related art, the brightness of a photographed image is positively correlated with the amount of light entering the sensor. When taking pictures in an environment with complex lighting, due to too long exposure time, the bright areas will be overexposed, and due to too short exposure time, the dark areas will be underexposed. As a result, the image obtained by the one-time exposure imaging method usually cannot guarantee the image quality.

[0003] To ensure the image quality of the photographed image, usually multiple short-exposure images are acquired, and image fusion is performed on all the entire images in the multiple short-exposure images to obtain the finally synthesized image, resulting in motion blur in the finally synthesized image. Summary of the Invention

[0004] The objective of the embodiments of this application is to provide an image fusion method, apparatus, electronic device, and storage medium, which avoid the problem of blur in the fused image caused by the fusion of some pixels with other pixels that do not match the optical parameters, and improve the clarity of the finally synthesized image.

[0005] In a first aspect, the embodiments of this application provide an image fusion method, including: collecting N first images, each of the N first images including M pixels, where N and M are both positive integers; dividing a first target image into P first pixel groups, the first target image being any one of the N first images, and P being a positive integer; obtaining the optical parameters of a target pixel group in the P first pixel groups, the target pixel group being any one of the P first pixel groups; determining at least one second target image in the N first images according to the optical parameters of the target pixel group, the second target image including a second pixel group that matches the optical parameters of the target pixel group, the second target image being an image in the N first images, the target pixel group including Q pixels, where Q < M; and performing a fusion process on the target pixel group and the second pixel group, the position of the target pixel group in the first target image corresponding to the position of the second pixel group in the second target image.

[0006] In a second aspect, an image fusion device provided by an embodiment of the present application includes: an acquisition module, configured to acquire N first images, each first image including M pixels, where N and M are both positive integers; a grouping module, configured to divide a first target image into P first pixel groups, the first target image being any one of the N first images, and P being a positive integer; an acquisition module, configured to acquire the optical parameters of a target pixel group among the P first pixel groups, the target pixel group being any one of the P first pixel groups; a determination module, configured to determine at least one second target image among the N first images according to the optical parameters of the target pixel group, the second target image including a second pixel group that matches the optical parameters of the target pixel group, the second target image being an image among the N first images, and the target pixel group including Q pixels; a fusion module, configured to perform a fusion process on the target pixel group and the second pixel group, where the position of the target pixel group in the first target image corresponds to the position of the second pixel group in the second target image.

[0007] In a third aspect, an electronic device provided by an embodiment of the present application includes a processor and a memory, the memory storing a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the method in the first aspect are implemented.

[0008] In a fourth aspect, a readable storage medium provided by an embodiment of the present application stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method in the first aspect are implemented.

[0009] In a fifth aspect, a chip provided by an embodiment of the present application includes a processor and a communication interface, the communication interface being coupled to the processor, and the processor being configured to run a program or instruction to implement the steps of the method in the first aspect.

[0010] In a sixth aspect, a computer program product provided by an embodiment of the present application is stored in a storage medium, and the program product is executed by at least one processor to implement the method in the first aspect.

[0011] In the embodiments of the present application, when an electronic device captures a captured object in a moving state, multiple frames of first images including the captured object are collected. The first target image among the N frames of first images is divided into multiple first pixel groups, and a target pixel group is obtained from the multiple first pixel groups. Through the optical parameters of the target pixel group, a second pixel group in a second target image that can be fused with the target pixel group can be found. Since the second pixel group is found according to the optical parameters of the target pixel group, it can be ensured that the optical parameters between the second pixel group and the target pixel group for fusion match. After fusing each target pixel group with the corresponding second pixel and then stitching, an image with motion blur removed can be obtained. In the embodiments of the present application, the pixel group is used as the unit of image fusion, so that each target pixel group is fused with a matching second pixel group, which can avoid the problem of blurring in the fused image caused by some pixels being fused with other pixels with mismatched optical parameters, and improve the clarity of the finally synthesized image.

[0012] In the embodiments of the present application, the pixel group is used as the unit of image fusion, and only the optical parameters of the whole pixel group need to be calculated, without fusing each pixel in the pixel group separately. Since the pixels in the target pixel group are strongly correlated, using the pixel group as the unit of image fusion can reduce the computational amount of the electronic device while ensuring the accuracy of image fusion processing. It realizes reducing the data processing amount required for image fusion while ensuring the clarity of the fused image. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 Shows a schematic flowchart of the image fusion method provided by the embodiments of the present application;

[0014] Figure 2 Shows the curve of the change in the light flux value provided by the embodiments of the present application;

[0015] Figure 3 Shows one of the schematic diagrams of the pixel arrangement of the target pixel group provided by the embodiments of the present application;

[0016] Figure 4 Shows another schematic diagram of the pixel arrangement of the target pixel group provided by the embodiments of the present application;

[0017] Figure 5 Shows a third schematic diagram of the pixel arrangement of the target pixel group provided by the embodiments of the present application;

[0018] Figure 6 Shows one of the schematic diagrams of the first pixel group in the first target image provided by the embodiments of the present application;

[0019] Figure 7Schematic diagram II of the first pixel group in the first target image provided by an embodiment of the present application;

[0020] Figure 8 Schematic block diagram of an image fusion device provided by an embodiment of the present application;

[0021] Figure 9 Schematic block diagram of an electronic device according to an embodiment of the present application;

[0022] Figure 10 Hardware structure schematic diagram of an electronic device for implementing an embodiment of the present application. Specific embodiments

[0023] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0024] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. generally belong to the same category, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the associated objects before and after.

[0025] Next, in conjunction with the attached Figures 1 to 10 , the image fusion method, image fusion device, electronic device, and storage medium provided by the embodiments of the present application will be described in detail through specific embodiments and their application scenarios.

[0026] In some embodiments of the present application, an image fusion method is provided. Figure 1 Schematic flowchart of the image fusion method provided by an embodiment of the present application. As Figure 1 shown, the image fusion method includes:

[0027] Step 102, collect N frames of first images, each frame of the first image includes M pixels, and both N and M are positive integers;

[0028] During the process of multi-segment exposure shooting by the electronic device at a preset frame rate, the electronic device can collect N first images. Since the pixels in the N first images are all collected by the same sensor, the number of pixels in each first image is the same and each includes M pixels.

[0029] Step 104: Divide the first target image into P first pixel groups. The first target image is any one of the N first images, and P is a positive integer.

[0030] The first target image is the image among the N first images for image fusion processing. The first target image can be selected as any one of the N first images. In subsequent processing, this first target image is used as the initial image for fusion processing with other images.

[0031] Divide all the pixels of each first image into P first pixel groups. The number of pixels in each first pixel group needs to be less than the number of pixels in the first target image, and each first pixel group includes at least one pixel.

[0032] It should be noted that the number of pixels in each of the P first pixel groups can be the same or different. And the pixel arrangement forms in multiple first pixel groups can be the same or different.

[0033] Exemplarily, when it is determined that the number of pixels in each first pixel group is 12, the arrangement form of the first pixel group can be 2×6, 3×4, or 1×12.

[0034] Step 106: Obtain the optical parameters of the target pixel group among the P first pixel groups. The target pixel group is any one of the P first pixel groups.

[0035] The target pixel group is any one of the P first pixel groups. During the process of fusing the first target image, each of the P first pixel groups can be used as the target pixel group, or some of the P first pixel groups can be used as the target pixel group.

[0036] Step 108: Determine at least one second target image among the N first images according to the optical parameters of the target pixel group. The second target image includes a second pixel group that matches the optical parameters of the target pixel group. The second target image is an image among the N first images. The target pixel group includes Q pixels, where Q < M.

[0037] The second target image is screened from the N first images according to the optical parameters of the target pixel group. It should be noted that the second target image is different from the first target image, and the number of second target images is at most N - 1 frames, that is, the second target image can be all the images among the N first images except the first target image.

[0038] The second pixel group in the second target image matches the optical parameters of the target pixel group. When the parameter difference between the optical parameters of the second pixel group and the optical parameters of the target pixel group is less than a preset difference, it is determined that the optical parameters of the second pixel group match the optical parameters of the target pixel group.

[0039] The position of the second pixel group in the second target image corresponds to the position of the target pixel group in the first target image. The pixel matrices corresponding to the first target image and the second target image are the same, and the position of the target pixel group in the pixel matrix is the same as the position of the first pixel group in the pixel matrix.

[0040] Step 110: Perform a fusion process on the target pixel group and the second pixel group. The position of the target pixel group in the first target image corresponds to the position of the second pixel group in the second target image.

[0041] In the embodiments of the present application, the electronic device acquires multiple frames of first images, divides the first target image in the multiple frames of first images into P first pixel groups, and searches for the second target image in the multiple frames of first images according to the target pixel group in the P first pixel groups. The target pixel group is fused according to the second pixel group in the corresponding second target image. After all the target pixel groups in the P first pixel groups are fused, a fused image is obtained. The image fused through the above process can avoid the problem of motion blur caused by the movement of the shooting object during the shooting process.

[0042] Specifically, in the process of searching for the second target image in the multiple frames of first images according to the target pixel group, it is necessary to first determine the third pixel group in the images other than the first target image in the multiple frames of first images according to the target pixel group. Obtain the optical parameters of the target pixel group and the optical parameters of multiple third pixel groups, and determine the image corresponding to the second pixel group that matches the optical parameters of the target pixel group among the multiple third pixel groups as the second target image.

[0043] Exemplarily, determining whether the optical parameters of the third pixel group match the optical parameters of the target pixel group includes: calculating the difference between the optical parameters of the third pixel group and the optical parameters of the target pixel group, and determining the third pixel group with a difference less than the preset difference as the second pixel group that matches the target pixel group.

[0044] The optical parameters include any one of the following: luminous flux value, number of photoelectrons.

[0045] In some possible embodiments, the optical parameter includes only one of the luminous flux value or the number of photo-electrons. When the difference between the luminous flux value of the third pixel group and that of the target pixel group is less than a preset difference, or when the difference between the number of photo-electrons of the third pixel group and that of the target pixel group is less than a preset difference, it is determined that the third pixel group is the second pixel group matching the target pixel group.

[0046] In some possible embodiments, the optical parameter includes the luminous flux value and the number of photo-electrons. When the difference between the luminous flux value of the third pixel group and that of the target pixel group is less than a preset difference, and when the difference between the number of photo-electrons of the third pixel group and that of the target pixel group is less than a preset difference, it is determined that the third pixel group is the second pixel group matching the target pixel group. It is also possible to determine that the third pixel group is the second pixel group matching the target pixel group when the difference between the luminous flux value of the third pixel group and that of the target pixel group is less than a preset difference, or when the difference between the number of photo-electrons of the third pixel group and that of the target pixel group is less than a preset difference.

[0047] It should be noted that the luminous flux value can be obtained by calculation. The number of photo-electrons is the number of pixels in the pixel group in the lit state, and can be collected and counted by a sensor.

[0048] Figure 2 The variation curve of the luminous flux value provided by the embodiment of the present application is shown, as Figure 2 shown, the electronic device acquires 1000 frames of the third image, and the luminous flux value of the target pixel group has two obvious changes in the 1000 frames of the third image. Among them, the change amount of the first change is 0.6, and the change amount of the second change is 1. When the preset change amount is 0.8, the moment corresponding to the second change is determined as the moment of sudden change of the luminous flux value, that is, the first moment.

[0049] In the embodiments of the present application, when an electronic device captures a photographed object in a moving state, multiple first images including the photographed object are collected. The first target image among N first images is divided into multiple first pixel groups, and a target pixel group is obtained from the multiple first pixel groups. Through the optical parameters of the target pixel group, a second pixel group in the second target image that can be fused with the target pixel group can be found. Since the second pixel group is found according to the optical parameters of the target pixel group, it can be ensured that the optical parameters between the second pixel group and the target pixel group for fusion match. After each target pixel group is fused with the corresponding second pixel and then stitched together, an image with motion blur removed can be obtained. In the embodiments of the present application, the pixel group is used as the unit for image fusion, so that each target pixel group is fused with a matching second pixel group, which can avoid the problem of blurring in the fused image caused by some pixels being fused with other pixels with unmatched optical parameters, and improve the clarity of the finally synthesized image.

[0050] In the embodiments of the present application, the pixel group is used as the unit for image fusion, and only the optical parameters of the pixel group as a whole need to be calculated, without fusing each pixel in the pixel group individually. Since the pixels in the target pixel group are strongly correlated, using the pixel group as the unit for image fusion can reduce the computational amount of the electronic device while ensuring the accuracy of the image fusion process. It realizes reducing the data processing amount required for image fusion while ensuring the clarity of the fused image.

[0051] In some embodiments of the present application, obtaining the optical parameters of the target pixel group in P first pixel groups includes: determining X target pixels in the target pixel group, where X is a positive integer and X ≤ Q; and determining the optical parameters of the target pixel group according to the optical parameters of the X target pixels.

[0052] In the embodiments of the present application, the X target pixels are partial pixels in the target pixel group. The value range of the number of target pixels is greater than 1 and less than or equal to Q. After determining the X target pixels, the optical parameters of the target pixel group are determined according to the optical parameters corresponding to the X target pixels.

[0053] It should be noted that the selection rule for the X target pixels can be set in advance before the electronic device leaves the factory.

[0054] In the case where the number of target pixels is 1, the optical parameters of the target pixel are directly used as the optical parameters of the target pixel group. Exemplarily, when the number of target pixels is 1 and the luminous flux value of the target pixel is 1.2, the luminous flux value of the target pixel group where the target pixel is located is determined to be 1.2.

[0055] When the number of target pixels is multiple, the optical parameters of the multiple target pixels are calculated to determine the optical parameters of the target pixel group to which the multiple target pixels belong. Exemplarily, when the number of target pixels is 2, the luminous flux value of one target pixel is 0.4, and the luminous flux value of the other target pixel is 0.6. By calculating the average value of the luminous flux values of the two target pixels, the luminous flux value of this target pixel group is determined to be 0.5. Other weighted calculation methods for the optical parameters of the multiple target pixels can also be selected to determine the optical parameters of this target pixel group.

[0056] It should be noted that when the optical parameter is the number of photo-electrons, the number of lit target pixels in the target pixel group is counted.

[0057] Figure 3 FIG. shows one of the schematic diagrams of the pixel arrangement of the target pixel group provided by the embodiment of the present application. As Figure 3 shown, the target pixel group 300 is a 2×2 pixel group. Selecting the single target pixel 302 in the upper left corner of the 2×2 pixel group as the basis for calculating the luminous flux value of the target pixel group 300, then there is no need to calculate the luminous flux values of other pixels, thereby reducing the calculation amount by three quarters.

[0058] Figure 4 FIG. shows another schematic diagram of the pixel arrangement of the target pixel group provided by the embodiment of the present application. As Figure 4 shown, the target pixel group 400 is a 2×2 pixel group. Selecting the two target pixels 402 on the diagonal of the 2×2 pixel group as the basis for calculating the luminous flux value of the target pixel group 400. Calculating the average value of the luminous flux values of the above two target pixels 402 and taking this average value as the luminous flux value of the target pixel group 400, thereby reducing the calculation amount.

[0059] Figure 5 FIG. shows a third schematic diagram of the pixel arrangement of the target pixel group provided by the embodiment of the present application. As Figure 5 shown, the target pixel group 500 is a 2×2 pixel group. Selecting the two target pixels 502 in the first row of the 2×2 pixel group as the basis for calculating the luminous flux value of the target pixel group 500. Calculating the average value of the luminous flux values of the above two target pixels 502 and taking this average value as the luminous flux value of the target pixel group 500, thereby reducing the calculation amount.

[0060] In the embodiment of the present application, after dividing the first image into multiple target pixel groups, by using some pixels in the target pixel group as target pixels and determining the optical parameters of the corresponding target pixel group according to the target pixels, it is realized that there is no need to calculate the optical parameters of all pixels in the target pixel group, reducing the calculation amount for determining the optical parameters corresponding to the target pixel group.

[0061] In some embodiments of the present application, determining X target pixels in a target pixel group includes: determining X target pixels in the target pixel group according to the image content corresponding to the target pixel group in the first target image.

[0062] In the embodiments of the present application, during the process of selecting X target pixels in the first pixel group, the number of target pixels in the first pixel group and the positions where the target pixels are located can be selected according to the image content corresponding to the pixels included in the first pixel group. When the image content corresponding to the first pixel group is relatively rich, to ensure the accuracy of determining the optical parameters in the first pixel group, a larger number of target pixels will be selected, and the target pixels at the corresponding positions of the image content in the first pixel group will be selected.

[0063] Specifically, after the electronic device finishes dividing the first pixel group, the electronic device determines the image content corresponding to the target pixel group through an image recognition algorithm, and based on different image contents, different target pixels are selected for the target pixel group.

[0064] Exemplarily, obtain the image content corresponding to the target pixel group, and determine the number of target pixels in the target pixel group according to the content volume of the image content corresponding to the target pixel group.

[0065] Wherein, the image content corresponding to the target pixel group is the target image feature in the image area of the target pixel group in the first target image. By performing image recognition on the above image area, the target image feature in the image area can be determined. The content volume of the image content is the proportion of the recognized target image feature in the image area. Exemplarily, the image area has a total of N pixels, and the recognized target image features in the image area have a total of n pixels, then the image content volume is n / N.

[0066] Specifically, when the content volume of the image content corresponding to the target pixel group is large, a larger number of target pixels will be selected from the target pixel group. When the content volume of the image content corresponding to the target pixel group is small, a smaller number of target pixels will be selected from the target pixels.

[0067] Exemplarily, obtain the image content corresponding to the target pixel group, and select the target pixels in the target pixel group according to the position information of the image content corresponding to the target pixel group.

[0068] Wherein, the position information of the image content is the position of the recognized target image feature in the image area. Exemplarily, the image area is a pixel matrix of M×N, and the position where the target image feature is located is a pixel matrix of m×n, then the pixels in the pixel matrix of m×n are used as target pixels.

[0069] In the embodiments of the present application, the electronic device adaptively selects target pixels in the first pixel group according to the image content, achieving the reduction of the computational complexity for determining the optical parameters while improving the accuracy of the optical parameters of the first pixel group obtained thereby.

[0070] In some embodiments of the present application, dividing the first target image into P first pixel groups includes: obtaining the first resolution of the first target image; determining the preset number of pixels in each first pixel group according to the first resolution; and dividing the first target image into P first pixel groups according to the preset number of pixels.

[0071] In the embodiments of the present application, before dividing the first target image into P first pixel groups, it is necessary to determine the first resolution of the first target image, determine the preset number of pixels in the first pixel group according to the first resolution, and divide the first target image into P first pixel groups according to the preset number of pixels. The number of pixels in the first pixel groups obtained by the above method is equal.

[0072] In the embodiments of the present application, the electronic device can automatically determine the number of pixels in the first pixel group according to the resolution of the first target image, and divide the first target image according to the number of pixels in the first pixel group. The division of the first target image can be completed without manual operation by the user, and the number of pixels in each of the divided first pixel groups is equal. During the process of dividing the first target image, there is no need to perform image recognition and feature extraction on the first target image, further reducing the computational complexity required by the electronic device and simplifying the process of the electronic device for performing fusion processing on the first target image.

[0073] In some embodiments of the present application, determining the preset number of pixels in each first pixel group according to the first resolution includes: when the first resolution is greater than the preset resolution, determining the preset number of pixels as the first value; when the first resolution is less than or equal to the preset resolution, determining the preset number of pixels as the second value; wherein, the first value is greater than the second value.

[0074] In the embodiments of the present application, when the resolution of the first target image is relatively high, the number of pixels in the first pixel group is set to be relatively high, and when the resolution of the first target image is relatively low, the number of pixels in the first pixel group is set to be relatively low.

[0075] The larger the resolution of the first target image is, the greater the correlation between adjacent pixels in the first target image is. Then, setting a relatively large number of pixels in the first pixel group can ensure that the computational load of the electronic device is reduced while not affecting the accuracy of determining the optical parameters of the first pixel group. The lower the resolution of the first target image is, the smaller the correlation between adjacent pixels in the first target image is. To ensure the accuracy of the optical parameters of the first pixel group, a relatively small number of pixels in the first pixel group is set.

[0076] Figure 6 FIG. 4 shows one of the schematic diagrams of the first pixel group in the first target image provided by the embodiments of the present application. As Figure 6 shown, when the resolution of the first target image 600 is relatively high, it is possible to select to set the preset number of pixels of the first pixel group 602 in the first target image 600 to 9, and the arrangement is 3×3.

[0077] In the embodiments of the present application, the electronic device can select different preset numbers of pixels for the first pixel group according to the comparison result between the preset resolution and the first resolution of the first target image, ensuring that when the first resolution is relatively high, the number of pixels in the first pixel group can be relatively large, and when the first resolution is relatively low, the number of pixels in the first pixel group can be relatively small. It realizes that the electronic device can automatically set the number of pixels of the first pixel group and divide the first target image according to the set number of pixels.

[0078] In some embodiments of the present application, dividing the first target image into P first pixel groups includes: identifying the image content of the first target image; dividing the first target image into P first pixel groups according to the image content; where, among the P first pixel groups, there are pixel groups with different numbers of pixels.

[0079] In the embodiments of the present application, during the process of dividing the first pixel group of the first target image, the electronic device can adaptively divide the first target image according to the identified image content.

[0080] Specifically, after acquiring the first target image, the electronic device determines the image content in the first target image through image recognition, and divides the first target image according to the recognized image content, so that the number of first pixel groups in the area with more image content in the first target image is smaller, and the number of first pixel groups in the area with less image content in the first target image is larger.

[0081] It should be noted that the image content is the target image feature in the first target image. By performing image recognition on the first target image, the target image feature within this image area can be determined. The area with more image content is the area where the target image feature is located in the first target image. The area with more image content can be used as the main area in the first target image. The area outside the area where the target image feature is located in the first target image is the area with less image content, and the area with less image content can be used as the background area in the first target image. Figure 7 FIG. 2 shows a second schematic diagram of the first pixel group in the first target image provided by an embodiment of the present application. As Figure 7 shown, in the first target image 700, the first area 702 is the background area of the first target image 700, and a shooting object 706 is shown in the second area 704 of the first target image 700. The electronic device sets the first pixel group 708 located in the first area 702 as a 5×5 pixel group, and sets the first pixel group 708 located in the second area 704 as a 2×2 pixel group.

[0082] In an embodiment of the present application, the electronic device can identify the main area with more image content and the background area with less image content in the first target image through image recognition, set a smaller number of pixels for the first pixel group in the main area, so as to ensure that the image content will not be lost after the first pixel groups located in the shooting main area are fused, and set a larger number of pixels for the first pixel group in the background area, further reducing the calculation amount of the electronic device.

[0083] In some embodiments of the present application, the optical parameter includes any one of the following: luminous flux value, number of photoelectrons.

[0084] In an embodiment of the present application, the optical parameter can be selected as the luminous flux value and the number of photoelectrons. When it is detected that the luminous flux value changes suddenly, or the number of photoelectrons changes suddenly, the moment of the mutation is taken as the first moment when the optical parameter changes.

[0085] The number of photoelectrons is the number of pixels in the first pixel group that are in the lit state. The number of photoelectrons can be directly collected by the sensor, and the luminous flux value can be obtained through calculation.

[0086] The calculation method of the luminous flux value is shown in formula (1):

[0087]

[0088] where, is the luminous flux value, N pf is the number of the first images, T pf is the acquisition time of a single exposure, t i is the moment when the pixel receives photons, and q is the photon detection efficiency of the sensor.

[0089] In the embodiments of the present application, setting the optical parameter to any one of the luminous flux value and the number of photo-electrons ensures the accuracy of the judgment of the first moment of the optical parameter mutation, and also simplifies the steps of obtaining the optical parameter.

[0090] It should be noted that the optical parameter is not limited to the above luminous flux value and the number of photo-electrons, and may also be other optical parameters, which are not specifically limited herein.

[0091] In some embodiments of the present application, determining at least one second target image in the N first images according to the optical parameter of the target pixel group includes: obtaining the optical parameter change amount of the third pixel group in the third target image adjacent to the acquisition moment, where the third target image is other images in the N first images except the first target image, and the position of the third pixel group in the third target image corresponds to the position of the first pixel group in the first target image; determining the acquisition moment when the optical parameter change amount is greater than the preset change amount as the target moment; and determining at least one second target image in the N first images according to the target moment, where the acquisition moment of the second target image is between the acquisition moment of the first target image and the target moment.

[0092] In the embodiments of the present application, after capturing the N first images, the acquisition moment of each first image is recorded. The images in the N first images except the first target image are all used as the third target images. The third target images are sorted according to the acquisition moment, and the third pixel group in each third target image is determined according to the position of the target pixel group in the first target image. The optical parameter change amount corresponding to the acquisition moment is obtained by calculating the difference between the optical parameters of the third pixel groups in every two adjacent third target images. When it is determined that the optical parameter change amount is greater than the preset change amount, the optical parameter change amount is used as the target moment. The target moment is the moment when the optical parameter of the third pixel group mutates during the process of capturing multiple first images.

[0093] If the acquisition moment of the first target image is before the target moment, the third target images with acquisition moments after the acquisition moment of the first target image and before the target moment are determined as the second target images.

[0094] Exemplarily, during the process of collecting N frames of first images, the first image with the earliest collection time can be used as the first target image, and the subsequent first images collected are all used as the third target images. Therefore, during the process of collecting N frames of first images, the target time can be determined. That is, for each frame of the third target image collected, the optical parameter change amount is calculated between the third pixel group therein and the third pixel group in the third target image at the previous collection time, and compared with the preset change amount. Thus, the target time can be determined during the shooting process, and the second target image is determined based on the target time, further improving the efficiency of determining the second pixel group that can be fused with the target pixel group, and thus improving the overall efficiency of image fusion.

[0095] If the collection time of the first target image is after the target time, then the third target images whose collection times are after the target image and before the collection time of the first target image are determined as the second target images.

[0096] It should be noted that the target time is the time when the optical parameters of the third pixel group mutate during the process of collecting N frames of first images. If the collection time of the first target image is before the target time, in order to ensure that the target pixel group in the first target image can be fused with the second pixel group with matching optical parameters, the image before the optical parameter mutation is determined as the second target image. If the collection time of the first target image is after the target time, in order to ensure that the target pixel group in the first target image can be fused with the second pixel group with matching optical parameters, the image after the optical parameter mutation is determined as the second target image.

[0097] In the embodiments of the present application, by calculating the difference in the optical parameters of the third target pixel groups adjacent in collection time, the optical parameter change amount corresponding to this collection time can be determined. By obtaining the optical parameter change amounts of the third pixel groups in every two adjacent third target images, the optical parameter change amount corresponding to each collection time can be obtained. When the optical parameter change amount exceeds the preset change amount, it is determined that a mutation in the optical change amount occurs at this collection time. The third pixel groups in the third target images collected between this target time and the collection time of the first target image all match the target pixel group. Therefore, all the first images between this target time and the collection time of the first target image are used as the second target images, ensuring that the image area of the target pixel group in the image after fusion according to the second pixel group and the target pixel group in the second target image will not appear blurred.

[0098] In some embodiments of the present application, after fusing the target pixel group with the second pixel group, it includes: generating a fourth target image according to a plurality of third pixel groups, where the third pixel group is a pixel group obtained by fusing the target pixel group and the second pixel group, and the number of third pixel groups is the same as the number of first pixel groups. In the embodiments of the present application, when each first pixel group is determined as the target pixel group, the second target image corresponding to each target pixel group is obtained. The corresponding target pixel group is fused with the corresponding second pixel group in the second target image to obtain a third pixel group. Repeating the fusion step for each target pixel group to obtain a plurality of third pixel groups, and generating a fourth target image by splicing the plurality of third pixel groups according to the positions of the corresponding first pixel groups in the first target image.

[0099] In the embodiments of the present application, by using the pixel group as the unit of fusion processing, the electronic device can ensure that targeted fusion processing is performed on each part of the first target image, avoid the existence of unclear areas in the fourth target image obtained after fusion processing, and without the need to perform separate processing on each pixel, reducing the computational amount of the electronic device and improving the image output efficiency.

[0100] For the image fusion method provided by the embodiments of the present application, the execution subject can be an image fusion device. In the embodiments of the present application, taking the image fusion device executing the image fusion method as an example, the image fusion device provided by the embodiments of the present application is described.

[0101] In some embodiments of the present application, an image fusion device is provided. Figure 8 The schematic block diagram of the image fusion device 800 provided by the embodiments of the present application is shown, as Figure 8 shown, the image fusion device 800 includes:

[0102] An acquisition module 802, configured to acquire N first images, each first image includes M pixels, and both N and M are positive integers;

[0103] A grouping module 804, configured to divide the first target image into P first pixel groups, the first target image is any one of the N first images, and P is a positive integer;

[0104] An acquisition module 806, configured to acquire the optical parameters of the target pixel group in the P first pixel groups, where the target pixel group is any one of the P first pixel groups;

[0105] A determination module 808, configured to determine at least one second target image in the N first images according to the optical parameters of the target pixel group, where the second pixel group matching the optical parameters of the target pixel group is included in the second target image, the second target image is an image in the N first images, and the target pixel group includes Q pixels;

[0106] A fusion module 810 is configured to perform a fusion process on the target pixel group and the second pixel group, where the position of the target pixel group in the first target image corresponds to the position of the second pixel group in the second target image.

[0107] In the embodiments of the present application, when the electronic device captures a captured object in a moving state, multiple frames of first images including the captured object are collected. The first target image in the N frames of first images is divided into multiple first pixel groups, and a target pixel group is obtained from the multiple first pixel groups. Through the optical parameters of the target pixel group, the second pixel group in the second target image that can be fused with the target pixel group can be found. Since the second pixel group is found according to the optical parameters of the target pixel group, it can be ensured that the optical parameters between the second pixel group and the target pixel group for fusion are matched. After each target pixel group is fused with the corresponding second pixel and then stitched together, an image free of motion blur can be obtained. In the embodiments of the present application, the pixel group is used as the unit of image fusion, so that each target pixel group is fused with the matching second pixel group, which can avoid the problem of blurring in the fused image caused by some pixels being fused with other pixels with unmatched optical parameters, and improve the clarity of the finally synthesized image.

[0108] In the embodiments of the present application, the pixel group is used as the unit of image fusion, and only the optical parameters of the whole pixel group need to be calculated, without fusing each pixel in the pixel group separately. Since the pixels in the target pixel group are strongly correlated, using the pixel group as the unit of image fusion can reduce the computational amount of the electronic device while ensuring the accuracy of the image fusion process. It realizes reducing the data processing amount required for image fusion while ensuring the clarity of the fused image.

[0109] In some embodiments of the present application, the determination module 808 is further configured to determine X target pixels in the target pixel group, where X is a positive integer and X ≤ Q;

[0110] The determination module 808 is further configured to determine the optical parameters of the target pixel group based on the optical parameters of the X target pixels.

[0111] In the embodiments of the present application, after the first image is divided into multiple target pixel groups, by using some pixels in the target pixel group as target pixels and determining the optical parameters of the corresponding target pixel group according to the target pixels, it is realized that there is no need to calculate the optical parameters of all pixels in the target pixel group, reducing the computational amount of determining the optical parameters corresponding to the target pixel group.

[0112] In some embodiments of the present application, the determination module 808 is further configured to determine X target pixels in the target pixel group according to the image content corresponding to the target pixel group in the first target image.

[0113] In the embodiments of the present application, the electronic device adaptively selects target pixels in the first pixel group according to the image content, achieving the reduction of the computational amount for determining the optical parameters while improving the accuracy of the determined optical parameters of the first pixel group.

[0114] In some embodiments of the present application, the obtaining module 806 is further configured to obtain the first resolution of the first target image;

[0115] The determining module 808 is further configured to determine the preset number of pixels in each first pixel group according to the first resolution;

[0116] The grouping module 804 is further configured to divide the first target image into P first pixel groups according to the preset number of pixels.

[0117] In the embodiments of the present application, the electronic device can automatically determine the number of pixels in the first pixel group according to the resolution of the first target image, and divide the first target image according to the number of pixels in the first pixel group. The division of the first target image can be completed without manual operation by the user, and the number of pixels in each divided first pixel group is equal. During the process of dividing the first target image, there is no need to perform image recognition and feature extraction on the first target image, further reducing the computational amount required by the electronic device and simplifying the process of the electronic device for fusing the first target image.

[0118] In some embodiments of the present application, the determining module 808 is further configured to determine that the preset number of pixels is the first value when the first resolution is greater than the preset resolution;

[0119] The determining module 808 is further configured to determine that the preset number of pixels is the second value when the first resolution is less than or equal to the preset resolution;

[0120] Wherein, the first value is greater than the second value.

[0121] In the embodiments of the present application, the electronic device can select different preset numbers of pixels for the first pixel group according to the comparison result between the preset resolution and the first resolution of the first target image, ensuring that when the first resolution is relatively high, the number of pixels in the first pixel group can be relatively large, and when the first resolution is relatively low, the number of pixels in the first pixel group can be relatively small. It realizes that the electronic device can automatically set the number of pixels in the first pixel group and divide the first target image according to the set number of pixels.

[0122] In some embodiments of the present application, the image fusion device 800 further includes:

[0123] An identification module, configured to identify the image content of the first target image;

[0124] The grouping module 804 is further configured to divide the first target image into P first pixel groups according to the image content;

[0125] Among them, the P first pixel groups include pixel groups with different numbers of pixels.

[0126] In the embodiments of the present application, the electronic device can identify the main area with more image content and the background area with less image content in the first target image through image recognition, set a smaller number of pixels for the first pixel groups in the main area, so as to ensure that the image content will not be lost after the first pixel groups located in the photographed main area are fused, and set a larger number of pixels for the first pixel groups in the background area, further reducing the calculation amount of the electronic device.

[0127] In some embodiments of the present application, the optical parameter includes any one of the following: luminous flux value, number of photo-electrons.

[0128] In the embodiments of the present application, setting the optical parameter to any one of the luminous flux value and the number of photo-electrons ensures the accuracy of judging the first moment of the optical parameter mutation, and also simplifies the steps of obtaining the optical parameter.

[0129] In some embodiments of the present application, the acquisition module 806 is configured to acquire the optical parameter change amount of the third pixel group in the third target image adjacent to the acquisition moment, where the third target image is other images in the N first images except the first target image, and the position of the third pixel group in the third target image corresponds to the position of the first pixel group in the first target image; the determination module 808 is configured to determine the acquisition moment with the optical parameter change amount greater than the preset change amount as the target moment; the determination module 808 is configured to determine at least one second target image in the N first images according to the target moment, and the acquisition moment of the second target image is between the acquisition moment of the first target image and the target moment.

[0130] In the embodiments of the present application, calculating the difference between the optical parameters of the third target pixel groups adjacent to the acquisition moment can determine the optical parameter change amount corresponding to the acquisition moment. By acquiring the optical parameter change amount of the third pixel groups in every two adjacent third target images, the optical parameter change amount corresponding to each acquisition moment can be obtained. When the optical parameter change amount exceeds the preset change amount, it is determined that a mutation of the optical change amount occurs at the acquisition moment. The third pixel groups in the third target images collected between the target moment and the acquisition moment of the first target image all match the target pixel group. Therefore, all the first images between the target moment and the acquisition moment of the first target image are used as the second target images, ensuring that the image area of the target pixel group in the image fused according to the second pixel group and the target pixel group in the second target image will not be blurred.

[0131] In some embodiments of the present application, the image fusion device 800 further includes:

[0132] A generation module, configured to generate a fourth target image according to a plurality of third pixel groups, where the third pixel groups are pixel groups obtained by fusing the target pixel groups and the second pixel groups, and the number of the third pixel groups is the same as the number of the first pixel groups.

[0133] In the embodiments of the present application, by using pixel groups as the unit of fusion processing, the electronic device can ensure that targeted fusion processing is performed on each part of the first target image, avoid the existence of unclear areas in the fourth target image obtained after fusion processing, and without the need to perform separate processing on each pixel, reducing the computational amount of the electronic device and improving the image output efficiency.

[0134] The image fusion device in the embodiments of the present application may be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or other devices other than terminals. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a handheld computer, an in-vehicle electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and may also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0135] The image fusion device in the embodiments of the present application may be a device with an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems. The embodiments of the present application do not make specific limitations.

[0136] The image fusion device provided in the embodiments of the present application can implement each process implemented in the above method embodiments. To avoid repetition, details are not described here again.

[0137] Optionally, an embodiment of the present application further provides an electronic device, which includes the image fusion device in any of the above embodiments, and thus has all the beneficial effects of the image fusion device in any of the embodiments, and will not be elaborated here too much.

[0138] Optionally, an embodiment of the present application further provides an electronic device. Figure 9 The block diagram of the electronic device according to an embodiment of the present application is shown. As Figure 9 shown, the electronic device 900 includes a processor 902, a memory 904, a program or instruction stored on the memory 904 and executable on the processor 902. When the program or instruction is executed by the processor 902, each process of the above image fusion method embodiment is implemented, and the same technical effects can be achieved. To avoid repetition, it will not be elaborated here.

[0139] It should be noted that the electronic device in the embodiment of the present application includes the above-mentioned mobile electronic device and non-mobile electronic device.

[0140] Figure 10 It is a schematic diagram of the hardware structure of an electronic device for implementing an embodiment of the present application.

[0141] The electronic device 1000 includes, but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010 and other components.

[0142] Those skilled in the art can understand that the electronic device 1000 may further include a power supply (such as a battery) for supplying power to each component. The power supply can be logically connected to the processor 1010 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. Figure 10 The structure of the electronic device shown in does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0143] Among them, the processor 1010 is used to collect N first images, each first image includes M pixels, and both N and M are positive integers;

[0144] The processor 1010 is used to divide the first target image into P first pixel groups, the first target image is any one of the N first images, and P is a positive integer;

[0145] The processor 1010 is used to obtain the optical parameters of the target pixel group in the P first pixel groups, and the target pixel group is any one of the P first pixel groups;

[0146] A processor 1010 is configured to determine at least one second target image in N first images according to the optical parameters of a target pixel group. The second target image includes a second pixel group that matches the optical parameters of the target pixel group. The second target image is an image in the N first images. The target pixel group includes Q pixels.

[0147] The processor 1010 is configured to perform a fusion process on the target pixel group and the second pixel group. The position of the target pixel group in the first target image corresponds to the position of the second pixel group in the second target image.

[0148] In an embodiment of the present application, when an electronic device captures a moving object, multiple first images including the object are collected. The first target image in the N first images is divided into multiple first pixel groups, and a target pixel group is obtained from the multiple first pixel groups. Through the optical parameters of the target pixel group, a second pixel group in a second target image that can be fused with the target pixel group can be found. Since the second pixel group is found according to the optical parameters of the target pixel group, it can be ensured that the optical parameters between the second pixel group and the target pixel group for fusion match. After fusing each target pixel group with the corresponding second pixel and then stitching, an image with motion blur removed can be obtained. In the embodiment of the present application, the pixel group is used as the unit of image fusion, so that each target pixel group is fused with a matching second pixel group, which can avoid the problem of blurring in the fused image caused by some pixels being fused with other pixels with unmatched optical parameters, reduce the motion blur in the image, and improve the clarity of the image of the moving object.

[0149] In the embodiment of the present application, the pixel group is used as the unit of image fusion, and only the optical parameters of the whole pixel group need to be calculated, without fusing each pixel in the pixel group separately. Since the pixels in the target pixel group are highly correlated, using the pixel group as the unit of image fusion can reduce the computational amount of the electronic device while ensuring the accuracy of the image fusion process. It realizes reducing the data processing amount required for image fusion while ensuring the clarity of the fused image.

[0150] Further, the processor 1010 is configured to determine X target pixels in the target pixel group, where X is a positive integer and X ≤ Q.

[0151] The processor 1010 is configured to determine the optical parameters of the target pixel group according to the optical parameters of the X target pixels.

[0152] In the embodiments of the present application, after dividing the first image into multiple target pixel groups, by using some pixels in the target pixel group as target pixels and determining the optical parameters of the corresponding target pixel group according to the target pixels, it is realized that there is no need to calculate the optical parameters of all pixels in the target pixel group, reducing the computational amount for determining the optical parameters of the target pixel group.

[0153] Further, the processor 1010 is configured to determine X target pixels in the target pixel group according to the image content corresponding to the target pixel group in the first target image.

[0154] In the embodiments of the present application, by adaptively selecting target pixels in the first pixel group by the electronic device according to the image content, it is realized that while reducing the computational amount for determining the optical parameters, the accuracy of the optical parameters of the first pixel group determined is improved.

[0155] Further, the processor 1010 is configured to obtain the first resolution of the first target image;

[0156] The processor 1010 is configured to determine the preset number of pixels in each first pixel group according to the first resolution;

[0157] The processor 1010 is configured to divide the first target image into P first pixel groups according to the preset number of pixels.

[0158] In the embodiments of the present application, the electronic device can automatically determine the number of pixels in the first pixel group according to the resolution of the first target image, and divide the first target image according to the number of pixels in the first pixel group. The division of the first target image can be completed without manual operation by the user, and the number of pixels in each divided first pixel group is equal. During the process of dividing the first target image, there is no need to perform image recognition and feature extraction on the first target image, further reducing the computational amount required by the electronic device and simplifying the process of the electronic device for fusing and processing the first target image.

[0159] Further, the processor 1010 is configured to determine that the preset number of pixels is the first value when the first resolution is greater than the preset resolution;

[0160] The processor 1010 is configured to determine that the preset number of pixels is the second value when the first resolution is less than or equal to the preset resolution;

[0161] Wherein, the first value is greater than the second value.

[0162] In the embodiments of the present application, the electronic device can select different preset pixel numbers for the first pixel group according to the comparison result between the preset resolution and the first resolution of the first target image, ensuring that when the first resolution is relatively high, the number of pixels in the first pixel group can be relatively large, and when the first resolution is relatively low, the number of pixels in the first pixel group can be relatively small. It realizes that the electronic device can automatically set the pixel number of the first pixel group and divide the first target image according to the set pixel number.

[0163] Further, the processor 1010 is configured to identify the image content of the first target image;

[0164] The processor 1010 is configured to divide the first target image into P first pixel groups according to the image content;

[0165] Among them, the P first pixel groups include pixel groups with different pixel numbers.

[0166] In the embodiments of the present application, the electronic device can identify the main body area with more image content and the background area with less image content in the first target image through image recognition, set a smaller number of pixels for the first pixel group in the main body area, so as to ensure that the image content will not be lost after the first pixel groups located in the shooting main body area are fused, and set a larger number of pixels for the first pixel group in the background area, further reducing the calculation amount of the electronic device.

[0167] Further, the optical parameter includes any one of the following: luminous flux value, number of photoelectrons.

[0168] In the embodiments of the present application, setting the optical parameter as any one of the luminous flux value and the number of photoelectrons ensures the accuracy of judging the first moment of the optical parameter mutation and also simplifies the steps of obtaining the optical parameter.

[0169] Further, the processor 1010 is configured to obtain the change amount of the optical parameter of the third pixel group in the third target image adjacent to the acquisition moment, where the third target image is other images in the N first images except the first target image, and the position of the third pixel group in the third target image corresponds to the position of the first pixel group in the first target image;

[0170] The processor 1010 is configured to determine the acquisition moment with the change amount of the optical parameter greater than the preset change amount as the target moment;

[0171] The processor 1010 is configured to determine at least one second target image in the N first images according to the target moment, and the acquisition moment of the second target image is between the acquisition moment of the first target image and the target moment.

[0172] In the embodiments of the present application, by calculating the difference in the optical parameters of the third target pixel groups adjacent to the acquisition moment, the optical parameter variation corresponding to the acquisition moment can be determined. By obtaining the optical parameter variations of the third pixel groups in every two adjacent third target images, the optical parameter variations corresponding to each acquisition moment can be obtained. When the optical parameter variation exceeds the preset variation, it is determined that a mutation of the optical variation occurs at the acquisition moment. The third pixel groups in the third target images acquired between the target moment and the acquisition moment of the first target image all match the target pixel group. Therefore, all the first images between the target moment and the acquisition moment of the first target image are used as the second target images, ensuring that the image area of the target pixel group in the image obtained by fusing the second pixel group and the target pixel group in the second target image will not be blurred.

[0173] Further, the processor 1010 is configured to generate a fourth target image according to a plurality of third pixel groups, where the third pixel groups are pixel groups obtained by fusing the target pixel group and the second pixel group, and the number of the third pixel groups is the same as the number of the first pixel groups.

[0174] In the embodiments of the present application, by using the pixel group as the unit of the fusion process, it can be ensured that targeted fusion processing is performed on each part of the first target image, avoiding unclear areas in the fourth target image obtained after the fusion process, and there is no need to perform separate processing on each pixel, reducing the computational amount of the electronic device and improving the image output efficiency.

[0175] It should be understood that in the embodiments of the present application, the input unit 1004 may include a Graphics Processing Unit (GPU) 10041 and a microphone 10042. The GPU 10041 processes the image data of the static pictures or videos obtained by an image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include a touch detection device and a touch controller. The other input devices 10072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0176] The memory 1009 can be used to store software programs and various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area may store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1009 may include a volatile memory or a non-volatile memory, or the memory 1009 may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synch link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 1009 in the embodiments of the present application includes, but is not limited to, these and any other suitable types of memories.

[0177] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 1010 either.

[0178] The embodiments of the present application further provide a readable storage medium. A program or instructions are stored on the readable storage medium. When the program or instructions are executed by a processor, each process of the above method embodiments is implemented, and the same technical effects can be achieved. To avoid repetition, it will not be elaborated here.

[0179] Among them, the processor is the processor in the electronic device in the above-mentioned embodiment. The readable storage medium includes computer-readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disk or optical disc, etc.

[0180] Another embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above-mentioned embodiment of the image fusion method, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0181] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0182] The embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement each process of the above-mentioned embodiment of the image fusion method, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0183] It should be noted that in this article, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including one..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0184] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present application.

[0185] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative rather than restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them fall within the protection scope of the present application.

Claims

1. An image fusion method, characterized in that, it includes: Collect N frames of first images, each of the first images includes M pixels, and both N and M are positive integers; Divide the first target image into P first pixel groups, the first target image is any one of the N frames of first images, and P is a positive integer; Obtain the optical parameters of the target pixel group among the P first pixel groups, the target pixel group is any one of the P first pixel groups; According to the optical parameters of the target pixel group, determine at least one second target image among the N frames of first images, the second target image includes a second pixel group that matches the optical parameters of the target pixel group, the second target image is an image among the N frames of first images, the target pixel group includes Q pixels, and Q < M; Perform fusion processing on the target pixel group and the second pixel group, and the position of the target pixel group in the first target image corresponds to the position of the second pixel group in the second target image.

2. The image fusion method according to claim 1, characterized in that, The obtaining of the optical parameters of the target pixel group among the P first pixel groups includes: Determine X target pixels in the target pixel group, X is a positive integer, and X ≤ Q; According to the optical parameters of the X target pixels, determine the optical parameters of the target pixel group.

3. The image fusion method according to claim 2, characterized in that, The determining of the X target pixels in the target pixel group includes: According to the image content corresponding to the target pixel group in the first target image, determine the X target pixels in the target pixel group.

4. The image fusion method according to any one of claims 1 to 3, characterized in that, The dividing of the first target image into P first pixel groups includes: Identify the image content of the first target image; According to the image content, divide the first target image into P first pixel groups; Wherein, the P first pixel groups include pixel groups with different numbers of pixels.

5. The image fusion method according to any one of claims 1 to 3, characterized in that, The determining of at least one second target image among the N frames of first images according to the optical parameters of the target pixel group includes: Obtain the optical parameter change amount of the third pixel group in the third target image at adjacent acquisition times, the third target image is other images among the N frames of first images except the first target image, and the position of the third pixel group in the third target image corresponds to the position of the first pixel group in the first target image; Determine the acquisition time when the optical parameter change amount is greater than the preset change amount as the target time; According to the target time, determine at least one of the N frames of first images as the second target image, and the acquisition time of the second target image is between the acquisition time of the first target image and the target time.

6. An image fusion device, characterized in that, it includes: An acquisition module, configured to acquire N frames of first images, each of the first images including M pixels, where N and M are both positive integers; A grouping module, configured to divide a first target image into P first pixel groups, where the first target image is any one of the N frames of first images, and P is a positive integer; An obtaining module, configured to obtain optical parameters of a target pixel group among the P first pixel groups, where the target pixel group is any one of the P first pixel groups; A determining module, configured to determine at least one second target image among the N frames of first images according to the optical parameters of the target pixel group, where the second target image includes a second pixel group that matches the optical parameters of the target pixel group, the second target image is an image among the N frames of first images, and the target pixel group includes Q pixels; A fusion module, configured to perform a fusion process on the target pixel group and the second pixel group, where the position of the target pixel group in the first target image corresponds to the position of the second pixel group in the second target image.

7. The image fusion device according to claim 6, wherein, the determining module is further configured to determine X target pixels in the target pixel group, where X is a positive integer and X ≤ Q; the determining module is further configured to determine the optical parameters of the target pixel group according to the optical parameters of the X target pixels.

8. The image fusion device according to claim 7, wherein, the determining module is further configured to determine the X target pixels in the target pixel group according to the image content corresponding to the target pixel group in the first target image.

9. The image fusion device according to any one of claims 6 to 8, wherein, further comprising: an identification module, configured to identify the image content of the first target image; the grouping module is further configured to divide the first target image into P first pixel groups according to the image content; wherein, among the P first pixel groups, there are pixel groups with different numbers of pixels.

10. The image fusion device according to any one of claims 6 to 8, wherein, the obtaining module is further configured to obtain a change amount of optical parameters of a third pixel group in a third target image adjacent to the acquisition time, where the third target image is an image other than the first target image among the N frames of first images, and the position of the third pixel group in the third target image corresponds to the position of the first pixel group in the first target image; the determining module is further configured to determine the acquisition time with the optical parameter change amount greater than a preset change amount as the target time; the determining module is configured to determine at least one of the second target images among the N frames of first images according to the target time, where the acquisition time of the second target image is between the acquisition time of the first target image and the target time.

11. An electronic device, wherein, comprising: A processor and a memory, the memory storing a program or instructions that can run on the processor, and when the program or instructions are executed by the processor, the steps of the image fusion method according to any one of claims 1 to 5 are implemented.

12. A readable storage medium having a program or instructions stored thereon, characterized in that when the program or instructions are executed by a processor, the steps of the image fusion method according to any one of claims 1 to 5 are implemented.

Images