Image fusion processing method, device and electronic system

Abstract

The present invention provides an image fusion processing method, device and electronic system. The method includes: determining a corresponding optical flow map based on a first image and a second image; determining an initial mask based on the pixel similarity of the first image and the second image; for a connected area of ​​pixels with a first value in the initial mask, correcting the pixel values ​​of the pixels in the connected area based on the pixels in the corresponding area of ​​the connected area in the optical flow map to obtain a corrected mask; fusing the first image and the second image based on the corrected mask to obtain a fused image. By correcting the initial mask through the optical flow map and obtaining a fused image based on the corrected mask, the detection error of the unmatched area in the fused image can be reduced, so that the registration area uses the telephoto image information in its entirety, improving the clarity of the fused image, and maximizing the fusion result.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image fusion processing method, device and electronic system. Background Art

[0002] Image registration is a very important step in image fusion. For example, in a binocular camera, wide-angle images and telephoto images need to be registered. The ideal situation in image registration is complete registration, that is, the telephoto information can be used within the telephoto FOV (Field Of View), and the wide-angle information is used in the area outside the telephoto camera FOV.

[0003] However, there are problems such as occluded areas, large differences in clarity, and large differences in brightness due to differences in ISP (Image Signal Processor) parameters, which bring great difficulties to the detection of unregistered areas. Due to occlusion, registration algorithm and other reasons, some areas within the telephoto camera FOV often cannot be registered, resulting in large detection errors. These unregistered areas can only use wide-angle information, and the incorrect use of too much wide-angle information will reduce the clarity of the image after registration and affect the image fusion effect. Summary of the invention

[0004] In view of this, an object of the present invention is to provide an image fusion processing method, device and electronic system to reduce the detection error of the unmatched area and maximize the fusion result.

[0005] In a first aspect, an embodiment of the present invention provides an image fusion processing method, the method comprising: determining a corresponding optical flow map based on a first image and a second image having the same field of view; determining an initial mask based on pixel similarity of the first image and the second image; wherein pixel points with a preset first value in the initial mask are used to characterize pixel points whose similarity is less than a set similarity threshold; for connected areas of pixel points with the first value in the initial mask, pixel values ​​of the pixel points of the connected area are corrected based on pixel points of the corresponding area of ​​the connected area in the optical flow map to obtain a corrected mask; and fusing the first image and the second image based on the corrected mask to obtain a fused image.

[0006] In a preferred embodiment of the present invention, the above-mentioned steps of correcting the pixel values ​​of the pixel points in the connected area with the first value in the initial mask based on the pixel points in the corresponding area of ​​the connected area in the optical flow map to obtain the corrected mask include: obtaining the first mask corresponding to the optical flow map; wherein the pixel points with the first value in the first mask represent the edge pixel points in the optical flow map; for the connected area of ​​the target pixel points with the first value in the initial mask, determining the pixel value of the target pixel points in the first mask; if the pixel value of the target pixel point in the first mask is the first value, setting the pixel value of the target pixel point in the initial mask to the first value to obtain the corrected mask.

[0007] In a preferred embodiment of the present invention, after the above-mentioned step of obtaining the first mask corresponding to the optical flow map, the method further includes: checking the first target area corresponding to the connected area of ​​the target pixel point of the first value in the first mask; if the pixel value of the target area in the second target area corresponding to the initial mask or the corrected mask is the second value, setting the pixel value in the second target area to the first value.

[0008] In a preferred embodiment of the present invention, the step of obtaining the first mask corresponding to the optical flow map includes: for each pixel point in the optical flow map, calculating the first standard deviation in the horizontal coordinate direction and the second standard deviation in the vertical coordinate direction; taking the larger value of the first standard deviation and the second standard deviation as the standard deviation of the pixel point; if the standard deviation of the pixel point is greater than a preset standard deviation threshold, setting the pixel value of the pixel point in the first mask to the first value.

[0009] In a preferred embodiment of the present invention, after the above-mentioned step of obtaining the first mask corresponding to the optical flow map, the method further includes: calculating the ratio of the area of ​​the pixel points whose pixel values ​​in the first mask are the first value to the area of ​​the first mask; if the ratio is less than a preset ratio threshold, continuing to execute the step of determining the pixel value of the target pixel point in the first mask for the connected area of ​​the target pixel point with the first value in the initial mask.

[0010] In a preferred embodiment of the present invention, the above method also includes: acquiring a first initial image and a second image having the same field of view; mapping the first initial image to obtain a first image; determining a mapping mask based on the first initial image and the first image; wherein the pixel points with the first value in the mapping mask are used to represent the pixel points in the first image whose coordinates exceed the boundary of the first initial image; after the step of obtaining the correction mask, the method also includes: setting the pixel points in the correction mask corresponding to the pixel points with the first value in the mapping mask to the first value to obtain an updated correction mask.

[0011] In a preferred embodiment of the present invention, after the above-mentioned step of obtaining the correction mask, the method further includes: calculating the difference between the pixel value of the pixel point of the first image and the pixel value of the pixel point of the second image; if the difference is greater than a preset pixel value threshold, setting the pixel value of the pixel point in the correction mask to the first value, to obtain an updated correction mask.

[0012] In a preferred embodiment of the present invention, after the above-mentioned step of obtaining the correction mask, the method further includes: performing gradient processing on the boundary area of ​​the correction mask so that the pixel value of the correction mask changes from the preset third value to the first value at a preset step value starting from the boundary of the correction mask, to obtain an updated correction mask.

[0013] In a preferred embodiment of the present invention, after the step of obtaining the correction mask, the method further comprises: performing Gaussian blur processing on the correction mask to add a transition area in the area of ​​the correction mask where the pixel value is the first value.

[0014] In a preferred embodiment of the present invention, the step of determining the initial mask based on the pixel similarity between the first image and the second image includes: calculating the structural similarity between the pixel value of the pixel in the first image and the pixel value in the second image; if the structural similarity of the pixel is less than a preset similarity threshold, setting the pixel value of the pixel in the initial mask to the first value.

[0015] In a preferred embodiment of the present invention, the above-mentioned step of fusing the first image and the second image based on the correction mask to obtain a fused image includes: inverting the correction mask to obtain a target mask; and fusing the first image and the second image based on the target mask to obtain a fused image.

[0016] In a preferred embodiment of the present invention, the above method also includes: obtaining the overlapping area corresponding to the same field of view of the telephoto image and the wide-angle image to be fused to obtain the first image and the second image; the above step of fusing the first image and the second image based on the correction mask to obtain the fused image includes: inverting the correction mask to obtain the target mask; fusing the first image and the second image based on the target mask to obtain a local fused image; fusing the local fused image with the wide-angle image to obtain a global fused image.

[0017] In a second aspect, an embodiment of the present invention further provides an image fusion processing device, the device comprising: an optical flow map determination module, used to determine a corresponding optical flow map based on a first image and a second image having the same field of view; an initial mask determination module, used to determine an initial mask based on the pixel similarity of the first image and the second image; wherein the pixel points with a preset first value in the initial mask are used to characterize the pixel points whose similarity is less than a set similarity threshold; a corrected mask determination module, used to correct the pixel values ​​of the pixel points in the connected area of ​​the pixel points with the first value in the initial mask based on the pixel points in the corresponding area of ​​the connected area in the optical flow map, to obtain a corrected mask; and a fused image determination module, used to fuse the first image and the second image based on the corrected mask to obtain a fused image.

[0018] In a third aspect, an embodiment of the present invention provides an electronic system, which includes: an image acquisition device, a processing device and a storage device; the image acquisition device is used to obtain wide-angle images and telephoto images; a computer program is stored on the storage device, and the computer program executes the image fusion processing method as any one of the aforementioned embodiments when the processed device is running.

[0019] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processing device, the steps of the image fusion processing method as described in any one of the aforementioned embodiments are executed.

[0020] The embodiments of the present invention bring the following beneficial effects:

[0021] An image fusion processing method, device and electronic system provided by an embodiment of the present invention determines an optical flow map based on a first image and a second image having the same field of view, and determines an initial mask based on the pixel similarity of the first image and the second image; based on the pixel points of the corresponding area of ​​the connected area in the optical flow map, the pixel values ​​of the pixel points in the connected area of ​​the pixel points with the first value in the initial mask are corrected to obtain a corrected mask; based on the corrected mask, the first image and the second image are fused to obtain a fused image. In this method, the initial mask is corrected by the optical flow map, and the fused image is obtained based on the corrected mask, which can reduce the detection error of the unmatched area in the fused image, so that the registration area uses the telephoto image information in its entirety, improves the clarity of the fused image, and maximizes the fusion result.

[0022] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present disclosure.

[0023] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A schematic diagram of the structure of an electronic system provided by an embodiment of the present invention;

[0026] Figure 2 A flowchart of an image fusion processing method provided by an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of a wide-angle image provided by an embodiment of the present invention;

[0028] Figure 4 A schematic diagram of a telephoto image provided by an embodiment of the present invention;

[0029] Figure 5 A schematic diagram of a fused weight map image provided by an embodiment of the present invention;

[0030] Figure 6 A flowchart of another image fusion processing method in a game provided by an embodiment of the present invention;

[0031] Figure 7 A schematic diagram of a Gaussian blurred post-correction mask provided by an embodiment of the present invention;

[0032] Figure 8 A schematic diagram of an image fusion processing method provided by an embodiment of the present invention;

[0033] Fig. 9 A schematic diagram of the structure of an image fusion processing device provided by an embodiment of the present invention;

[0034] Fig.10 A schematic diagram of the structure of another image fusion processing device provided by an embodiment of the present invention;

[0035] Fig.11 A schematic diagram of the structure of another electronic system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] In order to fuse the telephoto image with the wide-angle image, it is necessary to find matching areas and unmatched areas. The matching areas use the image information of the telephoto image, and the unmatched areas all use the image information of the wide-angle image. In the related art, due to differences in the brightness and clarity of the images themselves, the detection error of the unmatched areas is large, which affects the image fusion effect. Based on this, an image fusion processing method, device, and electronic system provided in an embodiment of the present invention can reduce the detection error of the unmatched areas in the fused image, so that the registration areas all use the telephoto image information, and the fusion result can be improved to the greatest extent possible.

[0038] To facilitate understanding of this embodiment, an image fusion processing method disclosed in an embodiment of the present invention is first introduced in detail.

[0039] Embodiment 1:

[0040] First, refer to Figure 1 The electronic system 100 is a schematic structural diagram of the electronic system 100. The electronic system can be used to implement the image fusion processing method, device and electronic system of the embodiment of the present invention.

[0041] like Figure 1 The electronic system 100 includes one or more processing devices 102, one or more storage devices 104, an input device 106, an output device 108, and one or more image acquisition devices 110. These components are interconnected via a bus system 112 and / or other forms of connection mechanisms (not shown). It should be noted that Figure 1 The components and structure of the electronic system 100 shown are merely exemplary and not limiting. The electronic system may also have other components and structures as required.

[0042] The processing device 102 can be a server, an intelligent terminal, or a device including a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities. It can process data of other components in the electronic system 100 and can also control other components in the electronic system 100 to perform image fusion processing functions.

[0043] The storage device 104 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processing device 102 may run the program instructions to implement the client functions and / or other desired functions in the embodiments of the present invention (implemented by the processing device) described below. Various applications and various data, such as various data used and / or generated by the application, may also be stored in the computer-readable storage medium.

[0044] The input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.

[0045] The output device 108 may output various information (eg, images or sounds) to the outside (eg, a user), and may include one or more of a display, a speaker, and the like.

[0046] The image acquisition device 110 can acquire images to be detected, and store the acquired images in the storage device 104 for use by other components.

[0047] Exemplarily, the components used to implement the image fusion processing method, device, and electronic system according to the embodiment of the present invention can be integrated or dispersed, such as integrating the processing device 102, the storage device 104, the input device 106, and the output device 108 into one body, and setting the image acquisition device 110 at a designated position where the image can be acquired. When the components in the above electronic system are integrated, the electronic system can be implemented as a smart terminal such as a camera, a smart phone, a tablet computer, a computer, a vehicle-mounted terminal, etc.

[0048] Embodiment 2

[0049] This embodiment provides an image fusion processing method. Figure 2 The flowchart of an image fusion processing method shown in FIG. 1 includes the following steps:

[0050] Step S202: determining a corresponding optical flow map based on the first image and the second image having the same field of view.

[0051] In this embodiment, the first image and the second image are images to be fused, and may be a telephoto image and a wide-angle image, for example, the first image is a telephoto image and the second image is a wide-angle image; or the first image is a wide-angle image and the second image is a telephoto image. In this embodiment, the first image and the second image are respectively a telephoto image and a wide-angle image, and no further description is given hereinafter.

[0052] Focal length affects how clearly objects appear in the picture. The smaller the focal length, the more of the scene will be included in the picture. Increasing the focal length will include less of the scene, but will show more detail in distant objects.

[0053] Focal length can be measured in millimeters. A 50mm camera is usually the standard camera for photography. Cameras with a focal length less than 50mm are called short or wide-angle cameras. Cameras with a focal length greater than 50mm are called long or telephoto cameras. Images taken by a telephoto camera are called telephoto images, and images taken by a wide-angle camera are called wide-angle images.

[0054] FOV characterizes the amount of scene that is visible. FOV is measured in degrees of the horizon and is directly related to the focal length of the camera. For example, a 50mm camera displays 46 degrees of the horizon. The longer the camera, the narrower the FOV. The shorter the camera, the wider the FOV. A short focal length (wide FOV) emphasizes perspective distortion, making objects appear deeper and blurrier toward the viewer; a long focal length (narrow FOV) reduces perspective distortion, making objects flattened or parallel to the viewer.

[0055] In summary, the FOV of wide-angle images is wider and the clarity is poorer; the FOV of telephoto images is narrower and the clarity is better. In order to ensure a better image fusion effect, telephoto images can be used as much as possible, and wide-angle images can be used outside the FOV of telephoto images.

[0056] Generally speaking, the wide-angle camera and the telephoto camera can be arranged in parallel, vertically or in other arrangements in a photographing device, wherein the photographing device can be a camera, a mobile phone, a computer, a tablet computer or other device with a photographing function. When the photographing device is taking a photo, the wide-angle camera and the telephoto camera can take a photo at the same time, and obtain a wide-angle image and a telephoto image respectively. Among them, the method provided in the embodiment of the present invention can be applied to the filter of the photographing device.

[0057] Since the wide-angle camera and the telephoto camera are located in different positions and their FOVs are different, there are some overlapping areas and some non-overlapping areas in the wide-angle image and the telephoto image; the overlapping area may refer to objects or scenery captured in both the wide-angle image and the telephoto image.

[0058] See also Figure 3A schematic diagram of a wide-angle image and Figure 4 A schematic diagram of a telephoto image is shown in FIG. Figure 4 The telephoto image in Figure 3 The medium and wide angle images show the same overlapping area, and it can be seen that the telephoto image has higher clarity. When merging the images, it is best to use the wide angle image for this area.

[0059] When fusing wide-angle images and telephoto images, the absolute value of the difference between the wide-angle image and the registered telephoto image is generally used as the initial value of the fusion weight. This method of fusion weight can be found in Figure 5 A schematic diagram of a fused weight map image is shown in FIG. Figure 5 The white color in the figure indicates that the telephoto image is used for fusion, and the black color indicates that the wide-angle image is used for fusion. Figure 5 It can be seen that due to the differences in brightness, clarity, etc. between the telephoto and wide-angle images, detection errors occur in the originally registered areas in the schematic diagram of the fusion weight map. The wide-angle image has a larger weight and the clarity improvement effect is not good.

[0060] Therefore, this embodiment hopes to reduce the weight of the wide-angle image and increase the weight of the telephoto image in the registered area. First, the corresponding optical flow map can be determined based on the overlapping area of ​​the telephoto image and the wide-angle image to be fused. Among them, the optical flow map can be calculated for the registered image pair through the optical flow algorithm. Common optical flow algorithms mainly include the Horn-Schunck algorithm, the Lucas-Kanade algorithm, etc., which can be selected according to actual needs.

[0061] Step S204, determining an initial mask based on the pixel similarity between the first image and the second image; wherein the pixel points with a preset first value in the initial mask are used to represent the pixel points whose similarity is less than a set similarity threshold.

[0062] After determining the mapping image, the initial mask can be determined according to the pixel similarity between the first image and the second image. Since the same object in the first image and the second image has the same coordinates, the similarity of each pixel point in the first image and the second image can characterize the similarity between the object in the first image and the object in the second image. The higher the pixel similarity, the higher the similarity of the object, and the more likely it is to use a telephoto image.

[0063] The preset first value may be 1, and the preset second value may be 0. In other embodiments, the first value may be 0 and the second value may be 1, which will not be described in detail. If the pixel point in the initial mask has the preset first value, that is, the pixel point set to 1, and the similarity is less than the set similarity threshold, the pixel point may use the wide-angle image instead of the telephoto image. If the pixel point in the initial mask has the preset second value, that is, the pixel point set to 0, and the similarity is greater than or equal to the similarity threshold, the pixel point may use the telephoto image instead of the wide-angle image.

[0064] Step S206, for the connected area of ​​the pixel points with the first value in the initial mask, based on the pixel points of the corresponding area of ​​the connected area in the optical flow map, the pixel values ​​of the pixel points in the connected area are corrected to obtain a corrected mask.

[0065] The low similarity of pixels in the initial mask may be caused by clarity difference, misalignment, exposure difference, etc. Reasons other than misalignment such as clarity difference and exposure difference need to be eliminated. Therefore, for the connected area of ​​pixels with the first value in the initial mask, the pixel values ​​of the pixels in the connected area can be corrected based on the pixels in the corresponding area of ​​the connected area in the optical flow map, that is, the pixel values ​​of the pixels in the connected area are changed from the first value to the second value. The corrected initial mask is called a corrected mask.

[0066] The specific idea may be: if the pixels of the connected area are relatively flat in the optical flow map, the low clarity is not caused by misalignment, so the pixel values ​​of the pixels of the connected area may be modified from the first value to the second value.

[0067] Step S208: fusing the first image and the second image based on the corrected mask to obtain a fused image.

[0068] After the correction of the initial mask is completed, the first image and the second image can be fused according to the correction mask to obtain a fused image. Specifically, taking the first image and the second image as an example, the correction mask has the first value, that is, the similarity is low, and the wide-angle image needs to be used for fusion; the correction mask has the second value, that is, the similarity is high, and the telephoto image needs to be used for fusion. The fused image obtained in this embodiment can use the telephoto image information as much as possible in the registration area, thereby improving the clarity of the fused image.

[0069] An image fusion processing method provided by an embodiment of the present invention determines an optical flow map based on a first image and a second image with the same field of view, and determines an initial mask based on the pixel similarity of the first image and the second image; based on the pixel points of the corresponding area of ​​the connected area in the optical flow map, the pixel values ​​of the pixel points in the connected area of ​​the pixel points with the first value in the initial mask are corrected to obtain a corrected mask; based on the corrected mask, the first image and the second image are fused to obtain a fused image. In this method, the initial mask is corrected by the optical flow map, and the fused image is obtained based on the corrected mask, which can reduce the detection error of the unmatched area in the fused image, so that the registration area uses the telephoto image information in its entirety, improves the clarity of the fused image, and maximizes the fusion result.

[0070] Embodiment 3

[0071] This embodiment provides another image fusion processing method, which is implemented on the basis of the above embodiment; this embodiment focuses on describing a specific implementation method of correcting the pixel values ​​of the connected area of ​​the pixel points with the first value in the initial mask based on the optical flow map. Figure 6 The flowchart of another image fusion processing method in the game is shown in FIG. 1 . The image fusion processing method in this embodiment includes the following steps:

[0072] Step S602: determining a corresponding optical flow map based on the first image and the second image having the same field of view.

[0073] Taking a telephoto image and a wide-angle image as an example, this embodiment can respectively use the overlapping area corresponding to the same field of view of the telephoto image and the wide-angle image as the first image and the second image. For example: obtain the overlapping area corresponding to the same field of view of the telephoto image and the wide-angle image to be fused to obtain the first image and the second image.

[0074] The first image may be an overlapping area of ​​the telephoto image, which may also be referred to as a first overlapping image, and the second image may be an overlapping area of ​​the wide-angle image, which may also be referred to as a second overlapping image.

[0075] Since the positions and FOVs of the wide-angle camera and the telephoto camera are different. Therefore, the coordinates of the same object in the first overlapping image and the second overlapping image are not the same, and pixel offset is required. In this embodiment, the pixels of the first overlapping image can be offset (i.e., mapping processing is performed) based on the optical flow map, and the image after the mapping processing can be called a mapping image, i.e., the above-mentioned first image. The same object in the mapping image and the second overlapping image has the same coordinates, i.e., the mapping image is aligned with the second overlapping image. In addition to offsetting the pixels of the first overlapping image, this embodiment can also offset the pixels of the second overlapping image.

[0076] The mapping process may also be referred to as remap: the first overlapping image is shifted pixel by pixel according to the optical flow map so that the shifted first overlapping image is aligned with the corresponding area of ​​the second overlapping image. The shifted first overlapping image may be referred to as a mapping image.

[0077] Step S604, determining an initial mask based on the pixel similarity between the first image and the second image; wherein the pixel points with a preset first value in the initial mask are used to represent the pixel points whose similarity is less than a set similarity threshold.

[0078] In some embodiments of the present application, if the first image and the second image are a telephoto image and a wide-angle image, respectively, an initial mask may be determined based on the pixel similarity between the first image and the second image. When calculating the pixel similarity between the first image and the second image, structural similarity (SSIM), irrelevance coefficient, absolute value of pixel difference, etc. may be used as pixel similarity. Taking structural similarity as an example, an initial mask (also called mask_SSIM) may be determined based on the pixel similarity between the first image and the second image through steps A1-A2:

[0079] Step A1, calculating the structural similarity between the pixel value of the first image and the pixel value of the second image.

[0080] The value range of SSIM can be -1 to 1; when the two images are exactly the same, the SSIM value is 1. The higher the SSIM, the higher the similarity between the pixel value in the second image and the pixel value in the first image, and the lower the SSIM, the lower the similarity. The reasons for low similarity may be: difference in clarity, misregistration, exposure difference, etc.

[0081] Step A2: if the structural similarity of the pixel point is less than a preset similarity threshold, the pixel value of the pixel point in the initial mask is set to a first value.

[0082] The similarity threshold (thr1) can be pre-set by the user to threshold the SSIM. If the SSIM is lower than thr1, the pixel value of the pixel in the initial mask is set to the first value (set to 1); if the SSIM is greater than or equal to thr1, the pixel value of the pixel in the initial mask is set to the second value (set to 0).

[0083] Through the above method, an initial mask can be obtained. The pixel value of each pixel of the initial mask is composed of a first value (1) and a second value (0). The pixel points set to the first value represent a low similarity, which may be caused by clarity differences, misalignment, exposure differences, etc.

[0084] Step S606, obtaining a first mask corresponding to the optical flow map; wherein the pixel points with the first value in the first mask represent the edge pixel points in the optical flow map.

[0085] In some embodiments of the present application, the first mask is calculated based on the optical flow map, and the value of each pixel in the first mask can characterize the flatness of the pixel in the optical flow map, and is characterized by the standard deviation of the optical flow map. For example: for the pixel in the optical flow map, the first standard deviation in the horizontal axis direction and the second standard deviation in the vertical axis direction are calculated; the larger value of the first standard deviation and the second standard deviation is used as the standard deviation of the pixel; if the standard deviation of the pixel is greater than a preset standard deviation threshold, the pixel value of the pixel in the first mask (also called mask) is set to the first value.

[0086] The coordinates of each pixel in the optical flow map are in the form of (x, y). By calculating the standard deviation of the coordinate values ​​of each pixel in the x direction and the y direction, the first standard deviation of each pixel in the x direction (horizontal coordinate direction) and the second standard deviation in the y direction (vertical coordinate direction) can be obtained; the larger value of the first standard deviation and the second standard deviation is selected as the standard deviation of the pixel.

[0087] If the standard deviation of the pixel is greater than the preset standard deviation threshold (thr), this indicates that the optical flow of the pixel is not flat, and the pixel value of the pixel in the first mask is set to the first value (set to 1); if the standard deviation of the pixel is less than or equal to thr, this indicates that the optical flow of the pixel is flat, and the pixel value of the pixel in the first mask is set to the second value (set to 0).

[0088] In some embodiments of the present invention, after obtaining the first mask, the ratio of the area of ​​the uneven optical flow in the first mask can be calculated. If the ratio is too high, it is difficult to perform image fusion. For example: calculate the ratio of the area of ​​the pixel point with the first value in the first mask to the area of ​​the first mask; if the ratio is less than the preset ratio threshold, continue to perform the step of determining the pixel value of the target pixel point in the first mask for the connected area of ​​the target pixel point with the first value in the initial mask.

[0089] In some embodiments of the present application, the area of ​​the pixel points whose pixel values ​​in the first mask are the first value is the area of ​​the pixel points that characterize the uneven optical flow, and the ratio ratio of the area to the area of ​​the first mask is calculated. If the ratio is less than the preset ratio threshold (ratioThr), it means that there are fewer pixels with uneven optical flow, and image fusion can continue. If the ratio is greater than or equal to the ratioThr, it means that there are more pixels with uneven optical flow, too many edges in the optical flow, and there is a high probability that the registration will be completely wrong. At this time, an error can be reported and image fusion will not continue; or the wide-angle image can be directly used as the fused image.

[0090] In some embodiments of the present invention, after obtaining the first mask, for some pixel points in the initial mask, these pixel points represent a high similarity in the initial mask, but in order to ensure that the optical flow is not flat in the first mask, the above-mentioned pixel points are also not aligned, and the initial mask needs to be modified. For example: check the first target area corresponding to the connected area of ​​the target pixel point of the first value in the first mask; if the pixel value of the target area in the second target area corresponding to the initial mask or the modified mask is the second value, set the pixel value in the second target area to the first value.

[0091] The pixels of the connected area in the first target area in the first mask are all of the first value, and the optical flows of the pixels representing the first target area are all uneven. The area corresponding to these pixels in the initial mask or the modified mask is the second target area. Setting the pixels of the second value in the second target area to the first value can ensure that all pixels with uneven optical flows are set to pixels that may not be registered. In addition to the above-mentioned connected area, the first target area may also include the vicinity of the above-mentioned connected area, and the pixels of the vicinity of the above-mentioned connected area may be the first value or the second value.

[0092] Specifically, first, in the connected areas with a value of 1 in the mask, these areas and the nearby areas are taken as the first target areas, and in the second target area of ​​the mask_SSIM corresponding to the first target area, the pixels of the second value are set to the first value.

[0093] In this method, for the initial mask or the modified mask, if the similarity of a pixel is very high, but the optical flow flatness of the pixel represented by the first mask is low, it is considered that the pixel may also be misaligned.

[0094] Step S608: for the connected areas of the target pixel points with the first value in the initial mask, the pixel value of the target pixel points in the first mask is determined.

[0095] The pixels with the first value in the initial mask are divided into multiple connected regions. The connected regions are generally divided into two ways: 4 adjacent connected domain labels and 8 adjacent connected domain labels. After the connected regions are divided, the pixels in the connected regions are called target pixels, and the pixel values ​​of the target pixels in the first mask can be obtained. It can be considered that the target pixels in the connected regions are all pixels with flat optical flow, which will not be distorted and will not be misaligned.

[0096] Step S610: If the pixel value of the target pixel in the first mask is the first value, the pixel value of the target pixel in the initial mask is set to the first value to obtain a modified mask.

[0097] If the pixel value of the target pixel in the first mask is the first value (i.e., the pixel value of the target pixel in the first mask is 1), it means that the similarity is low due to reasons such as clarity difference and exposure difference, and the clarity is not low due to misregistration. Therefore, the pixel value of the target pixel in the initial mask can be set to the first value to obtain a modified mask.

[0098] In this method, the initial mask can be corrected by the first mask, and the pixels with low similarity due to clarity difference, exposure difference, etc. in the initial mask are set from the second value to the first value to obtain a corrected mask, thereby increasing the fusion weight of the telephoto image for the subsequent fused image and increasing the clarity of the fused image.

[0099] In some embodiments of the present application, further updates can be performed on the corrected mask, for example: detection of areas beyond the boundary of the optical flow, that is, detection of areas whose coordinates exceed the image boundary after remap. First, the first image can be obtained by the following steps: acquiring a first initial image and a second image having the same field of view; mapping the first initial image to obtain the first image, so that the first image is aligned with the second image. The first initial image can be obtained by mapping the first image. Afterwards, the boundary area of ​​the first image can be detected, for example: determining a mapping mask mask_re based on the first initial image and the first image; wherein the pixel points with the first value in the mapping mask mask_re are used to characterize the pixel points in the first image whose coordinates exceed the boundary of the first initial image.

[0100] Since the first initial image will be moved during the mapping process, the coordinates of the first image will move out of the boundary of the first initial image, and the values ​​of the pixels in the first image whose coordinates exceed the boundary of the first initial image are set to 1 to obtain mask_re, where 1 represents the area beyond the boundary.

[0101] For the pixels in the area beyond the boundary, the pixel values ​​of these pixels can be set to 1 in the correction mask, and it is considered that they cannot be aligned. For example, the pixels in the correction mask corresponding to the pixels with the first value in the mapping mask are set to the first value to obtain an updated correction mask.

[0102] In some embodiments of the present application, if the difference in pixel value between the same pixel point of the mapped image and the second image is too large, it can also be considered that they cannot be aligned, for example: calculate the difference between the pixel value of the pixel point of the first image and the pixel value of the pixel point of the second image; if the difference is greater than a preset pixel value threshold, set the pixel value of the pixel point in the correction mask to the first value, and obtain an updated correction mask.

[0103] The user can pre-set a pixel value threshold (thr2). If the absolute value of the difference between the pixel value of a pixel point in the mapped image and the pixel value in the second image is greater than thr2, this indicates that the pixel value difference of the pixel point is too large and cannot be aligned. The pixel value of the pixel point in the correction mask can be set to the first value.

[0104] Step S612: fusing the first image and the second image based on the corrected mask to obtain a fused image.

[0105] In some embodiments of the present application, when performing image fusion, the correction mask can be first negated, and then the second overlapping image and the first overlapping image are fused according to the target mask obtained after the negation. For example: the correction mask is negated to obtain the target mask; the first image and the second image are fused based on the target mask to obtain a fused image.

[0106] In some embodiments of the present application, if the first image and the second image are respectively a telephoto image and a wide-angle image, the following steps may be performed: inverting the correction mask to obtain a target mask; fusing the first image and the second image based on the target mask to obtain a local fused image; and fusing the local fused image with the wide-angle image to obtain a global fused image.

[0107] The modified mask can be inverted by the following formula: mask_SSIM=1-mask_SSIM. In the inverted target mask, the pixel points set to the first value (1) represent that the registration is possible, and the pixel points set to the second value (0) represent that the registration is not possible.

[0108] In some embodiments of the present application, the corrected boundary area can be gradient processed to make the pixel value gradient of the correction mask more reasonable. For example, the boundary area of ​​the correction mask is gradient processed so that the pixel value of the correction mask starts from the boundary of the correction mask and changes from the preset third value to the first value with a preset step value to obtain an updated correction mask.

[0109] The mask_SSIM image boundary area is gradient processed. From the edges of the image to the middle, the pixel value increases from 0.0 to 1.0 with a preset step value (for example, 0.01). The reason for this is to increase or decrease the gradient fusion area when the wide-angle information outside the telephoto camera FOV is spliced ​​with the telephoto camera, so that there will be no obvious sense of disconnection visually.

[0110] In some embodiments of the present application, mask_SSIM may be Gaussian blurred thereafter, for example, Gaussian blurring may be performed on the modified mask to add a transition region in a region of the modified mask where the pixel value is the first value.

[0111] The purpose of Gaussian blur is to increase the transition area, see Figure 7 A schematic diagram of a correction mask after Gaussian blur is shown. In the correction mask after Gaussian blur, white represents the use of a telephoto camera, and black represents the use of a wide-angle camera.

[0112] It can be seen that the wide-angle camera is generally used near the edge of the optical flow. At the same time, it can solve the problem of actual alignment caused by reasons such as clarity difference but low image similarity, thereby incorrectly using the wide-angle camera information.

[0113] For specific steps of the image fusion processing method provided in this embodiment, please refer to Figure 8 A schematic diagram of an image fusion processing method is shown, in which feature point matching detection is performed on a telephoto image and a wide-angle image to obtain overlapping images obtained in the overlapping area, which are respectively a first overlapping image and a second overlapping image. An optical flow map is obtained based on the first overlapping image and the second overlapping image, and a mapping image is obtained based on the optical flow map and the first overlapping image. An initial mask mask_SSIM is obtained based on the mapping image and the second overlapping image, wherein the area below thr1 is set to 1, representing an unregistered candidate area.

[0114] Then update mask_SSIM, calculate the standard deviation according to the optical flow map, set the similarity with standard deviation greater than thr to 1 on the first mask mask, merge the mask to mask_SSIM, so that the area with mask 1 is also set to 1 on mask_SSIM. For the map and the second overlapping image summary, set the pixel points with the absolute value of the difference between the pixel values ​​greater than th2 to 1 on mask_SSIM.

[0115] The mapping mask mask_re is obtained according to the mapping image, and the pixel points whose mask_re coordinates exceed the boundary of the first overlapping image are set to 1. The mask_re is merged into the updated mask_SSIM, and the connected domain of the mask_SSIM is calculated. If each mask_SSIM has no pixel point with a value of 1 in the mask, the pixel values ​​of all the pixels in the connected domain are set to 0.

[0116] The updated mask_SSIM is inverted to obtain the target mask, the boundary of the target mask is gradient processed, and Gaussian blur is performed on the whole to obtain the final weight map. A fused image can be obtained according to the weight map, and the image fusion processing method in this embodiment is completed.

[0117] Embodiment 4

[0118] Corresponding to the above method embodiment, the embodiment of the present invention provides an image fusion processing device, such as Fig. 9 The structure diagram of an image fusion processing device shown in FIG. 1 includes:

[0119] An optical flow map determining module 91, configured to determine a corresponding optical flow map based on a first image and a second image having the same field of view;

[0120] An initial mask determination module 92, configured to determine an initial mask based on pixel similarity between the first image and the second image; wherein the pixel points of the initial mask with a preset first value are used to represent pixel points whose similarity is less than a set similarity threshold;

[0121] A modified mask determination module 93 is used to modify the pixel values ​​of the connected areas of the pixels of the first value in the initial mask based on the pixels of the corresponding area of ​​the connected areas in the optical flow map to obtain a modified mask;

[0122] The fused image determination module 94 is used to perform fusion processing on the first image and the second image based on the correction mask to obtain a fused image.

[0123] An image fusion processing device provided by an embodiment of the present invention determines an optical flow map based on a first image and a second image with the same field of view, and determines an initial mask based on the pixel similarity of the first image and the second image; based on the pixel points of the corresponding area of ​​the connected area in the optical flow map, the pixel values ​​of the pixel points in the connected area of ​​the pixel points with the first value in the initial mask are corrected to obtain a corrected mask; based on the corrected mask, the first image and the second image are fused to obtain a fused image. In this method, the initial mask is corrected by the optical flow map, and the fused image is obtained based on the corrected mask, which can reduce the detection error of the unmatched area in the fused image, so that the registration area uses the telephoto image information in its entirety, improves the clarity of the fused image, and maximizes the fusion result.

[0124] The above-mentioned modified mask determination module is used to obtain the first mask corresponding to the optical flow map; wherein the pixel points with the first value in the first mask represent the edge pixel points in the optical flow map; for the connected area of ​​the target pixel points with the first value in the initial mask, the pixel value of the target pixel point in the first mask is determined; if the pixel value of the target pixel point in the first mask is the first value, the pixel value of the target pixel point in the initial mask is set to the first value to obtain the modified mask.

[0125] The above-mentioned modified mask determination module is also used to check the first target area corresponding to the connected area of ​​the target pixel point of the first value in the first mask; if the pixel value of the target area in the second target area corresponding to the initial mask or the modified mask is the second value, the pixel value in the second target area is set to the first value.

[0126] The above-mentioned corrected mask determination module is used to calculate the first standard deviation in the horizontal coordinate direction and the second standard deviation in the vertical coordinate direction for each pixel point in the optical flow map; the larger value of the first standard deviation and the second standard deviation is used as the standard deviation of the pixel point; if the standard deviation of the pixel point is greater than the preset standard deviation threshold, the pixel value of the pixel point in the first mask is set to the first value.

[0127] The above-mentioned modified mask determination module is also used to calculate the ratio of the area of ​​the pixel points whose pixel values ​​in the first mask are the first value to the area of ​​the first mask; if the ratio is less than the preset ratio threshold, continue to execute the step of determining the pixel value of the target pixel point in the first mask for the connected area of ​​the target pixel point with the first value in the initial mask.

[0128] See also Fig.10 The structural schematic diagram of another image fusion processing device, the image fusion processing device also includes: a correction mask correction module 95, which is respectively connected to the correction mask determination module 93 and the fusion image determination module 94, the correction mask correction module 95, which is used to map the first image to obtain a mapping image; wherein the mapping image is aligned with the second image; the mapping mask is determined based on the first image and the mapping image; wherein the pixel points with the first value in the mapping mask are used to represent the pixel points in the mapping image whose coordinates exceed the boundary of the first image; the correction mask correction module 95 is also used to set the pixel points in the correction mask corresponding to the pixel points with the first value in the mapping mask to the first value, so as to obtain an updated correction mask.

[0129] The correction mask correction module is further used to calculate the difference between the pixel value of the pixel point of the first image and the pixel value of the pixel point of the second image; if the difference is greater than a preset pixel value threshold, the pixel value of the pixel point in the correction mask is set to the first value to obtain an updated correction mask.

[0130] The correction mask correction module is further used to perform gradient processing on the boundary area of ​​the correction mask so that the pixel value of the correction mask changes from the preset third value to the first value at a preset step value starting from the boundary of the correction mask to obtain an updated correction mask.

[0131] The correction mask correction module is further used to perform Gaussian blur processing on the correction mask to add a transition area in the area of ​​the correction mask where the pixel value is the first value.

[0132] The above-mentioned initial mask determination module is used to calculate the structural similarity between the pixel value of the first image and the pixel value in the second image; if the structural similarity of the pixel is less than a preset similarity threshold, the pixel value of the pixel in the initial mask is set to the first value.

[0133] The fused image determination module is used to invert the correction mask to obtain a target mask; and to fuse the first image and the second image based on the target mask to obtain a fused image.

[0134] like Fig.10 As shown, the image fusion processing device also includes: an image acquisition module 96, which is connected to the optical flow map determination module 91, and the image acquisition module 96 is used to obtain the overlapping area corresponding to the same field of view range of the telephoto image and the wide-angle image to be fused to obtain the first image and the second image; the above-mentioned fused image determination module is used to invert the correction mask to obtain the target mask; fuse the first image and the second image based on the target mask to obtain a local fused image; fuse the local fused image with the wide-angle image to obtain a global fused image.

[0135] The image fusion processing device provided in the embodiment of the present invention has the same technical features as the image fusion processing method provided in the above embodiment, and can therefore solve the same technical problems and achieve the same technical effects.

[0136] Embodiment 5

[0137] The embodiment of the present invention also provides an electronic system for executing the above-mentioned image fusion processing method; see Fig.11 A structural schematic diagram of another electronic system is shown, which includes a memory 10 and a processor 11, wherein the memory 10 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor 11 to implement the above-mentioned image fusion processing method.

[0138] Further, Fig.11 The electronic system shown further includes a bus 12 and a communication interface 13 , and the processor 11 , the communication interface 13 and the memory 10 are connected via the bus 12 .

[0139] The memory 10 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 13 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 12 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0140] The processor 11 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 11. The above processor 11 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 10, and the processor 11 reads the information in the memory 10 and completes the steps of the method of the above embodiment in combination with its hardware.

[0141] Embodiment 6

[0142] An embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned image fusion processing method. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0143] The computer program product of the image fusion processing method, device and electronic system provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method in the previous method embodiment. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0144] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and / or device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0145] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0146] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, an electronic system, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0147] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0148] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An image fusion processing method, characterized in that: The method comprises: Acquire an overlapping area corresponding to the same field of view of the telephoto image and the wide-angle image to be fused, to obtain a first image and a second image; Determine a corresponding optical flow map based on a first image and a second image having the same field of view; Determine an initial mask based on pixel similarity between the first image and the second image; wherein the pixel points with a preset first value in the initial mask are used to represent pixel points with a similarity less than a set similarity threshold; For a connected area of ​​pixels having the first value in the initial mask, pixel values ​​of the pixels in the connected area are corrected based on the pixels in the corresponding area of ​​the connected area in the optical flow map to obtain a corrected mask; The first image and the second image are fused based on the correction mask to obtain a fused image.

2. The method according to claim 1, characterized in that For a connected area of ​​pixels having the first value in the initial mask, the step of correcting the pixel values ​​of the pixels in the connected area based on the pixels in the corresponding area of ​​the connected area in the optical flow map to obtain a corrected mask includes: Acquire a first mask corresponding to the optical flow map; wherein the pixel points with the first value in the first mask represent edge pixel points in the optical flow map; For each connected area of ​​the target pixel point having the first value in the initial mask, determining the pixel value of the target pixel point in the first mask; If the pixel value of the target pixel in the first mask is the first value, the pixel value of the target pixel in the initial mask is set to the first value to obtain a modified mask.

3. The method according to claim 2, characterized in that After the step of acquiring the first mask corresponding to the optical flow map, the method further includes: Checking a first target area corresponding to a connected area of ​​target pixels of the first value in the first mask; If the pixel value of the target area in the second target area corresponding to the initial mask or the modified mask is a second value, the pixel value in the second target area is set to the first value.

4. The method according to claim 2 or 3, characterized in that: The step of obtaining a first mask corresponding to the optical flow map includes: For each pixel point in the optical flow map, a first standard deviation in the horizontal coordinate direction and a second standard deviation in the vertical coordinate direction are calculated; The larger value of the first standard deviation and the second standard deviation is used as the standard deviation of the pixel; If the standard deviation of the pixel point is greater than a preset standard deviation threshold, the pixel value of the pixel point in the first mask is set to the first value.

5. The method according to any one of claims 2 to 4, characterized in that: After the step of acquiring the first mask corresponding to the optical flow map, the method further includes: Calculating a ratio of an area of ​​a pixel point in the first mask whose pixel value is the first value to an area of ​​the first mask; If the ratio is less than a preset ratio threshold, continue to perform the step of determining the pixel value of the target pixel in the first mask for each connected area of ​​the target pixel having the first value in the initial mask.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Acquire a first initial image and a second image having the same field of view; perform mapping processing on the first initial image to obtain the first image; Determining a mapping mask based on the first initial image and the first image; wherein the pixel points of the first value in the mapping mask are used to represent the pixel points of the first image whose coordinates exceed the boundary of the first initial image; After the step of obtaining the correction mask, the method further includes: setting the pixel points in the correction mask corresponding to the pixel points of the first value in the mapping mask to the first value, so as to obtain an updated correction mask.

7. The method according to claim 6, characterized in that After the step of obtaining the correction mask, the method further comprises: Calculating a difference between a pixel value of a pixel point of the first image and a pixel value of a pixel point of the second image; If the difference is greater than a preset pixel value threshold, the pixel value of the pixel point in the correction mask is set to the first value to obtain an updated correction mask.

8. The method according to any one of claims 1 to 5, characterized in that: After the step of obtaining the correction mask, the method further comprises: A gradient process is performed on the boundary area of ​​the correction mask, so that the pixel value of the correction mask changes from a preset third value to the first value at a preset step value starting from the boundary of the correction mask, to obtain an updated correction mask.

9. The method according to any one of claims 1 to 5, characterized in that: After the step of obtaining the correction mask, the method further comprises: Gaussian blur processing is performed on the correction mask to add a transition area to a region of the correction mask where the pixel value is the first value.

10. The method according to any one of claims 1 to 9, characterized in that: The step of determining an initial mask based on pixel similarity between the first image and the second image comprises: Calculating the structural similarity between the pixel value of the pixel point in the first image and the pixel value in the second image; If the structural similarity of the pixel point is less than a preset similarity threshold, the pixel value of the pixel point in the initial mask is set to the first value.

11. The method according to any one of claims 1 to 9, characterized in that: The step of fusing the first image and the second image based on the correction mask to obtain a fused image includes: Inverting the correction mask to obtain a target mask; The first image and the second image are fused based on the target mask to obtain a fused image.

12. The method according to any one of claims 1 to 9, characterized in that: The step of fusing the first image and the second image based on the correction mask to obtain a fused image includes: Inverting the correction mask to obtain a target mask; fusing the first image and the second image based on the target mask to obtain a local fused image; The local fused image is fused with the wide-angle image to obtain a global fused image.

13. An image fusion processing device, characterized in that: The device comprises: An optical flow map determination module is used to obtain an overlapping area corresponding to the same field of view of the telephoto image and the wide-angle image to be fused, to obtain a first image and a second image; and to determine a corresponding optical flow map based on the first image and the second image having the same field of view; An initial mask determination module, configured to determine an initial mask based on pixel similarity between the first image and the second image; wherein the pixel points of the initial mask having a preset first value are used to represent pixel points whose similarity is less than a set similarity threshold; A modified mask determination module, configured to modify the pixel values ​​of the pixels of the connected area having the first value in the initial mask based on the pixels of the corresponding area of ​​the connected area in the optical flow map, so as to obtain a modified mask; The fused image determination module is used to perform fusion processing on the first image and the second image based on the correction mask to obtain a fused image.

14. An electronic system, characterized in that: The electronic system includes: an image acquisition device, a processing device and a storage device; The image acquisition device is used to acquire wide-angle images and telephoto images; The storage device stores a computer program, and when the computer program is executed by the processing device, the image fusion processing method according to any one of claims 1 to 12 is executed.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the image fusion processing method according to any one of claims 1 to 12 are executed.

Images