Image fusion method based on two-dimensional space information

Abstract

The invention provides an image fusion method based on two-dimensional space information. The image fusion method mainly comprises the following steps: carrying out image registration on an original image; pixels are expanded around the registered images A and B; performing binarization on the registered image A and the registered image B to obtain an image biImgA and an image biImgB, and adding pixel values of the image biImgA and the image biImgB to obtain an image G; performing corrosion operation on the image G to obtain an image E, performing subtraction on the image G and the image E to obtain an image TE, and finally summing the image E and the image TE to obtain an image O; calculating the minimum distances Da and Db from the pixel point p to the boundaries Ea and Eb of the overlapping region; the weight of the image A and the weight of the image B; calculating a pixel value p (x, y) of the overlapped region image E; according to the image fusion method based on the two-dimensional image space information, the fused image contains more comprehensive and accurate information, so that the quality and interpretability of the image are improved.

Description

Technical Field

[0001] The invention belongs to the technical field of image fusion, and in particular relates to an image fusion method based on two-dimensional spatial information. Background Art

[0002] Image fusion is a technology that combines two or more images into a new image through a specific algorithm. In the process of image fusion, the temporal and spatial correlation and information complementarity of the source images are utilized, so that the fused image can describe the scene more comprehensively and clearly. This not only helps the human eye to identify and judge, but also facilitates automatic detection by machines.

[0003] With the development of technology, image fusion technology has gone through several stages. Early image fusion methods mainly include averaging, HIS transform, principal component analysis, high-pass filtering, etc. These methods do not decompose and transform the image during fusion processing, and the fusion processing is only performed on one level, which is a relatively simple image fusion method. In the mid-1980s, image fusion methods based on pyramid decomposition began to appear, including Laplacian pyramid, gradient pyramid, ratio low-pass pyramid, etc. After the 1990s, the widespread application of wavelet transform technology provided new tools for image fusion, making the research and application of image fusion technology show a rising trend. Its application fields also widely involve remote sensing image processing, computer vision, automatic target detection, medical image processing and other fields.

[0004] In addition, deep learning technology has also been introduced into the field of image fusion, such as the research on polarization image fusion. Through deep learning, the overall quality of the image can be further improved by combining spectral information and polarization information, and the effect of image fusion can be improved. This technology has broad application prospects in image enhancement, space remote sensing, target recognition, military defense and other fields.

[0005] Different images may come from different sensors, different shooting angles or different shooting times, and their respective contrast and brightness may be different. Direct fusion will produce stitching gaps and the overlapping area will be too abrupt, reducing the visual intuitive experience. Summary of the invention

[0006] In order to solve the problem that different images may come from different sensors, different shooting angles or different shooting times, their respective contrast and brightness may be different. Direct fusion will produce stitching gaps and the overlapping area is too abrupt, reducing the visual intuitive experience.

[0007] To this end, the present invention provides an image fusion method based on two-dimensional spatial information, comprising the following steps:

[0008] Step 1: Obtain the original image A and the original image B to be fused;

[0009] Step 2: perform image registration on the original image A and the original image B;

[0010] Step 3: Expand the surroundings of the registered images A and B by one pixel and assign a value of 0;

[0011] Step 4: Determine the boundaries and internal pixels of the overlapping area, and binarize the registered images A and B to obtain images biImgA and biImgB;

[0012] Step 5, add the binary image biImgB to the pixel value of the binary image biImgA at the corresponding position according to the registered offset to obtain image G;

[0013] Step 6: Perform an erosion operation on the image G and a matrix of size 3*3 and value 1 to obtain the image E, then perform a difference operation on the image G and the image E to obtain the image TE, and finally sum the image E and the image TE to obtain the image O;

[0014] Step 7: Calculate the minimum distances Da and Db between each pixel point p in the overlapping area image E and the overlapping area boundaries Ea and Eb;

[0015] Step 8: Calculate the weights of the overlapping area pixels (x, y) in images A and B using the weighted average idea;

[0016] Step 9: Calculate the pixel value p(x, y) of the overlapping area image E.

[0017] Furthermore, the specific process of step 4, determining the boundary and internal pixel points of the overlapping area, binarizing the registered images A and B, and obtaining the images biImgA and biImgB is:

[0018] The registered images A and B are binarized, and the pixels greater than the threshold T are assigned values ​​of vA and vB, respectively, to obtain images biImgA and biImgB, that is:

[0019]

[0020] Furthermore, the image G and the matrix with a size of 3*3 and a value of 1 are corroded to obtain the image E, and then the image G and the image E are subtracted to obtain the image TE, and finally the image E and the image TE are summed to obtain the image O. At this time, the boundary pixel values ​​of the overlapping area in the image O, the values ​​of the pixels inside the overlapping area, and the pixel values ​​of the non-overlapping area will be different, that is:

[0021]

[0022] Furthermore, in step 8, the weights of the overlapping area pixels (x, y) in images A and B are calculated using the weighted average idea. The formula is as follows:

[0023]

[0024] Among them, W A (x,y),W B (x, y) are the weights of the overlapping pixel point (x, y) in images A and B, respectively. Da(x, y) and Db(x, y) are the distances from the overlapping pixel point (x, y) to images A and B, respectively. Least is a constant to prevent division by zero, Least = 0.000001.

[0025] Furthermore, in step 9, the pixel value p(x, y) of the overlapping area image E is calculated using the following formula:

[0026] p(x,y)=A(x,y)*W A (x,y)+B(x,y)*W B (x,y) (4)

[0027] Where A(x,y) is the pixel value of image A at the overlapping area (x,y), and B(x,y) is the pixel value of image B at the overlapping area (x,y).

[0028] The advantages of the present invention are: the present invention provides such an image fusion method based on two-dimensional image space information, which fully considers the information of the overlapping area in each dimension, thereby being able to solve the problems of unnatural transition of the overlapping area and splicing gaps at the boundary, so that the fused image contains more comprehensive and accurate information, thereby improving the quality and interpretability of the image.

[0029] The present invention is described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a flow chart of the image fusion method based on two-dimensional spatial information.

[0031] Figure 2 Schematic diagram of pixel values ​​in different areas of the image.

[0032] Figure 3a Schematic diagram of the first group of images A to be fused.

[0033] Figure 3b Schematic diagram of the first group of images B to be fused.

[0034] Figure 3c Schematic diagram of connecting the first group of images A to be fused and the image B to be fused.

[0035] Figure 3d Schematic diagram of the fusion connection between the first group of images A to be fused and the image B to be fused.

[0036] Figure 4a Schematic diagram of the second group of images A to be fused.

[0037] Figure 4b Schematic diagram of the second group of images B to be fused.

[0038] Figure 4c Schematic diagram of the second group of images C to be fused.

[0039] Figure 4d Schematic diagram of the fusion connection between the second group of images A to be fused and the image B to be fused.

[0040] Figure 4e Schematic diagram of the fusion connection of the second group of images A to be fused, images B to be fused, and images C to be fused. DETAILED DESCRIPTION

[0041] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose, the specific implementation methods, structural features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and embodiments.

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] All features disclosed in this specification, or steps in all methods or processes disclosed, except mutually exclusive features and / or steps, can be combined in any manner.

[0044] Any feature disclosed in this specification (including any additional claims, abstract and drawings), unless otherwise stated, may be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

[0045] Example 1

[0046] This embodiment proposes a Figure 1 As shown, in order to solve the problem that different images may come from different sensors, different shooting angles or different shooting times, their respective contrast and brightness may be different. Direct fusion will produce stitching gaps and the overlapping area is too abrupt, reducing the visual intuitive experience.

[0047] To this end, the present invention provides an image fusion method based on two-dimensional spatial information, comprising the following steps:

[0048] Step 1: Obtain the original image A and the original image B to be fused;

[0049] Step 2: perform image registration on the original image A and the original image B;

[0050] The image registration process of this embodiment adopts the SURF (Speeded Up Robust Features) image registration algorithm, which is a prior art. This embodiment is briefly described as follows: First, the SURF algorithm detects key points in the image through the Hessian matrix, and these key points have scale invariance and rotation invariance. Then, the algorithm generates a unique descriptor for each key point, which contains information about the area around the key point, so that the corresponding point can be accurately identified in the subsequent matching process. Next, the SURF algorithm uses an effective feature matching strategy, such as distance-based nearest neighbor matching, to match the key points in the two images. During the matching process, the algorithm considers the similarity and spatial relationship between the key points to eliminate incorrect matching. Finally, by estimating a transformation model (such as an affine transformation or a perspective transformation), the SURF algorithm calculates the geometric transformation relationship between the two images. This transformation model describes how to map one image to another image to achieve image registration.

[0051] Step 3: Expand the surroundings of the registered images A and B by one pixel and assign a value of 0 to facilitate subsequent boundary calculation;

[0052] Step 4: determine the boundary and internal pixel points of the overlapping area, binarize the registered images A and B, assign vA and vB to the pixels greater than the threshold T (T=1 in the present invention), and obtain images biImgA and biImgB;

[0053] Step 5, add the binary image biImgB to the pixel value of the binary image biImgA at the corresponding position according to the registered offset to obtain image G;

[0054] Step 6: Perform an erosion operation on the image G and a matrix of size 3*3 and value 1 to obtain the image E, then perform a difference operation on the image G and the image E to obtain the image TE, and finally sum the image E and the image TE to obtain the image O;

[0055] Step 7: Calculate the minimum distances Da and Db between each pixel point p in the overlapping area image E and the overlapping area boundaries Ea and Eb;

[0056] Step 8: Calculate the weights of the overlapping area pixels (x, y) in images A and B using the weighted average idea;

[0057] Step 9: Calculate the pixel value p(x, y) of the overlapping area image E.

[0058] Furthermore, the specific process of step 4, determining the boundary and internal pixel points of the overlapping area, binarizing the registered images A and B, and obtaining the images biImgA and biImgB is:

[0059] The registered images A and B are binarized, and the pixels greater than the threshold T are assigned values ​​of vA and vB, respectively, to obtain images biImgA and biImgB, that is:

[0060]

[0061] Further, such as Figure 2 As shown, the image G and the matrix with a size of 3*3 and a value of 1 are corroded to obtain the image E, and then the image G and the image E are subtracted to obtain the image TE, and finally the image E and the image TE are summed to obtain the image O. At this time, the boundary pixel values ​​of the overlapping area in the image O, the values ​​of the pixels inside the overlapping area, and the pixel values ​​of the non-overlapping area will be different, that is:

[0062] Furthermore, in step 8, the weights of the overlapping area pixels (x, y) in images A and B are calculated using the weighted average idea. The formula is as follows:

[0063]

[0064] Among them, W A (x,y),W B (x, y) are the weights of the overlapping area pixel point (x, y) in images A and B, respectively. Da(x, y) and Db(x, y) are the distances from the overlapping area pixel point (x, y) to images A and B, respectively. Least is a constant to prevent division by zero. In this embodiment, Least = 0.000001.

[0065] Furthermore, in step 9, the pixel value p(x, y) of the overlapping area image E is calculated using the following formula:

[0066] p(x,y)=A(x,y)*W A (x,y)+B(x,y)*W B (x,y) (4)

[0067] Where A(x,y) is the pixel value of image A at the overlapping area (x,y), and B(x,y) is the pixel value of image B at the overlapping area (x,y).

[0068] To sum up, the image fusion method based on two-dimensional image space information provided in this embodiment fully considers the information of the overlapping area in each dimension, thereby solving the problems of unnatural transition in the overlapping area and splicing gaps in the boundaries, so that the fused image contains more comprehensive and accurate information, thereby improving the quality and interpretability of the image.

[0069] Example 2

[0070] The image fusion method based on two-dimensional image space information shown in Example 1 is used to Figure 3a , Figure 3b The images shown in the figure are fused, and the fusion result is as follows Figure 3d As shown, the images are fused: Figure 3d Image with direct link Figure 3c By comparison, even if there are large differences in brightness and contrast between the images to be fused, the image fusion method based on two-dimensional image space information can still completely eliminate the seams at the boundaries of the image overlapping areas and achieve a smooth transition of the image overlapping areas. Overall, the image brightness, contrast and color balance in the fusion transition area are very natural and coordinated.

[0071] Example 3

[0072] The image fusion method based on two-dimensional image space information shown in Example 1 is used to Figure 4a , Figure 4b , Figure 4c The images shown in the figure are fused, and the fusion result is as follows Figure 4d , Figure 4e As shown, Figure 4d for Figure 4a , Figure 4b The fused image, Figure 4e for Figure 4a , Figure 4b , Figure 4c The fused image of Figure 4d and Figure 4e In comparison, both can solve the problems of unnatural transition in overlapping areas and stitching gaps in boundaries, so that the fused image contains more comprehensive and accurate information, thereby improving the quality and interpretability of the image. In other words, even if multiple images are fused, the image fusion method based on two-dimensional image space information can still achieve very good image fusion effects.

[0073] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. An image fusion method based on two-dimensional spatial information, characterized in that: The steps include: Step 1: Obtain the original image A and the original image B to be fused; Step 2: perform image registration on the original image A and the original image B; Step 3: Expand the surroundings of the registered images A and B by one pixel and assign a value of 0; Step 4: Determine the boundaries and internal pixels of the overlapping area, and binarize the registered images A and B to obtain images biImgA and biImgB; Step 5, add the binary image biImgB to the pixel value of the binary image biImgA at the corresponding position according to the registered offset to obtain image G; Step 6: Perform an erosion operation on the image G and a matrix of size 3*3 and value 1 to obtain the image E, then perform a difference operation on the image G and the image E to obtain the image TE, and finally sum the image E and the image TE to obtain the image O; Step 7: Calculate the minimum distances Da and Db between each pixel point p in the overlapping area image E and the overlapping area boundaries Ea and Eb; Step 8: Calculate the weights of the overlapping area pixels (x, y) in images A and B using the weighted average idea; Step 9: Calculate the pixel value p(x, y) of the overlapping area image E.

2. The image fusion method based on two-dimensional spatial information according to claim 1, characterized in that: The specific process of step 4, determining the boundary and internal pixel points of the overlapping area, binarizing the registered images A and B, and obtaining the images biImgA and biImgB is: The registered images A and B are binarized, and the pixels greater than the threshold T are assigned values ​​of vA and vB, respectively, to obtain images biImgA and biImgB, that is:

3. The image fusion method based on two-dimensional spatial information as claimed in claim 1, characterized in that: The image G and the matrix with a size of 3*3 and a value of 1 are corroded to obtain the image E, and then the image G and the image E are subtracted to obtain the image TE, and finally the image E and the image TE are summed to obtain the image O. At this time, the boundary pixel values ​​of the overlapping area in the image O, the values ​​of the pixels inside the overlapping area, and the pixel values ​​of the non-overlapping area will be different, that is:

4. The image fusion method based on two-dimensional spatial information according to claim 1, characterized in that: In step 8, the weights of the overlapping pixel points (x, y) in images A and B are calculated using the weighted average concept. The formula is as follows: Among them, W A (x,y),W B (x, y) are the weights of the overlapping pixel point (x, y) in images A and B, respectively. Da(x, y) and Db(x, y) are the distances from the overlapping pixel point (x, y) to images A and B, respectively. Least is a constant to prevent division by zero, Least = 0.000001.

5. The image fusion method based on two-dimensional spatial information as claimed in claim 1, characterized in that: In step 9, the pixel value p(x, y) of the overlapping area image E is calculated using the following formula: p(x,y)=A(x,y)*W A (x,y)+B(x,y)*W B (x,y) (4) Wherein, A(x, y) is the pixel value of image A at the overlapping area (x, y), and B(x, y) is the pixel value of image B at the overlapping area (x, y).

Images