A Multispectral Image Fusion Method for Urban Low-Altitude Remote Sensing Monitoring Targets

Abstract

The present invention provides a multispectral image fusion method for urban low-altitude remote sensing monitoring targets, comprising: a cloud observation device, the cloud observation device comprising an image acquisition module, a position and attitude acquisition module and a storage module, the storage module storing and recording digital information of the image acquisition module and exchangeable image file information; a control module for controlling the exposure interval of the image acquisition module; a fixing device, arranged on the top of an unmanned aerial vehicle, suitable for fixing the cloud observation device and the control module on the unmanned aerial vehicle, the fixing device being provided with a high reflectivity patch to achieve geometric calibration before data acquisition. Based on this method, multispectral image fusion can be performed for specific urban low-altitude remote sensing monitoring targets, ensuring the accuracy of partial fusion of the observed target, and solving the problem of image fusion failure caused by projection differences caused by height differences of urban surface objects.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a multispectral image fusion method and method for urban low-altitude remote sensing monitoring targets. Background Art

[0002] Low-altitude multispectral remote sensing is an emerging remote sensing technology in recent years. It is an observation technology that connects satellites and the ground. It can monitor ground targets over a large area, with high efficiency and high frequency, and provide effective technical support for quantitative high-frequency monitoring and evolution analysis of urban surface resources.

[0003] Multispectral image fusion can obtain more accurate and richer information than single-band data, thereby improving the accuracy of target recognition, classification and inversion. However, due to the large number of high-rise buildings in cities, there will be serious projection differences in the images obtained by low-altitude remote sensing. Traditional image fusion methods cannot meet the requirements of accurate fusion of images with large projection differences, resulting in partially overlapping pixels between channels not being the same target, which seriously affects the accuracy of target recognition, classification and inversion. Summary of the invention

[0004] The present invention provides a multispectral image fusion method for urban low-altitude remote sensing monitoring targets, so as to solve the technical problem of image fusion failure caused by projection difference due to height difference of surface objects in urban low-altitude remote sensing monitoring.

[0005] To solve the above problems, the present invention provides a multispectral image fusion method for urban low-altitude remote sensing monitoring targets, comprising:

[0006] S1: Elevation calculation,

[0007] By using the stereo binocular vision forward intersection method, the elevation of the feature points in the image in the photogrammetry coordinate system or the object space coordinate system is calculated;

[0008] S2: Matching elevation range calculation,

[0009] Get the elevation value of the monitoring target, calculate the projection difference caused by the heights of different objects based on the elevation, set the maximum value allowed for the projection difference, and determine the elevation range within which feature matching is allowed based on the value;

[0010] S3: Weak feature objects removal,

[0011] Classify images through deep learning algorithms and remove objects with weak features;

[0012] S4: Feature point matching,

[0013] Match the feature points of images in different bands at the same exposure point;

[0014] S5: Image space transformation,

[0015] A coordinate system is constructed based on a certain image in the synchronously exposed multispectral image, and the spatial correspondence between the fused shadow lines is established based on the matched feature points to calculate the spatial transformation parameters.

[0016] S6: Image fusion,

[0017] The spatially transformed image is resampled to have the same spatial resolution as the reference image, and finally multiple multispectral images are superimposed to obtain a multi-channel fused image.

[0018] Preferably, in step S1, the calculating the elevation of the feature point in the image in the photogrammetric coordinate system or the object space coordinate system specifically includes:

[0019] S 31 : Select two images of the same band from two adjacent exposure points on the same route to detect and match feature points;

[0020] S 32 : Obtain the interior and exterior orientation elements of the two images, and calculate the elevations of all feature points using the collinear equation formula;

[0021] S 33 : The elevation values ​​of the remaining non-feature points are estimated through interpolation calculation, and finally the elevation values ​​of all pixel points in the entire image can be obtained.

[0022] Preferably, the method for detecting and matching the feature points is a SIFT algorithm or a SURF algorithm.

[0023] Preferably, the calculation expression of the collinear equation formula is as follows:

[0024]

[0025]

[0026] in, , is the image plane coordinate of the image point; , , is the internal orientation element of the image; , , is the object space coordinate of the camera station; , , is the object space coordinate of the object point; , , ( = 1,2,3) are the 9 direction cosines composed of the 3 exterior azimuth elements of the image. This is the elevation value.

[0027] Preferably, in step S 33 In the method, the interpolation calculation method for estimating the elevation values ​​of the remaining non-feature point parts includes a bilinear interpolation method.

[0028] Preferably, in step S2, the calculation expression of the projection difference is as follows:

[0029]

[0030] in, is the projection difference, is the ground feature depth, is the projection length of the feature depth on the left image, is the projection length of the feature depth on the right slice, is the object distance, For Hanggao, is the distance between the bottom point of the feature and the main point of the image on the left image. is the focal length, is the baseline distance between the left and right slices.

[0031] Preferably, in step S2, the specific content of determining the elevation range for allowing feature matching includes:

[0032] Taking the monitoring target elevation as the reference plane, some elevation objects with a projection difference greater than 5 pixels in length will be eliminated during image feature matching.

[0033] Preferably, in step S3, the neural network of the deep learning algorithm is a network of the SegNet, MobileNet, or DeepLab series.

[0034] Preferably, in step S6, the specific method of resampling the spatially transformed image is bilinear interpolation.

[0035] Compared with the prior art, the present invention has significant advantages and beneficial effects, which are specifically embodied in the following aspects:

[0036] 1. This method determines the height range suitable for the fusion of target object images by calculating the projection difference of the same object in the image, and selects part of the images within the height range for fusion;

[0037] 2. Classify images through deep learning methods, determine the objects suitable for multi-band image matching, ignore objects with weak features, and achieve high-precision multispectral image fusion for specific monitoring targets;

[0038] 3. Multispectral image fusion can be performed for specific urban low-altitude remote sensing monitoring targets to ensure the accuracy of partial fusion of the observed target and solve the problem of image fusion failure caused by projection differences due to height differences of urban surface objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of a multispectral image fusion method for urban low-altitude remote sensing monitoring targets in an embodiment of the present invention;

[0040] Figure 2 Schematic diagram of a formula for calculating the projection difference of the same ground object in two images by base height ratio in an embodiment of the present invention;

[0041] Figure 3 exemplary image diagram of the same exposure point in three bands B1, B2, and B3 in an embodiment of the present invention;

[0042] Figure 4 Schematic diagram of feature point matching results before image fusion in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0044] like Figure 1-4 As shown, an embodiment of the present invention provides a multispectral image fusion method for urban low-altitude remote sensing monitoring targets, comprising the following steps:

[0045] Step S1: elevation calculation;

[0046] Select two images of the same band from two adjacent exposure points on the same route to detect and match feature points.

[0047] Preferably, SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features) algorithm is used to detect and match feature points.

[0048] Preferably, more overlapping images can be selected to adjust and calculate the elevation of feature points.

[0049] Get the interior and exterior orientation elements of the two images, and calculate the elevation of all feature points using the following collinear equation formula:

[0050]

[0051]

[0052] in, , is the image plane coordinate of the image point; , , is the internal orientation element of the image; , , is the object space coordinate of the camera station; , , is the object space coordinate of the object point; , , ( = 1,2,3) are the 9 direction cosines composed of the 3 exterior azimuth elements of the image.

[0053] That is the elevation value. The elevation values ​​of the remaining non-feature points can be estimated by interpolation calculation. Preferably, the bilinear interpolation method can be used for difference calculation, and finally the elevation values ​​of all pixel points in the entire image can be obtained.

[0054] Step S2: Matching elevation range calculation;

[0055] See also Figure 2 As shown in the figure, the elevation information of the monitored target is obtained from the image with elevation value, and the elevation value is used as the reference plane to calculate the projection difference caused by each base height ratio when it is above and below the reference plane. The projection difference calculation formula is as follows:

[0056]

[0057] in, is the projection difference, is the ground feature depth, is the projection length of the feature depth on the left image, is the projection length of the feature depth on the right slice, is the object distance, For Hanggao, is the distance between the bottom point of the feature and the main point of the image on the left image. is the focal length, is the baseline distance between the left and right slices.

[0058] Preferably, the projection difference d should be no greater than 5 pixels in length. Taking the elevation of the monitored target as the reference plane, the elevation features whose projection difference d is greater than 5 pixels in length are eliminated during image feature point matching.

[0059] Step S3: Eliminate weak feature objects;

[0060] In the embodiment of the present invention, a deep learning method is used to classify the objects in the image and eliminate weak feature objects, such as water surface, glass surface, etc. These object parts will be eliminated when matching the image feature points.

[0061] Preferably, the deep learning neural network can choose SegNet, MobileNet, and DeepLab series networks.

[0062] Step S4: feature point matching;

[0063] According to step S3 and step S4, the parts of the objects whose elevations do not meet the constraint conditions and have weak features in the image are eliminated, and image feature points are matched between different bands of the same exposure point.

[0064] Preferably, SIFT or SURF algorithm is selected for feature point matching.

[0065] See also Figure 3 As shown, in the embodiment of the present invention, three multispectral images are taken as an example. These three images are three different bands B1, B2, and B3 of the same exposure point. Figure 4 The figure shows the feature point matching result after removing some objects.

[0066] Step S5: image space transformation;

[0067] In the embodiment of the present invention, a certain band image is selected as a reference image, a spatial coordinate system is constructed, and the remaining band images are transformed into the four spatial transformation parameters of the reference image based on the matched feature points.

[0068] Step S6: image fusion;

[0069] The spatially transformed image is resampled so that the transformed image has the same spatial resolution as the reference image.

[0070] As the best preferred method of this embodiment, bilinear interpolation is used for image resampling, and finally multiple multispectral images are superimposed to obtain a multi-channel fused image.

[0071] Although the disclosure is disclosed as above, the protection scope of the disclosure is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the disclosure, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A multispectral image fusion method for urban low-altitude remote sensing monitoring targets, characterized in that: include: S1: Elevation calculation, By using the stereo binocular vision forward intersection method, the elevation of the feature points in the image in the photogrammetry coordinate system or the object space coordinate system is calculated; The calculation of the elevation of the feature point in the image in the photogrammetric coordinate system or the object space coordinate system specifically includes: S 31 : Select two images of the same band from two adjacent exposure points on the same route to detect and match feature points; S 32 : Obtain the interior and exterior orientation elements of the two images, and calculate the elevations of all feature points using the collinear equation formula; S 33 : Estimate the elevation values ​​of the remaining non-feature points through interpolation calculation, and finally obtain the elevation values ​​of all pixel points in the entire image; S2: Matching elevation range calculation, Get the elevation value of the monitoring target, calculate the projection difference caused by the heights of different objects based on the elevation, set the maximum value allowed for the projection difference, and determine the elevation range allowed for feature matching based on the value; The calculation expression of the projection difference is as follows: Where d is the projection difference, is the ground feature depth, is the projection length of the feature depth on the left image, is the projection length of the feature depth on the right slice, For Hanggao, is the distance between the bottom point of the feature and the main point of the image on the left image. is the focal length, is the baseline distance between the left and right slices; The specific content of determining the elevation range of the allowed feature matching includes: Taking the monitoring target elevation as the reference plane, the elevation features with a projection difference greater than 5 pixels are eliminated during image feature matching. S3: Weak feature objects removal, Classify images through deep learning algorithms and remove objects with weak features; S4: Feature point matching, Match the feature points of images in different bands at the same exposure point; S5: Image space transformation, A coordinate system is constructed based on a certain image in the synchronously exposed multispectral image, and the spatial correspondence between the fused shadow lines is established based on the matched feature points to calculate the spatial transformation parameters. S6: Image fusion, The spatially transformed image is resampled to have the same spatial resolution as the reference image, and finally multiple multispectral images are superimposed to obtain a multi-channel fused image.

2. The multispectral image fusion method for urban low-altitude remote sensing monitoring targets according to claim 1 is characterized in that: The method for detecting and matching the feature points is a SIFT algorithm or a SURF algorithm.

3. The multispectral image fusion method for urban low-altitude remote sensing monitoring targets according to claim 1 is characterized in that: The calculation expression of the collinear equation formula is as follows: in, , is the image plane coordinate of the image point; , , is the internal orientation element of the image; , , is the object space coordinate of the camera station; , , is the object space coordinate of the object point; , , are the 9 direction cosines composed of the 3 exterior azimuth elements of the image, This is the elevation value.

4. The multispectral image fusion method for urban low-altitude remote sensing monitoring targets according to claim 1 is characterized in that: In step S 33 In the method, the interpolation calculation method for estimating the elevation values ​​of the remaining non-feature point parts includes a bilinear interpolation method.

5. The multispectral image fusion method for urban low-altitude remote sensing monitoring targets according to claim 1 is characterized in that: In step S3, the neural network of the deep learning algorithm is a network of the SegNet, MobileNet, or DeepLab series.

6. The multispectral image fusion method for urban low-altitude remote sensing monitoring targets according to claim 1 is characterized in that: In step S6, the specific method of resampling the spatially transformed image is bilinear interpolation.

Images