A remote sensing image fusion method and system based on geographically weighted principal component analysis
Through the method based on geo-weighted principal component analysis, the weight matrix is calculated by point-by-point processing and local image matrix, the spectral distortion problem in traditional fusion methods is solved, the spatial and spectral resolution of remote sensing images is improved, and more efficient image fusion effect is achieved .
Patent Information
- Application Number
- CN202411320428.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-09-23
AI Technical Summary
The traditional remote sensing image fusion method cannot fully consider the spatial variation characteristics in the image, resulting in serious spectral distortion problems in the fusion result, and it is difficult to improve the spatial resolution and spectral resolution of the image at the same time.
Using a method based on geo-weighted principal component analysis (GWPCA), the weight matrix is calculated by point-by-point processing and local image matrix, geo-weighted mean substitution and PCA decomposition are performed to generate fusion images, and principal component substitution is combined with full-color images to improve the image spatial and spectral resolution.
It effectively overcomes the problem of spectral distortion, improves the spatial resolution and spectral resolution of the fusion image, significantly improves the image's interpretation accuracy and land object recognition capabilities, and shows better performance on multiple indicators than traditional methods and deep learning methods. Good results.
Smart Images

Figure CN118864277B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of remote sensing image fusion, and in particular relates to a remote sensing image fusion method system based on Geographically Weighted Principal Component Analysis (GWPCA). Background Art
[0002] With the continuous progress of earth observation technology and the diversified development of remote sensing platforms, remote sensing images have become an important means of obtaining information on the earth's surface. Remote sensing technology uses sensors installed on satellites, aircraft or other aerial carriers to achieve long-distance and efficient collection of earth surface image data. These image data not only cover a wide geographical range, but also have an extremely fast acquisition speed, and can provide a large amount of surface information in a short time. They can obtain various details of the earth's surface, including topography, vegetation cover, water distribution, urban expansion, etc., and provide strong data support for resource surveys, environmental monitoring, urban and rural planning, disaster assessment and other fields. However, in the actual detection process, it is often impossible to simultaneously obtain images with both high spectral resolution and high spatial resolution. This affects the interpretation accuracy and application effect of the image to a certain extent, which is mainly caused by sensor technology and physical limitations. First of all, high spectral resolution means that the image can capture and record a wider and more detailed electromagnetic spectrum information. This usually requires an increase in the instantaneous field of view, and the increase in the instantaneous field of view often leads to a decrease in spatial resolution. Increasing spatial resolution usually means reducing spectral resolution because the sensor needs to focus on a smaller spatial scale, so it is impossible to capture complete spectral information at the same time. Therefore, how to effectively improve the spatial resolution and spectral resolution of remote sensing images has always been a hot topic in the field of remote sensing research. Remote sensing image fusion technology is a method of combining image data from multiple sources or multiple time points to obtain richer and more accurate information about the earth's surface. Pansharpening, also known as panchromatic sharpening, is a widely used technique in the field of remote sensing image fusion. technology The core goal of this technology is to fuse a panchromatic image (PAN) with a high spatial resolution with a low resolution multispectral image (LRMS). Through data fusion, an image is generated that not only maintains the high spatial resolution of the panchromatic image, but also integrates the rich spectral information in the multispectral image. Panchromatic sharpening fusion technology is of great significance in improving the spatial details and object recognition capabilities of remote sensing images, and is a key link in remote sensing image processing and analysis.
[0003] Many traditional fusion methods based on component substitution (CS) are relatively mature in the field of image fusion, which can well present the spatial details of the fusion product, and are fast and easy to implement. However, such methods usually apply a unified fusion rule to the entire image or a fixed-size window, and fail to fully consider the spatial variation characteristics in the image, which may lead to serious spectral distortion problems in the fusion results. Summary of the invention
[0004] In view of the above shortcomings in the prior art, the present invention proposes a remote sensing image fusion method based on geographically weighted principal component analysis. Local fusion can be performed according to the spatial characteristics of different regions to improve the spectral distortion problem existing in the fusion algorithm based on CS. The following steps are included:
[0005] Step 1, obtaining an original multispectral image and an original panchromatic image of the same area, and resampling the original multispectral image to obtain a target multispectral image;
[0006] Step 2: Process the pixels of the target multispectral image point by point. , select The corresponding neighborhood , generate neighborhood The corresponding local image matrix , and then calculate the corresponding weight matrix ;
[0007] Step 3: Use the geographically weighted mean instead of the conventional mean to decentralize the sample points in the neighborhood, and obtain the decentralized image neighborhood and the corresponding local image matrix;
[0008] Step 4: Apply the weight matrix to the decentralized local image matrix to obtain the geographically weighted covariance matrix , and the geographically weighted covariance matrix Perform PCA decomposition to obtain the matrix of eigenvectors ;
[0009] Step 5: Combine the local image matrix after decentralized processing with the matrix of eigenvectors after PCA decomposition Calculate each principal component ;
[0010] Step 6: Select the neighboring regions from the original panchromatic image. Neighborhood corresponding to the location , and build The corresponding local image matrix ,Will and The first principal component of is used for histogram matching;
[0011] Step 7: Use the matched full-color image to replace the first principal component and obtain a new principal component matrix ,right Perform inverse transformation to generate fused neighborhood The corresponding local image matrix , and according to Build , The value of each pixel in is the final pixel value of the fused image.
[0012] Furthermore, the weight matrix The calculation formula is as follows;
[0013]
[0014] in, yes Middle Line The elements of the column represent The weight corresponding to the pixel point, Represents pixel The spatial distance between is the bandwidth value.
[0015] Furthermore, the leave-one-out method is used to select the appropriate bandwidth value. , the specific implementation method is as follows;
[0016] Use the residual matrix between the original data and the principal component The minimum value of The goodness of fit evaluation index, namely the goodness of fit GOF, is used for pixels, Neighborhood of bands , and the corresponding GOF calculation formula is as follows:
[0017]
[0018]
[0019]
[0020] in, yes The corresponding size is The local image matrix of The value of a pixel; each column corresponds to A band of for rank, Is A matrix composed of eigenvectors of , where each column of the matrix corresponds to an eigenvector; express Before Column, corresponding to Before feature vectors; therefore, we use Represents the principal component scores, using Before principal components, using express Best Rank approximation; It is The goodness of fit of pixels, It's a pixel No. The residual corresponding to the band is The ) elements; is the overall goodness of fit of the entire local image, When the value of is the smallest, the degree of fit is considered to be optimal. This is the optimal bandwidth value.
[0021] Furthermore, the following geographically weighted mean is used: Decentralization:
[0022]
[0023]
[0024] in, yes The corresponding geographically weighted mean, Represents the local image matrix after decentralized processing.
[0025] Furthermore, the following geographically weighted covariance matrix is used: Perform PCA decomposition;
[0026]
[0027]
[0028] in, represents the local image matrix after decentralized processing, is the eigenvalue after PCA decomposition, represents the corresponding eigenvector.
[0029] Further, calculate each principal component The calculation formula is as follows:
[0030]
[0031] in, represents the feature vector, Represents the local image matrix after decentralized processing.
[0032] Furthermore, the fusion neighborhood The corresponding local image matrix The calculation formula is as follows:
[0033] = .
[0034] Furthermore, step 8 is included, in which the fusion effect is evaluated by comparing with the existing image fusion method through simulation experiments and real experiments.
[0035] Furthermore, existing image fusion methods include an optimized Brovey transform method with haze correction, a GS transform method based on context adaptation, a BDSD method with physical constraints, a GLP high-pass modulation injection method based on MTF matched filter, a GLP method based on MTF matched filter, a panchromatic sharpening method based on a total variation model, a CNN panchromatic sharpening method based on detail injection, a panchromatic sharpening method based on bidirectional multi-scale learning, and a CNN panchromatic sharpening method based on machine learning and traditional methods.
[0036] The present invention also provides a remote sensing image fusion system based on geographically weighted principal component analysis, including a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a remote sensing image fusion method based on geographically weighted principal component analysis as described in the above scheme.
[0037] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: the present invention overcomes the problems of spectral distortion and neglect of local spatial information in traditional component replacement methods by introducing geographically weighted principal component analysis technology, thereby improving the spatial resolution and spectral resolution of the fused image. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 The figure is a flow chart of an embodiment of the present invention.
[0039] Figure 2 This is a diagram of the fusion results of a simulation experiment using the WV-3 dataset.
[0040] Figure 3 This is a diagram of the fusion results of a real experiment using the WV-3 dataset. DETAILED DESCRIPTION
[0041] The technical solution of the present invention is further described in detail below through specific implementation methods.
[0042] Figure 1 This is the process of the remote sensing image fusion method based on geographically weighted principal component analysis provided by an embodiment of the present invention. The specific implementation process is as follows:
[0043] Step 1, obtaining an original multispectral image and an original panchromatic image of the same area, resampling the original multispectral image to obtain a target multispectral image, wherein the target multispectral image has the same spatial resolution as the original panchromatic image;
[0044] Step 2: Process the pixels of the target multispectral image point by point and select the neighborhood of each pixel. , generate neighborhood The corresponding local image matrix , and then calculate the corresponding weight matrix .
[0045] First determine the optimal bandwidth value Bandwidth value Decide and The size of , thus affecting the results of GWPCA and image fusion. The leave-one-out method is used to select. A special case of the fold cross validation method - The value of is set to be equal to the total number of samples , that is, in each round of verification, samples are used as training set, and one sample is left as test set. In the present invention, each data point is a sample. For GWPCA, the residual matrix between the original data and the principal component is usually used. The minimum value of the bandwidth The evaluation index of goodness of fit is called “Goodness of Fit” (GOF). pixels, Neighborhood of bands , and the corresponding GOF calculation formula is as follows:
[0046]
[0047]
[0048]
[0049] in, yes The corresponding size is The local image matrix of The value of a pixel; each column corresponds to a band. for rank. Is A matrix composed of eigenvectors of , where each column of the matrix corresponds to an eigenvector; express Before Column, corresponding to Before feature vectors; therefore, we can use Represents the principal component scores, using Before principal components, using express The best (least squares) rank of Approximate value (Best(least squares) rank approximation). It is The goodness of fit of pixels, It's a pixel No. The residual corresponding to the band is The ) elements. is the overall goodness of fit for the entire data set (local image). When the value of is the smallest, the degree of fit is considered to be optimal. This is the optimal bandwidth.
[0050] Determine the bandwidth value Afterwards, for any pixel point in the target multispectral image , select The corresponding neighborhood , and construct the generating neighborhood The corresponding local image matrix Each row vector of the local image matrix corresponds to the value of a pixel in the neighborhood; each column vector corresponds to a band in the neighborhood. Calculate the corresponding weight matrix , is a diagonal matrix. pixels (corresponding to The The calculation formula of the corresponding weight is as follows:
[0051]
[0052] in, yes Middle Line The elements of the column represent The weight corresponding to each pixel; Indicates and The spatial distance between pixels.
[0053] Step 3: Perform preprocessing operations, that is, use the geographically weighted mean instead of the conventional mean to decentralize the sample points in the neighborhood. The calculation formula is as follows:
[0054]
[0055]
[0056] in, yes The corresponding geographically weighted mean, Represents the local image matrix after decentralized processing.
[0057] Step 4: Apply the weight matrix to the decentralized local image matrix , we get the geographically weighted covariance matrix , and the geographically weighted covariance matrix Perform PCA decomposition and obtain are eigenvalues (sorted from large to small), and Then it represents the matrix of the corresponding eigenvectors, and each column corresponds to an eigenvector. The PCA decomposition formula is as follows:
[0058]
[0059]
[0060] Step 5: Combine the local image matrix after decentralized processing and the matrix of the eigenvector after PCA decomposition , calculate each principal component, the calculation formula is as follows:
[0061]
[0062] in, It is a matrix composed of the principal components, and each column of the matrix corresponds to a principal component.
[0063] Step 6: Select the neighboring regions from the original panchromatic image. Neighborhood corresponding to the location , and build The corresponding local image matrix ,Will and The first principal component of is used for histogram matching;
[0064] Step 7: Replace the first principal component with the matched full-color image to obtain a new principal component matrix .right Perform inverse transformation to generate fused neighborhood The corresponding local image matrix , Each row vector in The value of a pixel in . Then according to Constructing fusion neighborhood , The value of each pixel is the final pixel value of the fused image, and the calculation formula for the inverse transformation is as follows:
[0065]
[0066] To illustrate the effect of the present invention, the following are experimental results and analysis:
[0067] Using GF-2 and WV-3 satellite images as data sets, simulation experiments and real experiments were conducted. Simulation experiments are low-resolution evaluations, and real experiments are full-resolution verifications. Some representative traditional algorithms and CNN-based panchromatic sharpening methods were selected as comparison algorithms, including:
[0068] 1) Components Substitution (CS).
[0069] BT-H: Brovey Transform with HazeCorrection;
[0070] C-GSA: Context-Adaptive-Based Gram-Schmidt Algorithm;
[0071] BDSD-PC: Band Dependent Spatial Detail with Physical Constrains.
[0072] 2) Multiresolution Analysis (MRA).
[0073] MTF-GLP-HPM-R: Regression-Based GLP based on MTF matchedfilter with HPM.
[0074] MTF-GLP-FS: Full-Scale Regression-Based GLPbased on MTF matched filter;
[0075] 3) Variational Optimization (VO) method.
[0076] TV: Panchromatic sharpening method based on the total variation (TV) model.
[0077] 4) Algorithms based on Deep Learning (DL).
[0078] DiCNN: Detail-Injection-Based Convolutional Neural Networks;
[0079] BDPN: Panchromatic sharpening method based on bidirectional multi-scale learning (Bidirectional PyramidNetwork);
[0080] FusionNet: CNN pan-sharpening method based on machine learning and traditional methods (Deep Convolutional Neural Network for Fusion).
[0081] Taking the experimental results of the WV-3 dataset as an example, the simulation results are as follows: Figure 2 As shown in the figure, the actual experimental results are as follows Figure 3 As shown in the figure. Two details are selected at the bottom of each image for enlargement, and the names of the methods used are marked above the images. In the simulation experiment, GT represents the reference image, that is, the real multispectral image; EXP represents the multispectral image after upsampling. In the real experiment, PAN represents the panchromatic image; EXP represents the multispectral image after upsampling.
[0082] The results show that in the simulation experiment, some traditional methods have spectral distortion or blurred spatial details. The images processed by BDSD-PC and TV methods have spectral distortion; the images processed by MTF-GLP-HPM-R and MTF-GLP-FS have more spatial details lost in the green box. Among the deep learning methods, BDPN has a certain spectral distortion problem. The present invention and the other two deep learning methods are relatively balanced in terms of spatial details and spectral fidelity, and the effect is relatively good.
[0083] In real experiments, most of the problems that traditional methods have in low-resolution evaluation still exist in full-resolution verification. However, deep learning methods cannot transfer the knowledge learned from low-resolution images to real experiments very well: the processing results of DiCNN and BDPN are blurred in some areas, and the clear spatial details in the input PAN image are not retained, and BDPN has obvious spectral distortion problems. The visual results of deep learning methods are not good, and most of them have significant color distortion and poor spatial details; FusionNet has significant spectral distortion and generates details that do not exist in PAN images. In full-resolution images, supervised deep learning methods have no significant advantages over traditional methods. Compared with other methods, the panchromatic sharpened images generated by the present invention have higher spatial and color quality, and better fusion effects in real scenes.
[0084] In summary, compared with existing traditional algorithms and deep learning methods, the present invention has achieved better evaluations in multiple indicators, especially in full-resolution verification, where the algorithm shows significant advantages in maintaining spectral information and ground object details. The effectiveness of the present invention in improving the quality of remote sensing images is demonstrated through the comparison of visual effects.
[0085] On the other hand, an embodiment of the present invention also provides a remote sensing image fusion system based on geographically weighted principal component analysis, including a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute a remote sensing image fusion method based on geographically weighted principal component analysis as described in the above scheme.
[0086] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.
Claims
1. A remote sensing image fusion method based on geographically weighted principal component analysis, characterized in that: The following steps are involved: Step 1, obtaining an original multispectral image and an original panchromatic image of the same area, and resampling the original multispectral image to obtain a target multispectral image; Step 2: Process the pixels of the target multispectral image point by point. , select The corresponding neighborhood , generate neighborhood The corresponding local image matrix , and then calculate the corresponding weight matrix ; Step 3: Use the geographically weighted mean instead of the conventional mean to decentralize the sample points in the neighborhood, and obtain the decentralized image neighborhood and the corresponding local image matrix; Step 4: Apply the weight matrix to the decentralized local image matrix , we get the geographically weighted covariance matrix , and the geographically weighted covariance matrix Perform PCA decomposition to obtain the eigenvector ; Step 5: Combine the local image matrix after decentralized processing and calculate the eigenvector after PCA decomposition , calculate each principal component ; Step 6: Select the neighboring regions from the original panchromatic image. Neighborhood corresponding to the location , and build The corresponding local image matrix ,Will and The first principal component of is used for histogram matching; Step 7: Use the matched full-color image to replace the first principal component and obtain a new principal component matrix ,right Perform inverse transformation to generate fused neighborhood The corresponding local image matrix , and according to Constructing fusion neighborhood , The value of each pixel in is the final pixel value of the fused image.
2. The remote sensing image fusion method based on geographically weighted principal component analysis according to claim 1, characterized in that: Weight Matrix The calculation formula is as follows; in, yes Middle Line The elements of the column represent The weight corresponding to the pixel point, Pixel The spatial distance between is the bandwidth value.
3. The remote sensing image fusion method based on geographically weighted principal component analysis according to claim 2, characterized in that: Use the leave-one-out method to determine the optimal bandwidth value , the specific implementation method is as follows; Use the residual matrix between the original data and the principal component The minimum value of The goodness of fit evaluation index, namely the goodness of fit GOF, is used for pixels, Neighborhood of bands , and the corresponding GOF calculation formula is as follows: in, yes The corresponding size is The local image matrix of The value of a pixel; each column corresponds to A band of for rank, Is A matrix composed of eigenvectors of , where each column of the matrix corresponds to an eigenvector; express Before Column, corresponding to Before feature vectors; therefore, we use Represents the principal component scores, using Before principal component components, using express Best Rank approximation; It is The goodness of fit of pixels, It's a pixel No. The residual corresponding to the band is The ) elements; is the overall goodness of fit of the entire local image, When the value of is the smallest, the degree of fit is considered to be optimal. This is the optimal bandwidth value.
4. The remote sensing image fusion method based on geographically weighted principal component analysis according to claim 1, characterized in that: Use the following geographically weighted mean Decentralization: in, yes The corresponding geographically weighted mean, Represents the local image matrix after decentralized processing.
5. The remote sensing image fusion method based on geographically weighted principal component analysis according to claim 1, characterized in that: Use the following geographically weighted covariance matrix Perform PCA decomposition; in, represents the local image matrix after decentralized processing, is the eigenvalue after PCA decomposition, represents the corresponding eigenvector.
6. The remote sensing image fusion method based on geographically weighted principal component analysis according to claim 1, characterized in that: Calculate each principal component The calculation formula is as follows: in, represents the feature vector, Represents the local image matrix after decentralized processing.
7. The remote sensing image fusion method based on geographically weighted principal component analysis according to claim 1, characterized in that: Dissolve Neighborhood The corresponding local image matrix The calculation formula is as follows: = 。 8. The remote sensing image fusion method based on geographically weighted principal component analysis according to claim 1, characterized in that: The method also includes step 8, which evaluates the fusion effect by comparing with the existing image fusion method through simulation experiments and real experiments.
9. The remote sensing image fusion method based on geographically weighted principal component analysis according to claim 8, characterized in that: Existing image fusion methods include the optimized Brovey transform method with haze correction, the GS transform method based on context adaptation, the BDSD method with physical constraints, the GLP high-pass modulation injection method based on MTF matched filter, the GLP method based on MTF matched filter, the panchromatic sharpening method based on total variation model, the CNN panchromatic sharpening method based on detail injection, the panchromatic sharpening method based on bidirectional multi-scale learning, and the CNN panchromatic sharpening method based on machine learning and traditional methods.
10. A remote sensing image fusion system based on geographically weighted principal component analysis, characterized in that: It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a remote sensing image fusion method based on geographically weighted principal component analysis as described in any one of claims 1 to 9.
