A remote sensing image space-spectrum fusion method and system based on quadratic programming weight optimization

Abstract

The application discloses a remote sensing image space-spectrum fusion method and system based on quadratic programming weight optimization, and the method comprises the following steps: S1, inputting low-resolution multispectral images and high-resolution panchromatic images; S2, performing up-sampling on the multispectral images so as to make the size of the multispectral images consistent with that of the panchromatic images, and obtaining the up-sampled multispectral images; S3, solving a global optimal weight vector according to the multispectral images and the panchromatic images; S4, simulating an optimal simulated panchromatic image by linearly weighting the multispectral bands according to the global optimal weight vector; and S5, performing component replacement processing on the multispectral images according to the simulated panchromatic image, and obtaining a high-resolution space-spectrum fusion image. By using the quadratic programming algorithm to perform weight optimization when calculating the simulated panchromatic band weight, the inherent defects of the least square method are fundamentally overcome, and the quality of remote sensing image fusion can be effectively improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method and system for spatial-spectral fusion of remote sensing images based on quadratic programming weight optimization. Background Technology

[0002] Spatial-spectral fusion technology aims to fuse low-resolution multispectral (MS) images with high-resolution panchromatic (PAN) images to obtain a new image with both high spatial and spectral resolution. In component substitution (CS)-based fusion methods (such as GS and PCA), a key step is to linearly simulate a high-resolution panchromatic band (intensity component) using the multispectral bands. The accuracy of this simulation directly determines the quality of the final fused image.

[0003] Multispectral low-resolution data is available. , ... Full-color high-resolution data .

[0004] In mainstream space-spectral fusion methods, such as the Gram-Schmidt fusion method (the mainstream method in ENVI software) and the UNB PanSharp method (used by PCI software), one step in improving the spatial resolution of low-resolution multispectral data using high-resolution panchromatic band data is to utilize multispectral band data... Generate a simulated panchromatic band data (Intensity component) should be made to make I as close as possible to the original panchromatic high-resolution data. .

[0005] The high-resolution panchromatic image is simulated using multispectral low spatial resolution imagery, and the simulated high-resolution panchromatic imagery is as follows:

[0006]

[0007] in,

[0008]

[0009]

[0010] The problem is transformed into finding the optimal weight parameters. , making As close as possible to the original panchromatic high-resolution data .

[0011] Traditional methods for simulating panchromatic bands typically employ the least squares (LS) method to determine the optimal weighting coefficients for each band, i.e., minimizing the mean square error between the simulated values ​​and the actual panchromatic bands. However, the least squares method has significant drawbacks:

[0012] 1. The weights cannot be guaranteed to be non-negative naturally: the least squares solution may produce negative weights, which does not conform to the physical meaning (the band contribution should not be negative) and may lead to spectral distortion and numerical instability in the fusion result.

[0013] 2. The weight sum cannot be guaranteed to be 1 naturally: If the weight sum is not 1, it will introduce brightness deviation, causing the simulated intensity component to be inconsistent with the brightness reference of the original multispectral image. This requires an additional normalization step, which increases the complexity and error of the algorithm. Summary of the Invention

[0014] In view of the above problems, the purpose of this invention is to provide a method and system for spatial-spectral fusion of remote sensing images based on quadratic programming for weight optimization. By employing a quadratic programming (QP) algorithm for weight optimization when calculating the weights of the simulated panchromatic (PAN) bands, the quality of remote sensing image fusion can be effectively improved.

[0015] The first aspect of this invention provides a spatial-spectral fusion method for remote sensing images based on quadratic programming weight optimization, the method comprising:

[0016] S1: Input registered low-resolution multispectral image and high-resolution panchromatic images ;

[0017] S2: For multispectral images Upsample it to the size of the panchromatic image. Consistent, resulting in upsampled multispectral images. ;

[0018] S3: Based on multispectral imagery and panchromatic images Solving for the global optimal weight vector ;

[0019] S4: Based on the globally optimal weight vector The optimal simulated panchromatic image was obtained by linear weighted simulation of multispectral bands. ;

[0020] S5: Based on simulated panchromatic image For multispectral images Component replacement processing was performed to obtain a high-resolution spatial-spectral fusion image.

[0021] Preferably, S3 specifically includes:

[0022] Use the upsampled multispectral of Each band linear simulated panchromatic image Construct the globally optimal weight vector and solve the objective function;

[0023] The objective function obtained by solving for the global optimal weight vector is expanded into a quadratic form;

[0024] The objective function for finding the globally optimal weight vector of a quadratic form is input into a quadratic programming solver to obtain the globally optimal weight vector. .

[0025] Preferably, the objective function for solving the global optimal weight vector is:

[0026]

[0027] The objective of solving the global optimal weight vector objective function is to make the simulated panchromatic image... Values ​​and panchromatic images The mean squared error (MSE) between them is minimized:

[0028]

[0029] Constraints:

[0030]

[0031]

[0032] in, To simulate panchromatic images, , where is the weight vector to be solved.

[0033] Preferably, the objective function for solving the global optimal weight vector is expanded into a quadratic form, specifically as follows:

[0034]

[0035] achievable

[0036]

[0037] in, Q is the total number of pixels in the panchromatic image. A symmetric positive definite matrix of order 1 is formed by the multispectral image matrix. The constructed quadratic core matrix determines the weight vector. The quadratic term distribution; For the weight vector Irrelevant constant terms; for The order column vector is the coefficient vector of the first-order term after expanding the objective function to solve for the global optimal weight vector.

[0038] Preferably, the optimal simulated panchromatic image The calculation formula is:

[0039]

[0040] in, Represents the optimal weight vector The corresponding number in the middle Weighting coefficients for each multispectral band.

[0041] Preferably, S5 includes:

[0042] Low-resolution multispectral images Projected onto the optimal weight vector Calculate the low-resolution intensity components of the defined vector. ;

[0043] For low-resolution intensity components Upsample it to the size of the panchromatic image. Consistent, resulting in the upsampled low-resolution intensity component. ;

[0044] Based on low-resolution intensity components Calculate the incremental detail information of the panchromatic image;

[0045] Incremental detail information from the panchromatic image is injected into the upsampled image. Image fusion is performed on the images, and the image fusion result is output, namely a high-resolution spatial-spectral fused image.

[0046] Preferably, the formula for calculating the detail information increment of the panchromatic image is:

[0047]

[0048] in, Low-resolution intensity components The size of the upsampled panchromatic image P.

[0049] Preferably, the step of incrementally injecting detailed information from the panchromatic image into the upsampled image... In image fusion within a video image, the expression for image fusion is:

[0050]

[0051] in, For high-resolution space-spectral fusion images, Indicates the first The injection gain factor for the band (can be 1 or calculated using other methods).

[0052] Preferably, the quadratic programming solver is Python CVXOPT or qpOASES.

[0053] A second aspect of the present invention provides a remote sensing image spatial-spectral fusion system based on quadratic programming weight optimization, comprising a memory and a processor. The memory includes a remote sensing image spatial-spectral fusion method program based on quadratic programming weight optimization. When the remote sensing image spatial-spectral fusion method program based on quadratic programming weight optimization is executed by the processor, it implements the steps of a remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization.

[0054] Compared with existing technologies, the beneficial effects of the technical solution of this invention are as follows: The purpose of this invention is to provide a method and system for spatial-spectral fusion of remote sensing images based on quadratic programming weight optimization. Compared with traditional remote sensing image spatial-spectral fusion methods, this invention has the following advantages:

[0055] (1) Strictly satisfy physical constraints: The weight coefficients obtained by the method described in this invention naturally satisfy the physical constraints that are non-negative and sum to 1, eliminating the spectral distortion caused by negative weights and ensuring the spectral fidelity of the fusion result.

[0056] (2) Improve fusion accuracy: Seeking the global optimal solution under strict physical constraints has higher mathematical rigor and accuracy than the least squares method with no constraints or post-processing constraints. The simulated intensity component is closer to the real panchromatic band, thereby improving the quality of the final fused image.

[0057] (3) Strong numerical stability: The present invention adopts a quadratic programming algorithm, which has good numerical stability and reliability, and avoids ill-conditioned or unstable solutions that may occur in the least squares method.

[0058] (4) Compatibility and universality: This invention can be seamlessly embedded into various space-spectrum fusion frameworks based on component substitution (such as GS, PCA, BT), improving the performance of these classic algorithms and having good compatibility and universality. Attached Figure Description

[0059] Figure 1 This is a flowchart of a remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization as described in Example 1. Detailed Implementation

[0060] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0061] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0062] Example 1

[0063] like Figure 1 As shown in the figure, this embodiment discloses a remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization, the method comprising:

[0064] S1: Input registered low-resolution multispectral image and high-resolution panchromatic images .

[0065] S2: For multispectral images Upsample it to the size of the panchromatic image. Consistent, resulting in upsampled multispectral images. .

[0066] It should be noted that this embodiment uses registered low-resolution multispectral images as input. , in, For the number of pixels, This refers to the number of bands.

[0067] It should be noted that this embodiment applies to multispectral images. Upsampling can be performed using bicubic interpolation. The upsampled multispectral image... , in For the number of pixels, This refers to the number of bands.

[0068] S3: Based on multispectral imagery and panchromatic images Solving for the global optimal weight vector .

[0069] S3.1: Use the upsampled multispectral data of Each band linear simulated panchromatic image We construct the globally optimal weight vector and solve the objective function.

[0070] In this embodiment, the objective function for solving the global optimal weight vector is:

[0071]

[0072] The objective of solving the global optimal weight vector objective function is to make the simulated panchromatic image... Values ​​and panchromatic images The mean squared error (MSE) between them is minimized:

[0073]

[0074] Constraints:

[0075]

[0076]

[0077] in, To simulate panchromatic images, , where is the weight vector to be solved.

[0078] S3.2: Expand the objective function obtained by solving the global optimal weight vector into a quadratic form.

[0079] In this embodiment, the step of expanding the objective function for solving the global optimal weight vector into a quadratic form is specifically as follows:

[0080]

[0081] achievable

[0082]

[0083] in, Q is the total number of pixels in the panchromatic image. A symmetric positive definite matrix of order 1 is formed by the multispectral image matrix. The constructed quadratic core matrix determines the weight vector. The quadratic term distribution; For the weight vector Irrelevant constant terms; for The order column vector is the coefficient vector of the first-order term after expanding the objective function to solve for the global optimal weight vector.

[0084] S3.3: Input the objective function for solving the quadratic form's globally optimal weight vector into the quadratic programming solver to obtain the globally optimal weight vector. .

[0085] It should be noted that in this embodiment, the objective function is solved by finding the global optimal weight vector of the quadratic form, that is, the quadratic programming problem of the global optimal weight vector is input into the quadratic programming solver for solving.

[0086] The quadratic programming solver can be, but is not limited to, Python libraries such as CVXOPT or qpOASES. Examples of quadratic programming solvers include: https: / / github.com / erikbrinkman / quadprog-rs; https: / / github.com / osqp / osqp.rs; https: / / github.com / oxfordcontrol / Clarabel.rs, etc.

[0087] S4: Based on the globally optimal weight vector The optimal simulated panchromatic image was obtained by linear weighted simulation of multispectral bands. .

[0088] In this embodiment, the optimal simulated panchromatic image The calculation formula is:

[0089]

[0090] in, Represents the optimal weight vector The corresponding number in the middle Weighting coefficients for each multispectral band.

[0091] S5: Based on simulated panchromatic image For multispectral images Component replacement processing was performed to obtain a high-resolution spatial-spectral fusion image.

[0092] In this embodiment, S5 includes:

[0093] S5.1: Convert low-resolution multispectral images Projected onto the optimal weight vector Calculate the low-resolution intensity components of the defined vector. ;

[0094] S5.2: For low-resolution intensity components Upsample it to the size of the panchromatic image. Consistent, resulting in the upsampled low-resolution intensity component. .

[0095] S5.3: Based on low-resolution intensity components Calculate the incremental detail information of the panchromatic image.

[0096] In this embodiment, the formula for calculating the detail information increment of the panchromatic image is:

[0097]

[0098] in, For low-resolution intensity components The size of the upsampled panchromatic image P.

[0099] It should be noted that in this embodiment, the low-resolution multispectral image is first processed. Projected onto the optimal weight vector Calculate the low-resolution intensity components on the defined vector. Low-resolution intensity components Single-band, representing multispectral imagery The overall luminance intensity; then the low-resolution intensity component Making with multispectral images The same upsampling operation (such as bicubic interpolation) makes its size / resolution similar to high-resolution panchromatic image P and upsampled multispectral image P. A perfect match is obtained. .

[0100] S5.4: Increment the detail information of the panchromatic image into the upsampled image. Image fusion is performed on the images, and the image fusion result is output, namely a high-resolution spatial-spectral fused image.

[0101] In this embodiment, the detailed information of the panchromatic image is incrementally injected into the upsampled image. In image fusion within a video image, the expression for image fusion is:

[0102]

[0103] in, For high-resolution space-spectral fusion images, Indicates the first The injection gain coefficient for the band can be 1 or calculated using other methods.

[0104] In this embodiment, during the spatial-spectral fusion process of remote sensing images, a quadratic programming (QP) optimization algorithm is used instead of the traditional least squares (LS) algorithm to solve for the optimal weight coefficients of each band in the simulated panchromatic band. This transforms the weight optimization problem into a standard quadratic programming problem. Under strict constraints (non-negative weights and a weight sum of 1), this method directly obtains the global optimal solution, fundamentally overcoming the inherent defects of the least squares method.

[0105] Example 2

[0106] This embodiment discloses a remote sensing image spatial-spectral fusion system based on quadratic programming weight optimization, including a memory and a processor. The memory includes a remote sensing image spatial-spectral fusion method program based on quadratic programming weight optimization. When the remote sensing image spatial-spectral fusion method program based on quadratic programming weight optimization is executed by the processor, it implements the steps of a remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization as described in Embodiment 1.

[0107] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0108] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0109] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

Claims

1. A spatial-spectral fusion method for remote sensing images based on quadratic programming weight optimization, characterized in that, The method includes: S1: Input registered low-resolution multispectral image and high-resolution panchromatic images ; S2: For multispectral images Upsample it to the size of the panchromatic image. Consistent, resulting in upsampled multispectral images. ; S3: Based on multispectral imagery and panchromatic images Solving for the global optimal weight vector ; S4: Based on the globally optimal weight vector The optimal simulated panchromatic image was obtained by linear weighted simulation of multispectral bands. ; S5: Based on simulated panchromatic image For multispectral images Component replacement processing was performed to obtain a high-resolution spatial-spectral fusion image.

2. The remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization according to claim 1, characterized in that, S3 specifically includes: Use the upsampled multispectral of Each band linear simulated panchromatic image Construct the globally optimal weight vector and solve the objective function; The objective function obtained by solving for the global optimal weight vector is expanded into a quadratic form; The objective function for finding the globally optimal weight vector of a quadratic form is input into a quadratic programming solver to obtain the globally optimal weight vector. .

3. The remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization according to claim 2, characterized in that, The objective function is solved by the globally optimal weight vector: The objective of solving the global optimal weight vector objective function is to make the simulated panchromatic image... Values ​​and panchromatic images The mean square error between them is the smallest: Constraints: in, To simulate panchromatic images, , where is the weight vector to be solved.

4. The remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization according to claim 3, characterized in that, The objective function for solving the global optimal weight vector is expanded into a quadratic form, specifically as follows: achievable in, Q is the total number of pixels in the panchromatic image. A symmetric positive definite matrix of order 1 is formed by the multispectral image matrix. The constructed quadratic core matrix determines the weight vector. The quadratic term distribution; For the weight vector Irrelevant constant terms; for The order column vector is the coefficient vector of the first-order term after expanding the objective function to solve for the global optimal weight vector.

5. The remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization according to claim 4, characterized in that, The optimal simulated panchromatic image The calculation formula is: in, Represents the optimal weight vector The corresponding number in the middle Weighting coefficients for each multispectral band.

6. A method for spatial-spectral fusion of remote sensing images based on quadratic programming weight optimization as described in claim 1 or 5, characterized in that, S5 include: Low-resolution multispectral images Projected onto the optimal weight vector Calculate the low-resolution intensity components of the defined vector. ; For low-resolution intensity components Upsample it to the size of the panchromatic image. Consistent, resulting in the upsampled low-resolution intensity component. ; Based on low-resolution intensity components Calculate the incremental detail information of the panchromatic image; Incremental detail information from the panchromatic image is injected into the upsampled image. Image fusion is performed on the images, and the image fusion result is output, namely a high-resolution spatial-spectral fused image.

7. The remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization according to claim 6, characterized in that, The formula for calculating the detail information increment of the panchromatic image is: in, For low-resolution intensity components The size of the upsampled panchromatic image P.

8. The remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization according to claim 7, characterized in that, The step involves injecting incremental detail information from the panchromatic image into the upsampled image. In image fusion within a video image, the expression for image fusion is: in, For high-resolution space-spectral fusion images, Indicates the first Injection gain coefficient for the band.

9. A remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization as described in claim 2 or 8, characterized in that, The quadratic programming solver is either Python CVXOPT or qpOASES.

10. A remote sensing image spatial-spectral fusion system based on quadratic programming weight optimization, characterized in that, The system includes a memory and a processor. The memory includes a program for a remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization. When the program for the remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization is executed by the processor, it implements the steps of a remote sensing image spatial-spectral fusion method based on quadratic programming weight optimization as described in any one of claims 1 to 6.

Images