A pan-sharpening method based on cross-scale variational constraints

Abstract

The application discloses a panchromatic sharpening method based on cross-scale variational constraint, comprising the following steps: S1, inputting a multispectral image MS and a panchromatic image PAN of the same area, and performing geometric registration processing on the two; S2, performing step-by-step downscaling on the panchromatic image processed in S1 by using a downscaling operator to obtain a panchromatic image sequence; specifically as follows: S3, generating a reference space component according to each band of the multispectral image of the current scale; S4, establishing a space component optimization model of the current scale, and solving to obtain an optimized space component; S5, calculating a regularization space residual of the current scale, and generating a fusion result; S6, if the current scale is smaller than a preset threshold, replacing the current scale with a next scale, and repeatedly performing S3-S5 until the current scale is larger than the preset threshold, and obtaining a fine-scale fusion result as a final multispectral fusion image. The method improves the spectral stability, spatial detail definition and cross-scale structure consistency of the fusion image.

Description

Technical Field

[0001] This invention relates to the fields of remote sensing image processing and computer vision, and in particular to a panchromatic sharpening method based on cross-scale variational constraints. Background Technology

[0002] Multispectral images (MS) contain reflectance information of ground features across multiple spectral bands, providing a good characterization of their spectral features. However, their spatial resolution is typically low, making it difficult to meet the needs of applications such as fine-grained feature identification, urban target extraction, road structure analysis, and high-resolution mapping. Panchromatic images (PAN) offer high spatial resolution and contain rich edge, texture, and geometric information, but their spectral information is limited. Panchromatic sharpening refers to using low-spatial-resolution multispectral images and high-spatial-resolution panchromatic images of the same area to generate a multispectral fusion image that combines high spatial resolution with good spectral preservation.

[0003] Existing panchromatic sharpening methods mainly include two commonly used categories: multi-resolution analysis methods and variational optimization methods. Among them:

[0004] Multi-resolution analysis methods typically involve first scaling the panchromatic image and then injecting the extracted spatial details into the multispectral image. These methods generally have good spectral preservation capabilities, but suffer from severe spatial distortion. Induction scaling techniques (Condat L, Montanvert A. A framework for image magnification: Inductionrevisited. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2005: 845–848) can maintain structural consistency across scales during image downscaling and reconstruction, which is beneficial for edge alignment and spatial information recovery between different scales. The Indusion method (Khan MM, Chanussot J, Condat L, Montanvert A. Indusion: Fusion of multispectral and panchromatic images using the induction scaling technique. IEEE Geoscience and Remote Sensing Letters, 2008, 5(1): 98–102) introduces the induction scaling technique into panchromatic sharpening. It constructs multispectral and panchromatic images at different scales using a set of orthogonal filters. Histogram matching within the scale is used to reduce the grayscale difference between MS and PAN. This approach may severely weaken the edge consistency between high and low scales of the original PAN image. The Inducion-TV method uses L1 norm total variation to constrain the spatial components before and after fusion at each scale. The fused image shows a significant improvement in spatial detail. However, this method does not impose strong constraints on edge alignment between scales. Therefore, it may still produce local artifacts and edge instability in areas with dense textures such as complex ground boundaries, buildings, and roads.

[0005] Variational optimization methods can jointly constrain the spectral and spatial structural information of fused images through objective functions, offering advantages in suppressing spectral distortion and controlling spatial detail transfer. However, in practice, these methods often optimize at a single scale, lacking modeling of the cross-scale propagation relationships of spatial components, making it difficult to simultaneously guarantee consistency between coarse-scale structure, meso-scale boundaries, and fine-scale textures. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a panchromatic sharpening method based on cross-scale variational constraints. This method can reduce local artifacts, stabilize edge structures, and coordinate the preservation of spectral and spatial information by introducing cross-scale constraints during the spatial detail transfer process.

[0007] Therefore, the present invention adopts the following technical solution:

[0008] A panchromatic sharpening method based on cross-scale variational constraints includes the following steps:

[0009] S1. Acquire the multispectral image MS and panchromatic image PAN of the same area, and perform geometric registration on the two to make them have consistent geographical coverage.

[0010] S2, using the downscaling operator to progressively downscale the panchromatic image after geometric registration in S1, to obtain a panchromatic image sequence containing panchromatic images at different induction scales;

[0011] S3, based on the multispectral image at the current scale Weighted generation of reference space components for each band ,in, As a scale, Initially, ;

[0012] S4, based on the reference space components , in terms of spatial components To optimize the target, a spatial component optimization model for the current scale is established, and an iterative optimization method is used to solve for the optimized spatial components at the current scale. ;

[0013] S5, based on the reference space components and optimize spatial components Calculate the regularized spatial residual at the current scale, and then generate the fusion result at the current scale. ;

[0014] S6, if the current scale ,by replace Repeat steps S3 through S5 until... The fine-scale fusion results obtained ,Will As the final multispectral fused image, full-color sharpening is completed.

[0015] In step S2 above:

[0016] Suppose that the panchromatic image after geometric registration of S1 is a fine-scale panchromatic image. By using the downscaling operator Obtain a mesoscale panchromatic image and coarse-scale panchromatic image ;

[0017] The panchromatic image sequence includes fine-scale panchromatic images. Mesoscale panchromatic image and coarse-scale panchromatic image .

[0018] The downscaling operator The functionality is achieved through the CDF9 filter and downsampling operation.

[0019] The reference space component mentioned in step S3 above The calculation formula is:

[0020] ,

[0021] in, For the number of multispectral bands, For current scale multispectral images The Multiple spectral bands, , For the first Weights of each multispectral band.

[0022] The weights for each band are taken to be the same. .

[0023] In step S3 above:

[0024] when At that time, coarse-scale multispectral images This is the multispectral image after S1 geometric registration;

[0025] when At that time, mesoscale multispectral images The coarse-scale fusion result obtained in the previous round The upsampling results ;

[0026] when At that time, fine-scale multispectral images The results of the previous round of mesoscale fusion The upsampling results ;

[0027] in, This represents the reconstruction operator, whose function is implemented through the CDF7 reconstruction filter and upsampling operation.

[0028] In step S4 above:

[0029] The formula for the spatial component optimization model is as follows:

[0030] when hour:

[0031] ,

[0032] when or hour:

[0033] ,

[0034] in:

[0035] This is the current-scale spectral fidelity term, used to maintain the consistent gray-level distribution of spatial components before and after fusion at the current scale, i.e., ensuring a consistent gray-level distribution in the multispectral image. Choose 1 or 2;

[0036] This is a spatial fidelity term for the current scale, used to incorporate edge and texture information from the current scale panchromatic image. Represents the gradient operator; These are the weight parameters corresponding to the current scale space fidelity term;

[0037] This is a cross-scale spectral fidelity term used to constrain the scale relationship between the current scale spatial component and the optimized spatial component at a lower scale. The weighting parameters corresponding to the cross-scale spectral fidelity term. Choose 1 or 2.

[0038] In step S4 above: the iterative optimization method adopts the iterative reweighting method, the alternating direction multiplier method, the primal-dual method, or any other numerical solution method applicable to L1 norm and total variation optimization problems.

[0039] Step S5 above includes:

[0040] Calculate the regularized spatial residual at the current scale :

[0041] ,

[0042] The Inject current-scale multispectral images Generate the current scale fusion result from each band. Among them, the current scale is the first The fusion results of each band The calculation formula is:

[0043] ,

[0044] in, To inject gain.

[0045] The injection gain .

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. The method of the present invention constructs a spatial component constraint model that coordinates cross-scale spectral and spatial structure, and explicitly constrains gray level and gradient between scales, so that the spatial structure can evolve continuously from coarse to fine, thereby enhancing the structural consistency between different scales; and by performing functional optimization on the spatial component variable, it can better balance spectral information and spatial information.

[0048] 2. The method of the present invention utilizes panchromatic image structure guidance and cross-scale spatial component constraints to control spatial detail transfer, avoiding problems such as ringing and artifacts caused by direct high-frequency injection, and reducing local artifacts and edge instability.

[0049] 3. This invention injects regularized spatial residuals into multispectral images, which improves spatial resolution while preserving the spectral information of the multispectral images, thus achieving both spatial enhancement and spectral preservation. Attached Figure Description

[0050] Figure 1 This is a flowchart of the pancolor sharpening method based on cross-scale variational constraints of the present invention;

[0051] Figure 2 This is a comparison image of full-color sharpened images generated by the method of this invention and existing technologies. Detailed Implementation

[0052] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0053] like Figure 1 As shown, this invention provides a panchromatic sharpening method based on cross-scale variational constraints, such as... Figure 1 As shown, it includes the following steps:

[0054] S1, acquire the image and perform geometric registration:

[0055] Acquire low spatial resolution multispectral images of the same region (MS) The image is a single-band image and a high spatial resolution panchromatic image (PAN). The two are geometrically registered to ensure consistent geographic coverage.

[0056] The multispectral image (MS) can provide relatively rich spectral information, but has low spatial resolution; the panchromatic image (PAN) has high spatial resolution and can provide clear edge, texture and geometric structure information.

[0057] S2, Construct a multi-scale panchromatic image sequence:

[0058] The panchromatic image after S1 geometric registration is downscaled stepwise using a downscaling operator to obtain a panchromatic image sequence containing panchromatic images at different induction scales, as follows:

[0059] Suppose that the panchromatic image after geometric registration of S1 is a fine-scale panchromatic image. (Fine scale) is achieved through the downscaling operator. Obtain a mesoscale panchromatic image and coarse-scale panchromatic image .

[0060] In one embodiment of the present invention, the downscaling operator The functionality is achieved through the CDF9 filter and downsampling operation.

[0061] The panchromatic image sequence includes fine-scale panchromatic images. Mesoscale panchromatic image and coarse-scale panchromatic image In subsequent fusion operations, panchromatic images at different scales serve as spatial structure guide images at the corresponding scales, providing edge, texture, and geometric structure guidance information.

[0062] S3, Construct the reference space components for the current scale:

[0063] At the current scale superior, (Initially, Based on the current scale of multispectral images Weighted generation of reference space components for each band The formula is:

[0064] ,

[0065] in, For the number of multispectral bands, For current scale multispectral images The Multiple spectral bands, , For the first The weights of each multispectral band are determined. In one embodiment of the invention, the weights of each band are the same: .

[0066] in:

[0067] when At that time, coarse-scale multispectral images The multispectral image obtained after geometric registration is S1;

[0068] when At that time, mesoscale multispectral images The coarse-scale fusion result obtained in the previous round The upsampling results ;

[0069] when At that time, fine-scale multispectral images The results of the previous round of mesoscale fusion The upsampling results .

[0070] in, This represents the reconstruction operator, whose function is implemented through the CDF7 reconstruction filter and upsampling operation.

[0071] S4. Establish the spatial component optimization model for the current scale and solve for the optimized spatial components at the current scale:

[0072] At the current scale Above, in terms of spatial components To optimize the target, a spatial component constraint model is constructed. An iterative optimization method is then used to solve this model, yielding the optimized spatial components at the current scale. The formula for the spatial component optimization model is as follows:

[0073] when hour:

[0074] ,

[0075] when or hour:

[0076] ,

[0077] in:

[0078] This is the current-scale spectral fidelity term, used to maintain the consistent gray-level distribution of spatial components before and after fusion at the current scale, i.e., ensuring a consistent gray-level distribution in the multispectral image. Either 1 or 2 can be chosen;

[0079] This is a spatial fidelity term for the current scale, used to incorporate edge and texture information from the current scale panchromatic image. Represents the gradient operator; These are the weight parameters corresponding to the current scale space fidelity term;

[0080] This is a cross-scale spectral fidelity term used to constrain the scale relationship between the current scale spatial component and the optimized spatial component at a lower scale. The weighting parameters corresponding to the cross-scale spectral fidelity term. Either 1 or 2 can be chosen.

[0081] The iterative optimization method can be any numerical solution method applicable to L1 norm and total variation optimization problems, such as iterative reweighting, alternating direction multiplier method, primal-dual method, or other methods.

[0082] S5, calculate the regularized spatial residual at the current scale, and then generate the fusion result at the current scale:

[0083] Regularized spatial residuals at the current scale The calculation formula is:

[0084] ,

[0085] The regularized spatial residual is not the original high-frequency component extracted directly from the panchromatic image, but a spatial component update obtained under the combined effect of multispectral fidelity, panchromatic image structure guidance, and cross-scale spatial component consistency constraints, which can reduce the risk of local artifacts and edge instability.

[0086] Inject the regularized spatial residual at the current scale into the current-scale multispectral image. Generate the current scale fusion result from each band. Among them, the current scale is the first The fusion results of each band The calculation formula is:

[0087] ,

[0088] in, To inject gain.

[0089] In one embodiment of the present invention, .

[0090] Figure 1 In this context, F represents the entire computational process of generating the fusion result using the optimized spatial components at the current scale.

[0091] S6, scale determination and output of multispectral fused image:

[0092] If the current scale ,by replace Repeat steps S3 through S5 until... The obtained fusion result is a fine-scale fusion result. This results in the final multispectral fused image, which completes panchromatic sharpening. The multispectral fused image has the same resolution as the panchromatic image, but contains spatial details injected at each scale, resulting in a more stable spatial structure and texture.

[0093] Figure 2 The images from left to right show the panchromatic sharpening results obtained by the Indusion method, the Induction-TV method, and the method of this invention. It can be seen that the fusion results generated by the Indusion method and the Induction-TV method both exhibit a certain degree of detail blurring, with relatively low discernibility of building outlines, roof boundaries, road markings, and vehicle targets. In contrast, the method of this invention, while maintaining the overall hue and spectral characteristics of the generated multispectral fusion result relatively stable, can more fully recover spatial details, making building edges more continuous and clear, parking lot vehicle and road textures easier to identify, and reducing spatial artifacts, demonstrating better spatial enhancement and structural preservation capabilities.

Claims

1. A panchromatic sharpening method based on cross-scale variational constraints, characterized in that, Includes the following steps: S1. Acquire the multispectral image MS and panchromatic image PAN of the same area, and perform geometric registration on the two to make them have consistent geographical coverage. S2, using the downscaling operator to progressively downscale the panchromatic image after geometric registration in S1, to obtain a panchromatic image sequence containing panchromatic images at different induction scales; S3, based on the multispectral image at the current scale Weighted generation of reference space components for each band ,in, As a scale, Initially, ; S4, based on the reference space components , in terms of spatial components To optimize the target, a spatial component optimization model for the current scale is established, and an iterative optimization method is used to solve for the optimized spatial components at the current scale. ; S5, based on the reference space components and optimize spatial components Calculate the regularized spatial residual at the current scale, and then generate the fusion result at the current scale. ; S6, if the current scale ,by replace Repeat steps S3 through S5 until... The fine-scale fusion results obtained ,Will As the final multispectral fused image, full-color sharpening is completed.

2. The full-color sharpening method according to claim 1, characterized in that, In S2: Suppose that the panchromatic image after geometric registration of S1 is a fine-scale panchromatic image. By using the downscaling operator Obtain a mesoscale panchromatic image and coarse-scale panchromatic image ; The panchromatic image sequence includes fine-scale panchromatic images. Mesoscale panchromatic image and coarse-scale panchromatic image .

3. The full-color sharpening method according to claim 2, characterized in that: The downscaling operator The functionality is achieved through the CDF9 filter and downsampling operation.

4. The full-color sharpening method according to claim 2, characterized in that, The reference space component described in S3 The calculation formula is: ， in, For the number of multispectral bands, For current scale multispectral images The Multiple spectral bands, , For the first Weights of each multispectral band.

5. The full-color sharpening method according to claim 4, characterized in that: The weights for each band are taken to be the same. .

6. The full-color sharpening method according to claim 4, characterized in that, In S3: when At that time, coarse-scale multispectral images This is the multispectral image after S1 geometric registration; when At that time, mesoscale multispectral images The coarse-scale fusion result obtained in the previous round The upsampling results ; when At that time, fine-scale multispectral images The results of the previous round of mesoscale fusion The upsampling results ; in, This represents the reconstruction operator, whose function is implemented through the CDF7 reconstruction filter and upsampling operation.

7. The full-color sharpening method according to claim 6, characterized in that, In S4: The formula for the spatial component optimization model is as follows: when hour: ， when or hour: ， in: This is the current-scale spectral fidelity term, used to maintain the consistent gray-level distribution of spatial components before and after fusion at the current scale, i.e., ensuring a consistent gray-level distribution in the multispectral image. Choose 1 or 2; This is a spatial fidelity term for the current scale, used to incorporate edge and texture information from the current scale panchromatic image. Represents the gradient operator; These are the weight parameters corresponding to the current scale space fidelity term; This is a cross-scale spectral fidelity term used to constrain the scale relationship between the current scale spatial component and the optimized spatial component at a lower scale. The weighting parameters corresponding to the cross-scale spectral fidelity term. Choose 1 or 2.

8. The full-color sharpening method according to claim 7, characterized in that, In S4: The iterative optimization method adopts the iterative reweighting method, the alternating direction multiplier method, the primal-dual method, or any other numerical solution method applicable to L1 norm and total variation optimization problems.

9. The full-color sharpening method according to claim 7, characterized in that, S5 include: Calculate the regularized spatial residual at the current scale : ， The Inject current-scale multispectral images Generate the current scale fusion result from each band. Among them, the current scale is the first The fusion results of each band The calculation formula is: ， in, To inject gain.

10. The full-color sharpening method according to claim 9, characterized in that: The injection gain .

Images