A hyperspectral and multispectral image fusion method and system based on dictionary expansion

Abstract

The present invention discloses a method and system for fusing hyperspectral and multispectral images based on dictionary expansion, the method comprising: obtaining a high-resolution multispectral image Y m and low-resolution hyperspectral image Y under incomplete overlapping scenes h , change Y m Divide the overlapping high-resolution multispectral images into non-overlapping high-resolution multispectral images to solve the spectral dictionary D of the overlapping area s and the overlapping area sparse coefficient A s , and fuse and reconstruct the overlapping area high-resolution hyperspectral image X s ; Extract the common spectrum with Use the common spectrum to expand the spectral dictionary to obtain the expanded non-overlapping region spectral dictionary D r and the non-overlapping sparse coefficient A r , reconstruct the high-resolution hyperspectral image X of the non-overlapping area r ; X s and X r Synthesize to obtain high-resolution multispectral image Y m High-resolution hyperspectral image X of the fully covered area. The present invention utilizes spectral similarity and spatial information of multispectral images to solve the sparse coefficient matrix of the non-overlapping area, thus realizing image fusion of the fully covered area of ​​multispectral images in incompletely overlapping scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and more particularly to a method and system for fusing hyperspectral and multispectral images based on dictionary expansion in an incompletely overlapping scene. Background Art

[0002] High-resolution hyperspectral images have rich spectral information and fine ground features, and can be widely used in fields such as feature recognition and ground feature classification. However, due to the mutual constraints between spatial resolution and spectral resolution during the imaging process, at present, only multispectral images with limited spectral resolution and hyperspectral images with limited spatial resolution can be obtained. Therefore, reconstructing high-resolution hyperspectral images by fusing high-resolution multispectral images with low-resolution hyperspectral images has become a problem of great concern. Various image fusion methods have been developed and released in recent years.

[0003] The current multispectral and hyperspectral image fusion methods use at least one pair of multispectral and hyperspectral images of the same scene for training or learning. However, in actual satellite imaging, the coverage of low-resolution hyperspectral images is much smaller than that of high-resolution multispectral images, which does not match the existing image fusion methods. As a result, when put into use, only overlapping areas can be cropped for fusion, so that the high-resolution multispectral image information in the non-overlapping areas cannot be used, and the size of the reconstructed high-resolution hyperspectral images is limited. Some deep learning methods reconstruct high-resolution hyperspectral images of non-overlapping areas by spectral mapping. However, since the pre-trained neural network model lacks the spectral information of the non-overlapping areas, this part of information cannot be incorporated into the pre-trained model, and its fusion performance is still limited. The above problems reflect that the existing methods cannot obtain satisfactory hyperspectral and multispectral image fusion effects in incompletely overlapping scenes.

[0004] Therefore, it is an urgent problem for those skilled in the art to propose a method for fusing hyperspectral and multispectral images in incompletely overlapping scenarios to improve the image fusion effect. Summary of the Invention

[0005] In view of this, the present invention provides a hyperspectral and multispectral image fusion method and system based on dictionary expansion. The multispectral and hyperspectral image fusion problem in incomplete overlapping scenarios is first decomposed into the image fusion problem of overlapping areas and the reconstruction problem of unreconstructed areas. On the basis of image fusion in overlapping areas, a spectral dictionary expansion strategy based on a common spectrum extraction strategy is proposed. The spectral dictionary is expanded to a spectral dictionary suitable for non-overlapping scenarios. The spectral similarity and the spatial information of the multispectral images are used to solve the sparse coefficient matrix of the non-overlapping areas, so as to realize image fusion of the full coverage area of ​​the multispectral images in incomplete overlapping scenarios, efficiently utilize image information, and facilitate subsequent image applications.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a method for fusing hyperspectral and multispectral images based on dictionary expansion, comprising the following steps:

[0008] S1. Obtain high-resolution multispectral image Y m and low-resolution hyperspectral image Y under incomplete overlapping scenes h , based on the low-resolution hyperspectral image Y h , the high-resolution multispectral image Y m Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images

[0009] S2, the low-resolution hyperspectral image Y h and overlapping high-resolution multispectral images The spectral dictionary D in the overlapping area is solved by using a fusion algorithm based on spectral dictionary learning. s and the overlapping area sparse coefficient A s , and fuse and reconstruct the overlapping area high-resolution hyperspectral image X s ;

[0010] S3, using a common spectrum extraction strategy to extract the overlapping high-resolution multispectral images using spectral similarity With non-overlapping high-resolution multispectral images Shared spectrum

[0011] S4, using the common spectrum Expand the spectral dictionary and add the spectral dictionary D s As the initial value of the non-overlapping region spectral dictionary, the extended non-overlapping region spectral dictionary D is obtained by joint optimization under the constraints of sparseness and low rank. r and the non-overlapping sparse coefficient A r , reconstruct the high-resolution hyperspectral image X of the non-overlapping area r ;

[0012] S5, the overlapping area high-resolution hyperspectral image X s and the non-overlapping region high-resolution hyperspectral image X r Synthesize to obtain the high-resolution multispectral image Y m High-resolution hyperspectral image X covering the entire area.

[0013] Furthermore, in step S1, the low-resolution hyperspectral image Y is defined as h for:

[0014]

[0015] Define the overlapping high-resolution multispectral image With non-overlapping high-resolution multispectral images for:

[0016]

[0017] Among them, D s is the spectral dictionary of the overlapping region, D r is the spectral dictionary of non-overlapping regions, A s is the sparse coefficient of the overlapping area, A r is the non-overlapping sparse coefficient, L is the spectral response function, B is the spatial degradation factor, and They are Gaussian noise in overlapping high-resolution multispectral images, low-resolution hyperspectral images and non-overlapping high-resolution multispectral images, respectively.

[0018] Furthermore, the spectral dictionary D of the overlapping region is solved in step S2. s and the overlapping area sparse coefficient A s The formula is:

[0019]

[0020] Among them, η1 is the balance coefficient, λ1 is the regularization parameter, || || F is the F-norm constraint, and || ||1 is the l1-norm constraint.

[0021] Furthermore, step S3 includes:

[0022] S31, using the spectral similarity index u(i) to measure the overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images similarity;

[0023] S32, by taking a threshold value for the spectrum similarity index u(i), complete the common spectrum Selection.

[0024] Furthermore, the step S31 includes:

[0025] The spectral similarity index u(i) is calculated as follows: With non-overlapping high-resolution multispectral images The similarity of spectral bands is expressed as follows:

[0026]

[0027] in, Overlapping high-resolution multispectral images The i-th spectral band of Non-overlapping high-resolution multispectral images For the j-th spectral band, || ||2 is the l2 norm constraint.

[0028] Furthermore, the step S32 includes:

[0029] The maximum value u of the first n items in the spectral similarity index is selected by the threshold σ s , complete the common matrix U n The selection of n is determined by the spectral similarity ratio The measure is:

[0030] U n (i,j)=δ(iu s (j))

[0031] st u s ={k|u(k)>σ}, k=1,…,M

[0032] Among them, δ is the Dirac function, and the spectral similarity index u with a similarity greater than the threshold σ is selected s Enter the common matrix, u s It is limited by the threshold σ, which is the nth maximum value of the similarity index u, and M is the total number of spectra in the overlapping area.

[0033] Through the common matrix U n , and obtain the common spectrum Selection:

[0034]

[0035] Among them, X s For the high-resolution hyperspectral image of the overlapping area reconstructed in step S3, the common matrix U n , and obtain the common spectrum and is the common spectrum sparse coefficient and noise under the dictionary representation.

[0036] Furthermore, the step S4 specifically includes:

[0037] Using the common spectrum and the common spectral sparsity coefficient With the non-overlapping area spectrum and non-overlapping sparse coefficient A r The similarity of , solves the sparse coefficient A of the non-overlapping region under the sparse and low-rank constraints. r With the extended non-overlapping region spectral dictionary D r ; The solution formula is:

[0038]

[0039] Among them, λ2, η2, and γ2 are regularization parameters, || || F is the F-norm constraint, || ||1 is the l1-norm constraint, || || * is the nuclear norm constraint; and represents the constraint for the common error in the overlapping region, The common error representation constraint for the non-overlapping region; and the overlapping region spectral dictionary D solved in step S2 can be s As the non-overlapping region spectral dictionary D r The initial value of .

[0040] According to the extended non-overlapping region spectral dictionary D r The sparse coefficient A of the non-overlapping area r , reconstruct the high-resolution hyperspectral image X in the non-overlapping area r .

[0041] Furthermore, the sparse coefficient A of the non-overlapping area r The solution process includes:

[0042] The overlapping region spectral dictionary D s As the initial value of the non-overlapping region spectral dictionary Dr, the sparse coefficient A of the non-overlapping region is calculated using a convergence iteration strategy. r Solution:

[0043]

[0044] It turns out that:

[0045]

[0046] Where H = [I (N-M)×(N-M) 0 (N-M)×n ] T is a block matrix, simplifying the constraint expression, D l =LD r For abbreviation, The singular value decomposition of U,∑ and V,E=A r and represents the splitting variable, V1 and V2 are Lagrange multipliers, μ1>0 is the Lagrange multiplier, soft(u, v) is the soft threshold function, and λ2 and γ2 are regularization parameters.

[0047] Furthermore, the extended non-overlapping region spectral dictionary D r The solution process includes:

[0048] For the extended non-overlapping region spectral dictionary D rPerform iterative solution:

[0049]

[0050] It turns out that:

[0051]

[0052] Where G = D r [A s A r ] represents a split variable, is an abbreviation, V3 is the Lagrange multiplier, μ2>0 is the Lagrange multiplier, and η2 is the regularization parameter.

[0053] In a second aspect, an embodiment of the present invention provides a hyperspectral and multispectral image fusion system based on dictionary expansion, comprising the following modules:

[0054] Module 1: Used to obtain high-resolution multispectral image Y m and low-resolution hyperspectral image Y under incomplete overlapping scenes h , based on the low-resolution hyperspectral image Y h , used to convert the high-resolution multispectral image Y m Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images

[0055] Module 2: for the low-resolution hyperspectral image Y h and overlapping high-resolution multispectral images The spectral dictionary D in the overlapping area is solved by using a fusion algorithm based on spectral dictionary learning. s and the overlapping area sparse coefficient A s , and used to fuse and reconstruct the overlapping area high-resolution hyperspectral image X s ;

[0056] Module 3: Used to extract the overlapping high-resolution multispectral images using a common spectrum extraction strategy and spectral similarity With non-overlapping high-resolution multispectral images Shared spectrum

[0057] Module 4: Utilizing the shared spectrum Expand the spectral dictionary and add the spectral dictionary D s As the initial value of the non-overlapping region spectral dictionary, the extended non-overlapping region spectral dictionary D is obtained by joint optimization under the constraints of sparseness and low rank. r and the non-overlapping sparse coefficient A r , used to reconstruct high-resolution hyperspectral images of non-overlapping areas X r ;

[0058] Module 5: for converting the overlapping area high-resolution hyperspectral image X s and the non-overlapping region high-resolution hyperspectral image X r Synthesize to obtain the high-resolution multispectral image Y m High-resolution hyperspectral image X covering the entire area.

[0059] Through the above technical solutions, it can be seen that compared with the prior art, the present invention discloses a method and system for fusion of hyperspectral and multispectral images based on dictionary expansion. First, the high-resolution multispectral and low-resolution hyperspectral images of the overlapping area are fused according to the fusion algorithm of spectral dictionary learning to obtain the spectral dictionary and the fused high-resolution hyperspectral image of the overlapping area; secondly, in order to ensure that the spectral dictionary obtained by solving the overlapping area is applicable to the non-overlapping area, a spectral dictionary expansion strategy is proposed, and a common spectrum extraction strategy is adopted based on the spectral similarity of the overlapping and non-overlapping areas, and the sparse coefficients of the expanded spectral dictionary and the non-overlapping area are jointly optimized to reconstruct the high-resolution hyperspectral image of the non-overlapping area; finally, the fusion result of the overlapping area and the reconstruction result of the non-overlapping area are synthesized into a high-resolution hyperspectral image under the full coverage area of ​​the multispectral image.

[0060] The present invention decomposes the multispectral and hyperspectral image fusion problem in incomplete overlapping scenarios into the image fusion problem of the overlapping area and the reconstruction problem of the unreconstructed area. On the basis of solving the overlapping area with the traditional method, a spectral dictionary expansion strategy based on the common spectrum extraction strategy is proposed. In the sparse and low-rank framework, the convergence iteration strategy is used to realize the closed-form solution of the sparse coefficients and the spectral dictionary. Combined with the overlapping area, the high-resolution hyperspectral image reconstruction of the full coverage area of ​​the high-resolution multispectral image is realized, which significantly improves the fusion effect of hyperspectral and multispectral images in incomplete overlapping scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0062] Figure 1 This is a schematic diagram of a hyperspectral and multispectral image fusion method based on dictionary expansion provided by the present invention.

[0063] FIG2( a ) shows an input low-resolution hyperspectral image in an embodiment of the present invention.

[0064] FIG2( b ) shows an input high-resolution multispectral image in an embodiment of the present invention.

[0065] FIG2( c ) is a reconstructed high-resolution hyperspectral image outputted in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0067] Example 1

[0068] Reference Figure 1 As shown, the embodiment of the present invention discloses a method for fusing hyperspectral and multispectral images based on dictionary expansion, comprising the following steps:

[0069] S1. Obtain high-resolution multispectral image Y m and low-resolution hyperspectral image Y under incomplete overlapping scenes h , based on the low-resolution hyperspectral image Y h , the high-resolution multispectral image Y m Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images

[0070] S2, low-resolution hyperspectral image Y h and overlapping high-resolution multispectral images The spectral dictionary D in the overlapping area is solved by using a fusion algorithm based on spectral dictionary learning. s and the overlapping area sparse coefficient A s , and fuse and reconstruct the overlapping area high-resolution hyperspectral image X s ;

[0071] S3, using the common spectrum extraction strategy to extract overlapping high-resolution multispectral images using spectral similarity With non-overlapping high-resolution multispectral images Shared spectrum

[0072] S4. Using the common spectrum Expand the spectral dictionary and add the spectral dictionary D s As the initial value of the non-overlapping region spectral dictionary, the extended non-overlapping region spectral dictionary D is obtained by joint optimization under the constraints of sparseness and low rank. r and the non-overlapping sparse coefficient A r , reconstruct the high-resolution hyperspectral image X of the non-overlapping area r ;

[0073] S5, the overlapping area high resolution hyperspectral image X s and non-overlapping area high-resolution hyperspectral image X r Synthesize to obtain high-resolution multispectral image Y m High-resolution hyperspectral image X covering the entire area.

[0074] The following are the detailed steps of this method:

[0075] In step S1, the input high-resolution multispectral image Y m The coverage area is large and the low-resolution hyperspectral image Y h The coverage area is small, refer to low-resolution hyperspectral images High-resolution multispectral images Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images Among them, m and N represent the low-resolution hyperspectral image Y h and high-resolution multispectral image Y m The number of pixels in each band, λ x and λ y is the low-resolution hyperspectral image Y h and high-resolution multispectral image Y m The number of spectral bands. Accordingly, the high-resolution hyperspectral image to be reconstructed can be Overlapping high-resolution hyperspectral images With non-overlapping high-resolution hyperspectral images

[0076] In the overlapping area, there is a spectral degradation and spatial degradation relationship between the target high-resolution hyperspectral image and the high-resolution multispectral image and the low-resolution hyperspectral image. The spectral response function L and the spatial degradation factor B are introduced, and the dictionary representation X=D of the high-resolution hyperspectral image is added. s A, available and In the non-overlapping area, there is spectral degradation between the high-resolution hyperspectral image and the high-resolution multispectral image, which can be expressed as Among them A s 、A r Represent the sparse coefficients of the overlapping area and the non-overlapping area respectively.

[0077] In this embodiment, refer to Figure 2(a) and Figure 2(b) for the input low-resolution hyperspectral image and high-resolution multispectral image. The spectral image is a spectral image taken by AVIRIS. The multispectral image size is 300×300×4, the spatial resolution is 20m, and the hyperspectral image size is 30×30×93, the spatial resolution is 4m, that is, Correspondingly, the size of the high-resolution hyperspectral image to be reconstructed is 300×300×93, that is,

[0078] In step S2, the low-resolution hyperspectral image Y h Overlapping high-resolution multispectral images The high-resolution hyperspectral image X of the overlapping area to be reconstructed under the spectral dictionary representation s =D s A s , there is a spectral degradation relationship Relationship with spatial degradation The spectral dictionary D is solved by joint optimization using a fusion method based on spectral dictionary learning. s and its sparse coefficient A s :

[0079]

[0080] Among them, η1 is the balance coefficient, λ1 is the regularization parameter, || || F is the F-norm constraint, and ||||1 is the l1-norm constraint.

[0081] In this embodiment, the spectral dictionary D s The number of columns K is 100.

[0082] In step S3, according to the overlapping high-resolution hyperspectral image X s With non-overlapping high-resolution hyperspectral image X r The proximity of the overlapping and non-overlapping regions is considered, and a common spectrum extraction strategy is proposed. The spectral similarity index u(i) is used to measure the overlapped high-resolution multispectral images. With non-overlapping high-resolution multispectral images The similarity of the common spectrum is selected by threshold

[0083] Among them, the spectral similarity index u(i) is calculated as the overlapping high-resolution multispectral image With non-overlapping high-resolution multispectral images The similarity of spectral bands is expressed as follows:

[0084]

[0085] After that, the maximum value u of the first n items in the spectral similarity index is selected by the threshold σ s , complete the selection of the common matrix U, where n is selected by the spectral similarity ratio Metrics can be further described as:

[0086] U n (i,j)=δ(iu s (j))

[0087] st u s={k|u(k)>σ}, k=1,…,M

[0088] Among them, δ is the Dirac function, and the spectral similarity index u with a similarity greater than the threshold σ is selected s Enter the common matrix, u s It is limited by the threshold σ, which is the nth maximum value of the similarity index u, and M is the total number of spectra in the overlapping area.

[0089] Through the common matrix U n , complete the common spectrum Selection:

[0090]

[0091] Among them, X s For the high-resolution hyperspectral image of the overlapping area reconstructed in step S3, the common matrix U n , and obtain the common spectrum and is the common spectrum sparse coefficient and noise under the dictionary representation.

[0092] In this embodiment, the common spectrum similarity ratio is set to ξ=0.2.

[0093] In step S4, the spectral dictionary expansion strategy is adopted, based on the common spectrum extraction strategy, under the constraints of sparseness and low rank, the spectral dictionary D r The sparse coefficient A of the non-overlapping area r Perform joint optimization, due to the shared spectrum The spectrum X of the non-overlapping region r The similarity of, accordingly, its corresponding sparse coefficient With A r It has low rank. The spectral dictionary D is r The sparse coefficient A of the non-overlapping area r Perform joint optimization to solve:

[0094]

[0095] Among them, λ2, η2, and γ2 are regularization parameters, || || F is the F-norm constraint, || ||1 is the l1-norm constraint, || || * is the nuclear norm constraint; and represents the constraint for the common error in the overlapping region, The common error representation constraint for the non-overlapping region; considering that the spectral dictionary D for the overlapping region is already solved in module S2 s , which is fixed as the spectral dictionary Dr The initial value of the sparse coefficient A is calculated using the convergence iteration strategy. r Solution:

[0096]

[0097] Thus we obtain the following exact closed-form solution:

[0098]

[0099] Where H = [I (N-M)×(N-M) 0 (N-M)×n ] T is a block matrix, simplifying the constraint expression, D l =LD r For abbreviation, The singular value decomposition of U,∑ and V,E=A r and represents the splitting variable, V1 and V2 are Lagrange multipliers, μ1>0 is the Lagrange multiplier, soft(u, v) is the soft threshold function, λ2 and γ2 are regularization parameters. Then fix the sparse coefficient A r , for the spectral dictionary D r Perform iterative solution:

[0100]

[0101] The following exact closed-form solution is obtained:

[0102]

[0103] Where G = D r [A s A r ] represents a split variable, is an abbreviation, V3 is the Lagrange multiplier, μ2>0 is the Lagrange multiplier, and η2 is the regularization parameter.

[0104] By extended spectral dictionary D r The sparse coefficient A of the non-overlapping area r , reconstruct the high-resolution hyperspectral image X in the non-overlapping area r .

[0105] In this embodiment, the regularization parameters are set to η2=1, γ2=10 -3 and η2=10 -4 .

[0106] In step S5, the reconstructed high-resolution hyperspectral image X of the non-overlapping area obtained in step S4 is used. r , and the fused high-resolution hyperspectral image X obtained in step S2 is the overlapping area s, synthesize high-resolution hyperspectral image X.

[0107] 2( c ) shows the reconstructed high-resolution hyperspectral image output in this embodiment.

[0108] This paper focuses on the differences in imaging areas of different sensors in actual satellite imaging and the incomplete applicability of existing image fusion methods. It proposes a hyperspectral and multispectral image fusion system for incomplete overlapping scenarios, which decomposes the incomplete fusion problem into the image fusion problem of the overlapping area and the reconstruction problem of the unreconstructed area.

[0109] A spectral dictionary expansion strategy based on a shared spectrum extraction strategy is proposed. Within a sparse and low-rank framework, the expanded spectral dictionary is solved through joint optimization of a common error representation constraint. Leveraging the spatial information of non-overlapping high-resolution multispectral images and spectral similarity constraints, a closed-form solution for the optimized sparse coefficient matrix in non-overlapping regions is theoretically achieved.

[0110] Example 2

[0111] The embodiment of the present invention discloses a hyperspectral and multispectral image fusion system based on dictionary expansion, including the following modules:

[0112] Module 1: Used to obtain high-resolution multispectral image Y m and low-resolution hyperspectral image Y under incomplete overlapping scenes h , based on the low-resolution hyperspectral image Y h , used to convert the high-resolution multispectral image Y m Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images

[0113] Module 2: For low-resolution hyperspectral image Y h and overlapping high-resolution multispectral images The spectral dictionary D in the overlapping area is solved by using a fusion algorithm based on spectral dictionary learning. s and the overlapping area sparse coefficient A s , and used to fuse and reconstruct the overlapping area high-resolution hyperspectral image X s ;

[0114] Module 3: Used to extract the overlapping high-resolution multispectral images using a common spectrum extraction strategy and spectral similarity With non-overlapping high-resolution multispectral images Shared spectrum

[0115] Module 4: Utilizing the shared spectrum Expand the spectral dictionary and add the spectral dictionary D s As the non-overlapping region spectral dictionary Dr The initial value of is jointly optimized under the sparse and low-rank constraints to obtain the extended non-overlapping region spectral dictionary D r and the non-overlapping sparse coefficient A r , used to reconstruct high-resolution hyperspectral images of non-overlapping areas X r ;

[0116] Module 5: Used to convert overlapping area high-resolution hyperspectral images into X s and non-overlapping area high-resolution hyperspectral image X r Synthesize to obtain high-resolution multispectral image Y m High-resolution hyperspectral image X covering the entire area.

[0117] The second embodiment is a system configuration based on the first embodiment, and its specific implementation process is based on the same inventive concept.

[0118] In summary, the present invention realizes the fusion of hyperspectral and multispectral images in incomplete overlapping scenarios, proposes a common spectrum extraction strategy and a spectral dictionary expansion strategy, and utilizes spectral similarity and the spatial information of multispectral images to expand the spectral dictionary solved in the overlapping area into a spectral dictionary suitable for non-overlapping scenarios. At the same time, the sparse coefficient matrix of the non-overlapping area is optimized from the high-resolution multispectral image, realizing image fusion of the entire area of ​​the multispectral image in incomplete overlapping scenarios, efficiently utilizing image information, and facilitating subsequent image applications.

[0119] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0120] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A hyperspectral and multispectral image fusion method based on dictionary expansion, characterized in that: The following steps are involved: S1. Obtain high-resolution multispectral image Y m and low-resolution hyperspectral image Y under incomplete overlapping scenes h , based on the low-resolution hyperspectral image Y h , the high-resolution multispectral image Y m Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images S2, the low-resolution hyperspectral image Y h and overlapping high-resolution multispectral images The spectral dictionary D in the overlapping area is solved by using a fusion algorithm based on spectral dictionary learning. s and the overlapping area sparse coefficient A s , and fuse and reconstruct the overlapping area high-resolution hyperspectral image X s ; S3, using a common spectrum extraction strategy to extract the overlapping high-resolution multispectral images using spectral similarity With non-overlapping high-resolution multispectral images Shared spectrum include: S31, using the spectral similarity index u(i) to measure the overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images similarity; S32, by taking a threshold value for the spectrum similarity index u(i), complete the common spectrum Selection; S4, using the common spectrum Expand the spectral dictionary and add the spectral dictionary D s As the initial value of the non-overlapping region spectral dictionary, the extended non-overlapping region spectral dictionary D is obtained by joint optimization under the constraints of sparseness and low rank. r Non-overlapping sparse coefficient A r , reconstruct the high-resolution hyperspectral image X of the non-overlapping area r ; S5, the overlapping area high-resolution hyperspectral image X s and the non-overlapping region high-resolution hyperspectral image X r Synthesize to obtain the high-resolution multispectral image Y m High-resolution hyperspectral image X covering the entire area.

2. The hyperspectral and multispectral image fusion method based on dictionary expansion according to claim 1, characterized in that: In step S1, the low-resolution hyperspectral image Y is defined h for: Define the overlapping high-resolution multispectral image With non-overlapping high-resolution multispectral images for: Among them, D s is the spectral dictionary of the overlapping region, D r is the spectral dictionary of non-overlapping regions, A s is the sparse coefficient of the overlapping area, A r is the non-overlapping sparse coefficient, L is the spectral response function, B is the spatial degradation factor, and They are Gaussian noise in overlapping high-resolution multispectral images, low-resolution hyperspectral images and non-overlapping high-resolution multispectral images, respectively.

3. The hyperspectral and multispectral image fusion method based on dictionary expansion according to claim 1, characterized in that: The spectral dictionary D of the overlapping region is obtained in step S2. s and the overlapping area sparse coefficient A s The formula is: Among them, η1 is the balance coefficient, λ1 is the regularization parameter, || || F is the F-norm constraint, and || ||1 is the l1-norm constraint.

4. The hyperspectral and multispectral image fusion method based on dictionary expansion according to claim 1, characterized in that: The step S31 includes: The spectral similarity index u(i) is calculated as follows: With non-overlapping high-resolution multispectral images The similarity of spectral bands is expressed as follows: in, Overlapping high-resolution multispectral images The i-th spectral band of Non-overlapping high-resolution multispectral images For the j-th spectral band, || ||2 is the l2 norm constraint.

5. The hyperspectral and multispectral image fusion method based on dictionary expansion according to claim 1, characterized in that: The step S32 includes: The maximum value u of the first n items in the spectral similarity index is selected by the threshold σ s , complete the common matrix U n The selection of n is determined by the spectral similarity ratio The measure is: IN n (i;j)=δ(iu s (j)) s.t.u s ={k|u(k)>σ};k=1;...;M Among them, ± is the Dirac function, and the spectral similarity index u with a similarity greater than the threshold σ is selected s Enter the common matrix, u s It is limited by the threshold σ, which is the nth maximum value of the similarity index u, and M is the total number of spectra in the overlapping area; Through the common matrix U n , and obtain the common spectrum Selection: Among them, X s For the high-resolution hyperspectral image of the overlapping area reconstructed in step S3, the common matrix U n , and obtain the common spectrum and is the common spectrum sparse coefficient and noise under the dictionary representation.

6. The hyperspectral and multispectral image fusion method based on dictionary expansion according to claim 1, characterized in that: The step S4 specifically includes: Using the common spectrum and the common spectral sparsity coefficient With the non-overlapping area spectrum and non-overlapping sparse coefficient A r The similarity of , solves the sparse coefficient A of the non-overlapping region under the sparse and low-rank constraints. r With the extended non-overlapping region spectral dictionary D r ; The solution formula is: Among them, λ2, η2, and γ2 are regularization parameters, |||| F is the F norm constraint, ||||1 is the l1 norm constraint, |||| * is the nuclear norm constraint; and represents the constraint for the common error in the overlapping region, The common error representation constraint for the non-overlapping region; and the overlapping region spectral dictionary D solved in step S2 can be s As the non-overlapping region spectral dictionary D r The initial value of According to the extended non-overlapping region spectral dictionary D r The sparse coefficient A of the non-overlapping area r , reconstruct the high-resolution hyperspectral image X in the non-overlapping area r .

7. The method for fusion of hyperspectral and multispectral images based on dictionary expansion according to claim 6, characterized in that: The sparse coefficient A of the non-overlapping area r The solution process includes: The overlapping region spectral dictionary D s As the non-overlapping region spectral dictionary D r The initial value of the non-overlapping area is calculated by using the convergence iteration strategy. r Solution: It turns out that: Where H = [I (N-M)×(N-M) 0 (N-M)×n ] T is a block matrix, simplifying the constraint expression, D l =LD r For abbreviation, The singular value decomposition of U,∑ and V,E=A r and represents the splitting variable, V1 and V2 are Lagrange multipliers, μ1>0 is the Lagrange multiplier, soft(u;v) is the soft threshold function, and λ2 and γ2 are regularization parameters.

8. The method for fusion of hyperspectral and multispectral images based on dictionary expansion according to claim 6, characterized in that: The extended non-overlapping region spectral dictionary D r The solution process includes: For the extended non-overlapping region spectral dictionary D r Perform iterative solution: It turns out that: Where G = D r [A s A r ] represents a split variable, is an abbreviation, V3 is the Lagrange multiplier, μ2>0 is the Lagrange multiplier, and η2 is the regularization parameter.

9. A hyperspectral and multispectral image fusion system based on dictionary expansion, characterized in that: Includes the following modules: Module 1: Used to obtain high-resolution multispectral image Y m and low-resolution hyperspectral image Y under incomplete overlapping scenes h , based on the low-resolution hyperspectral image Y h , used to convert the high-resolution multispectral image Y m Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images Module 2: for the low-resolution hyperspectral image Y h and overlapping high-resolution multispectral images The spectral dictionary D in the overlapping area is solved by using a fusion algorithm based on spectral dictionary learning. s and the overlapping area sparse coefficient A s , and used to fuse and reconstruct the overlapping area high-resolution hyperspectral image X s ; Module 3: Used to extract the overlapping high-resolution multispectral images using a common spectrum extraction strategy and spectral similarity With non-overlapping high-resolution multispectral images Shared spectrum include: Used to measure the overlapping high-resolution multispectral images using the spectral similarity index u(i) With non-overlapping high-resolution multispectral images similarity; It is also used to complete the common spectrum by taking a threshold value for the spectrum similarity index u(i) Selection; Module 4: Utilizing the shared spectrum Expand the spectral dictionary and add the spectral dictionary D s As the initial value of the non-overlapping region spectral dictionary, the extended non-overlapping region spectral dictionary D is obtained by joint optimization under the constraints of sparseness and low rank. r and the non-overlapping sparse coefficient A r , used to reconstruct high-resolution hyperspectral images of non-overlapping areas X r ; Module 5: for converting the overlapping area high-resolution hyperspectral image X s and the non-overlapping region high-resolution hyperspectral image X r Synthesize to obtain the high-resolution multispectral image Y m High-resolution hyperspectral image X covering the entire area.

Images