Hyperspectral and multispectral image fusion method and system based on dictionary extension

Abstract

The invention discloses a hyperspectral and multispectral image fusion method and system based on dictionary extension, and the method comprises the steps: obtaining a high-resolution multispectral image Ym and a low-resolution hyperspectral image Yh in an incomplete overlapping scene, dividing Ym into an overlapped high-resolution multispectral image # imgabs0 # and a non-overlapped high-resolution multispectral image # imgabs1 # to solve an overlapped region spectrum dictionary Ds and an overlapped region sparse coefficient As, and fusing and reconstructing an overlapped region high-resolution hyperspectral image Xs; the common spectrum # imgabs4 # of the # imgabs2 # and the # imgabs3 # is extracted, the common spectrum # imgabs5 # is used for carrying out spectrum dictionary expansion, an expanded non-overlapped region spectrum dictionary Dr and a non-overlapped sparse coefficient Ar are obtained, and a non-overlapped region high-resolution hyperspectral image Xr is reconstructed; and synthesizing Xs and Xr to obtain a high-resolution hyperspectral image X of a full coverage area of the high-resolution multispectral image Ym. According to the method, the sparse coefficient matrix of the non-overlapping area is solved by utilizing the spectral similarity and the spatial information of the multispectral image, and the image fusion of the full coverage area of the multispectral image in the incomplete overlapping scene is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and more particularly to a method and system for fusing hyperspectral and multispectral images based on dictionary expansion in an incompletely overlapping scene. Background Art

[0002] High-resolution hyperspectral images have rich spectral information and fine ground features, and can be widely used in fields such as feature recognition and ground classification. However, due to the mutual constraints between spatial resolution and spectral resolution in the imaging process, at present, only multispectral images with limited spectral resolution and hyperspectral images with limited spatial resolution can be obtained. Therefore, reconstructing high-resolution hyperspectral images by fusing high-resolution multispectral images with low-resolution hyperspectral images has become a problem of concern, and various image fusion methods have been developed and launched in recent years.

[0003] At present, the multispectral and hyperspectral image fusion methods use at least one pair of multispectral and hyperspectral images in the same scene for training or learning. However, in actual satellite imaging, the coverage of low-resolution hyperspectral images is much smaller than that of high-resolution multispectral images, which does not match the existing image fusion methods. As a result, when they are put into use, only overlapping areas can be cut out for fusion, so that the high-resolution multispectral image information in the non-overlapping areas cannot be used and the size of the reconstructed high-resolution hyperspectral images is limited. Some deep learning methods reconstruct high-resolution hyperspectral images of non-overlapping areas by spectral mapping, but because the pre-trained neural network model lacks spectral information of non-overlapping areas, this part of information cannot be incorporated into the pre-trained model, and its fusion performance is still limited. The above problems reflect that the existing methods cannot obtain satisfactory hyperspectral and multispectral image fusion effects in incompletely overlapping scenes.

[0004] Therefore, proposing a hyperspectral and multispectral image fusion method in incompletely overlapping scenarios to improve the image fusion effect is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0005] In view of this, the present invention provides a hyperspectral and multispectral image fusion method and system based on dictionary expansion. The multispectral and hyperspectral image fusion problem in incomplete overlapping scenes is first decomposed into the image fusion problem of overlapping areas and the reconstruction problem of unreconstructed areas. On the basis of image fusion in overlapping areas, a spectral dictionary expansion strategy based on a common spectrum extraction strategy is proposed. The spectral dictionary is expanded to a spectral dictionary suitable for non-overlapping scenes. The spectral similarity and the spatial information of the multispectral image are used to solve the sparse coefficient matrix of the non-overlapping area, so as to realize the image fusion of the full coverage area of ​​the multispectral image in the incomplete overlapping scene, and the image information is efficiently used, which is beneficial to the subsequent image application.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] In a first aspect, an embodiment of the present invention provides a method for fusing hyperspectral and multispectral images based on dictionary expansion, comprising the following steps:

[0008] S1. Obtain high-resolution multispectral image Y m and low-resolution hyperspectral image Y in incompletely overlapping scenes h , based on the low-resolution hyperspectral image Y h , the high-resolution multispectral image Y m Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images

[0009] S2, the low-resolution hyperspectral image Y h and overlapping high-resolution multispectral images The spectral dictionary D in the overlapping area is solved by using a fusion algorithm based on spectral dictionary learning. s and the overlap area sparse coefficient A s , and fuse and reconstruct the high-resolution hyperspectral image X in the overlapping area s ;

[0010] S3, adopting a common spectrum extraction strategy and using spectrum similarity to extract the overlapping high-resolution multispectral image With non-overlapping high-resolution multispectral images Common spectrum

[0011] S4, using the common spectrum Expand the spectral dictionary and add the spectral dictionary D s As the initial value of the non-overlapping region spectral dictionary, the extended non-overlapping region spectral dictionary D is obtained by joint optimization under sparse and low-rank constraints. r and the non-overlapping sparse coefficient A r , reconstruct the high-resolution hyperspectral image X of the non-overlapping area r ;

[0012] S5, the overlapping area high resolution hyperspectral image X s and the non-overlapping region high-resolution hyperspectral image X r Synthesize to obtain the high-resolution multispectral image Y m High-resolution hyperspectral image X covering the entire area.

[0013] Furthermore, in step S1, the low-resolution hyperspectral image Y is defined h for:

[0014]

[0015] Define the overlapping high-resolution multispectral image With non-overlapping high-resolution multispectral images for:

[0016]

[0017] Among them, D s is the spectral dictionary of the overlapping region, D r is the spectral dictionary of non-overlapping regions, A s is the sparse coefficient of the overlapping area, A r is the non-overlapping sparse coefficient, L is the spectral response function, B is the spatial degradation factor, and They are Gaussian noise in overlapping high-resolution multispectral images, low-resolution hyperspectral images, and non-overlapping high-resolution multispectral images, respectively.

[0018] Further, the spectral dictionary D of the overlapping region is solved in step S2 s and the overlap area sparse coefficient A s The formula is:

[0019]

[0020] Among them, η1 is the balance coefficient, λ1 is the regularization parameter, || || F is the F-norm constraint, and || ||1 is the l1-norm constraint.

[0021] Furthermore, the step S3 includes:

[0022] S31, using the spectral similarity index u(i) to measure the overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images similarity;

[0023] S32, by taking a threshold value for the spectrum similarity index u(i), complete the common spectrum Selection.

[0024] Furthermore, the step S31 includes:

[0025] The spectral similarity index u(i) is used to calculate the overlapping high-resolution multispectral image With non-overlapping high-resolution multispectral images The similarity of spectral bands is as follows:

[0026]

[0027] in, Overlapping high-resolution multispectral images The i-th spectral band of Non-overlapping high-resolution multispectral images For the j-th spectral band, || ||2 is the l2 norm constraint.

[0028] Furthermore, the step S32 includes:

[0029] The maximum value u of the first n items in the spectral similarity index is selected by threshold σ s , complete the common matrix U n The selection of n is determined by the spectral similarity ratio The measure is:

[0030] U n (i,j)=δ(iu s (j))

[0031] s s ={k|u(k)>σ}, k=1,…,M

[0032] Among them, δ is the Dirac function, and the spectral similarity index u with a similarity greater than the threshold σ is selected s Enter the common matrix, u s It is subject to a threshold σ, which is the nth maximum value of the similarity index u, and M is the total number of spectra in the overlapping area.

[0033] Through the common matrix U n , and obtain the common spectrum Selection:

[0034]

[0035] Among them, X s The high-resolution hyperspectral image of the overlapping area reconstructed in step S3 is obtained by using the common matrix U n , and obtain the common spectrum and are the common spectrum sparse coefficients and noise under dictionary representation.

[0036] Furthermore, the step S4 specifically includes:

[0037] Using the common spectrum and the common spectral sparse coefficient The non-overlapping area spectrum and non-overlapping sparse coefficient A r The similarity of , solves the sparse coefficient A of the non-overlapping region under the sparse and low-rank constraints. r With the extended non-overlapping region spectral dictionary D r ; The solution formula is:

[0038]

[0039] Among them, λ2, η2, and γ2 are regularization parameters, || || F is the F-norm constraint, || ||1 is the l1-norm constraint, || || * is the nuclear norm constraint; and represents the constraint for the common error in the overlapping region, The common error of the non-overlapping region represents the constraint; and the overlapping region spectral dictionary D solved in step S2 can be s As the non-overlapping region spectral dictionary D r The initial value of .

[0040] According to the extended non-overlapping region spectral dictionary D r The sparse coefficient A of the non-overlapping area r , reconstruct the high-resolution hyperspectral image X in the non-overlapping area r .

[0041] Furthermore, the sparse coefficient A of the non-overlapping area r The solution process includes:

[0042] The overlapping region spectral dictionary D s As the initial value of the spectral dictionary Dr of the non-overlapping region, the sparse coefficient A of the non-overlapping region is calculated using a convergence iteration strategy. r Solution:

[0043]

[0044] It turns out that:

[0045]

[0046] Where H = [I (N-M)×(N-M) 0 (N-M)×n ] T is a block matrix, simplifying the expression of constraints, D l =LD r For abbreviation, The singular value decomposition of U,∑ and V,E=A r and represents the splitting variable, V1 and V2 are Lagrange multipliers, μ1>0 is the Lagrange multiplier, soft(u, v) is the soft threshold function, and λ2 and γ2 are regularization parameters.

[0047] Furthermore, the extended non-overlapping region spectral dictionary D r The solution process includes:

[0048] The extended non-overlapping region spectral dictionary D rPerform an iterative solution:

[0049]

[0050] It turns out that:

[0051]

[0052] Where G = D r [A s A r ] represents a split variable, is an abbreviation, V3 is the Lagrange multiplier, μ2>0 is the Lagrange multiplier, and η2 is the regularization parameter.

[0053] In a second aspect, an embodiment of the present invention provides a hyperspectral and multispectral image fusion system based on dictionary expansion, comprising the following modules:

[0054] Module 1: Used to obtain high-resolution multispectral image Y m and low-resolution hyperspectral image Y in incompletely overlapping scenes h , based on the low-resolution hyperspectral image Y h , used to transform the high-resolution multispectral image Y m Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images

[0055] Module 2: for the low-resolution hyperspectral image Y h and overlapping high-resolution multispectral images The spectral dictionary D in the overlapping area is solved by using a fusion algorithm based on spectral dictionary learning. s and the overlap area sparse coefficient A s , and used to fuse and reconstruct the high-resolution hyperspectral images X in the overlapping areas s ;

[0056] Module 3: Used to extract the overlapping high-resolution multispectral images using a common spectrum extraction strategy and spectrum similarity With non-overlapping high-resolution multispectral images Common spectrum

[0057] Module 4: Using the common spectrum Expand the spectral dictionary and add the spectral dictionary D s As the initial value of the non-overlapping region spectral dictionary, the extended non-overlapping region spectral dictionary D is obtained by joint optimization under sparse and low-rank constraints. r and the non-overlapping sparse coefficient A r , used to reconstruct high-resolution hyperspectral images X in non-overlapping areas r ;

[0058] Module 5: Used to convert the overlapping area high-resolution hyperspectral image X s and the non-overlapping region high-resolution hyperspectral image X r Synthesize to obtain the high-resolution multispectral image Y m High-resolution hyperspectral image X covering the entire area.

[0059] Through the above technical solutions, it can be known that compared with the prior art, the present invention discloses a method and system for fusion of hyperspectral and multispectral images based on dictionary expansion. Firstly, the high-resolution multispectral and low-resolution hyperspectral images are fused in the overlapping area according to the fusion algorithm of spectral dictionary learning, so as to obtain the spectral dictionary and fused high-resolution hyperspectral image in the overlapping area; secondly, in order to ensure that the spectral dictionary obtained by solving the overlapping area is applicable to the non-overlapping area, a spectral dictionary expansion strategy is proposed, and a common spectrum extraction strategy is adopted according to the spectral similarity of the overlapping and non-overlapping areas, and the sparse coefficients of the expanded spectral dictionary and the non-overlapping areas are jointly optimized to reconstruct the high-resolution hyperspectral image in the non-overlapping area; finally, the fusion result of the overlapping area and the reconstruction result of the non-overlapping area are synthesized into a high-resolution hyperspectral image in the full coverage area of ​​the multispectral image.

[0060] The present invention decomposes the multispectral and hyperspectral image fusion problem in incomplete overlapping scenarios into the image fusion problem of overlapping areas and the reconstruction problem of unreconstructed areas. On the basis of solving the overlapping areas by traditional methods, a spectral dictionary expansion strategy based on a common spectrum extraction strategy is proposed. In the sparse and low-rank framework, a convergence iteration strategy is used to realize the closed-form solution of sparse coefficients and spectral dictionary. Combined with the overlapping areas, high-resolution hyperspectral image reconstruction of the full coverage area of ​​high-resolution multispectral images is realized, which significantly improves the fusion effect of hyperspectral and multispectral images in incomplete overlapping scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0062] Figure 1 A schematic diagram of a hyperspectral and multispectral image fusion method based on dictionary expansion provided by the present invention.

[0063] FIG. 2( a ) is an input low-resolution hyperspectral image in an embodiment of the present invention.

[0064] FIG. 2( b ) is an input high-resolution multispectral image in an embodiment of the present invention.

[0065] FIG. 2( c ) is a reconstructed high-resolution hyperspectral image outputted in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] Embodiment 1

[0068] Reference Figure 1 As shown, the embodiment of the present invention discloses a method for fusing hyperspectral and multispectral images based on dictionary expansion, comprising the following steps:

[0069] S1. Obtain high-resolution multispectral image Y m and low-resolution hyperspectral image Y in incompletely overlapping scenes h , based on the low-resolution hyperspectral image Y h , the high-resolution multispectral image Y m Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images

[0070] S2, low-resolution hyperspectral image Y h and overlapping high-resolution multispectral images The spectral dictionary D in the overlapping area is solved by using a fusion algorithm based on spectral dictionary learning. s and the overlap area sparse coefficient A s , and fuse and reconstruct the high-resolution hyperspectral image X in the overlapping area s ;

[0071] S3, adopt the common spectrum extraction strategy and use the spectral similarity to extract overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images Common spectrum

[0072] S4. Using the common spectrum Expand the spectral dictionary and add the spectral dictionary D s As the initial value of the non-overlapping region spectral dictionary, the extended non-overlapping region spectral dictionary D is obtained by joint optimization under sparse and low-rank constraints. r and the non-overlapping sparse coefficient A r , reconstruct the high-resolution hyperspectral image X of the non-overlapping area r ;

[0073] S5, high-resolution hyperspectral image X of the overlapping area s and non-overlapping area high-resolution hyperspectral image X r Synthesize to get high-resolution multispectral image Y m High-resolution hyperspectral image X covering the entire area.

[0074] The following are the detailed steps of this method:

[0075] In step S1, the input high-resolution multispectral image Y m The coverage area is large but the resolution is low. h Small coverage, refer to low-resolution hyperspectral images High-resolution multispectral images Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images Among them, m and N represent the low-resolution hyperspectral image Y h and high-resolution multispectral image Y m The number of pixels in each band, λ x and λ y is the low-resolution hyperspectral image Y h and high-resolution multispectral image Y m The number of spectral bands. Overlapping high-resolution hyperspectral images With non-overlapping high-resolution hyperspectral images

[0076] In the overlapping area, there is a spectral degradation and spatial degradation relationship between the target high-resolution hyperspectral image and the high-resolution multispectral image and the low-resolution hyperspectral image. The spectral response function L and the spatial degradation factor B are introduced, and the dictionary representation X=D of the high-resolution hyperspectral image is added. s A, available and In the non-overlapping area, there is spectral degradation between the high-resolution hyperspectral image and the high-resolution multispectral image, which can be expressed as Among them A s , A r Represent the sparse coefficients of the overlapping area and the non-overlapping area respectively.

[0077] In this embodiment, refer to Figure 2(a) and Figure 2(b) for the input low-resolution hyperspectral image and high-resolution multispectral image. The spectral image is a spectral image taken by AVIRIS. The size of the multispectral image is 300×300×4, the spatial resolution is 20m, and the size of the hyperspectral image is 30×30×93, the spatial resolution is 4m, that is, Correspondingly, the size of the high-resolution hyperspectral image to be reconstructed is 300×300×93, that is,

[0078] In step S2, the low-resolution hyperspectral image Y h Overlapping high-resolution multispectral images The high-resolution hyperspectral image X of the overlapping area to be reconstructed under the spectral dictionary representation s =D s A s , there is a spectral degradation relationship Relationship with spatial degradation The spectral dictionary D is jointly optimized using a fusion method based on spectral dictionary learning. s and its sparse coefficient A s :

[0079]

[0080] Among them, η1 is the balance coefficient, λ1 is the regularization parameter, || || F is the F-norm constraint, and ||||1 is the l1-norm constraint.

[0081] In this embodiment, the spectral dictionary D s The number of columns K is 100.

[0082] In step S3, according to the overlapping high-resolution hyperspectral image X s With non-overlapping high-resolution hyperspectral image X r The proximity of the overlapping and non-overlapping regions is considered, and a common spectrum extraction strategy is proposed. The spectral similarity index u(i) is used to measure the overlapped high-resolution multispectral images. With non-overlapping high-resolution multispectral images The similarity of

[0083] Among them, the spectral similarity index u(i) is calculated as follows: With non-overlapping high-resolution multispectral images The similarity of spectral bands is as follows:

[0084]

[0085] After that, the maximum value u of the first n items in the spectral similarity index is selected by the threshold σ s , complete the selection of the common matrix U, where n is selected by the spectral similarity ratio Metrics can be further described as:

[0086] U n (i,j)=δ(iu s (j))

[0087] s s={k|u(k)>σ}, k=1,…,M

[0088] Among them, δ is the Dirac function, and the spectral similarity index u with a similarity greater than the threshold σ is selected s Enter the common matrix, u s It is subject to a threshold σ, which is the nth maximum value of the similarity index u, and M is the total number of spectra in the overlapping area.

[0089] Through the common matrix U n , complete the common spectrum Selection:

[0090]

[0091] Among them, X s The high-resolution hyperspectral image of the overlapping area reconstructed in step S3 is obtained by using the common matrix U n , and obtain the common spectrum and are the common spectrum sparse coefficients and noise under dictionary representation.

[0092] In this embodiment, the common spectrum similarity ratio is set to ξ=0.2.

[0093] In step S4, the spectral dictionary expansion strategy is adopted, based on the common spectrum extraction strategy, under the sparse and low-rank constraints, the spectral dictionary D r The sparse coefficient A of the non-overlapping area r Perform joint optimization, because of the common spectrum The spectrum X of the non-overlapping region r The similarity of, accordingly, its corresponding sparse coefficient With A r It has low rank. The spectral dictionary D is r The sparse coefficient A of the non-overlapping area r Perform joint optimization to solve:

[0094]

[0095] Among them, λ2, η2, and γ2 are regularization parameters, || || F is the F-norm constraint, || ||1 is the l1-norm constraint, || || * is the nuclear norm constraint; and represents the constraint for the common error in the overlapping region, The common error of the non-overlapping region represents the constraint; considering that the spectral dictionary D for the overlapping region is already solved in module S2 s , which is fixed as the spectral dictionary Dr The initial value of the sparse coefficient A is calculated using the convergence iteration strategy. r Solution:

[0096]

[0097] Thus we obtain the following exact closed-form solution:

[0098]

[0099] Where H = [I (N-M)×(N-M) 0 (N-M)×n ] T is a block matrix, simplifying the expression of constraints, D l =LD r For abbreviation, The singular value decomposition of U,∑ and V,E=A r and represents the splitting variable, V1 and V2 are Lagrange multipliers, μ1>0 is the Lagrange multiplier, soft(u, v) is the soft threshold function, λ2 and γ2 are regularization parameters. Then fix the sparse coefficient A r , for the spectral dictionary D r Perform an iterative solution:

[0100]

[0101] The following exact closed-form solution is obtained:

[0102]

[0103] Where G = D r [A s A r ] represents a split variable, is an abbreviation, V3 is the Lagrange multiplier, μ2>0 is the Lagrange multiplier, and η2 is the regularization parameter.

[0104] By extended spectral dictionary D r The sparse coefficient A of the non-overlapping area r , reconstruct the high-resolution hyperspectral image X in the non-overlapping area r .

[0105] In this embodiment, the regularization parameters are set to η2=1, γ2=10 -3 and η2=10 -4 .

[0106] In step S5, the reconstructed high-resolution hyperspectral image X of the non-overlapping area obtained in step S4 is used. r , and the fused high-resolution hyperspectral image X obtained in step S2 is the overlapping area s, synthesize high-resolution hyperspectral image X.

[0107] Referring to FIG. 2( c ), this is the reconstructed high-resolution hyperspectral image output in this embodiment.

[0108] The present invention pays attention to the differences in imaging areas of different sensors in actual satellite imaging and the incomplete applicability of existing image fusion methods, and proposes a hyperspectral and multispectral image fusion system in incomplete overlapping scenes, which decomposes the incomplete fusion problem into the image fusion problem of the overlapping area and the reconstruction problem of the unreconstructed area.

[0109] A spectral dictionary expansion strategy based on a common spectrum extraction strategy is proposed. In a sparse and low-rank framework, the expanded spectral dictionary is solved by joint optimization through common error representation constraints. By using the spatial information and spectral similarity constraints of non-overlapping high-resolution multispectral images, a closed-form solution for the optimized sparse coefficient matrix in non-overlapping regions is theoretically achieved.

[0110] Embodiment 2

[0111] The embodiment of the present invention discloses a hyperspectral and multispectral image fusion system based on dictionary expansion, comprising the following modules:

[0112] Module 1: Used to obtain high-resolution multispectral image Y m and low-resolution hyperspectral image Y in incompletely overlapping scenes h , based on the low-resolution hyperspectral image Y h , used to transform the high-resolution multispectral image Y m Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images

[0113] Module 2: Used for low-resolution hyperspectral image Y h and overlapping high-resolution multispectral images The spectral dictionary D in the overlapping area is solved by using a fusion algorithm based on spectral dictionary learning. s and the overlap area sparse coefficient A s , and used to fuse and reconstruct the high-resolution hyperspectral images X in the overlapping areas s ;

[0114] Module 3: Used to extract the overlapping high-resolution multispectral images using a common spectrum extraction strategy and spectrum similarity With non-overlapping high-resolution multispectral images Common spectrum

[0115] Module 4: Using the common spectrum Expand the spectral dictionary and add the spectral dictionary D s As the non-overlapping region spectral dictionary Dr The initial value of is jointly optimized under the sparse and low-rank constraints to obtain the extended non-overlapping region spectral dictionary D r and the non-overlapping sparse coefficient A r , used to reconstruct high-resolution hyperspectral images X in non-overlapping areas r ;

[0116] Module 5: Used to convert overlapping areas into high-resolution hyperspectral images X s and non-overlapping area high-resolution hyperspectral image X r Synthesize to get high-resolution multispectral image Y m High-resolution hyperspectral image X covering the entire area.

[0117] Embodiment 2 is a system setting based on embodiment 1, and its specific implementation process is based on the same inventive concept.

[0118] In summary, the present invention realizes the image fusion of hyperspectral and multispectral in incomplete overlapping scenarios, proposes a common spectrum extraction strategy and a spectral dictionary expansion strategy, and uses spectral similarity and spatial information of multispectral images to expand the spectral dictionary solved in the overlapping area into a spectral dictionary suitable for non-overlapping scenarios. At the same time, the sparse coefficient matrix of the non-overlapping area is optimized from the high-resolution multispectral image, so as to realize the image fusion of the entire area of ​​the multispectral image in the incomplete overlapping scenario, efficiently utilize the image information, and facilitate the subsequent image application.

[0119] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0120] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for fusion of hyperspectral and multispectral images based on dictionary expansion, characterized in that: The following steps are involved: S1. Obtain high-resolution multispectral image Y m and low-resolution hyperspectral image Y in incompletely overlapping scenes h , based on the low-resolution hyperspectral image Y h , the high-resolution multispectral image Y m Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images S2, the low-resolution hyperspectral image Y h and overlapping high-resolution multispectral images The spectral dictionary D in the overlapping area is solved by using a fusion algorithm based on spectral dictionary learning. s and the overlap area sparse coefficient A s , and fuse and reconstruct the high-resolution hyperspectral image X in the overlapping area s ; S3, adopting a common spectrum extraction strategy and using spectrum similarity to extract the overlapping high-resolution multispectral image With non-overlapping high-resolution multispectral images Common spectrum S4. Using the common spectrum Expand the spectral dictionary and add the spectral dictionary D s As the initial value of the non-overlapping region spectral dictionary, the extended non-overlapping region spectral dictionary D is obtained by joint optimization under sparse and low-rank constraints. r Non-overlapping sparse coefficient A r , reconstruct the high-resolution hyperspectral image X of the non-overlapping area r ; S5, the overlapping area high resolution hyperspectral image X s and the non-overlapping region high-resolution hyperspectral image X r Synthesize to obtain the high-resolution multispectral image Y m High-resolution hyperspectral image X covering the entire area.

2. The method for fusion of hyperspectral and multispectral images based on dictionary expansion according to claim 1, characterized in that: In step S1, the low-resolution hyperspectral image Y is defined h for: Define the overlapping high-resolution multispectral image With non-overlapping high-resolution multispectral images for: Among them, D s is the spectral dictionary of the overlapping region, D r is the spectral dictionary of non-overlapping regions, A s is the sparse coefficient of the overlapping area, A r is the non-overlapping sparse coefficient, L is the spectral response function, B is the spatial degradation factor, and They are Gaussian noise in overlapping high-resolution multispectral images, low-resolution hyperspectral images, and non-overlapping high-resolution multispectral images, respectively.

3. The method for fusion of hyperspectral and multispectral images based on dictionary expansion as claimed in claim 1, characterized in that: Step S2 is used to solve the spectral dictionary D of the overlapping region. s and the overlap area sparse coefficient A s The formula is: Among them, η1 is the balance coefficient, λ1 is the regularization parameter, || || F is the F-norm constraint, and || ||1 is the l1-norm constraint.

4. The method for fusion of hyperspectral and multispectral images based on dictionary expansion according to claim 1, characterized in that: The step S3 comprises: S31, using the spectral similarity index u(i) to measure the overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images similarity; S32, by taking a threshold value for the spectrum similarity index u(i), complete the common spectrum Selection.

5. The method for fusion of hyperspectral and multispectral images based on dictionary expansion as claimed in claim 4, characterized in that: The step S31 comprises: The spectral similarity index u(i) is used to calculate the overlapping high-resolution multispectral image With non-overlapping high-resolution multispectral images The similarity of spectral bands is as follows: in, Overlapping high-resolution multispectral images The i-th spectral band of Non-overlapping high-resolution multispectral images For the j-th spectral band, || ||2 is the l2 norm constraint.

6. The method for fusion of hyperspectral and multispectral images based on dictionary expansion as claimed in claim 4, characterized in that: The step S32 comprises: The maximum value u of the first n items in the spectral similarity index is selected by threshold σ s , complete the common matrix U n The selection of n is determined by the spectral similarity ratio The measure is: IN n (i,j)=δ(iu s (j)) s.t.u s ={k|u(k)>σ};k=1;...;M Among them, δ is the Dirac function, and the spectral similarity index u with a similarity greater than the threshold σ is selected s Enter the common matrix, u s It is subject to a threshold σ, which is the nth maximum value of the similarity index u, and M is the total number of spectra in the overlapping area. Through the common matrix U n , and obtain the common spectrum Selection: Among them, X s The high-resolution hyperspectral image of the overlapping area reconstructed in step S3 is obtained by using the common matrix U n , and obtain the common spectrum and are the common spectrum sparse coefficients and noise under dictionary representation.

7. The method for fusion of hyperspectral and multispectral images based on dictionary expansion according to claim 1, characterized in that: The step S4 specifically includes: Using the common spectrum and the common spectral sparse coefficient With the non-overlapping area spectrum and non-overlapping sparse coefficient A r The similarity of , solves the sparse coefficient A of the non-overlapping region under the sparse and low-rank constraints. r With the extended non-overlapping region spectral dictionary D r ; The solution formula is: Among them, λ2, η2, γ2 are regularization parameters, || || F is the F-norm constraint, ||||1 is the l1-norm constraint, |||| * is the nuclear norm constraint; and represents the constraint for the common error in the overlapping region, The common error of the non-overlapping region represents the constraint; and the overlapping region spectral dictionary D solved in step S2 can be s As the non-overlapping region spectral dictionary D r The initial value of . According to the extended non-overlapping region spectral dictionary D r The sparse coefficient A of the non-overlapping area r , reconstruct the high-resolution hyperspectral image X in the non-overlapping area r .

8. The method for fusion of hyperspectral and multispectral images based on dictionary expansion as claimed in claim 7, characterized in that: The sparse coefficient A of the non-overlapping area r The solution process includes: The overlapping region spectral dictionary D s As the non-overlapping region spectral dictionary D r The initial value of the non-overlapping region is calculated by using the convergence iteration strategy. r Solution: Where H = [I (N-M)×(N-M) 0 (N-M)×n ] T is a block matrix, simplifying the expression of constraints, D l =LD r For abbreviation, The singular value decomposition of U,∑ and V,E=A r and represents the splitting variable, V1 and V2 are Lagrange multipliers, μ1>0 is the Lagrange multiplier, soft(u;v) is the soft threshold function, and λ2 and γ2 are regularization parameters.

9. The method for fusion of hyperspectral and multispectral images based on dictionary expansion according to claim 7, characterized in that: The extended non-overlapping region spectral dictionary D r The solution process includes: The extended non-overlapping region spectral dictionary D r Perform an iterative solution: It turns out that: Where G = D r [A s A r ] represents a split variable, is an abbreviation, V3 is the Lagrange multiplier, μ2>0 is the Lagrange multiplier, and η2 is the regularization parameter.

10. A hyperspectral and multispectral image fusion system based on dictionary expansion, characterized in that: Includes the following modules: Module 1: Used to obtain high-resolution multispectral image Y m and low-resolution hyperspectral image Y in incompletely overlapping scenes h , based on the low-resolution hyperspectral image Y h , used to transform the high-resolution multispectral image Y m Divided into overlapping high-resolution multispectral images With non-overlapping high-resolution multispectral images Module 2: for the low-resolution hyperspectral image Y h and overlapping high-resolution multispectral images The spectral dictionary D in the overlapping area is solved by using a fusion algorithm based on spectral dictionary learning. s and the overlap area sparse coefficient A s , and used to fuse and reconstruct the high-resolution hyperspectral images X in the overlapping areas s ; Module 3: Used to extract the overlapping high-resolution multispectral images using a common spectrum extraction strategy and spectrum similarity With non-overlapping high-resolution multispectral images Common spectrum Module 4: Using the common spectrum Expand the spectral dictionary and add the spectral dictionary D s As the initial value of the non-overlapping region spectral dictionary, the extended non-overlapping region spectral dictionary D is obtained by joint optimization under sparse and low-rank constraints. r and the non-overlapping sparse coefficient A r , used to reconstruct high-resolution hyperspectral images X in non-overlapping areas r ; Module 5: Used to convert the overlapping area high-resolution hyperspectral image X s and the non-overlapping region high-resolution hyperspectral image X r Synthesize to obtain the high-resolution multispectral image Y m High-resolution hyperspectral image X covering the entire area.

Images