A method and device for spectral image super-resolution

Abstract

The present invention discloses a method and apparatus for spectral image super-resolution, relating to the field of remote sensing image processing technology, to address the problem of poor accuracy in spectral super-resolution in reflecting ground object images in the prior art. The method comprises: obtaining a first image and a second image covering the imaging coverage of a target ground object, wherein the first image is a first hyperspectral image and a first multispectral image having the same imaging coverage of the ground object, and the second image is a second multispectral image of the area not covered by the hyperspectral image; based on the first image, optimizing the spectral dictionary according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; based on the target spectral dictionary and the second image, determining a target coefficient matrix according to a non-local sparse low-rank learning strategy; based on the target spectral dictionary and the target coefficient matrix, obtaining a second hyperspectral image covering the same area as the second multispectral image; thereby improving the accuracy of spectral super-resolution in reflecting ground object images.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a method and device for spectral image super-resolution. Background Art

[0002] Hyperspectral remote sensing image processing technology captures the rich electromagnetic wave signals reflected by ground objects and generates hundreds or even thousands of spectral bands, covering visible light, near-infrared, and far-infrared bands. This allows the detailed spectral characteristics of ground objects to be captured. Hyperspectral remote sensing image processing technology has been widely used in fields such as mineral identification and ground object classification. However, due to limitations in optical imaging hardware, hyperspectral satellite imaging is limited in imaging width and spatial resolution, making it difficult to simultaneously obtain hyperspectral images with both wide width and high spatial resolution. Compared to hyperspectral images, multispectral remote sensing images typically contain only a few discrete bands and are less able to represent the spectral information of ground objects, but they offer significant advantages in imaging width and spatial resolution. Therefore, by combining the complementary advantages of hyperspectral and multispectral imagery through methods such as super-resolution, higher-resolution images can be generated. This can fully leverage the complementary advantages of multi-source remote sensing data and expand the scope of remote sensing data applications.

[0003] Currently, the mainstream super-resolution methods proposed are divided into spatial image super-resolution variability methods and spectral image super-resolution variability methods. Both methods generate high-spatial-resolution hyperspectral images by combining low-spatial-resolution hyperspectral images with high-spatial-resolution multispectral images. The spatial image super-resolution variability method requires that the hyperspectral and multispectral images have the same surface coverage. Due to the limited imaging width of hyperspectral images, some areas covered by multispectral images may lack corresponding hyperspectral images, resulting in the hyperspectral and multispectral images not having the same surface coverage. In such scenarios, the spatial image super-resolution variability method cannot be used. Furthermore, the spectral image super-resolution variability method is usually targeted at natural images and only utilizes the redundant characteristics of the reconstructed image in the spectral dimension, ignoring the overall redundant characteristics of the reconstructed image in the spatial-spectral dimension. This results in the poor accuracy of the spectral image super-resolution variability method in reflecting ground object images.

[0004] In view of this, there is an urgent need to propose a more advanced spectral super-resolution method to solve the problem of poor accuracy of spectral super-resolution in existing technologies when reflecting ground object images. Summary of the Invention

[0005] The present invention aims to provide a method and apparatus for spectral image super-resolution, which utilizes hyperspectral and multispectral image data with the same ground object imaging coverage area, considers the overall redundancy of the reconstructed hyperspectral image in the spatial spectrum dimension, and reconstructs the hyperspectral image of the ground object that is missing in the hyperspectral image compared with the multispectral image; improves the accuracy of the hyperspectral image of the ground object that is missing in the hyperspectral image compared with the multispectral image; and solves the problem of poor accuracy of spectral super-resolution in reflecting ground object images in the prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for spectral image super-resolution, which may include:

[0008] Acquire a first image and a second image covering the target object, wherein the first image is a first hyperspectral image and a first multispectral image having the same object imaging coverage, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image;

[0009] Based on the first image, the spectral dictionary is optimized according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents a spectral dictionary of hyperspectral images and multispectral images with the same ground object scene coverage;

[0010] Determining a target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy;

[0011] Based on the target spectral dictionary and the target coefficient matrix, a second hyperspectral image is determined; the second hyperspectral image is a hyperspectral image covering the same area as the second multispectral image.

[0012] Preferably, the step of optimizing the spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary may include:

[0013] Sampling the first hyperspectral image to obtain a third hyperspectral image having the same spatial size as the first multispectral image;

[0014] Normalizing the third hyperspectral image and the first multispectral image to obtain a third multispectral image and a fourth hyperspectral image; pixel values ​​of the third multispectral image and the fourth hyperspectral image are target pixel values;

[0015] obtaining a plurality of first three-dimensional image blocks based on the third multispectral image and the fourth hyperspectral image;

[0016] The non-local low-rank spectral dictionary learning strategy is constructed based on multiple groups of the first three-dimensional image blocks.

[0017] Preferably, obtaining multiple groups of first three-dimensional image blocks based on the third multispectral image and the fourth hyperspectral image may include: spectrally superimposing the third multispectral image and the fourth hyperspectral image to obtain an intermediate image; performing three-dimensional tensor folding processing on the intermediate image to obtain a first three-dimensional image; performing image division on the first three-dimensional image to obtain multiple second three-dimensional image blocks; performing similarity grouping on the multiple second three-dimensional image blocks to obtain multiple groups of first target three-dimensional image blocks; the similarity between multiple second three-dimensional image blocks in the multiple groups of first target three-dimensional image blocks is greater than a preset threshold; and spatially stacking multiple second three-dimensional image blocks in the multiple groups of first target three-dimensional image blocks to obtain multiple groups of first three-dimensional image blocks.

[0018] Preferably, constructing a non-local low-rank spectral dictionary learning strategy based on the multiple groups of the first three-dimensional image blocks may include: performing a two-dimensional matrix expansion process on the multiple groups of the first three-dimensional image blocks to obtain multiple first target two-dimensional matrices;

[0019] Based on the first two-dimensional matrix, the formula is:

[0020]

[0021] Construct the non-local low-rank spectral dictionary learning strategy; wherein, D c is the spectral dictionary matrix of the first target two-dimensional matrix, A c is the coefficient matrix of the first target two-dimensional matrix, Y c For the intermediate image, is the coefficient matrix of the kth group of three-dimensional image blocks, symbol ||·|| F is the Fourier norm, the symbol ||·||1 is Norm, symbol ||·|| * is the nuclear norm, Q is the auxiliary variable, is the projection, λ is the regularization parameter, β is the regularization parameter, and γ is the regularization parameter; ||X c ||1 is used to promote the sparsity of the coefficient matrix, Used to promote the nonlocal similarity of the coefficient matrix, ||D c || * To promote the low rank of the spectral dictionary, Q is used as a non-negative constraint, which is given by Promote non-negativity of spectral dictionary matrices.

[0022] Preferably, optimizing the spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary may include: obtaining multiple groups of spectral dictionaries based on the non-local low-rank spectral dictionary learning strategy and combining global correlations and self-similarity of multiple groups of the first three-dimensional image blocks; and combining and reconstructing the multiple groups of spectral dictionaries to obtain the target spectral dictionary.

[0023] Preferably, the determining of the target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy may include:

[0024] Performing three-dimensional tensor folding processing on the second image to obtain a second three-dimensional image;

[0025] performing image segmentation on the second three-dimensional image to obtain a plurality of third three-dimensional image blocks;

[0026] performing similarity grouping on the plurality of third three-dimensional image blocks to obtain a plurality of groups of second target three-dimensional image blocks; wherein the similarity between the plurality of third three-dimensional image blocks in the plurality of groups of the second target three-dimensional image blocks is greater than a preset threshold;

[0027] performing spatial stacking processing on a plurality of the third three-dimensional image blocks in a plurality of groups of the second target three-dimensional image blocks to obtain a plurality of groups of fourth three-dimensional image blocks;

[0028] The non-local sparse low-rank learning strategy is constructed based on multiple groups of the fourth three-dimensional image blocks and the target spectral dictionary.

[0029] Preferably, constructing the non-local sparse low-rank learning strategy based on the target spectral dictionary of the multiple groups of the fourth three-dimensional image blocks may include: performing a two-dimensional matrix expansion process on the multiple groups of the fourth three-dimensional image blocks to obtain a plurality of second target two-dimensional matrices;

[0030] Based on the second target two-dimensional matrix, the formula is:

[0031]

[0032] Construct the non-local sparse low-rank learning strategy; where D H,c is the spectral dictionary of the first hyperspectral image, A s is the coefficient matrix of the second image, μ and κ are regularization parameters, is the coefficient matrix of the second target two-dimensional matrix of the g-th group, symbol ||·|| F is the Fourier norm, the symbol ||·||1 is Norm, symbol ||·|| * is the nuclear norm; ||A s ||1 is used to promote the coefficient matrix As The sparsity, The term is used to promote the coefficient matrix A s Non-local similarity.

[0033] Preferably, determining the target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy may include:

[0034] Based on the non-local low-rank spectral dictionary learning strategy, and in combination with the global correlation, self-similarity, and local low-rank properties of the plurality of fourth three-dimensional image blocks, a plurality of sets of intermediate coefficient matrices are obtained;

[0035] Multiple groups of intermediate coefficient matrices are combined and reconstructed to obtain the target coefficient matrix.

[0036] Preferably, determining the second hyperspectral image based on the target spectral dictionary and the target coefficient matrix may include:

[0037] The target spectral dictionary is multiplied by the target coefficient matrix to obtain the second hyperspectral image.

[0038] In a second aspect, the present invention provides a spectral image super-resolution device, which may include:

[0039] An acquisition module, the acquisition module being configured to acquire a first image and a second image covering the target object, wherein the first image is a first hyperspectral image and a first multispectral image having the same object imaging coverage, and the second image is a second multispectral image of an area not covered by the hyperspectral image; wherein the hyperspectral image is an image of low spatial resolution, and the multispectral image is an image of high spatial resolution;

[0040] an optimization module configured to optimize the spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents a spectral dictionary of hyperspectral images and multispectral images covering the same ground object scene;

[0041] a determination module, configured to determine a target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy;

[0042] An obtaining module is configured to obtain a second hyperspectral image based on the target spectral dictionary and the target coefficient matrix; the second hyperspectral image is a hyperspectral image covering the same area as the second multispectral image.

[0043] Compared with the prior art, the present invention provides a spectral image super-resolution method, which obtains a first image and a second image covering the imaging coverage of a target object, wherein the first image is a first hyperspectral image and a first multispectral image having the same imaging coverage, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image; based on the first image, the spectral dictionary is optimized according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents the spectral dictionary of the hyperspectral image and the multispectral image having the same object scene coverage; based on the target spectral dictionary and the second image, the target coefficient matrix is ​​determined according to a non-local sparse low-rank learning strategy; based on the target spectral dictionary and the second image, a second hyperspectral image is determined based on the target spectral dictionary and the target coefficient matrix; the second hyperspectral image is a hyperspectral image having the same coverage area as the second multispectral image. Based on this, using hyperspectral and multispectral image data with the same ground object imaging coverage area, considering the overall redundancy of the reconstructed hyperspectral image in the spatial spectral dimension, firstly, the target spectral dictionary is obtained according to the non-local low-rank spectral dictionary learning strategy, and further according to the non-local sparse low-rank learning strategy, the target coefficient matrix of the multispectral image in the area not covered by the hyperspectral image is obtained. Based on the target spectral dictionary and the target coefficient matrix, the hyperspectral image of the ground object missing in the hyperspectral image compared with the multispectral image is reconstructed, thereby obtaining a hyperspectral image with higher accuracy, which improves the accuracy of the hyperspectral image of the ground object missing in the hyperspectral image compared with the multispectral image. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0045] Figure 1 A schematic diagram of the main process of a spectral image super-resolution method provided by the present invention;

[0046] Figure 2 A schematic diagram showing the visual comparison between the super-resolution reconstruction results of 8 randomly selected bands and the real image of a spectral image super-resolution method provided by the present invention;

[0047] Figure 3 This is a schematic structural diagram of a spectral image super-resolution device provided by the present invention. DETAILED DESCRIPTION

[0048] To facilitate a clear description of the technical solutions of the embodiments of the present invention, the embodiments of the present invention use terms such as "first" and "second" to distinguish between identical or similar items with substantially the same functions and effects. For example, the first threshold and the second threshold are merely used to distinguish between different thresholds and do not limit their order of precedence. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that terms such as "first" and "second" do not necessarily define differences.

[0049] It should be noted that, in the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present invention should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0050] In the present invention, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects in the preceding time are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, the combination of a and b, the combination of a and c, the combination of b and c, or the combination of a, b and c, where a, b, c can be single or multiple.

[0051] At present, research on multispectral image spectral super-resolution technology is being carried out based on the imaging width and resolution characteristics of remote sensing hyperspectral images and multispectral images. By utilizing hyperspectral and multispectral images with the same ground object imaging coverage area, the spectral resolution of multispectral images that lack hyperspectral imaging coverage can be improved, which can give full play to the complementary advantages of multispectral and hyperspectral images. However, the existing spectral image super-resolution variability methods are usually aimed at natural images, and only use the redundant characteristics of the reconstructed image in the spectral dimension, ignoring the overall redundant characteristics of the reconstructed image in the spatial spectrum dimension, which leads to the problem of poor accuracy of spectral image super-resolution variability in reflecting ground object images.

[0052] In view of this, the present invention provides a method and device for spectral image super-resolution, which utilizes some hyperspectral and multispectral images with the same ground object imaging coverage as basic data, introduces non-local similarity constraints on images in the spatial spectrum dimension under a low-rank dictionary learning framework, and solves the spectral dictionary of hyperspectral images and multispectral images under the same ground object scene coverage through rigorous theoretical derivation; further utilizes the solved spectral dictionary of multispectral images under the same ground object scene coverage as the hyperspectral image, and solves the coefficient matrix of the multispectral image in the scene without hyperspectral image coverage under the non-local low-rank constraint framework; finally, the high spatial resolution hyperspectral image of the target is reconstructed through the spectral dictionary and the coefficient matrix; a more accurate high spatial resolution hyperspectral image is obtained; and the problem of poor accuracy of spectral super-resolution in reflecting ground object images in the prior art is solved.

[0053] Next, the technical solution of the present invention is described in detail with reference to the accompanying drawings:

[0054] In the first aspect, the present invention provides a method for spectral image super-resolution. Figure 1 , Figure 1 This is a schematic diagram of the main process of a spectral image super-resolution method provided by the present invention; its execution entity is a server or terminal device equipped with the technical solution disclosed in the embodiment of the present invention, such as a service platform or handheld device, etc.

[0055] exist Figure 1 In the method, the method may include:

[0056] Step 110: Acquire a first image and a second image covering the target object, wherein the first image is a first hyperspectral image and a first multispectral image having the same object imaging coverage, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image.

[0057] In step 110, three types of image data are acquired: a first hyperspectral image and a first multispectral image having the same ground object imaging coverage, and a second multispectral image of the area not covered by the hyperspectral image. It is understood that before acquiring these image data, a hyperspectral image and a multispectral image of the ground object imaging are acquired, and data processing is performed on each of the hyperspectral image and the multispectral image, including but not limited to image preprocessing operations such as atmospheric correction, radiometric calibration, and image registration. After data processing, the first hyperspectral image and the first multispectral image having the same ground object imaging coverage, as well as the second multispectral image of the area not covered by the hyperspectral image, are obtained, thereby obtaining the basic image data required by the present invention.

[0058] Step 120: Based on the first image, the spectral dictionary is optimized according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents a spectral dictionary of hyperspectral images and multispectral images with the same ground object scene coverage.

[0059] Step 130: Based on the target spectral dictionary and the second image, a target coefficient matrix is ​​determined according to a non-local sparse low-rank learning strategy.

[0060] In steps 120 to 130, since the non-local low-rank spectral dictionary learning strategy and the non-local sparse low-rank learning strategy are learning strategies constructed by considering the overall redundancy in the spatial spectral dimension, the optimization of the spectral dictionary according to the non-local low-rank spectral dictionary learning strategy and the determination of the target coefficient matrix according to the non-local sparse low-rank learning strategy will achieve higher accuracy in reflecting the spectral image of the ground object imaging. The target spectral dictionary is the spectral dictionary corresponding to the hyperspectral image.

[0061] Step 140: Determine a second hyperspectral image based on the target spectral dictionary and the target coefficient matrix; the second hyperspectral image is a hyperspectral image covering the same area as the second multispectral image.

[0062] In step 140, since the target spectral dictionary is a spectral dictionary obtained by optimizing hyperspectral and multispectral image data with the same ground feature imaging coverage area, and the target coefficient matrix is ​​a coefficient matrix constructed based on the target spectral dictionary and the second multispectral image of the area not covered by the hyperspectral image, there is data coupling between the two in spatial latitude, that is, the hyperspectral image with the same coverage area as the second multispectral image can be obtained by solving the target coefficient matrix and the target spectral dictionary.

[0063] Based on this, the present invention provides a spectral image super-resolution method, which considers the overall redundant characteristics of reconstructing hyperspectral images in the spatial spectral dimension, and utilizes hyperspectral and multispectral image data with the same ground object imaging coverage area to construct a non-local low-rank spectral dictionary learning strategy; and based on the second multispectral image of the area not covered by the hyperspectral image, a non-local sparse low-rank learning strategy is constructed; first, a target spectral dictionary is obtained according to the non-local low-rank spectral dictionary learning strategy, and further, a target coefficient matrix of the multispectral image of the area not covered by the hyperspectral image is obtained according to the non-local sparse low-rank learning strategy; finally, based on the target spectral dictionary and the target coefficient matrix, the hyperspectral image of the ground object missing in the hyperspectral image compared with the multispectral image is reconstructed, thereby obtaining a hyperspectral image with higher accuracy, thereby improving the accuracy of the hyperspectral image of the ground object missing in the hyperspectral image compared with the multispectral image.

[0064] It should be noted that the framework adopted in this invention is a non-local low-rank dictionary learning framework, where non-locality refers to the appearance of similar image structures or patterns at different locations in an image, a self-similarity that is widely present in remote sensing images. Due to the wide imaging area of ​​remote sensing images and the characteristics of natural and artificial scenes, hyperspectral and multispectral remote sensing images are prone to repeated similar structural content and texture information, i.e., non-local self-similarity. In different regions of hyperspectral and multispectral remote sensing images, there are many similar image patches, and the tensor composed of these similar patches has a stronger low-rank property. The low-rank representation of the image can project high-dimensional data into a low-dimensional linear subspace, expressing all data with a small number of basis vectors. Hyperspectral images have strong redundancy in the spectral dimension, so using a low-rank representation can well describe the strong correlation and structural information of hyperspectral data in the spectral dimension. For the spectral super-resolution task of multispectral images, although non-local similarity can maintain the global structure, it is insufficient for preserving the local structure of the image and extracting the fine spectral information of hyperspectral images. While a low-rank representation alone can remove redundant information, it also excessively loses spatial and spectral information of image details. Combining the two, non-local similarity clustering is used to obtain similar blocks, enhancing the low-rank characteristics of image blocks, and then low-rank decomposition is performed on each type of similar blocks. This can not only better remove redundant information, but also retain the spatial detail information of the image and reconstruct high-dimensional spectral information.

[0065] Preferably, data preparation and preprocessing may be included before step 110, first performing image preprocessing operations including atmospheric correction, radiation calibration, image registration, etc. on the hyperspectral image and the multispectral image; thereby obtaining a first image and a second image covering the imaging coverage of the target object, the first image being a first hyperspectral image and a first multispectral image having the same imaging coverage of the object, and the second image being a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image.

[0066] For example, high spatial resolution multispectral images can be divided into areas covered by the same imaging as the hyperspectral images. That is, the first multispectral image, which contains l bands and W c pixels; and multispectral images of areas not covered by hyperspectral images That is, the second multispectral image, which contains l bands and W s pixels; the input low spatial resolution hyperspectral image is only c Common coverage area That is, the first hyperspectral image, which contains L bands and w c pixels, where l <L,w s <W s .

[0067] It should be noted that the present invention provides a spectral image super-resolution method, the purpose of which is to use high spatial resolution multispectral image M c , M s and low spatial resolution hyperspectral image H c Considering the overall redundancy of the reconstructed hyperspectral image in the spatial spectrum dimension, the area not covered by the hyperspectral image (i.e., the multispectral image M s High spatial resolution hyperspectral imagery of the area Contains L bands and W s pixels.

[0068] Preferably, before step 120, based on the first image, the spectral dictionary is optimized according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary. This may include: spatially upsampling the first hyperspectral image to obtain a third hyperspectral image of the same spatial size as the first multispectral image; normalizing the third hyperspectral image and the first multispectral image to obtain a third multispectral image and a fourth hyperspectral image; pixel values ​​of the third multispectral image and the fourth hyperspectral image are target pixel values; obtaining multiple groups of first three-dimensional image blocks based on the third multispectral image and the fourth hyperspectral image; and constructing a non-local low-rank spectral dictionary learning strategy based on the multiple groups of first three-dimensional image blocks. Based on this, a non-local low-rank spectral dictionary learning strategy is constructed based on the overall redundancy in the spatial spectral dimension. Compared with the existing technology, this non-local low-rank spectral dictionary learning strategy has higher accuracy.

[0069] Preferably, obtaining multiple groups of first three-dimensional image blocks based on the third multispectral image and the fourth hyperspectral image may include: spectrally superimposing the third multispectral image and the fourth hyperspectral image to obtain an intermediate image; performing three-dimensional tensor folding processing on the intermediate image to obtain a first three-dimensional image; performing image division on the first three-dimensional image to obtain multiple second three-dimensional image blocks; performing similarity grouping on the multiple second three-dimensional image blocks to obtain multiple groups of first target three-dimensional image blocks; the similarity between multiple second three-dimensional image blocks in the multiple groups of first target three-dimensional image blocks is greater than a preset threshold; and spatially stacking multiple second three-dimensional image blocks in the multiple groups of first target three-dimensional image blocks to obtain multiple groups of first three-dimensional image blocks.

[0070] Preferably, constructing a non-local low-rank spectral dictionary learning strategy based on multiple groups of first three-dimensional image blocks may include: performing two-dimensional matrix expansion processing on the multiple groups of first three-dimensional image blocks to obtain multiple first target two-dimensional matrices; based on the multiple first target two-dimensional matrices, using the formula:

[0071]

[0072] Construct a non-local low-rank spectral dictionary learning strategy; where D c is the spectral dictionary matrix of the first target two-dimensional matrix, A c is the coefficient matrix of the first target two-dimensional matrix, Y c For the intermediate image, is the coefficient matrix of the kth group of three-dimensional image blocks, symbol ||·|| F is the Fourier norm, the symbol ||·||1 is Norm, symbol ||·|| * is the nuclear norm, Q is the auxiliary variable, is the projection, λ is the regularization parameter, β is the regularization parameter, and γ is the regularization parameter; ||X c ||1 is used to promote the sparsity of the coefficient matrix, Used to promote the nonlocal similarity of the coefficient matrix, ||D c || * To promote the low rank of the spectral dictionary, Q is used as a non-negative constraint, which is given by Promote non-negativity of spectral dictionary matrices.

[0073] More specifically, the solution step of constructing a non-local low-rank spectral dictionary learning strategy can be decomposed into the following steps:

[0074] First introduce auxiliary variables: Z1=A c , Z2=D c , Z3=D c , and Lagrange multipliers Υ1, Υ2, Υ3, corresponding to auxiliary variables Z1 = A c , Z2=D c , Z3=D c The constraints of , then the Lagrangian function of formula (1) is given as:

[0075]

[0076] The alternating direction multiplier method is used to alternately optimize the variables of the Lagrangian function (2).

[0077] Furthermore, for the spectral dictionary D c , the subproblem corresponding to the Lagrangian function can be defined as:

[0078]

[0079] Let (3) be D c Taking the partial derivative and setting it to 0, we can get:

[0080]

[0081] Formula (4) has D cThe analytical solution can be:

[0082]

[0083] For Y c The coefficient matrix A c , the sub-problem corresponding to the Lagrangian function can be:

[0084]

[0085] Let (6) be A c Taking the partial derivative and setting it to 0, we can get:

[0086]

[0087] Formula (7) has A c The analytical solution can be:

[0088]

[0089] For auxiliary variables The optimization sub-problem is:

[0090]

[0091] Formula (9) is a composite optimization problem, including the Fourier norm, Norm and nuclear norm, first deal with The norm term is processed using soft thresholding to obtain a temporary solution, which can be obtained:

[0092]

[0093] In the above formula is a soft threshold operator, which is defined as for each element x ij .

[0094] x ij The calculation process is:

[0095]

[0096] Where τ is the threshold parameter.

[0097] Furthermore, the singular value threshold (SVT) method is used to process the nuclear norm term for the temporary solution, which requires solving the Z1 group, that is, for the kth group Temporary solution Applying nuclear norm optimization can be:

[0098]

[0099] in, is the singular value threshold operator.

[0100] The specific optimization steps of formula (12) are:

[0101] (1) For the matrix X, first perform singular value decomposition on the matrix: X = UΣV T , matrix U, Σ, V T They are the left singular vector matrix, singular value matrix, and right singular vector matrix respectively.

[0102] (2) Soft threshold processing of singular values: Σ * =diag(max(σ i -τ,0)),σ i is the i-th singular value of the singular value matrix Σ.

[0103] Furthermore, for the auxiliary variable Z2, the optimization sub-problem can be:

[0104]

[0105] Taking the partial derivative of (13) with respect to Z2 and setting it to 0, we can obtain:

[0106] 2Υ(Z2-Q)-ρ2(D c -Z2+Y2)=0 (14)

[0107] Arrange equation (14) and obtain the closed-form solution:

[0108]

[0109] Furthermore, for the auxiliary variable Z3, the optimization subproblem can be:

[0110]

[0111] Formula (16) is a standard nuclear norm problem, which can be solved using the singular value threshold (SVT) according to formula (17):

[0112]

[0113] For Lagrange multipliers Υ1, Υ2, Υ3, the update rules are:

[0114]

[0115] In summary, the algorithm can be terminated by iteratively executing Formula (2) to Formula (18) until convergence or the maximum number of iterations is reached, and the solution is output to obtain a non-local low-rank spectral dictionary learning strategy.

[0116] Furthermore, in step 120, optimizing the spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary may include: obtaining multiple sets of spectral dictionaries based on the non-local low-rank spectral dictionary learning strategy and combining the global correlation and self-similarity of multiple sets of first three-dimensional image blocks; and combining and reconstructing the multiple sets of spectral dictionaries to obtain the target spectral dictionary.

[0117] Based on this, a non-local low-rank spectral dictionary learning strategy is constructed on the basis of considering the overall redundancy characteristics in the empty spectrum dimension. By optimizing the spectral dictionary based on this learning strategy, a hyperspectral dictionary with higher accuracy can be obtained.

[0118] Specifically, for the first input hyperspectral image H c Perform upsampling processing to obtain the same image as the multispectral image M c The third hyperspectral image of the spatial size is obtained Then the third hyperspectral image and the first multispectral image are normalized, and then and M c Normalize and scale the pixel values ​​to the target pixel values ​​to obtain the third multispectral image M 1c and the fourth hyperspectral image The target pixel value is preferably in the range of [0, 1].

[0119] Furthermore, the third multispectral image M 1c and the fourth hyperspectral image Superposition in the spectral dimension gives the intermediate image Y c ,Right now It contains (L+1) bands and W c pixels.

[0120] Y c Fold into a three-dimensional tensor form to obtain the first three-dimensional image Y 1c ,Right now For U c and N c The relationship between them is: U c ×N c =W c ; Then Y 1c The image is divided into p×p×(L+l) three-dimensional image blocks to obtain a plurality of second three-dimensional image blocks.

[0121] Furthermore, the multiple second 3D image blocks are divided into K groups according to their similarity to obtain multiple groups of first target 3D image blocks, wherein the similarity between the multiple second 3D image blocks in the multiple groups of first target 3D image blocks is greater than a preset threshold, such as greater than 95%.

[0122] Furthermore, multiple groups of first target three-dimensional image blocks are spatially stacked to obtain multiple groups of first three-dimensional image blocks. in represents the kth group of similar image blocks, represents the kth similar image block in the kth group i image blocks.

[0123] Further, for multiple groups of the first three-dimensional image blocks Perform two-dimensional matrix expansion processing to obtain multiple first target two-dimensional matrices Based on multiple first target two-dimensional matrices, a non-local low-rank spectral dictionary learning strategy is constructed using formula (1).

[0124] Based on this, we can capture global correlation and self-similarity by enforcing non-local low-rank structure and accurately learn Spectral Dictionary The learned Combined together and reconstructed into Y c Spectral Dictionary Among them D H,c Hyperspectral imagery for solution Spectral dictionary, D M,c To solve the multispectral image M c Spectral dictionary.

[0125] In summary, the present invention The dual constraints of the norm term and the nuclear norm simultaneously exploit the non-local similarity and low-rank properties of the image. In its specific implementation, the image blocks are first grouped according to their similarity based on their structural characteristics, fully exploiting the inherent similarity structure of the image and significantly enhancing the spatial and spectral correlations between the image blocks. Subsequently, a low-rank constraint is introduced to decompose each group of similar image blocks. This dual constraint collaborative mechanism not only effectively removes data redundancy, but also preserves the spatial and spectral structure of the multispectral and hyperspectral images to the greatest extent possible, resulting in high-quality super-resolution reconstructed hyperspectral images.

[0126] Preferably, before step 130, that is, before determining the target coefficient matrix based on the target spectral dictionary and the second image according to the non-local sparse low-rank learning strategy, the process may include: performing three-dimensional tensor folding processing on the second image to obtain a second three-dimensional image; performing image division on the second three-dimensional image to obtain a plurality of third three-dimensional image blocks; performing similarity grouping on the plurality of third three-dimensional image blocks to obtain a plurality of groups of second target three-dimensional image blocks; the similarity between the plurality of third three-dimensional image blocks in the plurality of groups of second target three-dimensional image blocks is greater than a preset threshold; performing spatial stacking processing on the plurality of third three-dimensional image blocks in the plurality of groups of second target three-dimensional image blocks to obtain a plurality of groups of fourth three-dimensional image blocks; and constructing a non-local sparse low-rank learning strategy based on the plurality of groups of fourth three-dimensional image blocks and the target spectral dictionary.

[0127] Preferably, constructing a non-local sparse low-rank learning strategy based on multiple sets of fourth three-dimensional image block target spectral dictionaries may include: performing two-dimensional matrix expansion processing on the multiple sets of fourth three-dimensional image blocks to obtain multiple second target two-dimensional matrices; based on the multiple second target two-dimensional matrices, using the formula:

[0128]

[0129] Construct a non-local sparse low-rank learning strategy; where D H,c is the spectral dictionary of the first hyperspectral image, A s is the coefficient matrix of the second image, μ and κ are regularization parameters, is the coefficient matrix of the second target two-dimensional matrix of the g-th group, symbol ||·|| F is the Fourier norm, the symbol ||·||1 is Norm, symbol ||·|| * is the nuclear norm; ||A s ||1 is used to promote the coefficient matrix A s The sparsity, The term is used to promote the coefficient matrix A s Non-local similarity.

[0130] More specifically, in the solution step of constructing a non-local sparse low-rank learning strategy, it can also be decomposed into the following steps:

[0131] First introduce auxiliary variables: Z4=A s For processing Norm term, Z5=A s Used to process the nuclear norm term, as well as the Lagrange multipliers Υ4, Υ5, and give the Lagrange function of formula (19):

[0132]

[0133] For the coefficient matrix A sTo solve, its sub-problems are:

[0134]

[0135] For the optimization problem in formula (21), the closed-form solution can be obtained by solving the least squares problem:

[0136]

[0137] For the auxiliary variable Z4, by fixing other variables, the subproblem about Z4 is:

[0138]

[0139] Similar to formula (23), the solution for the auxiliary variable Z1 is solved using the soft threshold operator:

[0140]

[0141] For auxiliary variables By fixing other variables, we can get The sub-problems are:

[0142]

[0143] For the problem of solving the nuclear norm in formula (25), the singular value operator is also used to solve it

[0144] For the Lagrange multiplier Υ4, The update rules can be:

[0145]

[0146] In summary, by iteratively executing Formula (20) to Formula (26) until convergence or reaching the maximum number of iterations, the algorithm terminates and outputs the solution, obtaining a non-local sparse low-rank learning strategy.

[0147] Furthermore, after optimizing and solving the non-local low-rank spectral dictionary learning strategy (1) and the non-local sparse low-rank learning strategy (19), the hyperspectral image obtained is Spectral Dictionary D H,c And the multispectral image M obtained s The coefficient matrix A s , we can reconstruct the area not covered by the hyperspectral image (i.e., the multispectral image M s High spatial resolution hyperspectral image X s =D H,c A s .

[0148] Preferably, in step 130, based on the target spectral dictionary and the second image, according to the non-local sparse low-rank learning strategy, determining the target coefficient matrix may include: based on the non-local low-rank spectral dictionary learning strategy, combining the global correlation, self-similarity and local low-rank of multiple groups of fourth three-dimensional image blocks to obtain multiple groups of intermediate coefficient matrices; combining and reconstructing the multiple groups of intermediate coefficient matrices to obtain the target coefficient matrix.

[0149] Specifically, the second multispectral image M s Fold it into a three-dimensional tensor form to obtain the second three-dimensional image M 1s for: For U s and N s , there is U s ×N s =W s ,W s is the multispectral image M 1s The number of pixels contained; further M s The image is divided into a plurality of second three-dimensional image blocks of p×p×l.

[0150] The plurality of second three-dimensional image blocks are divided into G groups according to their similarity to obtain a plurality of groups of second target three-dimensional image blocks; the similarity between the plurality of third three-dimensional image blocks in the plurality of groups of second target three-dimensional image blocks is greater than a preset threshold, for example, the similarity is greater than or equal to 95%.

[0151] Furthermore, multiple sets of second target three-dimensional image blocks are spatially stacked to obtain multiple sets of fourth three-dimensional image blocks. Right now in represents the g-th group of similar image blocks, represents the gth similar image block in the gth group i image blocks.

[0152] Furthermore, for each group of similar image blocks Expand into a two-dimensional matrix form to obtain multiple second target two-dimensional matrices Based on multiple second target two-dimensional matrices, formula (19) is used to construct the non-local sparse low-rank learning strategy.

[0153] Finally, based on the non-local low-rank spectral dictionary learning strategy, the global correlation and self-similarity of the fourth 3D image blocks, as well as the local low-rank property, are captured by enforcing the non-local low-rank structure to accurately solve Multiple sets of intermediate coefficient matrices are finally learned from each set of image blocks. Combined together to reconstruct the target coefficient matrix, the target coefficient matrix is ​​the second multispectral image M s The coefficient matrix As In other words, the multispectral image area M corresponding to the missing hyperspectral image area is obtained compared with the hyperspectral image coverage area. s The coefficient matrix A s .

[0154] Based on this, since the overall redundancy characteristics in the spatial spectral dimension are taken into account in the process of determining the non-local low-rank spectral dictionary learning strategy, compared with the hyperspectral image super-resolution method in the existing technology, which requires consistent imaging coverage of hyperspectral images and multispectral images in the study area, only some areas need to have the registration data of multispectral images and hyperspectral images at the same time, so as to realize high spatial resolution hyperspectral image reconstruction of the area covered only by multispectral images; the reconstructed target coefficient matrix is ​​more accurate when used in the image super-resolution processing process.

[0155] Preferably, in step 140 , determining the second hyperspectral image based on the target spectral dictionary and the target coefficient matrix may include: multiplying the target spectral dictionary and the target coefficient matrix to obtain the second hyperspectral image.

[0156] Specifically, the hyperspectral image obtained by the solution is used Spectral Dictionary D H,c And the multispectral image M obtained s The coefficient matrix A s , reconstruct the area not covered by the hyperspectral image (i.e., the multispectral image M s High spatial resolution hyperspectral image X s =D H,c A s .

[0157] It should be noted that only some areas within the research area of ​​the present invention have both types of data. In order to make full use of the registration data of this partial area, the present invention introduces two prior constraints, non-locality and low rank, and constructs a non-local low rank solution framework. Among them, non-locality can search for similar image blocks within the entire image range, break through the limitations of partial areas, and make full use of all available multispectral image data. This constraint mines the self-similarity within the image and expands the effective sample data. Low rank characterizes the global correlation of hyperspectral images in the spectral domain. Through this constraint, the spectral prior knowledge provided by the hyperspectral images of partial areas can be used to guide the reconstruction of hyperspectral images of the target area.

[0158] Through the organic combination of the non-local low-rank dictionary optimization module and the non-local low-rank sparse matrix optimization module, efficient spectral super-resolution reconstruction is achieved. The dual constraints of the norm term and the nuclear norm simultaneously exploit the non-local similarity and low-rank properties of the images. In its implementation, image blocks are first grouped according to similarity based on their structural features, fully exploiting the inherent similarity structure of the images and significantly enhancing the spatial and spectral correlations between them. Subsequently, a low-rank constraint is introduced to decompose each group of similar image blocks. This dual-constraint synergy not only effectively removes data redundancy but also maximizes the preservation of the spatial and spectral structure of the multispectral and hyperspectral images, resulting in high-quality super-reconstructed hyperspectral images. Therefore, the present invention achieves simultaneous improvements in spatial and spectral resolution through a dual-module optimization strategy. Specifically, a non-local low-rank dictionary optimization module processes the hyperspectral and multispectral image dictionaries in the overlapping regions to obtain rich spectral features. Simultaneously, a non-local low-rank sparse matrix optimization module processes the multispectral image coefficient matrix in regions without hyperspectral coverage to preserve high spatial resolution information. This strategy fully utilizes the spatial information inherent in the coefficient matrix. Since multispectral images typically have higher spatial resolution than hyperspectral images, the obtained coefficient matrix of the multispectral image can well preserve spatial detail information. The hyperspectral image in the overlapping regions provides rich spectral features. By combining the high spatial resolution multispectral image coefficient matrix with the high spectral accuracy spectral dictionary matrix, this method not only achieves the reconstruction of the target area hyperspectral image, but also significantly improves the spatial resolution of the reconstruction results and improves the accuracy.

[0159] Further, in order to verify the actual effect of the method of this embodiment, please refer to Figure 2 , Figure 2 A schematic diagram showing the visual comparison between 8 randomly selected bands of the super-resolution reconstruction results of a spectral image super-resolution method provided by the present invention and the real image.

[0160] Figure 2 The data corresponding to the image shown in the figure is the commonly used public dataset Indiane Pine, and the data from this dataset is used to verify the spectral image super-resolution method provided by the present invention. It should be noted that this dataset is hyperspectral data with 220 spectral bands and a spatial resolution of 145×145. To be suitable for the super-resolution task of the present invention, the spectral response function of Sentinel-2 is applied to this hyperspectral data to obtain a 10-band multispectral image. The 145×80 pixel area of ​​the dataset is then divided into an area covered by both hyperspectral and multispectral images, and the remaining 145×65 pixel area is used as a multispectral image-only area. The hyperspectral image of this area is reconstructed using the method proposed in the present invention, and the hyperspectral image of this area is used as the real image for comparison.

[0161] exist Figure 2In the figure, the upper row of images are the images corresponding to the 8 bands of the real hyperspectral image; the lower row are the images corresponding to the 8 bands of the hyperspectral image reconstructed using the spectral image super-resolution method provided by the present invention. Figure 2 The visualization of the reconstruction results shown in the figure can be obtained: by randomly selecting 8 bands, the band diagrams of the real image and the reconstructed image are drawn. The reconstructed image is very close to the real image in space and spectrum, and the accuracy of the image is guaranteed while achieving effective reconstruction of the missing hyperspectral image. It can be seen that the spectral image super-resolution method provided by the present invention is effective, improves the accuracy of spectral super-resolution in reflecting ground object images, and solves the problem of poor accuracy of spectral super-resolution in reflecting ground object images in the prior art.

[0162] In the second aspect, the present invention provides a spectral image super-resolution device, see Figure 3 , Figure 3 This is a schematic structural diagram of a spectral image super-resolution device provided by the present invention.

[0163] exist Figure 3 In the embodiment, the apparatus may include:

[0164] The acquisition module 310 is used to acquire a first image and a second image covering the imaging coverage of the target object, where the first image is a first hyperspectral image and a first multispectral image having the same imaging coverage of the object, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image.

[0165] The optimization module 320 is configured to optimize the spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents a spectral dictionary of hyperspectral images and multispectral images with the same ground object scene coverage.

[0166] The determination module 330 is configured to determine a target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy.

[0167] The obtaining module 340 is configured to obtain a second hyperspectral image based on the target spectral dictionary and the target coefficient matrix; the second hyperspectral image is a hyperspectral image covering the same area as the second multispectral image.

[0168] Based on this, the present invention provides a spectral image super-resolution device, which obtains a first image and a second image covering the imaging coverage of a target object through an acquisition module 310, wherein the first image is a first hyperspectral image and a first multispectral image having the same imaging coverage of the object, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image; and then the optimization module 320 optimizes the spectral dictionary based on the first image according to the non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents the spectral dictionary of the hyperspectral image and the multispectral image having the same scene coverage of the object. Dictionary; further utilizing the determination module 330 to determine the target coefficient matrix based on the target spectral dictionary and the second image according to the non-local sparse low rank learning strategy; finally, adopting the obtaining module 340 to obtain the second hyperspectral image based on the target spectral dictionary and the target coefficient matrix; the second hyperspectral image is a hyperspectral image covering the same area as the second multispectral image; thus, based on the organic combination of the non-local low rank dictionary optimization module and the non-local low rank sparse matrix optimization module, only part of the area needs to have the registration data of the multispectral image and the hyperspectral image at the same time, so as to realize the reconstruction of the high spatial resolution hyperspectral image of the area covered only by the multispectral image, and improve the accuracy of the ground object image.

[0169] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art can understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0170] Although the present invention has been described with reference to specific features and embodiments thereof, it will be apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the invention. Accordingly, this specification and drawings are merely illustrative of the invention as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the invention. It will be apparent that various modifications and variations may be made to the present invention by those skilled in the art without departing from the spirit and scope of the invention. Thus, the present invention is intended to include such modifications and variations as fall within the scope of the claims of the present invention and their equivalents.

Claims

1. A spectral image super-resolution method, characterized in that: include: Acquire a first image and a second image covering the target object, wherein the first image is a first hyperspectral image and a first multispectral image having the same object imaging coverage, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image; Based on the first image, a spectral dictionary is optimized according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents a spectral dictionary of a hyperspectral image and a multispectral image having the same ground object scene coverage; the optimization of the spectral dictionary according to the non-local low-rank spectral dictionary learning strategy based on the first image to obtain a target spectral dictionary includes: spatially upsampling the first hyperspectral image to obtain a third hyperspectral image having the same spatial size as the first multispectral image; normalizing the third hyperspectral image and the first multispectral image to obtain a third multispectral image and a fourth hyperspectral image; pixel values ​​of the third multispectral image and the fourth hyperspectral image are target pixel values; based on the third multispectral image and the fourth hyperspectral image, multiple groups of first three-dimensional image blocks are obtained; and the non-local low-rank spectral dictionary learning strategy is constructed based on the multiple groups of the first three-dimensional image blocks. The construction of the non-local low-rank spectral dictionary learning strategy based on the multiple groups of the first three-dimensional image blocks includes: Perform two-dimensional matrix expansion processing on multiple groups of the first three-dimensional image blocks to obtain multiple first target two-dimensional matrices; based on the multiple first target two-dimensional matrices, use the formula: Construct the non-local low-rank spectral dictionary learning strategy; wherein, D c is the spectral dictionary matrix of the first target two-dimensional matrix, A c is the coefficient matrix of the first target two-dimensional matrix, Y c For the intermediate image, is the coefficient matrix of the kth group of three-dimensional image blocks, symbol ||·|| F is the Fourier norm, the symbol ||·||1 is Norm, symbol ||·|| * is the nuclear norm, Q is the auxiliary variable, is the projection, λ is the regularization parameter, β is the regularization parameter, and γ is the regularization parameter; ||X c ||1 is used to promote the sparsity of the coefficient matrix, Used to promote the nonlocal similarity of the coefficient matrix, ||D c || * To promote the low rank of the spectral dictionary, Q is used as a non-negative constraint, which is given by Promote the non-negativity of spectral dictionary matrix; Determining a target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy; Based on the target spectral dictionary and the target coefficient matrix, a second hyperspectral image is determined; the second hyperspectral image is a hyperspectral image covering the same area as the second multispectral image.

2. The method according to claim 1, wherein The obtaining of a plurality of first three-dimensional image blocks based on the third multispectral image and the fourth hyperspectral image comprises: performing spectral superposition on the third multispectral image and the fourth hyperspectral image to obtain an intermediate image; performing a three-dimensional tensor folding process on the intermediate image to obtain a first three-dimensional image; performing image segmentation on the first three-dimensional image to obtain a plurality of second three-dimensional image blocks; performing similarity grouping on the plurality of second three-dimensional image blocks to obtain a plurality of groups of first target three-dimensional image blocks; wherein the similarity between the plurality of second three-dimensional image blocks in the plurality of groups of the first target three-dimensional image blocks is greater than a preset threshold; A spatial stacking process is performed on a plurality of the second three-dimensional image blocks in a plurality of groups of the first target three-dimensional image blocks to obtain a plurality of groups of the first three-dimensional image blocks.

3. The method according to claim 1, wherein The step of optimizing the spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary includes: Based on the non-local low-rank spectral dictionary learning strategy, multiple groups of spectral dictionaries are obtained by combining the global correlation and self-similarity of multiple groups of the first three-dimensional image blocks; Multiple groups of spectral dictionaries are combined and reconstructed to obtain the target spectral dictionary.

4. The method according to claim 1, wherein The determining of a target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy includes: Performing three-dimensional tensor folding processing on the second image to obtain a second three-dimensional image; performing image segmentation on the second three-dimensional image to obtain a plurality of third three-dimensional image blocks; performing similarity grouping on the plurality of third three-dimensional image blocks to obtain a plurality of groups of second target three-dimensional image blocks; wherein the similarity between the plurality of third three-dimensional image blocks in the plurality of groups of the second target three-dimensional image blocks is greater than a preset threshold; performing spatial stacking processing on a plurality of the third three-dimensional image blocks in a plurality of groups of the second target three-dimensional image blocks to obtain a plurality of groups of fourth three-dimensional image blocks; The non-local sparse low-rank learning strategy is constructed based on multiple groups of the fourth three-dimensional image blocks and the target spectral dictionary.

5. The method according to claim 4, wherein The constructing of the non-local sparse low-rank learning strategy based on the target spectral dictionary of the plurality of fourth three-dimensional image blocks includes: performing two-dimensional matrix expansion processing on the plurality of groups of the fourth three-dimensional image blocks to obtain a plurality of second target two-dimensional matrices; Based on the second target two-dimensional matrix, the formula is: Construct the non-local sparse low-rank learning strategy; where D H,c is the spectral dictionary of the first hyperspectral image, A s is the coefficient matrix of the second image, μ and κ are regularization parameters, is the coefficient matrix of the second target two-dimensional matrix of the g-th group, symbol ||·|| F is the Fourier norm, symbol ||·||1 is the l1 norm, symbol ||·|| * is the nuclear norm; ||A s ||1 is used to promote the coefficient matrix A s The sparsity, The term is used to promote the coefficient matrix A s Non-local similarity.

6. The method according to claim 5, wherein The determining of a target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy includes: Based on the non-local low-rank spectral dictionary learning strategy, and in combination with the global correlation, self-similarity, and local low-rank properties of the plurality of fourth three-dimensional image blocks, a plurality of sets of intermediate coefficient matrices are obtained; Multiple groups of intermediate coefficient matrices are combined and reconstructed to obtain the target coefficient matrix.

7. The method according to claim 1, wherein The determining of a second hyperspectral image based on the target spectral dictionary and the target coefficient matrix includes: The target spectral dictionary is multiplied by the target coefficient matrix to obtain the second hyperspectral image.

8. A spectral image super-resolution device, characterized in that: include: An acquisition module, the acquisition module being configured to acquire a first image and a second image covering the target object, wherein the first image is a first hyperspectral image and a first multispectral image having the same object imaging coverage, and the second image is a second multispectral image of an area not covered by the hyperspectral image; wherein the hyperspectral image is an image of low spatial resolution, and the multispectral image is an image of high spatial resolution; An optimization module is configured to optimize a spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents a spectral dictionary of a hyperspectral image and a multispectral image having the same ground object scene coverage; the optimization of the spectral dictionary based on the first image according to the non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary includes: spatially upsampling the first hyperspectral image to obtain a third hyperspectral image with the same spatial size as the first multispectral image; normalizing the third hyperspectral image and the first multispectral image to obtain a third multispectral image and a fourth hyperspectral image; pixel values ​​of the third multispectral image and the fourth hyperspectral image are target pixel values; obtaining multiple groups of first three-dimensional image blocks based on the third multispectral image and the fourth hyperspectral image; constructing the non-local low-rank spectral dictionary learning strategy based on the multiple groups of the first three-dimensional image blocks; wherein the constructing of the non-local low-rank spectral dictionary learning strategy based on the multiple groups of the first three-dimensional image blocks includes: Perform two-dimensional matrix expansion processing on multiple groups of the first three-dimensional image blocks to obtain multiple first target two-dimensional matrices; based on the multiple first target two-dimensional matrices, use the formula: Construct the non-local low-rank spectral dictionary learning strategy; wherein, D c is the spectral dictionary matrix of the first target two-dimensional matrix, A c is the coefficient matrix of the first target two-dimensional matrix, Y c For the intermediate image, is the coefficient matrix of the kth group of three-dimensional image blocks, symbol ||·|| F is the Fourier norm, symbol ||·||1 is the l1 norm, symbol ||·|| * is the nuclear norm, Q is the auxiliary variable, is the projection, λ is the regularization parameter, β is the regularization parameter, and γ is the regularization parameter; ||X c ||1 is used to promote the sparsity of the coefficient matrix, Used to promote the nonlocal similarity of the coefficient matrix, ||D c || * To promote the low rank of the spectral dictionary, Q is used as a non-negative constraint, which is given by Promote the non-negativity of spectral dictionary matrix; a determination module, configured to determine a target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy; An obtaining module is configured to obtain a second hyperspectral image based on the target spectral dictionary and the target coefficient matrix; the second hyperspectral image is a hyperspectral image covering the same area as the second multispectral image.

Images