Spectral image super-resolution method and device

Abstract

The invention discloses a spectral image super-resolution method and a spectral image super-resolution device, relates to the technical field of remote sensing image processing, and aims to solve the problem that the accuracy is poor when a ground object image is reflected by spectral super-resolution in the prior art. The method comprises the steps that a first image and a second image covered by target ground object imaging are acquired, the first image is a first hyperspectral image and a first multispectral image which have the same ground object imaging coverage, and the second image is a second multispectral image of an uncovered area of the hyperspectral image; based on the first image, optimizing the spectrum dictionary according to a non-local low-rank spectrum dictionary learning strategy to obtain a target spectrum dictionary; based on the target spectrum dictionary and the second image, determining a target coefficient matrix according to a non-local sparse low-rank learning strategy; based on the target spectrum dictionary and the target coefficient matrix, obtaining a second hyperspectral image having the same coverage area as the second multispectral image; and the accuracy of reflecting the ground object image by the spectral super-resolution is improved.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing image processing, and in particular to a method and a device for super-resolution of spectral images. Background Art

[0002] Hyperspectral remote sensing image processing technology is to obtain rich electromagnetic wave signals reflected by the ground objects, generate hundreds or even hundreds of spectral bands, covering visible light, near infrared to far infrared bands, so as to reflect the detailed spectral characteristics of the ground objects; hyperspectral remote sensing image processing technology has been widely used in the fields of mineral identification and ground object classification. However, due to the limitations of optical imaging hardware facilities, hyperspectral satellite imaging is limited in imaging width and spatial resolution, and it is difficult to obtain hyperspectral images with wide width and high spatial resolution at the same time. Compared with hyperspectral images, multispectral remote sensing images usually only contain a few discrete bands, which are insufficient in expressing the spectral information of the ground objects, but have significant advantages in imaging width and spatial resolution. Therefore, by combining the complementary advantages of hyperspectral and multispectral images through super-resolution and other methods, images with higher resolution can be generated, which can give full play to the complementary advantages of multi-source remote sensing data and broaden the application scope of remote sensing data.

[0003] At present, the mainstream super-resolution methods proposed are divided into spatial image super-resolution variability method and spectral image super-resolution variability method, both of which generate high-spatial resolution hyperspectral images by integrating low-spatial resolution hyperspectral images and high-spatial resolution multispectral images; among them, the spatial image super-resolution variability method requires that the hyperspectral and multispectral images have the same surface coverage. Due to the limitation of the imaging width of hyperspectral images, there will be a lack of corresponding hyperspectral images in some areas covered by multispectral images, resulting in the hyperspectral and multispectral images not having the same surface coverage; in such scenarios, the spatial image super-resolution variability method cannot be used; and the spectral image super-resolution variability method is usually aimed at natural images, only using the redundant characteristics of the reconstructed image in the spectral dimension, ignoring the overall redundant characteristics of the reconstructed image in the spatial spectrum dimension, resulting in the poor accuracy of the spectral image super-resolution variability in reflecting the ground object image.

[0004] In view of this, there is an urgent need to propose a more advanced spectral super-resolution method to solve the problem of poor accuracy of spectral super-resolution in existing technologies when reflecting ground object images. Summary of the invention

[0005] The present invention aims to provide a method and device for spectral image super-resolution, which utilizes hyperspectral and multispectral image data with the same ground object imaging coverage area, considers the overall redundant characteristics of reconstructing hyperspectral images in the spatial-spectral dimension, and reconstructs the hyperspectral images of ground objects that are missing in the hyperspectral images compared with the multispectral images; improves the accuracy of the hyperspectral images of ground objects that are missing in the hyperspectral images compared with the multispectral images; and solves the problem of poor accuracy of spectral super-resolution in reflecting ground object images in the prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for spectral image super-resolution, which may include:

[0008] Acquire a first image and a second image covering the imaging coverage of the target object, wherein the first image is a first hyperspectral image and a first multispectral image having the same imaging coverage of the object, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image;

[0009] Based on the first image, the spectral dictionary is optimized according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents a spectral dictionary of a hyperspectral image and a multispectral image with the same ground object scene coverage;

[0010] Based on the target spectral dictionary and the second image, determining a target coefficient matrix according to a non-local sparse low-rank learning strategy;

[0011] Based on the target spectral dictionary and the target coefficient matrix, a second hyperspectral image is determined; the second hyperspectral image is a hyperspectral image covering the same area as the second multispectral image.

[0012] Preferably, the step of optimizing the spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary may include:

[0013] Sampling the first hyperspectral image to obtain a third hyperspectral image with the same spatial size as the first multispectral image;

[0014] Normalizing the third hyperspectral image and the first multispectral image to obtain a third multispectral image and a fourth hyperspectral image; the pixel values ​​of the third multispectral image and the fourth hyperspectral image are target pixel values;

[0015] Based on the third multispectral image and the fourth hyperspectral image, a plurality of groups of first three-dimensional image blocks are obtained;

[0016] Based on multiple groups of the first three-dimensional image blocks, the non-local low-rank spectral dictionary learning strategy is constructed.

[0017] Preferably, obtaining a plurality of groups of first three-dimensional image blocks based on the third multi-spectral image and the fourth hyperspectral image may include: spectrally superimposing the third multi-spectral image and the fourth hyperspectral image to obtain an intermediate image; performing three-dimensional tensor folding processing on the intermediate image to obtain a first three-dimensional image; performing image division on the first three-dimensional image to obtain a plurality of second three-dimensional image blocks; performing similarity grouping on the plurality of second three-dimensional image blocks to obtain a plurality of groups of first target three-dimensional image blocks; the similarity between the plurality of second three-dimensional image blocks in the plurality of groups of the first target three-dimensional image blocks is greater than a preset threshold; and spatially stacking the plurality of second three-dimensional image blocks in the plurality of groups of the first target three-dimensional image blocks to obtain a plurality of groups of first three-dimensional image blocks.

[0018] Preferably, the constructing of the non-local low-rank spectral dictionary learning strategy based on the multiple groups of the first three-dimensional image blocks may include: performing two-dimensional matrix expansion processing on the multiple groups of the first three-dimensional image blocks to obtain multiple first target two-dimensional matrices;

[0019] Based on the first two-dimensional matrix of multiple targets, the formula is adopted:

[0020]

[0021] Construct the non-local low-rank spectral dictionary learning strategy; wherein D c is the spectral dictionary matrix of the first target two-dimensional matrix, A c is the coefficient matrix of the first target two-dimensional matrix, Y c For the intermediate image, is the coefficient matrix of the kth group of three-dimensional image blocks, symbol ||·|| F is the Fourier norm, the symbol ||·||1 is Norm, symbol ||·|| * is the nuclear norm, Q is the auxiliary variable, is the projection, λ is the regularization parameter, β is the regularization parameter, and γ is the regularization parameter; ||X c ||1 is used to promote the sparsity of the coefficient matrix, To promote the non-local similarity of the coefficient matrix, ||D c || * To promote the low rank of the spectral dictionary, Q is used as a non-negative constraint, given by Promote non-negativity of spectral dictionary matrices.

[0022] Preferably, optimizing the spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary may include: obtaining multiple groups of spectral dictionaries based on the non-local low-rank spectral dictionary learning strategy and combining global correlations and self-similarity of multiple groups of the first three-dimensional image blocks; and combining and reconstructing the multiple groups of spectral dictionaries to obtain the target spectral dictionary.

[0023] Preferably, the step of determining a target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy may include:

[0024] Performing three-dimensional tensor folding processing on the second image to obtain a second three-dimensional image;

[0025] performing image segmentation on the second three-dimensional image to obtain a plurality of third three-dimensional image blocks;

[0026] performing similarity grouping on the plurality of the third three-dimensional image blocks to obtain a plurality of groups of second target three-dimensional image blocks; wherein the similarity between the plurality of third three-dimensional image blocks in the plurality of groups of the second target three-dimensional image blocks is greater than a preset threshold;

[0027] Performing spatial stacking processing on a plurality of the third three-dimensional image blocks in a plurality of groups of the second target three-dimensional image blocks to obtain a plurality of groups of fourth three-dimensional image blocks;

[0028] The non-local sparse low-rank learning strategy is constructed based on multiple groups of the fourth three-dimensional image blocks and the target spectral dictionary.

[0029] Preferably, constructing the non-local sparse low-rank learning strategy based on the target spectral dictionary of the multiple groups of the fourth three-dimensional image blocks may include: performing two-dimensional matrix expansion processing on the multiple groups of the fourth three-dimensional image blocks to obtain multiple second target two-dimensional matrices;

[0030] Based on the second target two-dimensional matrix, the formula is:

[0031]

[0032] Construct the non-local sparse low-rank learning strategy; where D H,c is the spectral dictionary of the first hyperspectral image, A s is the coefficient matrix of the second image, μ and κ are regularization parameters, is the coefficient matrix of the second target two-dimensional matrix of the g-th group, symbol ||·|| F is the Fourier norm, the symbol ||·||1 is Norm, symbol ||·|| * is the nuclear norm; ||A s ||1 is used to promote the coefficient matrix As The sparsity of The term is used to promote the coefficient matrix A s Non-local similarity.

[0033] Preferably, determining the target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy may include:

[0034] Based on the non-local low-rank spectral dictionary learning strategy, multiple groups of intermediate coefficient matrices are obtained by combining the global correlation, self-similarity and local low-rank of multiple groups of the fourth three-dimensional image blocks;

[0035] Multiple groups of intermediate coefficient matrices are combined and reconstructed to obtain the target coefficient matrix.

[0036] Preferably, determining the second hyperspectral image based on the target spectral dictionary and the target coefficient matrix may include:

[0037] The target spectral dictionary is multiplied by the target coefficient matrix to obtain the second hyperspectral image.

[0038] In a second aspect, the present invention provides a spectral image super-resolution device, which may include:

[0039] An acquisition module, the acquisition module is used to acquire a first image and a second image covered by the imaging of the target object, the first image is a first hyperspectral image and a first multispectral image having the same imaging coverage of the object, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image;

[0040] An optimization module, wherein the optimization module is used to optimize the spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents a spectral dictionary of a hyperspectral image and a multispectral image with the same ground object scene coverage;

[0041] A determination module, the determination module is used to determine a target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy;

[0042] The obtaining module is used to obtain a second hyperspectral image based on the target spectral dictionary and the target coefficient matrix; the second hyperspectral image is a hyperspectral image with the same coverage area as the second multispectral image.

[0043] Compared with the prior art, the present invention provides a spectral image super-resolution method, which obtains a first image and a second image covered by the imaging of the target object, wherein the first image is a first hyperspectral image and a first multispectral image with the same imaging coverage of the object, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image; based on the first image, the spectral dictionary is optimized according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents the spectral dictionary of the hyperspectral image and the multispectral image with the same object scene coverage; based on the target spectral dictionary and the second image, the target coefficient matrix is ​​determined according to a non-local sparse low-rank learning strategy; based on the target spectral dictionary and the second image, the second hyperspectral image is determined based on the target spectral dictionary and the target coefficient matrix; the second hyperspectral image is a hyperspectral image with the same coverage area as the second multispectral image. Based on this, using hyperspectral and multispectral image data with the same ground object imaging coverage area, considering the overall redundant characteristics of reconstructed hyperspectral images in the spatial spectral dimension, firstly, the target spectral dictionary is obtained according to the non-local low-rank spectral dictionary learning strategy, and further according to the non-local sparse low-rank learning strategy, the target coefficient matrix of the multispectral image in the area not covered by the hyperspectral image is obtained. Based on the target spectral dictionary and the target coefficient matrix, the hyperspectral image of the ground objects missing in the hyperspectral image compared with the multispectral image is reconstructed, thereby obtaining a hyperspectral image with higher accuracy, which improves the accuracy of the hyperspectral image of the ground objects missing in the hyperspectral image compared with the multispectral image. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0045] Figure 1 A schematic diagram of the main process of a spectral image super-resolution method provided by the present invention;

[0046] Figure 2 A schematic diagram showing a visual comparison between 8 randomly selected bands of a spectral image super-resolution reconstruction result and a real image provided by the present invention;

[0047] Figure 3 A schematic diagram of the structure of a spectral image super-resolution device provided by the present invention. DETAILED DESCRIPTION

[0048] In order to facilitate the clear description of the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, the words "first", "second" and the like are used to distinguish the same items or similar items with substantially the same functions and effects. For example, the first threshold and the second threshold are only used to distinguish different thresholds, and do not limit their order of precedence. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not necessarily limit them to be different.

[0049] It should be noted that, in the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0050] In the present invention, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects in the previous time are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, the combination of a and b, the combination of a and c, the combination of b and c, or the combination of a, b and c, where a, b, c can be single or multiple.

[0051] At present, according to the imaging width and resolution characteristics of remote sensing hyperspectral images and multispectral images, research on spectral super-resolution technology of multispectral images is carried out. By utilizing hyperspectral and multispectral images with the same ground object imaging coverage area, the spectral resolution of multispectral images that lack hyperspectral imaging coverage can be improved, which can give full play to the complementary advantages of multispectral and hyperspectral images. However, the existing spectral image super-resolution variability methods are usually aimed at natural images, and only use the redundant characteristics of the reconstructed image in the spectral dimension, ignoring the overall redundant characteristics of the reconstructed image in the spatial spectrum dimension, which leads to the problem of poor accuracy of spectral image super-resolution variability in reflecting ground object images.

[0052] In view of this, the present invention provides a method and device for spectral image super-resolution, which utilizes part of hyperspectral and multispectral images with the same ground object imaging coverage as basic data, introduces non-local similarity constraints on images in the spatial spectrum dimension under the framework of low-rank dictionary learning, and solves the spectral dictionary of hyperspectral images and multispectral images under the same ground object scene coverage through rigorous theoretical derivation; further utilizes the solved spectral dictionary of multispectral images under the same ground object scene coverage as the hyperspectral image to solve the coefficient matrix of the multispectral image in the scene without hyperspectral image coverage under the framework of non-local low-rank constraints; finally, reconstructs a high spatial resolution hyperspectral image of the target through the spectral dictionary and the coefficient matrix; obtains a more accurate high spatial resolution hyperspectral image; solves the problem of poor accuracy of spectral super-resolution in reflecting ground object images in the prior art.

[0053] Next, the technical solution of the present invention is described in detail with reference to the accompanying drawings:

[0054] In a first aspect, the present invention provides a method for spectral image super-resolution. Figure 1 , Figure 1 A schematic diagram of the main process of a spectral image super-resolution method provided by the present invention; its execution subject is a server or terminal device equipped with the technical solution disclosed in the embodiment of the present invention, such as a service platform or a handheld device, etc.

[0055] exist Figure 1 In, the method may include:

[0056] Step 110: Acquire a first image and a second image covering the imaging coverage of the target object, wherein the first image is a first hyperspectral image and a first multispectral image having the same imaging coverage of the object, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image.

[0057] In step 110, three types of image data are obtained, namely, a first hyperspectral image and a first multispectral image with the same object imaging coverage, and a second multispectral image of the area not covered by the hyperspectral image. It can be understood that before obtaining these image data, the hyperspectral image and the multispectral image of the object imaging are respectively obtained, and the hyperspectral image and the multispectral image are respectively processed, including but not limited to atmospheric correction, radiation calibration, image registration and other image preprocessing operations; after data processing, the first hyperspectral image and the first multispectral image with the same object imaging coverage, and the second multispectral image of the area not covered by the hyperspectral image can be obtained; thereby obtaining the basic image data required by the present invention.

[0058] Step 120: Based on the first image, the spectral dictionary is optimized according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents a spectral dictionary of a hyperspectral image and a multispectral image covering the same ground object scene.

[0059] Step 130: Based on the target spectral dictionary and the second image, determine a target coefficient matrix according to a non-local sparse low-rank learning strategy.

[0060] In step 120 to step 130, since the non-local low-rank spectral dictionary learning strategy and the non-local sparse low-rank learning strategy are learning strategies constructed by considering the overall redundancy characteristics in the space spectrum dimension, the spectral dictionary is optimized according to the non-local low-rank spectral dictionary learning strategy and the target coefficient matrix is ​​determined according to the non-local sparse low-rank learning strategy, which will be more accurate in reflecting the spectral image of the ground object imaging. Among them, the target spectral dictionary is the spectral dictionary corresponding to the hyperspectral image.

[0061] Step 140: Determine a second hyperspectral image based on the target spectral dictionary and the target coefficient matrix; the second hyperspectral image is a hyperspectral image covering the same area as the second multispectral image.

[0062] In step 140, since the target spectral dictionary is a spectral dictionary obtained by optimizing the hyperspectral and multispectral image data with the same ground feature imaging coverage area, and the target coefficient matrix is ​​a coefficient matrix constructed based on the target spectral dictionary and the second multispectral image of the area not covered by the hyperspectral image, there is data coupling between the two in spatial latitude, that is, the hyperspectral image with the same coverage area as the second multispectral image can be obtained by solving the target coefficient matrix and the target spectral dictionary.

[0063] Based on this, the present invention provides a spectral image super-resolution method, which considers the overall redundant characteristics of reconstructing hyperspectral images in the spatial spectrum dimension, and constructs a non-local low-rank spectral dictionary learning strategy using hyperspectral and multispectral image data with the same ground object imaging coverage area; and constructs a non-local sparse low-rank learning strategy based on the second multispectral image of the area not covered by the hyperspectral image; firstly, a target spectral dictionary is obtained according to the non-local low-rank spectral dictionary learning strategy, and further, a target coefficient matrix of the multispectral image of the area not covered by the hyperspectral image is obtained according to the non-local sparse low-rank learning strategy; finally, based on the target spectral dictionary and the target coefficient matrix, the hyperspectral image of the ground object missing in the hyperspectral image compared with the multispectral image is reconstructed, thereby obtaining a hyperspectral image with higher accuracy, and improving the accuracy of the hyperspectral image of the ground object missing in the hyperspectral image compared with the multispectral image.

[0064] It should be noted that the framework adopted in the present invention is a non-local low-rank dictionary learning framework, wherein non-locality is the appearance of similar image structures or patterns at different positions in the image, which is a self-similarity widely present in remote sensing images. Due to the wide imaging area of ​​remote sensing images, as well as the characteristics of natural scenes and artificial scenes, hyperspectral and multispectral remote sensing images widely have repeated similar structural content and texture information, i.e., non-local self-similarity. In different regions of hyperspectral and multispectral remote sensing images, there are many similar image blocks, and the tensors composed of these similar blocks have stronger low-rank characteristics. The low-rank representation of the image can project high-dimensional data into a low-dimensional linear subspace, and express all data with a small number of basis vectors. Hyperspectral images have strong redundancy characteristics in the spectral dimension, so the use of low-rank representation is a good description of the strong correlation and structural information of hyperspectral data in the spectral dimension. For the spectral super-resolution task of multispectral images, although non-local similarity can maintain the global structure, it is insufficient for preserving the local structure of the image and finely extracting the spectral information of the hyperspectral image. Although a single low-rank representation can remove redundant information, it will also excessively lose the spatial and spectral information of the image details. Combining the two, non-local similarity clustering is used to obtain similar blocks, the low-rank characteristics of image blocks are enhanced, and then each type of similar blocks is decomposed into low-rank blocks. This can not only better remove redundant information, but also retain the spatial detail information of the image and reconstruct high-dimensional spectral information.

[0065] Preferably, data preparation and preprocessing may be included before step 110, firstly, image preprocessing operations including atmospheric correction, radiation calibration, image registration, etc. are performed on the hyperspectral image and the multispectral image; thereby obtaining a first image and a second image covering the imaging of the target object, wherein the first image is a first hyperspectral image and a first multispectral image having the same imaging coverage of the object, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image.

[0066] For example, high spatial resolution multispectral images can be divided into areas that are imaged together with hyperspectral images. That is, the first multispectral image, which contains l bands and W c pixels; and multispectral images of areas not covered by hyperspectral images That is, the second multispectral image, which contains l bands and W s pixels; the input low spatial resolution hyperspectral image has only c Common coverage area That is, the first hyperspectral image, which contains L bands and w c pixels, where l <L,w s <W s .

[0067] It should be noted that the present invention provides a spectral image super-resolution method, the purpose of which is to use high spatial resolution multispectral image M c , M s and low spatial resolution hyperspectral images H c Considering the overall redundancy of the reconstructed hyperspectral image in the spatial spectrum dimension, the area not covered by the hyperspectral image (i.e., the multispectral image M s High spatial resolution hyperspectral image of the area Contains L bands and W s pixels.

[0068] Preferably, before step 120, based on the first image, according to the non-local low-rank spectral dictionary learning strategy, the spectral dictionary is optimized to obtain a target spectral dictionary, which may include: spatially upsampling the first hyperspectral image to obtain a third hyperspectral image with the same spatial size as the first multispectral image; normalizing the third hyperspectral image and the first multispectral image to obtain a third multispectral image and a fourth hyperspectral image; the pixel values ​​of the third multispectral image and the fourth hyperspectral image are target pixel values; based on the third multispectral image and the fourth hyperspectral image, multiple groups of first three-dimensional image blocks are obtained; based on the multiple groups of first three-dimensional image blocks, a non-local low-rank spectral dictionary learning strategy is constructed. Based on this, a non-local low-rank spectral dictionary learning strategy is constructed on the basis of considering the overall redundancy characteristics in the spatial spectrum dimension, and the accuracy of the non-local low-rank spectral dictionary learning strategy is higher than that of the prior art.

[0069] Preferably, obtaining multiple groups of first three-dimensional image blocks based on the third multispectral image and the fourth hyperspectral image may include: spectrally superimposing the third multispectral image and the fourth hyperspectral image to obtain an intermediate image; performing three-dimensional tensor folding processing on the intermediate image to obtain a first three-dimensional image; performing image division on the first three-dimensional image to obtain multiple second three-dimensional image blocks; performing similarity grouping on the multiple second three-dimensional image blocks to obtain multiple groups of first target three-dimensional image blocks; the similarity between multiple second three-dimensional image blocks in the multiple groups of first target three-dimensional image blocks is greater than a preset threshold; and spatially stacking multiple second three-dimensional image blocks in the multiple groups of first target three-dimensional image blocks to obtain multiple groups of first three-dimensional image blocks.

[0070] Preferably, constructing a non-local low-rank spectral dictionary learning strategy based on multiple groups of first three-dimensional image blocks may include: performing two-dimensional matrix expansion processing on the multiple groups of first three-dimensional image blocks to obtain multiple first target two-dimensional matrices; based on the multiple first target two-dimensional matrices, using the formula:

[0071]

[0072] Construct a non-local low-rank spectral dictionary learning strategy; where D c is the spectral dictionary matrix of the first target two-dimensional matrix, A c is the coefficient matrix of the first target two-dimensional matrix, Y c For the intermediate image, is the coefficient matrix of the kth group of three-dimensional image blocks, symbol ||·|| F is the Fourier norm, the symbol ||·||1 is Norm, symbol ||·|| * is the nuclear norm, Q is the auxiliary variable, is the projection, λ is the regularization parameter, β is the regularization parameter, and γ is the regularization parameter; ||X c ||1 is used to promote the sparsity of the coefficient matrix, To promote the non-local similarity of the coefficient matrix, ||D c || * To promote the low rank of the spectral dictionary, Q is used as a non-negative constraint, given by Promote non-negativity of spectral dictionary matrices.

[0073] More specifically, in the solution step of constructing a non-local low-rank spectral dictionary learning strategy, it can also be decomposed into the following steps:

[0074] First introduce auxiliary variables: Z1 = A c , Z2=D c , Z3=D c , and Lagrange multipliers Υ1, Υ2, Υ3, corresponding to auxiliary variables Z1 = A c , Z2=D c , Z3=D c The constraints are given in the following formula (1):

[0075]

[0076] The alternating direction multiplier method is used to alternately optimize the variables of the Lagrangian function (2).

[0077] Furthermore, for the spectral dictionary D c , the subproblem corresponding to the Lagrangian function can be defined as:

[0078]

[0079] Let (3) be equal to D c Taking the partial derivative and setting it to 0, we can get:

[0080]

[0081] Formula (4) has D cThe analytical solution can be:

[0082]

[0083] For Y c The coefficient matrix A c , the sub-problem corresponding to the Lagrangian function can be:

[0084]

[0085] Let (6) be A c Taking the partial derivative and setting it to 0, we can get:

[0086]

[0087] Formula (7) has A c The analytical solution can be:

[0088]

[0089] For auxiliary variables The optimization sub-problem is:

[0090]

[0091] Formula (9) is a composite optimization problem, including the Fourier norm, Norm and nuclear norm, first deal with The norm term is processed using a soft threshold operation to obtain a temporary solution, which can be obtained:

[0092]

[0093] In the above formula is a soft threshold operator, which is defined as ij .

[0094] x ij The calculation process is:

[0095]

[0096] Where τ is the threshold parameter.

[0097] Furthermore, the singular value threshold (SVT) method is used to process the nuclear norm term for the temporary solution. It is necessary to solve the Z1 grouping, that is, for the kth group Temporary solution Applying nuclear norm optimization can be:

[0098]

[0099] in, is the singular value threshold operator.

[0100] The specific optimization steps of formula (12) are:

[0101] (1) For the matrix X, first perform singular value decomposition on the matrix: X = UΣV T , matrix U, Σ, V T They are the left singular vector matrix, the singular value matrix, and the right singular vector matrix respectively.

[0102] (2) Soft threshold processing of singular values: Σ * =diag(max(σ i -τ,0)),σ i is the i-th singular value of the singular value matrix Σ.

[0103] Furthermore, for the auxiliary variable Z2, its optimization subproblem can be:

[0104]

[0105] Taking the partial derivative of (13) with respect to Z2 and setting it equal to 0, we can obtain:

[0106] 2Υ(Z2-Q)-ρ2(D c -Z2+Y2)=0 (14)

[0107] Arrange equation (14) to obtain the closed-form solution:

[0108]

[0109] Furthermore, for the auxiliary variable Z3, its optimization subproblem can be:

[0110]

[0111] Formula (16) is a standard nuclear norm solution problem, which can be solved using the singular value threshold (SVT) according to formula (17):

[0112]

[0113] For Lagrange multipliers Υ1, Υ2, Υ3, the update rules are:

[0114]

[0115] In summary, the algorithm can be terminated by iteratively executing formula (2) to formula (18) until convergence or the maximum number of iterations is reached, and the solution is output to obtain a non-local low-rank spectral dictionary learning strategy.

[0116] Further, in step 120, based on the first image, according to the non-local low-rank spectral dictionary learning strategy, the spectral dictionary is optimized to obtain a target spectral dictionary, which may include: based on the non-local low-rank spectral dictionary learning strategy, combining the global correlation and self-similarity of multiple groups of first three-dimensional image blocks to obtain multiple groups of spectral dictionaries; combining and reconstructing the multiple groups of spectral dictionaries to obtain the target spectral dictionary.

[0117] Based on this, a non-local low-rank spectral dictionary learning strategy is constructed on the basis of considering the overall redundancy in the empty spectrum dimension. By optimizing the spectral dictionary based on this learning strategy, a hyperspectral dictionary with higher accuracy can be obtained.

[0118] Specifically, for the first input hyperspectral image H c Perform upsampling to obtain the same image as the multispectral image M c The third hyperspectral image of the spatial size is obtained Then the third hyperspectral image and the first multispectral image are normalized. and M c Normalize and scale the pixel values ​​to the target pixel values ​​to obtain the third multispectral image M 1c and the fourth hyperspectral image The target pixel value is preferably in the range of [0, 1].

[0119] Furthermore, the third multispectral image M 1c and the fourth hyperspectral image Superposition in the spectral dimension gives the intermediate image Y c ,Right now It contains (L+l) bands and W c pixels.

[0120] Y c Fold into a three-dimensional tensor form to obtain the first three-dimensional image Y 1c ,Right now For U c and N c The relationship between them is: c ×N c =W c ; Then Y 1c The image is divided into p×p×(L+l) three-dimensional image blocks to obtain a plurality of second three-dimensional image blocks.

[0121] Further, the multiple second 3D image blocks are divided into K groups according to their similarity to obtain multiple groups of first target 3D image blocks, wherein the similarity between the multiple second 3D image blocks in the multiple groups of first target 3D image blocks is greater than a preset threshold, such as greater than 95%.

[0122] Further, multiple groups of first target three-dimensional image blocks are spatially stacked to obtain multiple groups of first three-dimensional image blocks. in represents the kth group of similar image blocks, represents the kth similar image block in the kth group i image blocks.

[0123] Further, for multiple groups of the first three-dimensional image blocks Perform two-dimensional matrix expansion processing to obtain multiple first target two-dimensional matrices Based on multiple of the first target two-dimensional matrices, formula (1) is used to construct a non-local low-rank spectral dictionary learning strategy.

[0124] Based on this, we can capture global correlation and self-similarity by enforcing a non-local low-rank structure and accurately learn Spectral Dictionary The learned Combined together to reconstruct into Y c Spectral Dictionary Where D H,c Hyperspectral images for solving Spectral dictionary, D M,c To solve the multispectral image M c Spectral dictionary.

[0125] In summary, the present invention The dual constraints of the norm term and the nuclear norm simultaneously utilize the non-local similarity and low-rank characteristics of the image. In the specific implementation, the image blocks are first grouped according to similarity based on their structural features, fully exploring the inherent similarity structure of the image and significantly enhancing the spatial and spectral correlation between the image blocks; then, a low-rank constraint is introduced to each group of similar image blocks for decomposition. This dual constraint collaborative mechanism not only effectively removes data redundancy, but also retains the spatial and spectral structure of multispectral images and hyperspectral images to the greatest extent, and can obtain high-quality super-resolution reconstructed hyperspectral images.

[0126] Preferably, before step 130, that is, before determining the target coefficient matrix based on the target spectral dictionary and the second image according to the non-local sparse low-rank learning strategy, the process may include: performing three-dimensional tensor folding processing on the second image to obtain a second three-dimensional image; performing image division on the second three-dimensional image to obtain a plurality of third three-dimensional image blocks; performing similarity grouping on the plurality of third three-dimensional image blocks to obtain a plurality of groups of second target three-dimensional image blocks; the similarity between the plurality of third three-dimensional image blocks in the plurality of groups of second target three-dimensional image blocks is greater than a preset threshold; performing spatial stacking processing on the plurality of third three-dimensional image blocks in the plurality of groups of second target three-dimensional image blocks to obtain a plurality of groups of fourth three-dimensional image blocks; and constructing a non-local sparse low-rank learning strategy based on the plurality of groups of fourth three-dimensional image blocks and the target spectral dictionary.

[0127] Preferably, constructing a non-local sparse low-rank learning strategy based on multiple groups of fourth three-dimensional image block target spectral dictionaries may include: performing two-dimensional matrix expansion processing on multiple groups of fourth three-dimensional image blocks to obtain multiple second target two-dimensional matrices; based on the multiple second target two-dimensional matrices, using the formula:

[0128]

[0129] Construct a non-local sparse low-rank learning strategy; where D H,c is the spectral dictionary of the first hyperspectral image, A s is the coefficient matrix of the second image, μ and κ are regularization parameters, is the coefficient matrix of the second target two-dimensional matrix of the g-th group, symbol ||·|| F is the Fourier norm, the symbol ||·||1 is Norm, symbol ||·|| * is the nuclear norm; ||A s ||1 is used to promote the coefficient matrix A s The sparsity of The term is used to promote the coefficient matrix A s Non-local similarity.

[0130] More specifically, in the solution step of constructing a non-local sparse low-rank learning strategy, it can also be decomposed into the following steps:

[0131] First introduce auxiliary variables: Z4 = A s For processing Norm term, Z5 = A s It is used to process the nuclear norm terms, as well as the Lagrangian multipliers Υ4, Υ5, and gives the Lagrangian function of formula (19):

[0132]

[0133] For the coefficient matrix A sTo solve, its sub-problem is:

[0134]

[0135] For the optimization problem in formula (21), the closed-form solution can be obtained by solving the least squares problem:

[0136]

[0137] For the auxiliary variable Z4, by fixing other variables, the sub-problem about Z4 is:

[0138]

[0139] Similar to equation (23), the solution for the auxiliary variable Z1 is obtained using the soft threshold operator:

[0140]

[0141] For auxiliary variables By fixing other variables, we can get The sub-problems are:

[0142]

[0143] For the problem of solving the nuclear norm in equation (25), the singular value operator is also used to solve it

[0144] For the Lagrange multiplier Υ4, The update rule can be:

[0145]

[0146] In summary, by iteratively executing formula (20) to formula (26) until convergence or reaching the maximum number of iterations, the algorithm terminates and outputs the solution to obtain a non-local sparse low-rank learning strategy.

[0147] Furthermore, after optimizing and solving the non-local low-rank spectral dictionary learning strategy (1) and the non-local sparse low-rank learning strategy (19), the hyperspectral image obtained is Spectral Dictionary D H,c And the obtained multispectral image M s The coefficient matrix A s , we can reconstruct the area not covered by the hyperspectral image (i.e., the multispectral image M s High spatial resolution hyperspectral imagery of the area) s =D H,c A s .

[0148] Preferably, in step 130, based on the target spectral dictionary and the second image, according to a non-local sparse low-rank learning strategy, determining the target coefficient matrix may include: based on the non-local low-rank spectral dictionary learning strategy, combining the global correlation, self-similarity and local low-rank of multiple groups of fourth three-dimensional image blocks to obtain multiple groups of intermediate coefficient matrices; combining and reconstructing the multiple groups of intermediate coefficient matrices to obtain the target coefficient matrix.

[0149] Specifically, the second multispectral image M s Fold it into a three-dimensional tensor form to obtain the second three-dimensional image M 1s for: For U s and N s , there is U s ×N s =W s ,W s is the multispectral image M 1s The number of pixels contained; further M s The second three-dimensional image is divided into a plurality of p×p×l blocks.

[0150] The multiple second three-dimensional image blocks are divided into G groups according to their similarity to obtain multiple groups of second target three-dimensional image blocks; the similarity between multiple third three-dimensional image blocks in the multiple groups of second target three-dimensional image blocks is greater than a preset threshold, for example, the similarity is greater than or equal to 95%.

[0151] Further, multiple groups of second target three-dimensional image blocks are spatially stacked to obtain multiple groups of fourth three-dimensional image blocks. Right now in represents the g-th group of similar image blocks, represents the gth similar image block in the gth group i image blocks.

[0152] Furthermore, for each group of similar image blocks Expand into a two-dimensional matrix form to obtain multiple second target two-dimensional matrices Based on multiple second target two-dimensional matrices, formula (19) is used to construct the non-local sparse low-rank learning strategy.

[0153] Finally, based on the non-local low-rank spectral dictionary learning strategy, the global correlation and self-similarity of the fourth 3D image patch, as well as the local low-rank property, are captured by enforcing the non-local low-rank structure to accurately solve Finally, the learned Combined together to reconstruct the target coefficient matrix, the target coefficient matrix is ​​the second multispectral image M s The coefficient matrix As In other words, the multispectral image area M corresponding to the missing hyperspectral image area is obtained compared with the hyperspectral image coverage area. s The coefficient matrix A s .

[0154] Based on this, since the overall redundancy characteristics in the spatial spectral dimension are taken into account in the process of determining the non-local low-rank spectral dictionary learning strategy, compared with the hyperspectral image super-resolution method in the existing technology that requires consistent imaging coverage of hyperspectral images and multispectral images in the study area, only some areas need to have the registration data of multispectral images and hyperspectral images at the same time, so as to realize high spatial resolution hyperspectral image reconstruction of the area covered only by multispectral images; the reconstructed target coefficient matrix is ​​more accurate when used in the image super-resolution processing process.

[0155] Preferably, in step 140, determining the second hyperspectral image based on the target spectral dictionary and the target coefficient matrix may include: multiplying the target spectral dictionary and the target coefficient matrix to obtain the second hyperspectral image.

[0156] Specifically, the hyperspectral image obtained by the solution is used Spectral Dictionary D H,c And the obtained multispectral image M s The coefficient matrix A s , reconstruct the area not covered by the hyperspectral image (i.e., the multispectral image M s High spatial resolution hyperspectral imagery of the area) s =D H,c A s .

[0157] It should be noted that only some areas in the research area of ​​the present invention have both types of data. In order to make full use of the registration data of this part of the area, the present invention introduces two prior constraints of non-locality and low rank, and constructs a non-local low rank solution framework. Among them, non-locality can search for similar image blocks in the entire image range, break through the limitations of some areas, and make full use of all available multispectral image data. This constraint mines the self-similarity within the image and expands the effective sample data. Low rank characterizes the global correlation of hyperspectral images in the spectral domain. Through this constraint, the spectral prior knowledge provided by the hyperspectral images of some areas can be used to guide the reconstruction of hyperspectral images of the target area.

[0158] Through the organic combination of the non-local low-rank dictionary optimization module and the non-local low-rank sparse matrix optimization module, efficient spectral super-resolution reconstruction is achieved. The dual constraints of the norm term and the nuclear norm simultaneously utilize the non-local similarity and low-rank characteristics of the image. In the specific implementation, the image blocks are first grouped according to similarity based on the structural characteristics, and the intrinsic similarity structure of the image is fully explored, which significantly enhances the spatial and spectral correlations between the image blocks; then the low-rank constraint is introduced into each group of similar image blocks for decomposition. This dual constraint synergy mechanism not only effectively removes data redundancy, but also retains the spatial and spectral structure of the multispectral image and the hyperspectral image to the greatest extent, and can obtain high-quality super-resolution reconstructed hyperspectral images. Therefore, the present invention achieves the simultaneous improvement of spatial resolution and spectral resolution through a dual-module optimization strategy. Specifically, the hyperspectral and multispectral image dictionaries in the overlapping area are processed by the non-local low-rank dictionary optimization module to obtain rich spectral features; at the same time, the multispectral image coefficient matrix in the area without hyperspectral coverage is processed by the non-local low-rank sparse matrix optimization module to retain high spatial resolution information. This strategy makes full use of the spatial information characteristics contained in the coefficient matrix. Since multispectral images usually have higher spatial resolution than hyperspectral images, the coefficient matrix of the multispectral image obtained can well preserve the spatial detail information; and the hyperspectral image in the overlapping area provides rich spectral features. By combining the multispectral image coefficient matrix with high spatial resolution and the spectral dictionary matrix with high spectral accuracy, this method not only realizes the reconstruction of the hyperspectral image of the target area, but also significantly improves the spatial resolution and accuracy of the reconstruction results.

[0159] Further, in order to verify the actual effect of the method of this embodiment, please refer to Figure 2 , Figure 2 A schematic diagram of the visual comparison between 8 randomly selected bands of the super-resolution reconstruction results of a spectral image super-resolution method provided by the present invention and the real image.

[0160] Figure 2 The data corresponding to the image shown in is the commonly used public dataset Indiane Pine, and the data in the dataset is used to verify a spectral image super-resolution method provided by the present invention. It should be noted that the dataset is a hyperspectral data with 220 spectral bands and a spatial resolution of 145×145; in order to be applicable to the super-resolution task of the present invention, the spectral response function of Sentinel-2 is applied to the hyperspectral data to obtain a 10-band multispectral image, and then the 145×80 pixel area of ​​the dataset is divided as a common coverage area of ​​the hyperspectral image and the multispectral image, and the remaining 145×65 pixel area is used as a multispectral image area only; the hyperspectral image of the area is reconstructed using the method proposed in the present invention, and the hyperspectral image of the area is used as a real image for comparison.

[0161] exist Figure 2In the figure, the upper row of images are images corresponding to the 8 bands of the real hyperspectral image; the lower row are images corresponding to the 8 bands of the hyperspectral image reconstructed using a spectral image super-resolution method provided by the present invention. Figure 2 The visualization of the reconstruction result shown can be obtained: by randomly selecting 8 bands, the band diagrams of the real image and the reconstructed image are drawn, and the reconstructed image is very close to the real image in space and spectrum, while achieving effective reconstruction of the missing hyperspectral image while ensuring the accuracy of the image; it can be seen that the method of spectral image super-resolution provided by the present invention is effective, improves the accuracy of spectral super-resolution in reflecting ground object images, and solves the problem of poor accuracy of spectral super-resolution in reflecting ground object images in the prior art.

[0162] In a second aspect, the present invention provides a spectral image super-resolution device, see Figure 3 , Figure 3 A schematic diagram of the structure of a spectral image super-resolution device provided by the present invention.

[0163] exist Figure 3 In the embodiment, the device may include:

[0164] The acquisition module 310 is used to acquire a first image and a second image covering the imaging coverage of the target object, wherein the first image is a first hyperspectral image and a first multispectral image having the same imaging coverage of the object, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image.

[0165] The optimization module 320 is used to optimize the spectral dictionary based on the first image according to the non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents the spectral dictionary of the hyperspectral image and the multispectral image with the same ground object scene coverage.

[0166] The determination module 330 is configured to determine a target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy.

[0167] The obtaining module 340 is used to obtain a second hyperspectral image based on the target spectral dictionary and the target coefficient matrix; the second hyperspectral image is a hyperspectral image covering the same area as the second multispectral image.

[0168] Based on this, the present invention provides a spectral image super-resolution device, which obtains a first image and a second image covered by a target object imaging through an acquisition module 310, wherein the first image is a first hyperspectral image and a first multispectral image with the same object imaging coverage, and the second image is a second multispectral image of an area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image; and then the optimization module 320 optimizes the spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents the spectral dictionary of the hyperspectral image and the multispectral image with the same object scene coverage. Dictionary; further using the determination module 330 to determine the target coefficient matrix based on the target spectral dictionary and the second image according to the non-local sparse low rank learning strategy; finally, using the obtaining module 340 to obtain the second hyperspectral image based on the target spectral dictionary and the target coefficient matrix; the second hyperspectral image is a hyperspectral image with the same coverage area as the second multispectral image; thus, based on the organic combination of the non-local low rank dictionary optimization module and the non-local low rank sparse matrix optimization module, only part of the area needs to have the registration data of the multispectral image and the hyperspectral image at the same time, so as to realize the reconstruction of the high spatial resolution hyperspectral image of the area covered only by the multispectral image, and improve the accuracy of the ground object image.

[0169] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0170] Although the present invention has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present invention. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present invention as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present invention. Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, the present invention is intended to include such modifications and variations if they fall within the scope of the claims of the present invention and their equivalents.

Claims

1. A method for spectral image super-resolution, characterized in that: include: Acquire a first image and a second image covering the imaging coverage of the target object, wherein the first image is a first hyperspectral image and a first multispectral image having the same imaging coverage of the object, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image; Based on the first image, the spectral dictionary is optimized according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents a spectral dictionary of a hyperspectral image and a multispectral image with the same ground object scene coverage; Based on the target spectral dictionary and the second image, determining a target coefficient matrix according to a non-local sparse low-rank learning strategy; Based on the target spectral dictionary and the target coefficient matrix, a second hyperspectral image is determined; the second hyperspectral image is a hyperspectral image covering the same area as the second multispectral image.

2. The method according to claim 1, characterized in that The step of optimizing the spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary includes: Performing spatial upsampling processing on the first hyperspectral image to obtain a third hyperspectral image with the same spatial size as the first multispectral image; Normalizing the third hyperspectral image and the first multispectral image to obtain a third multispectral image and a fourth hyperspectral image; the pixel values ​​of the third multispectral image and the fourth hyperspectral image are target pixel values; Based on the third multispectral image and the fourth hyperspectral image, a plurality of groups of first three-dimensional image blocks are obtained; Based on multiple groups of the first three-dimensional image blocks, the non-local low-rank spectral dictionary learning strategy is constructed.

3. The method according to claim 2, characterized in that The obtaining of a plurality of groups of first three-dimensional image blocks based on the third multi-spectral image and the fourth hyperspectral image comprises: Superimposing the third multispectral image and the fourth hyperspectral image in a spectral dimension to obtain an intermediate image; Performing three-dimensional tensor folding processing on the intermediate image to obtain a first three-dimensional image; Performing image division on the first three-dimensional image to obtain a plurality of second three-dimensional image blocks; performing similarity grouping on the plurality of second three-dimensional image blocks to obtain a plurality of groups of first target three-dimensional image blocks; wherein the similarity between the plurality of second three-dimensional image blocks in the plurality of groups of the first target three-dimensional image blocks is greater than a preset threshold; A spatial stacking process is performed on a plurality of the second three-dimensional image blocks in a plurality of groups of the first target three-dimensional image blocks to obtain a plurality of groups of the first three-dimensional image blocks.

4. The method according to claim 2, characterized in that The constructing of a non-local low-rank spectral dictionary learning strategy based on multiple groups of the first three-dimensional image blocks includes: Performing two-dimensional matrix expansion processing on multiple groups of the first three-dimensional image blocks to obtain multiple first target two-dimensional matrices; Based on the first two-dimensional matrix of multiple targets, the formula is adopted: Construct the non-local low-rank spectral dictionary learning strategy; wherein D c is the spectral dictionary matrix of the first target two-dimensional matrix, A c is the coefficient matrix of the first target two-dimensional matrix, Y c For the intermediate image, is the coefficient matrix of the kth group of three-dimensional image blocks, symbol ||·|| F is the Fourier norm, symbol ||·||1 is the l1 norm, symbol ||·|| * is the nuclear norm, Q is the auxiliary variable, is the projection, λ is the regularization parameter, β is the regularization parameter, and γ is the regularization parameter; ||X c ||1 is used to promote the sparsity of the coefficient matrix, To promote the non-local similarity of the coefficient matrix, ||D c || * To promote the low rank of the spectral dictionary, Q is used as a non-negative constraint, given by Promote non-negativity of spectral dictionary matrices.

5. The method according to claim 4, characterized in that The step of optimizing the spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary includes: Based on the non-local low-rank spectral dictionary learning strategy, multiple groups of spectral dictionaries are obtained by combining the global correlation and self-similarity of multiple groups of the first three-dimensional image blocks; Multiple groups of spectral dictionaries are combined and reconstructed to obtain the target spectral dictionary.

6. The method according to claim 1, characterized in that The step of determining a target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy comprises: Performing three-dimensional tensor folding processing on the second image to obtain a second three-dimensional image; performing image segmentation on the second three-dimensional image to obtain a plurality of third three-dimensional image blocks; performing similarity grouping on the plurality of the third three-dimensional image blocks to obtain a plurality of groups of second target three-dimensional image blocks; wherein the similarity between the plurality of third three-dimensional image blocks in the plurality of groups of the second target three-dimensional image blocks is greater than a preset threshold; Performing spatial stacking processing on a plurality of the third three-dimensional image blocks in a plurality of groups of the second target three-dimensional image blocks to obtain a plurality of groups of fourth three-dimensional image blocks; The non-local sparse low-rank learning strategy is constructed based on multiple groups of the fourth three-dimensional image blocks and the target spectral dictionary.

7. The method according to claim 6, characterized in that The constructing the non-local sparse low-rank learning strategy based on the target spectral dictionary of the plurality of groups of the fourth three-dimensional image blocks comprises: Performing two-dimensional matrix expansion processing on the plurality of groups of the fourth three-dimensional image blocks to obtain a plurality of second target two-dimensional matrices; Based on the second target two-dimensional matrix, the formula is: Construct the non-local sparse low-rank learning strategy; where D H,c is the spectral dictionary of the first hyperspectral image, A s is the coefficient matrix of the second image, μ and κ are regularization parameters, is the coefficient matrix of the second target two-dimensional matrix of the g-th group, symbol ||·|| F is the Fourier norm, symbol ||·||1 is the l1 norm, symbol ||·|| * is the nuclear norm; ||A s ||1 is used to promote the coefficient matrix A s The sparsity of The term is used to promote the coefficient matrix A s Non-local similarity.

8. The method according to claim 7, characterized in that The step of determining a target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy includes: Based on the non-local low-rank spectral dictionary learning strategy, multiple groups of intermediate coefficient matrices are obtained by combining the global correlation, self-similarity and local low-rank of multiple groups of the fourth three-dimensional image blocks; Multiple groups of intermediate coefficient matrices are combined and reconstructed to obtain the target coefficient matrix.

9. The method according to claim 1, characterized in that The determining of a second hyperspectral image based on the target spectral dictionary and the target coefficient matrix comprises: The target spectral dictionary is multiplied by the target coefficient matrix to obtain the second hyperspectral image.

10. A spectral image super-resolution device, characterized in that: include: An acquisition module, the acquisition module is used to acquire a first image and a second image covered by the imaging of the target object, the first image is a first hyperspectral image and a first multispectral image having the same imaging coverage of the object, and the second image is a second multispectral image of the area not covered by the hyperspectral image; wherein the hyperspectral image is a low spatial resolution image, and the multispectral image is a high spatial resolution image; An optimization module, wherein the optimization module is used to optimize the spectral dictionary based on the first image according to a non-local low-rank spectral dictionary learning strategy to obtain a target spectral dictionary; the target spectral dictionary represents a spectral dictionary of a hyperspectral image and a multispectral image with the same ground object scene coverage; A determination module, the determination module is used to determine a target coefficient matrix based on the target spectral dictionary and the second image according to a non-local sparse low-rank learning strategy; The obtaining module is used to obtain a second hyperspectral image based on the target spectral dictionary and the target coefficient matrix; the second hyperspectral image is a hyperspectral image with the same coverage area as the second multispectral image.

Images