Hyperspectral image super-resolution method based on robust principal component and tensor ring decomposition
By using robust principal component and tensor ring decomposition methods, a sparse noise model is constructed and solved iteratively, which solves the problems of high model complexity and low computational efficiency in hyperspectral image super-resolution and achieves efficient image reconstruction.
Patent Information
- Application Number
- CN202510710663.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-29
- Publication Date
- 2025-09-12
AI Technical Summary
Existing hyperspectral image super-resolution methods face challenges in approximating spatial and spectral degradation matrices. The models are highly complex and computationally complex, making it difficult to effectively capture cross-band and cross-space structural correlations. Traditional matrix completion algorithms are inefficient and cannot balance detail preservation and computational efficiency.
A method based on robust principal component analysis and tensor ring decomposition is adopted. By constructing a sparse noise model, it is converted into a robust principal component analysis (RPCA) problem. The ADMM algorithm is used to split it into two sub-problems and solve them iteratively. The tensor ring Transformer model is combined for feature mapping to reduce computational complexity and retain global information.
While reducing computational complexity, the coordinated optimization of spatial details and spectral coherence is achieved, improving the reconstruction quality and efficiency of hyperspectral images.
Smart Images
Figure CN120634858A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image processing, and in particular relates to a hyperspectral image super-resolution method based on robust principal component and tensor ring decomposition. Background Art
[0002] Image super-resolution (SR), also known as super-resolution (SR), is the process of using optics and related optical knowledge to restore image details and other data based on known image information. Simply put, it increases the image resolution, making it clearer. At the same time, the image super-resolution process must maintain consistency with the actual image to avoid the problem of unknown details after super-resolution.
[0003] Traditional hyperspectral image super-resolution methods primarily rely on deep unwrapped networks (DUNs), which combine the nonlinear expressive power of deep learning with a model-driven optimization framework to map low-resolution images to high-resolution ones. However, existing methods face significant challenges in approximating the spatial and spectral degradation matrices. In particular, complex spatial degradation modeling requires the introduction of multiple convolutional networks, which dramatically increases model complexity and makes them susceptible to simulation errors. For example, while convolutional neural network (CNN)-based methods can effectively extract local features, their limited receptive field makes it difficult to model long-range dependencies. Transformer-based models, while capturing global information through self-attention mechanisms, are computationally prohibitive for large-scale hyperspectral data due to their high computational complexity. Furthermore, existing methods often simplify the problem through matrix completion or low-rank decomposition when processing high-dimensional spectral data. However, traditional matrix completion algorithms are inefficient when processing multi-dimensional, highly complex data, making it difficult to strike a balance between detail preservation and computational efficiency.
[0004] In recent years, the problem of low-rank matrix approximation has attracted considerable attention in the fields of image processing and computer vision. Its core goal is to recover missing or degraded information by exploiting the structural properties of data matrices (such as low rank and sparsity). Image super-resolution can be abstracted as an inverse problem, and a relatively mature theoretical framework has emerged. Traditional inverse problem methods face significant challenges due to their limitation to two-dimensional modeling: computational complexity increases exponentially with dimensionality, and they struggle to effectively capture cross-band and cross-spatial structural correlations. To address this challenge, researchers have turned to low-rank tensor completion techniques to address high-dimensional data. For example, Xu et al. proposed a Hankel tensor restoration model based on image patch clustering. This model constructs multidimensional patches into a three-dimensional Hankel tensor and clusters similar Hankel tensors to fully exploit the low-rank nature of the image. However, existing methods still have shortcomings in modeling dynamic hyperspectral data, such as neglecting the coupled spatial-spectral degradation characteristics and limited computational efficiency. Summary of the Invention
[0005] The purpose of the present invention is to solve the problems raised in the background technology and propose a hyperspectral image super-resolution method based on robust principal component and tensor ring decomposition.
[0006] To achieve the above object, the technical solution adopted by the present invention is: The present invention proposes a hyperspectral image super-resolution method based on robust principal component and tensor ring decomposition, comprising: Acquire low-resolution hyperspectral images of the same target and high-resolution multispectral images , and upsample the low-resolution hyperspectral image to obtain the same high-resolution multispectral image Hyperspectral images of the same resolution ; Constructing upsampled hyperspectral images A model of sparse noise between the image and the high-resolution hyperspectral image to be reconstructed; Convert the constructed sparse noise model into the robust principal component analysis (RPCA) problem form; The ADMM algorithm is introduced to split the robust principal component analysis problem into two sub-problems: sparse noise and high-resolution hyperspectral image to be reconstructed, and the two sub-problems are solved iteratively. During the iterative solution process, when stable sparse noise and high-resolution hyperspectral images are obtained, the iteration is stopped, and the high-resolution hyperspectral image obtained by the last iteration is used as the reconstructed high-resolution hyperspectral image.
[0007] Preferably, the low-resolution hyperspectral image , high-resolution multispectral imagery , hyperspectral image ,in, and Both represent spectral dimensions, and , and All indicate size; The hyperspectral image obtained by constructing the up-sampled The model of sparse noise between the high-resolution hyperspectral image to be reconstructed is expressed as follows: ; in, is the high-resolution hyperspectral image to be reconstructed, is sparse noise.
[0008] Preferably, the constructed sparse noise model is converted into a robust principal component analysis (RPCA) problem form, and the specific form is as follows: ; in, To tune the hyperparameters of the sparse noise weights, For Norm, used to emphasize The sparsity of is the spectral constraint mapping, for norm, Indicates that it is restricted.
[0009] Preferably, the ADMM algorithm is introduced to split the robust principal component analysis problem into two sub-problems regarding sparse noise and the high-resolution hyperspectral image to be reconstructed, and the two sub-problems are iteratively solved, wherein the two sub-problems are as follows: The first sub-problem is The sub-problem is expressed as: ; The second sub-problem is The sub-problem is expressed as: ; ; in, for The sub-problem The Lagrange multiplier of the wheel, for The sub-problem The Lagrange multiplier of the wheel, and are penalty parameters, is the coefficient for adjusting the intensity, is the nuclear norm, Indicates the The sparse noise of the wheel, Indicates the The sparse noise of the wheel, Indicates the High-resolution hyperspectral image of the wheel to be reconstructed, Indicates the High-resolution hyperspectral image of the wheel to be reconstructed, Indicates the A priori copy of the wheel, Indicates the A priori copy of the wheel; In the iterative solution process, the first round, i.e. , solve When solving the subproblem, first initialize The up-sampled , Initialize to a tensor of all zeros; Using the initialized and , and through soft thresholding The subproblem is solved directly to get ; Solution When solving the sub-problem, the tensor ring Transformer model is used to solve it. Input into the tensor ring Transformer model, and get ; The second round, , solve When solving the sub-problem, use the and , and then through the soft threshold The subproblem is solved directly to get ; Solution When solving the sub-problem, the tensor ring Transformer model is used to solve the problem. Input into the tensor ring Transformer model, and get ; The loop iterates until it reaches the When the round is completed, and When all converge, that is, when all are stable, the iteration is stopped and the obtained As a reconstructed high-resolution hyperspectral image.
[0010] Preferably, the tensor ring Transformer model includes a linear mapping module, a tensor ring decomposition module, an attention mechanism and a feedforward neural network connected in sequence, wherein the input of the tensor ring Transformer model is added to the output of the attention mechanism, and the result of the addition is used as the input of the feedforward neural network.
[0011] Preferably, in each round of solving When the sub-problem As the input of the tensor ring Transformer model, in the linear mapping module, it passes through the parallel linear layer q, linear layer k and linear layer v respectively, and obtains vector, Vector Sum vector; In the tensor ring decomposition module, the obtained vector, Vector Sum The vector is decomposed according to the tensor ring rank 、 and ,and , , ,in is the intermediate amount; Then decompose the obtained 、 and The attention mechanism is used to calculate the first feature, which is then combined with the input of the tensor ring Transformer model. Perform addition; Finally, the added result is input into the feedforward neural network to obtain .
[0012] Compared with the prior art, the present invention has the following beneficial effects: This hyperspectral image super-resolution method based on robust principal component and tensor ring decomposition regards the difference between the upsampled hyperspectral image and the high-resolution hyperspectral image to be reconstructed as sparse noise, which is used to remove complex spatial degradation factors and convert the sparse noise into a robust principal component analysis (RPCA) problem. The problem is then converted into two sub-problems for iterative solution. During the solution, the tensor ring Transformer model is used to map features to a low-rank factor space to reduce the amount of attention calculation and obtain global information. At the same time, the tensor ring Transformer model is introduced to replace the dot product operation in the traditional self-attention with a multilinear tensor ring product, which compresses redundant information while retaining the expressive power of the global attention mechanism, thereby reducing the computational complexity to a certain extent and achieving the coordinated optimization of spatial details and spectral coherence. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 This is a flow chart of the hyperspectral image super-resolution method based on robust principal component and tensor ring decomposition of the present invention.
[0014] Figure 2 Schematic diagram of the structure of the tensor ring Transformer model of the present invention; Figure 3 This is a diagram showing an example of the sparse noise model of the present invention. DETAILED DESCRIPTION
[0015] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.
[0016] like Figure 1-Figure 2 As shown, a hyperspectral image super-resolution method based on robust principal component and tensor ring decomposition is provided, including: Step 1: Obtain a low-resolution hyperspectral image of the same target and high-resolution multispectral images (The low-resolution hyperspectral image and the high-resolution multispectral image are images of the same target, which can be a specific scene. The two images are reconstructed to obtain a reconstructed high-resolution hyperspectral image), and the low-resolution hyperspectral image is upsampled (or interpolated) to obtain the same high-resolution multispectral image. Hyperspectral images of the same resolution ; Among them, low-resolution hyperspectral images , high-resolution multispectral imagery , hyperspectral image ,in, and Both represent spectral dimensions, and , and All indicate size.
[0017] Step 2: Construct the upsampled hyperspectral image The model of sparse noise between the high-resolution hyperspectral image to be reconstructed (due to the upsampling of the hyperspectral image Already contains high-resolution hyperspectral images to be reconstructed The difference between the two lies in a small number of outlier pixels such as edge artifacts and bad lines, which present the sparse characteristics of "small number and large amplitude". These outlier pixel errors are called sparse noise), and the model expression is as follows: ; in, is the high-resolution hyperspectral image to be reconstructed, is sparse noise (expected to be highly sparse, used to describe the residual caused by spatial degradation and interpolation error). Figure 3 Give specific examples to show The corresponding image.
[0018] Step 3: Convert the constructed sparse noise model into the robust principal component analysis (RPCA) problem form (RPCA (Robust Principal Component Analysis) is a data analysis technique for processing sparse noise, namely robust principal component analysis), and the specific form is as follows: ; in, To tune the hyperparameters of the sparse noise weights, For Norm, used to emphasize The sparsity of Spectral constrained mapping (specifically, 1×1 convolution or fully connected layer can be used to implement spectral constrained mapping, mapping from C spectral channels to c channels), for norm, Indicates limited to (abbreviated as st), Used to measure and By minimizing the pixel-wise squared error between To ensure recovery Maintaining the correct spectral characteristics while inheriting High spatial resolution details.
[0019] Step 4: Introduce the ADMM (Alternating Direction Method of Multipliers) algorithm to split the robust principal component analysis problem into two sub-problems: sparse noise and the high-resolution hyperspectral image to be reconstructed. The two sub-problems are solved iteratively. The two sub-problems are as follows: The first sub-problem is Sub-problem (i.e., the RPCA problem form ), the formula is: ; The second sub-problem is Sub-problem (i.e., the RPCA problem form ), the formula is: ; ; in, for The sub-problem The Lagrange multiplier of the wheel, the initial value is 0, for The sub-problem The Lagrange multiplier of the wheel has an initial value of 0 and will be gradually updated during the iterative solution process (the specific update process belongs to the existing technology). and are penalty parameters, is the coefficient for adjusting the intensity, is the nuclear norm, Indicates the The sparse noise of the wheel, Indicates the The sparse noise of the wheel, Indicates the High-resolution hyperspectral image of the wheel to be reconstructed, Indicates the High-resolution hyperspectral image of the wheel to be reconstructed, Indicates the A priori copy of the wheel, Indicates the A priori copies of the wheel to help with block optimization; In the iterative solution process, the first round, i.e. , solve When solving the subproblem, first initialize The up-sampled , Initialize to a tensor of all zeros; Using the initialized and , and through soft-thresholding The subproblem is solved directly to get (The solution process is well known to those skilled in the art); Solution When solving the subproblem, the tensor ring Transformer model is used to solve it (because it is often desirable to introduce stronger image priors in super-resolution scenarios, the update of this subproblem can be replaced by a deep network to improve the reconstruction capability). Input into the tensor ring Transformer model, and get ; The second round, , solve When solving the sub-problem, use the and , and then through the soft threshold The subproblem is solved directly to get ; Solution When solving the sub-problem, the tensor ring Transformer model is used to solve the problem. Input into the tensor ring Transformer model, and get ; The loop iterates until it reaches the When the round is completed, and When all converge, that is, when all are stable, the iteration is stopped and the obtained As a reconstructed high-resolution hyperspectral image.
[0020] Among them, the tensor ring Transformer model includes a linear mapping module, a tensor ring decomposition module, an attention mechanism and a feedforward neural network connected in sequence. The input of the tensor ring Transformer model is added to the output of the attention mechanism, and the result of the addition is used as the input of the feedforward neural network.
[0021] Solve in each round When the sub-problem As the input of the tensor ring Transformer model, in the linear mapping module, it passes through the parallel linear layer q, linear layer k and linear layer v respectively, and obtains vector, Vector Sum vector, where the formula is expressed as: ; in, 、 and Both are trainable parameter matrices; In the tensor ring decomposition module, the obtained vector, Vector Sum The vector is decomposed according to the tensor ring rank 、 and (Using tensor ring Transformer to vector, Vector Sum The vector is projected into the low-rank tensor ring factor space, achieving global attention expression at lower complexity), and , , ,in is the intermediate amount, 、 and are all tensor ring ranks; Then decompose the obtained 、 and Perform attention mechanism calculation (by reducing the dimension of channels and tokens to reduce the large-scale matrix multiplication during attention) to obtain the first feature, which is then combined with the input of the tensor ring Transformer model Perform addition (element-wise addition); Finally, the added result is input into the feedforward neural network to obtain By using the tensor ring Transformer vector, Vector Sum The vector is projected into the low-rank tensor ring factor space, achieving global attention expression with lower complexity.
[0022] Step 5: During the iterative solution process, when stable sparse noise and high-resolution hyperspectral images are obtained, the iteration is stopped, and the high-resolution hyperspectral image obtained by the last iteration is used as the reconstructed high-resolution hyperspectral image.
[0023] This hyperspectral image super-resolution method based on robust principal component and tensor ring decomposition regards the difference between the upsampled hyperspectral image and the high-resolution hyperspectral image to be reconstructed as sparse noise, which is used to remove complex spatial degradation factors and convert the sparse noise into a robust principal component analysis (RPCA) problem. The problem is then converted into two sub-problems for iterative solution. During the solution, the tensor ring Transformer model is used to map features to a low-rank factor space to reduce the amount of attention calculation and obtain global information. At the same time, the tensor ring Transformer model is introduced to replace the dot product operation in the traditional self-attention with a multilinear tensor ring product, which compresses redundant information while retaining the expressive power of the global attention mechanism, thereby reducing the computational complexity to a certain extent and achieving the coordinated optimization of spatial details and spectral coherence.
[0024] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0025] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.
Claims
1. A hyperspectral image super-resolution method based on robust principal component and tensor ring decomposition, characterized by: The hyperspectral image super-resolution method based on robust principal component and tensor ring decomposition includes: Acquire low-resolution hyperspectral images of the same target and high-resolution multispectral images , and upsample the low-resolution hyperspectral image to obtain the same high-resolution multispectral image Hyperspectral images of the same resolution ; Constructing upsampled hyperspectral images A model of sparse noise between the image and the high-resolution hyperspectral image to be reconstructed; Convert the constructed sparse noise model into the robust principal component analysis (RPCA) problem form; The ADMM algorithm is introduced to split the robust principal component analysis problem into two sub-problems: sparse noise and high-resolution hyperspectral image to be reconstructed, and the two sub-problems are solved iteratively. During the iterative solution process, when stable sparse noise and high-resolution hyperspectral images are obtained, the iteration is stopped, and the high-resolution hyperspectral image obtained by the last iteration is used as the reconstructed high-resolution hyperspectral image.
2. The hyperspectral image super-resolution method based on robust principal component analysis and tensor ring decomposition according to claim 1, characterized in that: The low-resolution hyperspectral image , high-resolution multispectral imagery , hyperspectral image ,in, and Both represent spectral dimensions, and , and All indicate size; The hyperspectral image obtained by constructing the up-sampled The model of sparse noise between the high-resolution hyperspectral image to be reconstructed is expressed as follows: ； in, is the high-resolution hyperspectral image to be reconstructed, is sparse noise.
3. The hyperspectral image super-resolution method based on robust principal component analysis and tensor ring decomposition according to claim 2, characterized in that: The constructed sparse noise model is converted into the robust principal component analysis RPCA problem form, and the specific form is as follows: ； in, To tune the hyperparameters of the sparse noise weights, For Norm, used to emphasize The sparsity of is the spectral constraint mapping, for norm, Indicates that it is restricted.
4. The hyperspectral image super-resolution method based on robust principal component analysis and tensor ring decomposition according to claim 3, characterized in that: The ADMM algorithm is introduced to split the robust principal component analysis problem into two sub-problems: sparse noise and high-resolution hyperspectral image to be reconstructed. The two sub-problems are solved iteratively. The two sub-problems are as follows: The first sub-problem is The sub-problem is expressed as: ； The second sub-problem is The sub-problem is expressed as: ； ； in, for The sub-problem The Lagrange multiplier of the wheel, for The sub-problem The Lagrange multiplier of the wheel, and are penalty parameters, is the coefficient for adjusting the intensity, is the nuclear norm, Indicates the The sparse noise of the wheel, Indicates the The sparse noise of the wheel, Indicates the High-resolution hyperspectral image of the wheel to be reconstructed, Indicates the High-resolution hyperspectral image of the wheel to be reconstructed, Indicates the A priori copy of the wheel, Indicates the A priori copy of the wheel; In the iterative solution process, the first round, i.e. , solve When solving the subproblem, first initialize The up-sampled , Initialize to a tensor of all zeros; Using the initialized and , and through soft thresholding The subproblem is solved directly to get ; Solution When solving the sub-problem, the tensor ring Transformer model is used to solve it. Input into the tensor ring Transformer model, and get ; The second round, , solve When solving the sub-problem, use the and , and then through the soft threshold The subproblem is solved directly to get ; Solution When solving the sub-problem, the tensor ring Transformer model is used to solve the problem. Input into the tensor ring Transformer model, and get ; The loop iterates until it reaches the When the round is completed, and When all converge, that is, when all are stable, the iteration is stopped and the obtained As a reconstructed high-resolution hyperspectral image.
5. The hyperspectral image super-resolution method based on robust principal component and tensor ring decomposition according to claim 4, characterized in that: The tensor ring Transformer model includes a linear mapping module, a tensor ring decomposition module, an attention mechanism and a feedforward neural network connected in sequence, wherein the input of the tensor ring Transformer model is added to the output of the attention mechanism, and the result of the addition is used as the input of the feedforward neural network.
6. The hyperspectral image super-resolution method based on robust principal component analysis and tensor ring decomposition according to claim 5, characterized in that: Solve in each round When the sub-problem As the input of the tensor ring Transformer model, in the linear mapping module, it passes through the parallel linear layer q, linear layer k and linear layer v respectively, and obtains vector, Vector Sum vector; In the tensor ring decomposition module, the obtained vector, Vector Sum The vector is decomposed according to the tensor ring rank 、 and ,and , , ,in is the intermediate amount; Then decompose the obtained 、 and The attention mechanism is used to calculate the first feature, which is then combined with the input of the tensor ring Transformer model. Perform addition; Finally, the added result is input into the feedforward neural network to obtain .