Hyperspectral remote sensing image destriping method based on low-rank tensor multimodal approximation
Through the low-rank tensor multimodal approximation method, the weighted nuclear norm and truncated nuclear norm are used to constrain the spectral redundancy and rank 1 property of stripe noise in hyperspectral images in the tensor domain. Combined with the alternating iterative multiplier method, the problems of residual stripe noise and spatial spectrum consistency in hyperspectral remote sensing images are solved, and an efficient image destriping effect is achieved.
Patent Information
- Application Number
- CN202411769939.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-09-23
- Estimated Expiration
- 2044-12-04
AI Technical Summary
Existing hyperspectral remote sensing image destriping technologies have problems such as residual noise, over-smoothing of images, and difficulty in maintaining spatial-spectral consistency, especially when dealing with complex random stripes.
A method based on low-rank tensor multimodal approximation is adopted. The spectral redundancy of hyperspectral images and the rank-1 property of stereo stripe noise are constrained in the tensor domain by weighted nuclear norm and truncated nuclear norm. The alternating iterative multiplier method is combined to optimize the model to achieve the separation of clean image and stripe noise.
On the premise of maintaining the spatial-spectral consistency of the hyperspectral image, the striping noise is effectively removed, the parameter adjustment process is avoided, and an efficient image striping effect is achieved.
Smart Images
Figure CN119228681B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of hyperspectral remote sensing image denoising, and is a hyperspectral remote sensing image destriping method based on low-rank tensor multimodal approximation, which is used to eliminate stripe noise and generate clean hyperspectral images, providing effective data for watershed hyperspectral remote sensing observation tasks. Background Art
[0002] Hyperspectral remote sensing with nanometer-scale spectral resolution can provide diagnostic spectra of targets, thus playing a vital role in Earth observation. Hyperspectral remote sensing, which mostly uses push-broom and swing-broom scanning methods, is prone to detector mismatch, resulting in vertical or horizontal banding artifacts in the captured data. This severely degrades the interpretation and analysis performance of hyperspectral remote sensing images. Therefore, research on techniques for removing banding noise from hyperspectral remote sensing images has important theoretical and practical applications.
[0003] Traditional de-banding techniques include statistical and filtering methods. Statistical methods often assume that the banding or image conforms to certain statistical laws, resulting in limited effectiveness and being primarily applied to single-band images. Filtering methods utilize a transformation to convert the image to a transform domain, achieving image-noise separation. Appropriate filters are then designed to remove noise components, and the resulting image is then converted to the spatial domain. These methods are effective against periodic banding noise. While simple and fast, these traditional de-banding methods suffer from residual noise and over-smoothing when dealing with the densely distributed random banding found in real-world hyperspectral remote sensing images. Popular de-banding methods include deep learning and regularization. Deep learning-based de-banding techniques are mostly supervised and rely on clean ground-truth image labels, which are unknown. Therefore, these methods use clean data from public libraries as labels, generate simulated noisy data by adding simulated noise, and then remove the noise by learning a mapping from the noisy data to the corresponding ground-truth values. However, the distribution of real banding noise is complex and random, making it difficult to simulate artificially. Therefore, the practical effectiveness of supervised learning-based de-banding techniques is often unsatisfactory.
[0004] Regularization-based de-banding methods regard the de-banding problem as an underdetermined inverse problem of estimating potential clean data based on observed noise data, and constrain the solution space of the inverse problem through appropriate prior knowledge. Finally, an effective optimization algorithm is designed to obtain the optimal solution or local optimal solution. This type of method has attracted much attention due to the flexibility and effectiveness of the prior selection. Since the introduction of stripe noise intuitively destroys the smoothness of hyperspectral data, many variation-based smoothing regularization strategies have been proposed, such as Huber-Markov variation with edge preservation ability, one-way total variation, anisotropic space-spectrum total variation, and adaptive anisotropic space-spectrum total variation. However, variational smoothing has the risk of over-smoothing the image, and secondly, it is difficult to avoid the staircase effect. Based on linear unmixing theory, hyperspectral data can be decomposed into the product of a sparse abundance matrix and a finite end-member matrix. Therefore, some people use norm or its convex relaxation Sparse regularization such as the norm imposes sparse constraints on the abundance component, solves the clean abundance and endmembers, and then estimates the potential image components. In addition, the high spectral redundancy unique to hyperspectral data has attracted a wave of scholars to use low-rank matrix decomposition, nuclear norm, and weighted nuclear norm to perform low-rank matrix regularization on the Casorati matrix of hyperspectral images. However, image matrixization has the risk of destroying the consistency of the intrinsic spatial-spectral structure of hyperspectral data. Therefore, a variety of low-rank tensor approximation strategies that preserve the three-dimensional structure of the data have been used in the task of destriping hyperspectral data, such as CP decomposition, Tucker decomposition, tensor chain decomposition, tensor ring decomposition, and tensor singular value decomposition. However, studies have shown that due to the high low-rank nature of the strip components, methods based on image low-rank regularization have the problem of residual striping.
[0005] Some scholars have pointed out that the image and noise separation method that uses collaborative image prior and stripe prior regularization can achieve better de-striping performance than image estimation methods that rely solely on image prior regularization. Subsequently, some methods for low-rank and sparse regularization of stripes were proposed. Given the defect that most prior regularization methods applied to stripes require manual parameter adjustment, and any stripe can theoretically be represented as a linear superposition of other stripes, a regularization strategy that does not require parameter adjustment has been proposed, namely, the rank-1 constraint based on the matrix truncated nuclear norm. However, the stripe components are also three-dimensional structures with spectral correlation. This low-rank matrix strategy cannot well characterize the spatial-spectral consistency of three-dimensional stripes, which is not conducive to preserving the spatial-spectral consistency of the image. Summary of the Invention
[0006] In order to solve the problem of residual striping caused by low-rank regularization of images, maintain the three-dimensional structure of the stripping components, and remove the stripping noise while maintaining the spatial-spectral consistency of hyperspectral data and without the need for parameter adjustment, the present invention proposes a hyperspectral remote sensing image stripping method based on low-rank tensor multimodal approximation. This method extends the weighted nuclear norm and the truncated nuclear norm to the tensor domain, which are used to characterize the spectral redundancy of hyperspectral images and the high low-rank nature of three-dimensional stripping noise, respectively. Then, the above image spectral redundancy and stripe low-rank characterization terms are used to jointly regularize the additive stripping noise degradation problem, and a hyperspectral image stripping model based on low-rank tensor multimodal approximation is constructed. Finally, the present invention optimizes the model under the framework of the alternating iterative multiplier method to achieve accurate separation of clean images and striping noise.
[0007] The specific technical solution is a hyperspectral remote sensing image destriping method based on low-rank tensor multimodal approximation, which includes the following steps:
[0008] S1, will observe hyperspectral data Each pixel in is divided by the maximum value of all pixels to achieve the normalization of the observed hyperspectral data amplitude. h 、 w and b These refer to the number of rows, columns, and bands of hyperspectral data, respectively.
[0009] S2, through the tensor mode-n singular value decomposition (SVD), the matrix weighted nuclear norm is extended to the tensor domain, and a weighted nuclear norm based on tensor mode-n singular value decomposition (WNN-TnSVD) is proposed to constrain the spectral redundancy of potential hyperspectral images.
[0010] S3, through tensor SVD, the matrix truncated nuclear norm is extended to the tensor domain, and a truncated nuclear norm based on tensor singular value decomposition (TNN-TSVD) is proposed to restrict the rank 1 of the three-dimensional strip components.
[0011] S4, using WNN-TnSVD minimization of clean hyperspectral images and TNN-TSVD minimization of stereo stripe noise, collaboratively constrained the degradation problem of observed hyperspectral images contaminated by additive stripe noise: , a hyperspectral image destriping model based on low-rank tensor multimodal approximation is established. Represent the clean hyperspectral image and stripe noise components respectively.
[0012] S5, based on the alternating direction multiplier method, after splitting the above stripping model into several simple sub-problems, by alternately solving the sub-problems, the clean hyperspectral image and stripe noise are obtained, and the image and stripe separation is achieved.
[0013] Furthermore, step S2 specifically includes the following steps:
[0014] S201: Clean hyperspectral image Perform modulo-n SVD: ,in, and is an orthogonal tensor, is a diagonal tensor, n Finger modulus -n, represents the nth dimension of the tensor, Denoted as Fourier domain t product, superscript In order to constrain the spectral redundancy of the hyperspectral image, let .
[0015] Specifically, n =1 For example: ,in, , , So, about The modulo-1 singular value decomposition of can be done by Fast Fourier transform (FFT) and SVD of the matrix: Each horizontal slice of Perform FFT to get Afterwards, Each horizontal slice of Perform SVD: , and finally Each horizontal slice of Perform an inverse fast Fourier transform (IFFT) to obtain .in, .
[0016] S202: Singular value tensor FFT Each level of n =1 or side n Perform a weighted sum of all diagonal elements in the =2 slice: ;in, WNN-TnSVD representing hyperspectral images, and They refer to slices and diagonal elements, Represents a weighted sequence. To retain important image information, i.e., large singular values, let Elements , c is a positive constant, Used to avoid invalid operations. Specifically, n =1 as an example: ,by n =2 as an example: .
[0017] Furthermore, step S3 specifically includes the following steps:
[0018] S301: Stereo stripe noise Perform tensor SVD (modulo-3 SVD): ,in, , is an orthogonal tensor, is a diagonal tensor.
[0019] Specifically, Each forward slice of Perform FFT to get Afterwards, Each forward slice of Perform SVD: , and finally 、 and Each forward slice of 、 and Perform IFFT to get 、 and .in, .
[0020] S302: Singular value tensor FFT Each forward slice of Middle After (including ) perform a weighted sum of the smaller diagonal elements: .in, TNN-TSVD representing stripe noise, and They refer to forward slice and the diagonal elements. is the cutoff value, which is set to 2 to ensure the rank 1 property of the stripe noise.
[0021] Furthermore, step S4 specifically includes the following steps:
[0022] S401: Using the above WNN-TnSVD and TNN-TSVD minimization, collaboratively constrain the hyperspectral image noise degradation problem , and then we get the following de-banding model based on low-rank tensor multimodal approximation:
[0023] (1)
[0024] S402: Introduction of Lagrange multipliers , transform the above WNN-TnSVD and TNN-TSVD constrained destriping models into their Lagrangian functions:
[0025] (2)
[0026] in, represents the square of the Frobenius norm. For one greater than The maximum value of the largest element in can ensure the rank 1 of the strip tensor. Therefore, It's just a constant that's large enough, no tuning required.
[0027] Furthermore, step S5 specifically includes the following steps:
[0028] S501: Separate orders about and The partial derivative of is zero, then we can get:
[0029] (3)
[0030] (4)
[0031] Among them, the superscript t Indicates the number of iterations. For example, Representation function No. The result obtained in the iterative optimization The solution.
[0032] S502: Execute model initialization: ,in, Represents a tensor of all zeros.
[0033] S503: Alternately solve the two sub-problems obtained in step S501 until the convergence condition is met: .
[0034] Specifically, For example: , .in, , is a soft threshold function, , is a truncated soft threshold function.
[0035] The technical effects of the present invention are as follows:
[0036] 1. The hyperspectral remote sensing image destriping method provided by the present invention introduces the matrix weighted nuclear norm into the tensor domain and proposes a new low-rank tensor multimodal approximation strategy, which can achieve parameter-free characterization of the spectral redundancy of the hyperspectral image while maintaining the spatial-spectral consistency of the image.
[0037] 2. The hyperspectral remote sensing image destriping method provided by the present invention introduces the matrix truncated nuclear norm into the tensor domain and proposes a new tensor rank 1 regularization strategy, which can achieve highly linear parameter-free characterization of the striping noise while maintaining the spatial-spectral consistency of the striping noise.
[0038] 3. The hyperspectral remote sensing image destriping method provided by the present invention designs an alternating iterative optimization algorithm based on the alternating direction multiplier method, which can realize the alternating solution of hyperspectral images and stripe noise, and then realize image and noise separation, and eliminate stripe noise in hyperspectral images. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The present invention will be further described below with reference to the accompanying drawings and examples.
[0040] Figure 1 is a flow chart of the method of the present invention;
[0041] Figure 2 Schematic diagram of the hyperspectral image tensor modulus-n SVD of the present invention;
[0042] Figure 3 Schematic diagram of the strip noise tensor SVD of the present invention. DETAILED DESCRIPTION
[0043] In view of the widespread strip noise pollution problem in push-broom and swing-broom hyperspectral remote sensing, the present invention provides a hyperspectral remote sensing image striping method based on low-rank tensor multimodal approximation, which can achieve parameter-free estimation of potential hyperspectral images and strip noise while maintaining data spatial-spectral consistency, thereby obtaining clean hyperspectral images and laying a data foundation for the field of hyperspectral remote sensing earth observation. The hyperspectral remote sensing image striping method provided by the present invention is as follows: Figure 1 shown.
[0044] The process of the present invention is described below with specific embodiments:
[0045] In this embodiment, a hyperspectral remote sensing image contaminated by random longitudinal stripe noise is used as an example to specifically illustrate the striping removal method provided by the present invention. The striping removal method provided by the present invention is specifically as follows:
[0046] Step S1: observe hyperspectral data Each pixel in is divided by the maximum value of all pixels to achieve the normalization of the observed hyperspectral data amplitude. h 、 w and b These refer to the number of rows, columns, and bands of hyperspectral data, respectively.
[0047] In step S2, the matrix weighted nuclear norm is extended to the tensor domain through the tensor modulo-n singular value decomposition (SVD), and a weighted nuclear norm based on tensor mode-n SVD (WNN-TnSVD) is proposed, as shown in Figure 2 , which is used to constrain the spectral redundancy of potential hyperspectral images. Specifically, it includes the following steps:
[0048] S201: Clean hyperspectral image Perform modulo-n SVD: ,in, and is an orthogonal tensor, is a diagonal tensor, n Finger modulus -n, represents the nth dimension of the tensor, Denoted as Fourier domain t product, superscript In order to constrain the spectral redundancy of the hyperspectral image, let .
[0049] Specifically, For example: ,in, , , So, about The modulo-1 singular value decomposition of can be done by Fast Fourier transform (FFT) and SVD of the matrix: Each horizontal slice of Perform FFT to get Afterwards, Each horizontal slice of Perform SVD: , and finally Each horizontal slice of Perform an inverse fast Fourier transform (IFFT) to obtain .in, .
[0050] S202: Singular value tensor FFT Each level of n =1) or side ( n =2) Perform a weighted sum of all diagonal elements in the slice: .in, WNN-TnSVD representing hyperspectral images, and They refer to slices and diagonal elements, Represents a weighted sequence. To retain important image information, i.e., large singular values, let Elements , is a positive constant, Used to avoid invalid operations. Specifically, For example: ,by For example: .
[0051] Step S3, the matrix truncated nuclear norm is extended to the tensor domain through tensor SVD, and the truncated nuclear norm based on tensor singular value decomposition (TNN-TSVD) is proposed, as shown in Figure 3 , which is used to limit the rank 1 of the stereo strip components. Specifically, it includes the following steps:
[0052] S301: Stereo stripe noise Perform tensor SVD (modulo-3 SVD): ,in, , is an orthogonal tensor, is a diagonal tensor.
[0053] Specifically, Each forward slice of Perform FFT to get Afterwards, Each forward slice of Perform SVD: , and finally 、 and Each forward slice of 、 and Perform IFFT to get 、 and .in, .
[0054] S302: Singular value tensor FFT Each forward slice of Middle After (including ) perform a weighted sum of the smaller diagonal elements: .in, TNN-TSVD representing stripe noise, and They refer to forward slice and the diagonal elements. is the cutoff value, which is set to 2 to ensure the rank 1 property of the stripe noise.
[0055] In step S4, the WNN-TnSVD minimization of the clean hyperspectral image and the TNN-TSVD minimization of the stereo stripe noise are used to collaboratively constrain the degradation problem of the observed hyperspectral image contaminated by additive stripe noise: , a hyperspectral image destriping model based on low-rank tensor multimodal approximation is established. Represent the clean hyperspectral image and stripe noise components respectively. The specific steps include:
[0056] S401: Using the above WNN-TnSVD and TNN-TSVD minimization, collaboratively constrain the hyperspectral image noise degradation problem , and then we get the following de-banding model based on low-rank tensor multimodal approximation:
[0057] (1)
[0058] S402: Introduction of Lagrange multipliers , transform the above WNN-TnSVD and TNN-TSVD constrained destriping models into their Lagrangian functions:
[0059] (2)
[0060] in, represents the square of the Frobenius norm. For one greater than The maximum value of the largest element in can ensure the rank 1 of the strip tensor. Therefore, It's just a constant that's large enough, no tuning required.
[0061] Step S5, based on the alternating direction multiplier method, splits the above striping model into several simple sub-problems, and then alternately solves the sub-problems to obtain a clean hyperspectral image and stripe noise, thereby achieving image and stripe separation. Specifically, the steps include:
[0062] S501: Separate orders about and The partial derivative of is zero, then we can get:
[0063] (3)
[0064] (4)
[0065] Among them, the superscript Indicates the number of iterations. For example, Representation function No. The result obtained in the iterative optimization The solution.
[0066] S502: Execute model initialization: ,in, Represents a tensor of all zeros.
[0067] S503: Alternately solve the two sub-problems obtained in step S501 until the convergence condition is met: .
[0068] Specifically, For example: , .in, , is a soft threshold function, , is a truncated soft threshold function.
Claims
1. A hyperspectral remote sensing image destriping method based on low-rank tensor multimodal approximation, characterized in that: The steps include: S1, will observe hyperspectral data Each pixel in is divided by the maximum value of all pixels to achieve the normalization of the amplitude of the observed hyperspectral data; h 、 w and b These refer to the number of rows, columns, and bands of hyperspectral data, respectively; S2, through the tensor modulo-n singular value decomposition (SVD), the matrix weighted nuclear norm is extended to the tensor domain, and a weighted nuclear norm WNN-TnSVD based on the tensor modulo-n SVD is proposed to constrain the spectral redundancy of potential hyperspectral images; Step S2 specifically includes the following steps: S201: Clean hyperspectral image Perform modulo-n SVD: ,in, and is an orthogonal tensor, is a diagonal tensor, n Finger modulus -n, represents the nth dimension of the tensor, Denoted as Fourier domain t product, superscript represents the conjugate transpose; in order to constrain the spectral redundancy of the hyperspectral image, let ; S202: Singular value tensor FFT Each level of n =1 or side n Perform a weighted sum of all diagonal elements in the =2 slice: ;in, WNN-TnSVD representing hyperspectral images, i and j They refer to i slices and j diagonal elements, Represents a weighted sequence. To retain important image information, i.e., large singular values, let Elements , c is a positive constant, Used to avoid invalid operations; S3, through tensor SVD, the matrix truncated nuclear norm is extended to the tensor domain, and the truncated nuclear norm TNN-TSVD based on tensor SVD is proposed to restrict the rank 1 of the three-dimensional strip component; Step S3 specifically includes the following steps: S301: Stereo stripe noise Perform tensor SVD, i.e. modulo-3 SVD: ,in, , is an orthogonal tensor, is a diagonal tensor; Specifically, Each forward slice of Perform FFT to get Afterwards, Perform SVD on each forward slice of , and finally 、 and Each forward slice of 、 and Perform IFFT to get 、 and ;in, ; S302: Singular value tensor FFT Each forward slice of Middle r Then perform weighted summation on the small diagonal elements: ;in, TNN-TSVD representing stripe noise, k and j They refer to k forward slice and the j diagonal elements; among them, r is the cutoff value, which is set to 2 to ensure the rank 1 property of the stripe noise; S4, using WNN-TnSVD minimization of clean hyperspectral images and TNN-TSVD minimization of stereo stripe noise, collaboratively constrained the degradation problem of observed hyperspectral images contaminated by additive stripe noise: , a hyperspectral image destriping model based on low-rank tensor multimodal approximation is established; where, represent the clean hyperspectral image and stripe noise components respectively; S5, based on the alternating direction multiplier method, after splitting the above stripping model into several simple sub-problems, by alternately solving the sub-problems, the clean hyperspectral image and stripe noise are obtained, and the image and stripe separation is achieved.
2. The hyperspectral remote sensing image destriping method based on low-rank tensor multimodal approximation according to claim 1, characterized in that: Step S4 specifically includes the following steps: S401: Using the above WNN-TnSVD and TNN-TSVD minimization, collaboratively constrain the hyperspectral image noise degradation problem , and then we get the following de-banding model based on low-rank tensor multimodal approximation: （1） S402: Introduction of Lagrange multipliers , transform the above WNN-TnSVD and TNN-TSVD constrained destriping models into their Lagrangian functions: （2） in, represents the square of the Frobenius norm.
3. The hyperspectral remote sensing image destriping method based on low-rank tensor multimodal approximation according to claim 2, characterized in that: Step S5 specifically includes the following steps: S501: Separate orders about and The partial derivative of is zero, then: （3） （4） Among them, the superscript t Indicates the number of iterations; S502: Execute model initialization: ,in, represents an all-zero tensor; S503: Alternately solve the two sub-problems obtained in step S501 until the convergence condition is met: .