Hyperspectral image restoration method
Patent Information
- Application Number
- TW114112455
- Authority / Receiving Office
- TW · TW
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-03-31
- Publication Date
- 2026-10-16
- Estimated Expiration
- 2045-03-30
AI Technical Summary
Existing hyperspectral image restoration methods fail to effectively utilize multi-dimensional characteristics and struggle with diverse noise types, particularly when both noise and image are low-rank, leading to suboptimal restoration results.
A method involving block extraction and traversal expansion of noisy hyperspectral images into sub-block two-dimensional matrices, followed by iterative piecewise noise removal using low-rank matrix updating, L1 norm sparse noise removal, and smoothing proportional form penalty matrix noise removal, and finally merging the restored matrices to achieve high-quality images.
The method effectively restores noisy hyperspectral images by leveraging sparsity and low-rank properties, improving image quality and adaptability to mixed noise types.
Smart Images
Figure TWG2TA001079913_001 
Figure TWG2TA001079913_002 
Figure TWG2TA001079913_003
Abstract
Description
[Technical Field]
[0001] This invention relates to an image restoration method, and more particularly to a hyperspectral image restoration method. [Previous Technology]
[0002] Hyperspectral images contain rich spectral signals and are of great value to the field of remote sensing. Hyperspectral images are generated by spectral sensors with hundreds of continuous narrow spectral bands. However, noise interference is inevitably introduced during the imaging process, which affects the imaging quality.
[0003] In the prior art, methods for restoring hyperspectral images can be mainly divided into two categories: methods based on independent processing of a single band and methods based on the spatial domain. Methods based on a single band treat each band as an independent grayscale image and can use traditional grayscale image restoration methods to restore images for each band. However, these methods ignore the correlation between spectra and cannot utilize the multi-dimensional characteristic signals of the hyperspectral spectrum for image restoration. Methods based on the spatial domain consider the priori information between space and spectrum, thus significantly improving the image restoration results. These methods include sparse representations that represent images as linear combinations of finite basis elements, tensor decomposition models that extract useful information through matrix / tensor decomposition, and low-rank matrix restoration. Sparse representations rely on dictionaries to capture data structure, but a single dictionary is difficult to adapt to all data features and noise. Tensor decomposition models rely excessively on prior information and are difficult to effectively restore images when encountering diverse noise. Low-rank matrix restoration restores images through the low-rank properties of the matrix, but when both noise and the image are low-rank, it is difficult to separate them due to a lack of consideration for sparsity. [Summary of the Invention]
[0004] The main objective of this invention is to restore a noisy hyperspectral image to a high-quality hyperspectral image.
[0005] A hyperspectral image restoration method according to the present invention includes: inputting a noisy hyperspectral image into a computing device; the computing device performing block extraction and traversal expansion on the noisy hyperspectral image to obtain a plurality of sub-block two-dimensional matrices; the computing device performing iterative piecewise noise removal on each of the sub-block two-dimensional matrices, wherein the piecewise noise removal includes low-rank matrix image updating, L1 norm sparse noise removal and smoothing proportional form penalty matrix noise removal; and the computing device merging the piecewise noise-removed sub-block two-dimensional matrices into a hyperspectral image.
[0006] The present invention extracts and expands the noisy hyperspectral image into a two-dimensional matrix by segmenting and traversing the block, and then restores the sparsity and low-rank characteristics of each sub-block two-dimensional matrix. Finally, the restored sub-block two-dimensional matrices are merged to obtain the hyperspectral image of image restoration.
Implementation Method
[0007] Please refer to Figure 1, which is a flowchart of a hyperspectral image restoration method 10 according to one embodiment of the present invention. The hyperspectral image restoration method 10 includes: "inputting a noisy hyperspectral image 11", "segmenting and expanding the noisy hyperspectral image 12", "segmenting the sub-block two-dimensional matrix for noise removal 13", and "merging the segmented noise-removed sub-block two-dimensional matrix into a hyperspectral image 14".
[0008] Please refer to Figure 1. In step 11, a noisy hyperspectral image is input into a computing device. Please refer to Figure 2. In this embodiment, the matrix of the noisy hyperspectral image is represented as , where , , and represent the height, width, and number of bands of the noisy hyperspectral image, respectively. The computing device is a computer or a portable mobile device capable of performing logical operations.
[0009] Please refer to Figures 1 and 2. In step 12, the computing device performs block extraction and traversal expansion on the noisy hyperspectral image to obtain a plurality of sub-block two-dimensional matrices. Please refer to Figure 2. Block extraction of the noisy hyperspectral image can be represented as: where is the matrix of the noisy hyperspectral image, are the height, width and number of bands of the noisy hyperspectral image, respectively, are the plurality of sub-blocks obtained by block extraction of the matrix image of the noisy hyperspectral image, where , are the size of each sub-block, and are the index values along the height and width of the noisy hyperspectral image, respectively, the symbol [:] indicates taking all bands, and each sub-block of the noisy hyperspectral image is used as a general representation.
[0010] After the computing device extracts the sub-blocks, in this embodiment, the computing device performs a traversal unfolding of the two-dimensional matrix of each sub-block using a modulo-3 tensor. The traversal unfolding of each sub-block using a modulo-3 tensor can be represented as , where is the two-dimensional matrix of each sub-block.
[0011] In order to improve the accuracy and adaptability of hyperspectral image restoration with mixed noise, in this embodiment, in step 13, the computing device performs segmented noise removal on each of the sub-block two-dimensional matrices. The segmented noise removal includes low-rank matrix image update, L1 norm sparse noise removal and smoothing proportional form penalty matrix noise removal, so as to retain the advantages of different image restorations and give different penalty strengths to different mixed noises.
[0012] The objective function for piecewise noise removal of each sub-block two-dimensional matrix by the computing device can be expressed as: st where is a low-rank matrix to be obtained, containing the main features of the original hyperspectral image, is a sparse noise matrix reflecting the regularization norm, is a sparse noise matrix reflecting the regularization norm, is the Frobenius norm, is a norm regularization parameter, is the norm, is a smoothing norm regularization parameter, is a vectorization operator used to convert the matrix into a vector, is a penalty function, is a rank function, and is the upper bound of the rank. In this embodiment, the objective function minimizes the differences between the known sub-block two-dimensional matrices and the recovered low-rank matrix, the sparse noise matrix reflecting the regularization norm, and the sparse noise matrix reflecting the regularization norm by minimizing the reconstruction error term. The low-rank constraint ensures that the low-rank matrix can capture the main structural features of the image, while the sparse noise matrix reflecting the regularization norm is used to reflect the sparsity of different types of noise, and appropriate penalties can be applied to different noise sources to achieve better image restoration results.
[0013] In this embodiment, in order to improve the computational efficiency of the computing device, the computing device solves the objective function by converting the objective function into a first sub-objective function, a second sub-objective function, and a third sub-objective function. The computing device then performs iterative alternating projection optimization on the low-rank matrix, the sparse noise matrix reflecting the regularization norm, and the sparse noise matrix reflecting the regularization norm using the first sub-objective function, the second sub-objective function, and the third sub-objective function, respectively. The sub-objective functions and their optimization objectives are: where is the first sub-objective function, is the second sub-objective function, and is the third sub-objective function.
[0014] Before the computing device performs the iterative alternating projection optimization, the computing device initializes the low-rank matrix, the sparse noise matrix reflecting the regularization norm, and the sparse noise matrix reflecting the regularization norm. The initial value of the low-rank matrix is set to the sub-block two-dimensional matrix. The sparse noise matrix reflecting the regularization norm and the sparse noise matrix reflecting the regularization norm are set to all-zero matrices. The rank of the low-rank matrix, the maximum number of iterations K, the regularization parameter, and the smoothing regularization parameter are also set.
[0015] The computing device optimizes the low-rank matrix using the first sub-objective function to capture the low-rank structure of the data. Through appropriate matrix decomposition and update strategies, it gradually approximates the low-rank portion of the original data, providing an accurate foundation for subsequent sparse noise recovery. The steps for optimizing the low-rank matrix in this embodiment include: establishing a random matrix; performing QR decomposition on the random matrix and the low-rank matrix, where the QR decomposition can be expressed as: , where is an orthogonal matrix, , and is an upper triangular matrix; multiplying the orthogonal matrix with the low-rank matrix to obtain a new low-rank matrix, which can be expressed as:
[0016] Next, the computing device optimizes the sparse noise matrix reflecting the regularization norm using the second sub-objective function, so as to update the sparse noise matrix reflecting the regularization norm by utilizing the characteristics of the regularization norm. This can effectively identify and separate sparse noise components and improve the accuracy of image restoration. However, since the norm regularization function is not differentiable, this embodiment uses near-end gradient descent (PGD) to solve it. The steps for optimizing the sparse noise matrix reflecting the regularization norm in this embodiment include: dividing the second sub-objective function into a differentiable part and a non-differentiable part, which can be expressed as: , , where is the trace function, and is the differentiable part; differentiating the differentiable part of the second sub-objective function and calculating the gradient, which can be expressed as:; substituting the calculated gradient into the gradient descent method and setting the step size of the gradient descent method to 1 to obtain a differentiable sparse noise matrix, which can be expressed as:; approximating the non-differentiable part of the second sub-objective function using the subgradient. The gradient can be expressed as: , where is the sign function; and the gradient of the differentiable part of the second sub-objective function and the norm regularization parameter are substituted to obtain a new sparse noise matrix reflecting the regularization norm, which can be expressed as: , where is the soft thresholding function, and is the element index of the matrix. In this embodiment, elements below the threshold are set to 0 to avoid elements below the threshold being retained. The regularization norm is converted into a soft thresholding function through the proximal operator. Finally, the sparse noise matrix reflecting the regularization norm can be optimized and solved by the soft thresholding function.
[0017] The computing device optimizes the sparse noise matrix reflecting the regularization norm using the third sub-objective function, including: dividing the third sub-objective function into a differentiable part and a non-differentiable part, which can be expressed as: , , where is the trace function, and is the differentiable part; updating the third sub-objective function for the first time using non-monotonic line search and variable metric near-end gradient descent as: , where is a positive definite symmetric matrix, , , and are metric matrix construction operators; differentiating the differentiable part of the first updated third sub-objective function and calculating the gradient, which can be expressed as: ; substituting the calculated gradient into the gradient descent method to obtain a . The differentially derived sparse noise matrix can be expressed as: ; the positive definite symmetric metric matrix calculated using the differentiable partially sparse noise matrix can be expressed as: ; substituting the differentiable partially sparse noise matrix into the matrix yields the third sub-objective function updated for the second time: ; differentiating the new third sub-objective function and setting it to 0 yields: , which is a zero matrix of size . The result of the differentiation is rearranged to obtain the third sub-objective function updated for the third time: ; and the sparse noise matrix reflecting the regularization norm obtained by iteratively alternating between the positive definite symmetric metric matrix and the third updated third sub-objective function can be expressed as: and .
[0018] After completing the iterative optimization of the alternating projection of the low-rank matrix, the sparse noise matrix reflecting the regularization norm, and the sparse noise matrix reflecting the regularization norm, the computing device determines whether the number of iterations has reached the maximum number of iterations K or whether the convergence condition has been met. The determination of whether the convergence condition has been met can be expressed as: , where is the main iteration number, and and are the thresholds for the changes of the low-rank matrix, the sparse noise matrix reflecting the regularization norm, and the sparse noise matrix reflecting the regularization norm, respectively. If the computing device determines whether the number of iterations has reached the maximum number of iterations K or whether the convergence condition has been met, then step 14 is performed; otherwise, step 13 is performed to iterate the alternating projection optimization of the low-rank matrix, the sparse noise matrix reflecting the regularization norm, and the sparse noise matrix reflecting the regularization norm.
[0019] Please refer to Figure 1. In step 14, the computing device removes segmented noise, that is, it merges the optimized sub-block two-dimensional matrices into a hyperspectral image to complete the hyperspectral image restoration.
[0020] Please refer to Figures 3a and 3b, where Figure 3a is a false color map of the noisy hyperspectral image, where is the spatial size of the image and is the number of bands in the image. Each band of the noisy hyperspectral image contains Gaussian noise with a variation of 0.10 and 10.00% pulse noise, and 25.00% of the bands also contain 5.00% stripe noise. Figure 3b is a false color map of the hyperspectral image recovered through this embodiment, showing that this embodiment can indeed recover the noisy hyperspectral image into a high-quality hyperspectral image.
[0021] Please refer to Figures 4a and 4b, where Figure 3a is a false color map of the noisy hyperspectral image, where is the spatial size of the image and is the number of bands in the image. Each band of the noisy hyperspectral image contains Gaussian noise with a variation of 0.10 and 10.00% pulse noise, and 25.00% of the band also contains 5.00% dead-line noise. Figure 4b is a false color map of the hyperspectral image recovered through this embodiment, showing that this embodiment can indeed recover the noisy hyperspectral image into a high-quality hyperspectral image.
[0022] The present invention uses the computing device to segment and extract the noisy hyperspectral image into blocks and expand it to convert it into a two-dimensional matrix form. Then, the sparsity and low-rank characteristics of each sub-block two-dimensional matrix are restored. Finally, the restored sub-block two-dimensional matrices are merged to obtain the hyperspectral image of image restoration.
[0023] The scope of protection of this invention shall be determined by the appended claims. Any changes and modifications made by those skilled in the art without departing from the spirit and scope of this invention shall fall within the scope of protection of this invention. [Simplified Explanation of the Diagram]
[0024] Figure 1: A flowchart of a hyperspectral image restoration method according to one embodiment of the present invention. Figure 2: A schematic diagram of block extraction and traversal expansion of a noisy hyperspectral image according to one embodiment of the present invention. Figure 3a: A noisy hyperspectral image. Figure 3b: A hyperspectral image obtained by image restoration of Figure 3a according to one embodiment of the present invention. Figure 4a: A noisy hyperspectral image. Figure 4b: A hyperspectral image obtained by image restoration of Figure 4a according to one embodiment of the present invention.
Claims
1. A hyperspectral image restoration method, comprising: a computing device receiving a noisy hyperspectral image; the computing device performing block extraction and traversal expansion on the noisy hyperspectral image to obtain a plurality of sub-block two-dimensional matrices; the computing device performing iterative piecewise noise removal on each of the sub-block two-dimensional matrices, wherein the piecewise noise removal includes low-rank matrix image updating, L1 norm sparse noise removal, and smoothing proportional form penalty matrix noise removal; and the computing device merging the piecewise noise-removed sub-block two-dimensional matrices into a hyperspectral image.
2. As in the hyperspectral image restoration method of request item 1, the block extraction of the noisy hyperspectral image can be expressed as: where, Let be the matrix of the noisy hyperspectral image, be a height, a width and a number of bands of the noisy hyperspectral image, respectively, and be a plurality of sub-blocks obtained by block extraction of the matrix image of the noisy hyperspectral image, where , are the size of each sub-block, and and are the index values along the height and width of the noisy hyperspectral image, respectively, and the symbol [:] indicates taking all bands.
3. The hyperspectral image restoration method of claim 2, wherein the traversal expansion of each sub-block by the computing device can be represented as , where is a two-dimensional matrix of each sub-block, and is a general representation of each sub-block of the noisy hyperspectral image.
4. The hyperspectral image restoration method of request item 1, wherein the computing device performs traversal unfolding of each sub-block two-dimensional matrix using a modulo-3 tensor unfolding.
5. The hyperspectral image restoration method of claim 1, wherein the objective function of the computing device for piecewise noise removal of each sub-block two-dimensional matrix is: st, where is a low-rank matrix to be obtained, is a sparse noise matrix reflecting the regularization norm, is a sparse noise matrix reflecting the regularization norm, is the Frobenius norm, is a norm regularization parameter, is the norm, is a smoothing norm regularization parameter, is a vectorization operator, is a penalty function, is a rank function, and is the upper bound of the rank.
6. The hyperspectral image restoration method of claim 5, wherein the computing device solves the objective function by converting the objective function into a first sub-objective function, a second sub-objective function and a third sub-objective function, and the computing device performs alternating projection optimization on the low-rank matrix, the sparse noise matrix reflecting the regularization norm and the sparse noise matrix reflecting the regularization norm by the first sub-objective function, the second sub-objective function and the third sub-objective function respectively.
7. The hyperspectral image restoration method as described in Request 6, wherein the sub-objective functions and their optimization objectives are: wherein is the first sub-objective function, is the second sub-objective function, and is the third sub-objective function.
8. The hyperspectral image restoration method of claim 7, wherein the computing device optimizes the low-rank matrix by the first sub-objective function by: establishing a random matrix; performing QR decomposition on the random matrix and the low-rank matrix, wherein the QR decomposition can be expressed as: , is an orthogonal matrix, , is an upper triangular matrix, ; multiplying the orthogonal matrix with the low-rank matrix to obtain a new low-rank matrix, expressed as:.
9. The hyperspectral image restoration method of claim 7, wherein the computing device optimizes the sparse noise matrix reflecting the regularization norm using the second sub-objective function, comprising: dividing the second sub-objective function into a differentiable part and a non-differentiable part, which can be expressed as: , , where is the trace function, and is the differentiable part; differentiating the differentiable part of the second sub-objective function and calculating the gradient, which can be expressed as:; substituting the calculated gradient into the gradient descent method to obtain a differentiable sparse noise matrix, which can be expressed as:; approximating the gradient of the non-differentiable part of the second sub-objective function with the subgradient, which can be expressed as: , where is the sign function; and substituting the gradient of the differentiable part of the second sub-objective function and the norm regularization parameter into the matrix to obtain a new sparse noise matrix reflecting the regularization norm, which can be expressed as: , where is the soft thresholding function.
10. The hyperspectral image restoration method of claim 7, wherein the computing device optimizes the sparse noise matrix reflecting the regularization norm using the third sub-objective function, comprising: dividing the third sub-objective function into a differentiable part and a non-differentiable part, which can be expressed as: , , where is the trace function, and is the differentiable part; updating the third sub-objective function for the first time using non-monotonic line search and variable metric near-end gradient descent as: , where is a positive definite symmetric metric matrix, , , and are metric matrix construction operators; differentiating the differentiable part of the first updated third sub-objective function and calculating the gradient, which can be expressed as: ; and calculating the gradient of the differentiable part of the third sub-objective function. The gradient is substituted into the gradient descent method to obtain a differentiable partially sparse noise matrix, which can be expressed as: ; the positive definite symmetric metric matrix is calculated using the differentiable partially sparse noise matrix, which can be expressed as: ; the differentiable partially sparse noise matrix is substituted to obtain the third sub-objective function updated for the second time: ; the new third sub-objective function is differentiated and set to 0 to obtain the third sub-objective function updated for the third time: ; and the sparse noise matrix reflecting the regularization norm is obtained by iteratively alternating between the positive definite symmetric metric matrix and the third updated third sub-objective function, which can be expressed as: and , , are the iteration numbers.