A neural network and optimization model combined driving hyperspectral image restoration method

Abstract

The application discloses a hyperspectral image restoration method combined with a neural network and an optimization model, and belongs to the technical field of digital image processing. The method comprises the following steps: S1, constructing a low-rank tensor restoration model by taking a low-rank prior item in a spectral dimension and a non-local dimension of a hyperspectral image as a constraint; S2, building a neural network, inputting a tensor block T of the hyperspectral image into the neural network, and outputting optimal subspace dimensions in the spectral dimension and the non-local dimension of the hyperspectral image; S3, calculating the optimal subspace; and S4, solving the low-rank tensor restoration model by using an ADMM (Alternating Direction Method of Multipliers) method based on the optimal subspace in the spectral dimension and the optimal subspace in the non-local dimension, and restoring the clear hyperspectral image. The application improves the effect of hyperspectral image restoration, embeds the neural network into the optimization model in a new way, combines the advantages of the optimization model and the neural network, and provides a new idea for the hyperspectral image restoration.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of digital image processing, and more particularly relates to a hyperspectral image restoration method jointly driven by a neural network and an optimization model. BACKGROUND

[0002] In recent years, hyperspectral images have attracted widespread attention in the field of remote sensing. Compared with RGB images, hyperspectral images have rich spectral information, which is related to the physical properties of different materials. Due to this characteristic, hyperspectral images have been widely used in military, agriculture, medicine and other related applications. However, hyperspectral images are easily affected by noise, blur and other common damage caused by photon effects, calibration errors, atmospheric turbulence and equipment limitations, which in turn affects subsequent applications such as target recognition and classification. Therefore, it is crucial to restore hyperspectral images before subsequent applications.

[0003] In the prior art, hyperspectral image restoration methods mainly include model-based methods and network learning-based methods.

[0004] In the model-based method, it includes low-rank matrix restoration method and low-rank tensor restoration method; the low-rank matrix method mainly expands the hyperspectral data form into a matrix form, and then finds the prior information of the hyperspectral image in the matrix form data, such as the low-rank matrix restoration model (LRMR) and the total variation regularization low-rank matrix decomposition model (LRTV). However, the low-rank matrix restoration method destroys the correlation between the spectrum and space of the hyperspectral image when expanding the 3-dimensional form of the hyperspectral image data into a matrix form, and the restored hyperspectral image may be distorted. In order to better utilize the correlation between the spectrum and space of the hyperspectral image, the low-rank tensor restoration method is proposed, which directly processes the hyperspectral image data as a 3-dimensional tensor, such as the total variation regularization low-rank tensor decomposition model (LRTDTV), the weighted low-rank tensor restoration model (WLRTR), and the subspace low-rank tensor restoration model (NGM).

[0005] The network learning-based method realizes the end-to-end restoration of the hyperspectral image, can learn the difference of different structures of the image, and implicitly models the prior information of the hyperspectral image in the network, such as the HSI-DeNet model based on the hollow convolution and residual learning, the HSID-CNN model using the correlation between a certain waveband of the hyperspectral image and its adjacent waveband, and the QRNN3D model using 3D convolution to extract the spectral-space correlation and spectral global correlation of the hyperspectral image.

[0006] However, model-based methods cannot learn the differences in different structures of an image like network methods. Moreover, some hyperparameters in the model have a significant impact on the restoration results and cannot be adaptive. Network methods not only require a large amount of hyperspectral data for learning, but more importantly, their generalization ability is insufficient compared to model-based methods, and they can often only handle a single task. Summary of the Invention

[0007] To address the shortcomings and improvement needs of existing technologies, this invention provides a hyperspectral image restoration method jointly driven by neural networks and optimization models. The aim is to combine the strong expressive power of neural networks with the strong generalization ability of optimization models, utilize the neural network to learn important parameters in the optimization model, characterize the differences in different structures in the image, and thereby apply different constraints to different image structures to achieve adaptive restoration of different structures, thus improving the effect of hyperspectral image restoration.

[0008] To achieve the above objectives, according to one aspect of the present invention, a hyperspectral image restoration method jointly driven by a neural network and an optimization model is provided, comprising:

[0009] Step S1: Construct a low-rank tensor restoration model using the low-rank prior terms of the spectral dimension and non-local dimension of the hyperspectral image as constraints. The input of the low-rank tensor restoration model is the degraded hyperspectral image.

[0010] Step S2: Construct a neural network by inputting the hyperspectral image reconstruction tensor block T into the neural network, and outputting the optimal subspace dimension of the hyperspectral image's spectral dimension and non-local dimension; wherein, Represents a clear hyperspectral image. The operator represents the operation of recombining similar blocks, where i represents the i-th recombined tensor block;

[0011] Step S3: Perform SVD decomposition on the tensor block: in,(·) (j) This indicates that the tensor block is expanded into a matrix along pattern j, and the number of orthogonal vectors in vector V is consistent with the dimension of the optimal subspace; making Q i j =V, calculate the optimal subspace When j∈{2,3}; and j=2, pattern 2 represents the nonlocal dimension. Mode 3 represents the spectral dimension, where j=3, and represents the optimal subspace of the nonlocal dimension. Represents the optimal subspace of the spectral dimension;

[0012] Step S4: Based on the optimal subspace of the spectral dimension and the optimal subspace of the nonlocal dimension, the low-rank tensor restoration model is solved using the ADMM alternating direction multiplier method to restore the clear hyperspectral image.

[0013] Further, the degraded hyperspectral image is:

[0014]

[0015] wherein, denotes a clear hyperspectral image, f(.) represents different degradation operators, denotes random noise.

[0016] Further, the degradation operator f(.) is one or more of: denoising, deblurring or inpainting of the hyperspectral image.

[0017] Further, the low-rank tensor recovery model is:

[0018]

[0019] wherein, is a data fidelity term, H j (x) denotes a prior term of the hyperspectral image, ω j denotes a weighting coefficient.

[0020] Further, according to the non-local similarity prior and the low-rank prior of the hyperspectral image, a prior term H j (x) is constructed in the spectral dimension and the non-local dimension of the hyperspectral image, respectively.

[0021]

[0022] wherein, is an intermediate variable, rank(.) represents a rank operator, (× j ) represents a mode j product of a tensor, λ i is a hyperparameter.

[0023] Further, in step S2, the neural network comprises two RegNet classification convolutional neural networks with the same structure.

[0024] Further, in step S2, the method further comprises training the two RegNet classification convolutional neural networks;

[0025] The data set of the two RegNet classification convolutional neural networks is a CAVE data set.

[0026] The labels are respectively: the optimal subspace dimension in the spectral dimension and the optimal subspace dimension in the non-local dimension of the hyperspectral image tensor block searched by the low-rank optimization algorithm.

[0027] According to a second aspect of the present application, a hyperspectral image recovery system driven by a neural network and an optimization model is provided, comprising:

[0028] a low-rank tensor restoration model construction module, configured to construct a low-rank tensor restoration model by taking a low-rank prior term in a spectral dimension and a non-local dimension of the hyperspectral image as a constraint, and taking the degraded hyperspectral image as an input of the low-rank tensor restoration model;

[0029] an optimal subspace dimension acquisition module, configured to build a neural network, input a tensor block T of the hyperspectral image into the neural network, and output optimal subspace dimensions in the spectral dimension and the non-local dimension of the hyperspectral image; wherein, represents a clear hyperspectral image, represents an operation operator of reorganizing similar blocks, and i represents an i-th reorganized tensor block;

[0030] an optimal subspace calculation module, configured to perform SVD decomposition on the tensor block: wherein, (·) (j) represents that the tensor block is unfolded into a matrix along a mode j, the number of orthogonal vectors in the vector V is consistent with the optimal subspace dimension; and Q i j = V, the optimal subspace is calculated j is in {2, 3}; when j = 2, the mode 2 represents the non-local dimension, represents a non-local dimension optimal subspace, when j = 3, the mode 3 represents the spectral dimension, represents a spectral dimension optimal subspace;

[0031] a hyperspectral image restoration module, configured to solve the low-rank tensor restoration model by using an ADMM alternating direction multiplier method based on the spectral dimension optimal subspace and the non-local dimension optimal subspace, and restore the clear hyperspectral image.

[0032] According to a third aspect of the present application, an information data processing terminal is provided, which is used to implement the neural network and optimization model joint-driven hyperspectral image restoration method according to any one of the first aspect.

[0033] According to a fourth aspect of the present application, a computer storage medium is provided, including instructions, wherein the instructions make the computer execute the neural network and optimization model joint-driven hyperspectral image restoration method according to any one of the first aspect when the instructions are run on the computer.

[0034] Overall, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0035] (1) The application provides a hyperspectral image restoration method combining a network and an optimization model, a low-rank tensor restoration model is established, a low-rank prior item is constructed as a constraint of the low-rank tensor restoration model, and the optimal subspace dimension of a reorganized hyperspectral image block in a spectral dimension and a non-local dimension is learned in the low-rank tensor restoration model under the optimization framework of the low-rank tensor restoration model; the optimal subspace in the spectral dimension and the non-local dimension is calculated based on the optimal subspace dimension learned by the neural network, and the low-rank tensor restoration model is solved by numerical optimization. That is, the neural network is embedded in the optimization model, the global difference and the local similarity of the image can be learned in the unified optimization model framework, the optimal subspace dimension in the spectral dimension and the non-local dimension is adaptively obtained, and the optimal subspace is constructed, the difference of different structures of the image is perceived by the network, different constraints are applied to different image structures, the different structures of the image are equally treated in the simple optimization model, the global difference and the local similarity of the hyperspectral image are fully tapped, and the hyperspectral image restoration effect is improved. Meanwhile, the combination of the new neural network and the optimization model utilizes the strong generalization ability of the optimization model and the strong expression ability of the neural network, and provides a new idea for the hyperspectral image restoration.

[0036] (2) The hyperspectral image restoration method combining a network and an optimization model is analyzed, a low-rank tensor restoration model combining a network and an optimization is designed, and the low-rank tensor restoration model can restore the multi-task hyperspectral image in the unified optimization framework of the model, and can realize denoising, deblurring and repairing of the hyperspectral image. Simulation experiments show that, in the degradation scenes of random noise, blur and repair, the application is superior to existing algorithms in subjective visual effect and objective evaluation index, and can obtain good restoration effect. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 It is a flowchart of the hyperspectral image restoration method combining a network and an optimization provided by the embodiment of the application.

[0038] Figure 2 It is the result of denoising, deblurring and repairing of images in a CAVE data set by the method, wherein (a)-(f) in the figure respectively represent a noise image, a denoised image, a blurred image, a deblurred image, a repaired degraded image and a repaired image.

[0039] Figure 3These are simulation experiment diagrams comparing the method provided in the embodiments of the present invention with the method in the prior art. In the diagrams, (a)-(h) represent the original clear image, the noisy image, the LRMR restored image, the BM4D restored image, the TDL restored image, the NGM restored image, the KBRreg restored image, and the restored image of the present invention, respectively. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0041] like Figure 1 As shown, this invention provides a hyperspectral image restoration method jointly driven by a neural network and an optimization model, comprising the following steps:

[0042] Step S1: Construct a low-rank tensor restoration model using the low-rank prior terms of the spectral dimension and non-local dimension of the hyperspectral image as constraints. The input of the low-rank tensor restoration model is the degraded hyperspectral image.

[0043] Step S2: Construct a neural network. Input the hyperspectral image reconstruction tensor block T into the neural network, and output the optimal subspace dimension of the spectral dimension and nonlocal dimension of the hyperspectral image in the low-rank tensor reconstruction model; where, Represents a clear hyperspectral image. Let i represent the operator for recombining similar blocks, where i represents the i-th recombined tensor block; This is the third-order low-rank tensor obtained by recombining similar blocks;

[0044] Step S3: For tensor blocks After expanding the matrix along pattern j, perform SVD decomposition: in,(·) (j) This means that the tensor block is expanded into a matrix along pattern j, and the number of orthogonal vectors in vector V is consistent with the dimension of the optimal subspace, such that Q... i j =V, thus obtaining the optimal subspace of the spectral dimension and the nonlocal subspace. j∈{2,3}; when j=2, mode 2 represents the nonlocal dimension, and when j=3, mode 3 represents the spectral dimension;

[0045] Step S4: Based on the optimal subspace of the spectral dimension and the non-local subspace, the low-rank tensor restoration model is solved using the ADMM alternating direction multiplier method to obtain the restored clear hyperspectral image.

[0046] Specifically, in step S1, the degraded hyperspectral image wherein, denotes the clear hyperspectral image, denotes the degraded hyperspectral image observed under different degradation operators, n l , n c , n b denote the height, width and band number of the hyperspectral image respectively, denotes random noise, denotes different degradation operators, such as denoising, deblurring, inpainting;

[0047] The degraded hyperspectral image In the maximum posterior probability framework, a low-rank tensor restoration model is determined:

[0048]

[0049] In the formula, is a data fidelity term, H j (x) represents a regularization term, ω j denotes a weighting coefficient, j∈{2,3}, j=2 represents a non-local dimension, j=3 represents a spectral dimension, such as: H2(x), H3(x) respectively represent non-local dimension, spectral dimension regularization prior item information; ω2, ω3 respectively represent the weighting coefficients of the non-local dimension and the spectral dimension prior item information.

[0050] Specifically, according to the image non-local similarity prior, the following is constructed: According to the image low-rank prior, the following is constructed: wherein, is an intermediate variable, and rank(.) represents a rank operator.

[0051] The spectral dimension and non-local dimension prior items of the hyperspectral image are H j (x):

[0052]

[0053] wherein, is a third-order low-rank tensor obtained by reorganizing similar blocks, (× j ) represents a mode-j product of a tensor, λ i is a hyperparameter.

[0054] The low-rank tensor restoration model can be converted into:

[0055]

[0056] Specifically, in step S2, a neural network is built, and in the present embodiment, two RegNet classification convolutional neural networks with the same structure are selected to learn the optimal subspace dimension of the spectral dimension and the non-local dimension respectively.

[0057] In the process of training two networks, the CAVE dataset is used, and the labels are the optimal subspace dimensions of spectral dimension and non-local dimension of each picture tensor block in the dataset searched by low-rank optimization algorithm. The dataset is input into two neural networks respectively, and the network is trained until the network converges, and the optimal subspace dimensions of spectral dimension and non-local dimension are obtained:

[0058]

[0059]

[0060] Where b represents the optimal subspace dimension of spectral dimension, k represents the optimal subspace dimension of non-local dimension, and net() is the trained neural network.

[0061] Specifically, in step S3, the mode j expansion is performed on and then SVD decomposition is performed:

[0062]

[0063] The number of orthogonal vectors in V is learned by the network, which is consistent with the optimal subspace dimension of spectral dimension and non-local dimension, and Q is taken as i j = V, then the optimal non-local subspace is: The optimal spectral subspace is

[0064] Specifically, in step S4, it includes:

[0065] Solve the intermediate variable Take out the two items related to , and put the low-rank constraint term into the core norm to constrain, and the following equation can be obtained:

[0066]

[0067] Where, represents the low-rank constraint, and the above equation is solved by WNNM algorithm.

[0068] Solve the clear image to be restored After taking out the two items related to , the following equation can be obtained:

[0069]

[0070] Introduce auxiliary variable Let By using the augmented Lagrange algorithm, the constrained problem is changed into an unconstrained problem, and the following equation is obtained:

[0071]

[0072] Where β is the penalty parameter. It is a Lagrange multiplier.

[0073] Take out the above formula and After considering the two relevant terms, we obtain the following equation:

[0074]

[0075] Find the closed-form solution to the above equation:

[0076]

[0077] in, This represents the inverse fast Fourier transform. The symbol represents the Fast Fourier Transform, and * represents the conjugate operation. Represents the element-wise multiplication operation. Represents a unit tensor.

[0078] For auxiliary variables After taking the relevant terms, we have:

[0079]

[0080] Solving the above equation is equivalent to solving the following equation:

[0081]

[0082] Therefore, its closed-form solution is:

[0083]

[0084] During each iteration, its value is updated:

[0085]

[0086] β (l+1) =τβ (l)

[0087] We obtain the penalty parameter β and the Lagrange multiplier.

[0088] This invention also provides a hyperspectral image restoration system jointly driven by a neural network and an optimization model, mainly comprising:

[0089] The low-rank tensor restoration model construction module is configured to construct a low-rank tensor restoration model by taking a low-rank prior item in a spectral dimension and a non-local dimension of the hyperspectral image as a constraint, and the input of the low-rank tensor restoration model is the degraded hyperspectral image.

[0090] The optimal subspace dimension acquisition module is configured to build a neural network, input the reorganized tensor block T of the hyperspectral image into the neural network, and output optimal subspace dimensions in the spectral dimension and the non-local dimension of the hyperspectral image. represents a clear hyperspectral image, represents a reorganization similar block operation operator, and i represents the i-th reorganized tensor block.

[0091] The optimal subspace calculation module is configured to perform SVD decomposition on the tensor block. wherein (·) (j) represents unfolding the tensor block into a matrix along mode j, the number of orthogonal vectors in the vector V should be consistent with the optimal subspace dimension, and the optimal subspace dimension is obtained by the network; and Q i j = V, and the optimal subspace is calculated j ∈ {2, 3}; when j = 2, mode 2 represents the non-local dimension, represents the non-local optimal subspace, and when j = 3, mode 3 represents the spectral dimension, represents the spectral optimal subspace.

[0092] The hyperspectral image restoration module is configured to solve the low-rank tensor restoration model by using an ADMM alternating direction multiplier method based on the spectral optimal subspace and the non-local optimal subspace, and restore the clear hyperspectral image.

[0093] Specifically, the degraded hyperspectral image is:

[0094]

[0095] wherein, represents a clear hyperspectral image, f(.) represents different degradation operators, represents random noise.

[0096] Specifically, the low-rank tensor restoration model is:

[0097]

[0098] In the formula, is a data fidelity term, H j (x) represents a hyperspectral image spectral dimension and non-local dimension prior item, ω j represents a weighting coefficient.

[0099] The application further provides a computer readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the hyperspectral image restoration method driven by the neural network and the optimization model.

[0100] The specific simulation experiment in the embodiment is as follows:

[0101] (1) Simulation data construction process

[0102] The random noise degraded image is generated by two steps: first, the image gray range is scaled to the range [0, 255] to obtain an original clear image; and second, the Matlab function rand is used to generate an image containing random noise, and the noise level can be set to multiple levels.

[0103] (2) Data set

[0104] In order to evaluate the effect of the method, the embodiment of the application selects the CAVE data set for experiment. The CAVE data set contains 32 different scenes, and the picture size is 512*512*31, and the wavelength range is 400nm-700nm.

[0105] Training data set construction: first, in the CAVE data set, the noise level range is set to 1-50, with an interval of 1, and a total of 50 noise levels, and 50 degraded images of different noise levels are simulated for each image; second, for each clear image, 500 25*100*31 (spatial dimension*non-local dimension*spectral dimension) tensor blocks are randomly sampled, and 500 noisy tensor blocks are also selected at the corresponding positions in the 50 noise level degraded images corresponding to the clear image. For the 32 images in the CAVE data set, there are a total of 800,000 noisy data; third, for each noisy data, all possible subspace dimensions of the non-local dimension are enumerated, and the restoration result under different subspace dimensions is obtained by using a low-rank optimization algorithm. The best subspace dimension of the restoration result is the optimal subspace dimension of the non-local dimension, and the optimal subspace dimension of the spectral dimension can also be obtained in the same way. Fourth, the obtained optimal subspace dimension of the non-local dimension and the optimal subspace dimension of the spectral dimension are used as labels to train two networks, one network learns the optimal subspace dimension of the non-local dimension, and the other network learns the optimal subspace dimension of the spectral dimension. The 800,000 noisy data are used as network input, and the optimal subspace dimension is used as ground truth to train the network.

[0106] (3) Modeling and solving the model according to the above steps.

[0107] (4) Algorithm implementation details

[0108] The compromise is made in the recovery effect and algorithm speed, the iteration number of denoising is set to 10, the iteration number of deblurring and restoration is set to 100, and the proportional regularization parameters of the non-local dimension recovery route and the spectral dimension recovery route are respectively γ1=0.6 and γ2=0.4.

[0109] (5) Five hyperspectral image restoration algorithms are selected as comparative examples for comparative experiments, and the comparative algorithms include: LRMR algorithm, BM4D algorithm, TDL algorithm, NGM algorithm and KBRreg algorithm.

[0110] (6) Performance evaluation index

[0111] The peak signal-to-noise ratio (PSNR) and the ERGAS index are used to evaluate the restoration performance of the algorithm:

[0112]

[0113] wherein, R max is the maximum value of the image radian range, such as the maximum value R max of an 8-bit image, and MSE is the mean square error, which is defined as shown in the following formula,

[0114]

[0115] wherein, M is the number of pixels in the image, i is the pixel index, x is the restored image, x is the reference image, the value of PSNR is larger, the quality of the restored image is higher, and the unit of PSNR is dB.

[0116]

[0117] wherein, λ is a constant, N is the number of bands, RMSE is the root mean square error, and μ i is an average value of a band. Generally, ERGAS mainly considers the overall situation of spectral variation, and reflects the spectral quality, and the smaller the value is, the better the spectral quality is.

[0118] (7) Simulation experiment effect: Figure 2 (a)-(f) in the figure are the results of denoising, deblurring and restoration of images in the CAVE data set, and are used to show that the method of the application can be used for various different hyperspectral image restoration tasks. As can be seen from the figure, the algorithm of the application restores well for the above-mentioned degradation tasks, and the restoration effect is clear.

[0119] Figure 3are visual comparisons of different algorithm denoising results, and the evaluation indexes of each subgraph are PSNR and ERGAS values, which are used to simultaneously show the recovery indexes and visual quality of the algorithm. (a)-(h) in the figure respectively represent the original clear image, the noise image, the LRMR recovery image, the BM4D recovery image, the TDL recovery image, the NGM recovery image, the KBRreg recovery image and the recovery image of the present application, and Table 1 is the PSNR and ERGAS values obtained by simulation of several methods, which respectively correspond to (b)-(h) in the figure. It can be seen from the table that the method of the present application not only has better image denoising indexes than other methods, but also has better image detail preservation than other methods.

[0120] Table 1 simulation results of several methods

[0121]

[0122] Those skilled in the art can easily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of hyperspectral image restoration jointly driven by a neural network and an optimization model, characterized in that, The method comprises the following steps: Step S1, constructing a low-rank tensor restoration model by taking a low-rank prior item in a spectral dimension and a non-local dimension of a hyperspectral image as a constraint, wherein an input of the low-rank tensor restoration model is a degraded hyperspectral image; Step S2, building a neural network, inputting the hyperspectral image reorganization tensor block T into the neural network, and outputting the optimal subspace dimension of the spectral dimension and the non-local dimension of the hyperspectral image; wherein, , represents a clear hyperspectral image, represents the operation operator of reorganizing similar blocks, and i represents the i th reorganization tensor block; Step S3, SVD decomposition is performed on the tensor block: wherein, denotes unfolding the tensor block along mode j into a matrix, the number of orthogonal vectors in vector V is consistent with the optimal subspace dimension; make , calculate the optimal subspace , ; when j=2, mode 2 represents a non-local dimension, represent the optimal subspace of non-local dimension, when j=3, mode 3 represents a spectral dimension, represent the optimal subspace of spectral dimension; Step S4, solving the low-rank tensor restoration model by using an ADMM (Alternating Direction Method of Multipliers) method based on an optimal subspace in the spectral dimension and an optimal subspace in the non-local dimension, and restoring the clear hyperspectral image; The low-rank tensor restoration model is as follows: In the formula, is a data fidelity term, represents a prior term of the hyperspectral image, represents a weighting coefficient.

2. The neural network and optimization model jointly driven hyperspectral image restoration method according to claim 1, characterized in that, The degraded hyperspectral image Is: wherein, represents a clear hyperspectral image, represent different degradation operators, represents random noise.

3. The neural network and optimization model jointly driven hyperspectral image restoration method according to claim 2, characterized in that, The degradation operator is one or more of denoising, deblurring, or inpainting of the hyperspectral image.

4. The neural network and optimization model jointly driven hyperspectral image restoration method according to claim 1, characterized in that, According to the non-local similarity prior and the low-rank prior of the hyperspectral image, a prior term is constructed in the spectral dimension and the non-local dimension of the hyperspectral image : wherein is an intermediate variable, denotes the rank operator, denotes the mode j product of the tensor, is a hyperparameter.

5. The neural network and optimization model jointly driven hyperspectral image restoration method according to claim 1, characterized in that, In step S2, the neural network comprises two RegNet classification convolutional neural networks with the same structure.

6. The neural network and optimization model jointly driven hyperspectral image restoration method according to claim 5, characterized in that, In step S2, the two RegNet classification convolutional neural networks are trained. A data set of the two RegNet classification convolutional neural networks is a CAVE data set. The labels are respectively a spectral dimension optimal subspace dimension and a non-local dimension optimal subspace dimension of a hyperspectral image tensor block searched by using a low-rank optimization algorithm.

7. A hyperspectral image restoration system jointly driven by a neural network and an optimization model, characterized in that, The method comprises the following steps: A low-rank tensor restoration model construction module is configured to construct a low-rank tensor restoration model by taking a low-rank prior item in a spectral dimension and a non-local dimension of a hyperspectral image as a constraint, wherein an input of the low-rank tensor restoration model is a degraded hyperspectral image; An optimal subspace dimension obtaining module is configured to build a neural network, input a hyperspectral image reorganized tensor block T into the neural network, and output optimal subspace dimensions of a spectral dimension and a non-local dimension of the hyperspectral image. , represents a clear hyperspectral image, represents an operation operator of reorganizing similar blocks, and i represents an i-th reorganized tensor block. An optimal subspace calculation module is configured to perform SVD decomposition on the tensor block: , wherein, represents unfolding the tensor block into a matrix along mode j, the number of orthogonal vectors in the vector V is consistent with the optimal subspace dimension; so that , the optimal subspace is calculated , ; when j = 2, mode 2 represents a non-local dimension, represents a non-local dimension optimal subspace, when j = 3, mode 3 represents a spectral dimension, represents a spectral dimension optimal subspace; A hyperspectral image restoration module is configured to solve the low-rank tensor restoration model by using an ADMM (Alternating Direction Method of Multipliers) method based on an optimal subspace in the spectral dimension and an optimal subspace in the non-local dimension, and restore the clear hyperspectral image; The low-rank tensor restoration model is as follows: wherein is a data fidelity term, represents a prior term for the hyperspectral image, represents a weighting coefficient.

8. An information data processing terminal, characterized by The information data processing terminal is configured to implement the neural network and optimization model jointly driven hyperspectral image restoration method according to any one of claims 1 to 6.

9. A computer storage medium comprising instructions, wherein, The instructions, when executed on the computer, cause the computer to perform the neural network and optimization model jointly driven hyperspectral image restoration method according to any one of claims 1 to 6.

Images