Hyperspectral fusion imaging method and device based on deep nonlinear transformation tensor low-rank representation

Abstract

The invention discloses a hyperspectral fusion imaging method and device based on deep nonlinear transformation tensor low-rank representation, and the method comprises the following steps: collecting a priori multispectral image while collecting hyperspectral compression measurement in an observation scene, and taking the two data as input at the same time; based on tensor representation and deep learning theories, novel deep nonlinear transformation tensor low-rank regularization for the hyperspectral image is established, and global high-dimensional low-rank correlation of the hyperspectral image in a deep nonlinear transformation domain is fully described; taking the regularization item as a target function, modeling compression imaging and a spectrum degradation process into two data fidelity items, and constructing a fusion imaging model and a loss function; and minimizing a loss function through a learning algorithm to optimize the model, and fusing and reconstructing a complete hyperspectral image. According to the novel method and the novel device provided by the invention, high-precision fusion imaging of the observation scene can be realized without manual intervention and explicit intermediate steps.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral intelligent imaging processing, and particularly relates to a hyperspectral fusion imaging method and device based on deep non-linear transformation tensor low-rank representation. Background Art

[0002] Hyperspectral imaging technology can image in hundreds of continuous bands from visible light to near-infrared, obtaining fine spectral information of the observed scene. With the help of these spectral information, accurate analysis and recognition of the observed ground objects can be achieved, and high-precision results can be obtained in applications such as target detection, land cover classification, and change detection. However, traditional imaging systems usually need to expose the observed scene multiple times to obtain a complete hyperspectral data cube, which not only greatly sacrifices the acquisition time resolution, making it difficult to capture the spectral information of dynamic scenes, but also causes huge pressure on storage, transmission, etc. due to the large amount of data collected for hyperspectral images, severely limiting the application in important scenarios such as military and aerospace. The coded aperture snapshot spectral imaging technology obtains hyperspectral compressed measurements through dimensionality reduction acquisition of the observed scene, and then reconstructs a complete hyperspectral image from the compressed measurements through imaging algorithms, achieving the ability to quickly capture rich spectral information in a single exposure. At the same time, the compressed acquisition mode greatly reduces the storage and transmission costs of large-scale hyperspectral information.

[0003] Reconstructing a complete hyperspectral image from the observed compressed measurements through imaging algorithms is a typical compressive sensing problem with high underdetermination. Currently, common solutions include model optimization methods and deep learning methods. The former completes the reconstruction process by manually mining the linear representation of the intrinsic prior of the hyperspectral image, establishing an optimization model, and iteratively solving it; the latter, in a data-driven manner, constructs a deep network to learn the non-linear mapping relationship from the compressed measurements to the complete image for reconstruction. However, the method of manually designing prior regularization in the model optimization method often lacks the ability to represent the correlation of the high-dimensional structure of complex hyperspectral images, resulting in insufficient reconstruction accuracy; the deep learning method not only requires a large amount of high-quality labeled data for training, but also has the problem of poor interpretability, resulting in limited application effects in real scenarios. In addition, due to the relatively limited information carried by single-source compressed measurements itself, the reconstruction equation is overly underdetermined, making it difficult to achieve high-fidelity imaging of spatial details. Summary of the Invention

[0004] The object of the present invention is to provide a hyperspectral fusion imaging method and device based on the low-rank representation of deep non-linear transformation tensors. Aiming at the problems existing in the existing imaging algorithms, such as the insufficient representation ability of the linear prior optimization model, the poor interpretability of the deep non-linear network and the excessive dependence on a large amount of high-quality labeled data, as well as the defect of insufficient available information in the single-source compressive imaging system, the idea of fusion imaging is used in the compressive imaging system. A multi-spectral imaging sensor that can collect data at low cost is introduced, and the obtained multi-spectral image is used as the prior image. A fusion imaging method that can combine the advantages of model optimization methods and deep learning is designed to realize the in-depth exploration of the high-dimensional structure correlation of hyperspectral images while fully integrating the information of compressive measurements and prior multi-spectral images, and finally achieve high-precision hyperspectral fusion imaging.

[0005] To achieve the above object, the technical solution adopted by the present invention is: a hyperspectral fusion imaging method based on the low-rank representation of deep non-linear transformation tensors, including the following steps:

[0006] Step 1: While collecting hyperspectral compressive measurements in the observation scene, collect a prior multi-spectral image, and use the two data as inputs at the same time. Represent the hyperspectral compressive measurements and the prior multi-spectral image as a second-order matrix and a third-order tensor respectively;

[0007] Step 2: Based on tensor representation and deep learning theory, establish a new deep non-linear transformation tensor low-rank regularization for hyperspectral images. This regularization includes a deep low-rank transformation network and a tensor nuclear norm to fully characterize the global high-dimensional low-rank correlation of hyperspectral images in the deep non-linear transformation domain;

[0008] Step 3: Take the deep non-linear transformation tensor low-rank regularization term as the objective function, and model the compressive imaging and spectral degradation processes as two data fidelity terms respectively to construct a fusion imaging model and a loss function;

[0009] Step 4: Minimize the loss function through the adaptive moment estimation learning algorithm to optimize the fusion imaging model, fully integrate the information of compressive measurements and prior multi-spectral images, and reconstruct a complete hyperspectral image.

[0010] The present invention also provides a hyperspectral fusion imaging device based on the low-rank representation of deep non-linear transformation tensors for implementing the above-mentioned hyperspectral fusion imaging method based on the low-rank representation of deep non-linear transformation tensors. The device includes:

[0011] An encoded aperture spectral imaging system for converting the incident light energy obtained from the observation scene into compressive measurement data;

[0012] A multi-spectral sensor for converting the incident light energy obtained from the observation scene into multi-spectral image data;

[0013] A beam splitter, which is used to capture the incident light energy of an observation scene and divide it into two parts, and input them into a coded aperture spectral imaging system and a multispectral sensor respectively;

[0014] A storage unit: which is used to store the hyperspectral compressed measurement and prior multispectral image data pairs collected by the coded aperture spectral imaging system and the multispectral sensor in the observation scene;

[0015] A calculation unit: which is used to read the hyperspectral compressed measurement converted prior multispectral image data pairs in the storage unit as input, execute the aforementioned hyperspectral fusion imaging method based on deep non-linear transformation tensor low-rank representation, output the reconstructed high-precision complete hyperspectral image, and store the fusion imaging result on the storage unit.

[0016] An electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the above method are implemented.

[0017] A computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the above method are implemented.

[0018] A computer program product, including a computer program. When the computer program is executed by a processor, the steps of the above method are implemented.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The hyperspectral fusion imaging method based on deep non-linear transformation tensor low-rank representation provided by the present invention introduces prior multispectral image information into the compressive imaging system, and combines the advantages of model optimization methods and deep learning methods in the imaging algorithm, greatly improving the disadvantages of the existing hyperspectral imaging system with low reconstruction accuracy and narrow application range. (2) The hyperspectral fusion imaging method based on deep non-linear transformation tensor low-rank representation provided by the present invention improves the limitation of the existing tensor low-rank representation theory that only considers the high-dimensional low-rank correlation of images in the fixed linear transformation domain when dealing with hyperspectral fusion imaging problems by establishing a new type of deep non-linear transformation tensor low-rank regularization, and obtains a more compact low-rank representation of hyperspectral images through effective deep transformation network design, significantly improving the accuracy of the existing imaging methods. (3) The hyperspectral fusion imaging method based on deep non-linear transformation tensor low-rank representation provided by the present invention improves the limitation of the existing deep learning theory that requires a large amount of labeled data to effectively learn the deep non-linear features of hyperspectral images when dealing with hyperspectral fusion imaging problems through reasonable optimization objective function and unsupervised loss function design, and at the same time maintains the advantage of strong interpretability of the tensor low-rank representation theory, and significantly improves the generalization of the existing imaging methods in a data-adaptive manner. (4) The hyperspectral fusion imaging device based on deep non-linear transformation tensor low-rank representation provided by the present invention couples the coded aperture spectral imaging system, multispectral sensor, storage unit and computing unit, and can effectively and conveniently realize hyperspectral fusion imaging, and has significant application value in real-scene hyperspectral imaging scenarios. Description of the Drawings

[0020] Figure 1 is the overall flowchart of the hyperspectral fusion imaging method based on deep non-linear transformation tensor low-rank representation of the present invention.

[0021] Figure 2 is the structural diagram of the deep non-linear low-rank transformation network designed in the fusion imaging method proposed by the present invention.

[0022] Figure 3 is the schematic structural diagram of the hyperspectral fusion imaging device with deep non-linear transformation tensor low-rank representation of the present invention.

[0023] Figure 4 is the schematic diagram of hyperspectral compressive measurement collected by the coded aperture spectral imaging system.

[0024] Figure 5 is the schematic diagram of the prior multispectral image collected by the multispectral sensor.

[0025] Figure 6 is the high-precision fusion imaging result diagram obtained by the present invention. Detailed Embodiments

[0026] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. According to the following description and claims, the advantages and features of the present invention will be clearer. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise scales, only for the purpose of facilitating and clearly assisting in explaining the purpose of the embodiments of the present invention.

[0027] As Figure 1 shown, the hyperspectral fusion imaging method based on deep non-linear transformation tensor low-rank representation of the present invention includes the following steps:

[0028] Step 1: While collecting hyperspectral compressive measurements in the observation scene, collect a prior multispectral image, take the two data as inputs at the same time, and represent the hyperspectral compressive measurements and the prior multispectral image as a second-order matrix and a third-order tensor respectively;

[0029] Step 2: Based on tensor representation and deep learning theory, establish a new type of deep non-linear transformation tensor low-rank regularization for hyperspectral images, which includes a deep low-rank transformation network and a tensor nuclear norm to fully characterize the global high-dimensional low-rank correlation of hyperspectral images in the deep non-linear transformation domain;

[0030] Step 3: Take this deep non-linear transformation tensor low-rank regularization term as the objective function, and at the same time model the compressive imaging and spectral degradation processes as two data fidelity terms respectively, and construct a fusion imaging model and a loss function;

[0031] Step 4: Minimize the loss function through the adaptive moment estimation learning algorithm to optimize this fusion imaging model, fully fuse the information of the compressive measurements and the prior multispectral image, and reconstruct a complete hyperspectral image.

[0032] Specifically, in Step 1, while collecting hyperspectral compressive measurements in the observation scene, collect a prior multispectral image, take the two data as inputs at the same time, and represent the hyperspectral compressive measurements and the prior multispectral image as a second-order matrix and a third-order tensor respectively, which specifically includes the following steps:

[0033] Step 1.1: Represent the collected hyperspectral compressive measurements as a second-order matrix where, H and W represent the sizes of the spatial dimensions of the hyperspectral image, and B represents the size of the spectral dimension.

[0034] Step 1.2: Represent the collected prior multispectral image as a third-order tensor where, b is the number of channels of the prior multispectral image, and satisfies the condition b << B.

[0035] Specifically, based on tensor representation and deep learning theory, step 2 establishes a new deep non-linear transformation tensor low-rank regularization for hyperspectral images. This regularization includes a deep low-rank transformation network and a tensor nuclear norm to fully characterize the global high-dimensional low-rank correlation of hyperspectral images in the deep non-linear transformation domain. It specifically includes the following steps:

[0036] Step 2.1: Represent the complete hyperspectral image to be obtained by fusion imaging as a third-order tensor

[0037] Step 2.2: As Figure 2 shown, construct a deep non-linear low-rank transformation network for the hyperspectral image tensor This network consists of three parts, where represents the number of feature dimensions after transformation. The first part of this network contains k branches to extract non-linear transformation features of the high-dimensional redundant hyperspectral image tensor at different scales. Each branch consists of the following two-layer network structure: (1) 1×1 convolutions and a LeakyReLU linear rectification unit. This layer of network structure projects the hyperspectral image tensor along the spectral dimension into a high-dimensional feature space, fully considering the correlation of the spectral dimension; (2) 3×3 convolutions and a LeakyReLU linear rectification unit. This layer of network structure further enhances the non-linear representation of the hyperspectral image features while fully considering the correlation of the spatial dimension. The second part of this network is a concatenation operator that concatenates the k non-linear transformation features obtained from the k branches along the spectral dimension. The concatenated feature is dimensional. The third part of this network repeats the following structure 2 times: 3×3 convolutions and a LeakyReLU linear rectification unit; this part of the network structure compresses the concatenated non-linear transformation features into dimensions and further enhances its non-linear representation.

[0038] Step 2.3: Use this deep non-linear low-rank transformation network to project the hyperspectral image tensor into a non-linear transformation domain to obtain the deep non-linear transformation representation of the hyperspectral image tensor The mathematical expression of this process is That is, by replacing the fixed linear transformation used in the existing tensor singular value decomposition algebraic framework, such as Fourier transform or discrete cosine transform, etc., with the data-adaptive deep non-linear network designed in step 2.2 the compactness and robustness of the tensor low-rank representation in the transformation domain are enhanced.

[0039] ​Step 2.4: Construct a deep non-linear transformation tensor low-rank regularization term in combination with the tensor nuclear norm. The mathematical expression of this new regularization term is That is, the hyperspectral image tensor The deep non-linear transformation tensor low-rank regularization of is computationally equivalent to the nuclear norm of all the i-th forward slice matrices under its deep non-linear transformation . While maintaining the algebraic structure of the tensor nuclear norm minimization method, this new regularization term further characterizes the global high-dimensional low-rank correlation of the hyperspectral image tensor in the deep non-linear transformation domain with stronger representation ability.

[0040] Step 2.5: Construct a deep non-linear inverse transformation network for the obtained deep non-linear transformation representation tensor to project the deep non-linear low-rank representation of the hyperspectral image tensor back to the original space. The deep non-linear inverse transformation network consists of the following network structures: (1) 3×3 convolutions and a LeakyReLU linear rectification unit; (2) B 1×1 convolutions and a LeakyReLU linear rectification unit. That is, by using the deep non-linear network to replace the fixed linear inverse transformation used in the tensor singular value decomposition algebraic framework, such as the inverse Fourier transform or the inverse discrete cosine transform, etc., the hyperspectral image tensor after low-rank approximation is obtained.

[0041] Specifically, Step 3 takes the deep non-linear transformation tensor low-rank regularization term constructed in Step 2 as the objective function, and models the compressive imaging and spectral degradation processes as two data fidelity terms respectively, to construct a fused imaging model and a loss function, which specifically includes the following steps:

[0042] Step 3.1: Model the process of obtaining compressed measurements by hyperspectral image compressive imaging as the first data fidelity term, which is composed of the reconstructed fused imaging result and the observed compressed measurement X. Its mathematical expression is: where the function Ψ(·): represents the mathematical process of the coded aperture spectral imaging system for compressive imaging of the observed scene, and the vector represents the vectorized result of the compressed measurement X, and its dimension T = H×(W + B - 1).

[0043] Step 3.2: Model the process of the hyperspectral image degrading into a multi-spectral image in the spectral dimension as the second data fidelity term, which is composed of the reconstructed fused imaging result and the prior multi-spectral image . Its mathematical expression is: where the matrix denotes the spectral response function that degrades a hyperspectral image with dimension B into a multispectral image with dimension b along the spectral dimension, × 3 denotes the modulo-3 product of a tensor and a matrix.

[0044] Step 3.3: Combine the deep non-linear transformation tensor low-rank objective function and the data fidelity term to construct a fused imaging model. The mathematical expression of this model is:

[0045]

[0046] where ε 1 and ε 2 respectively represent the noise perturbation intensities of the compressive imaging and spectral degradation processes. While maintaining the concise structure of the typical tensor rank minimization model that is easy to optimize and solve, this model fully considers the global tensor low-rankness of hyperspectral images in a more compact and robust deep non-linear transformation domain, and at the same time fully integrates the information of hyperspectral compressive measurements and prior multispectral images to obtain accurate fused imaging results.

[0047] Step 3.4: Transform this fused imaging model into a composite loss function suitable for optimization and solution using learning algorithms where α and β are trade-off parameters. The first part of this loss function is the deep non-linear transformation tensor low-rank objective function of the fused imaging model, and its mathematical expression is The second part of this loss function is the first data fidelity term of the fused imaging model, and its mathematical expression is The third part of this loss function is the second data fidelity term of the fused imaging model, and its mathematical expression is

[0048] Specifically, Step 4 optimizes this fused imaging model by minimizing the loss function using the adaptive moment estimation learning algorithm, fully integrating the information of compressive measurements and prior multispectral images, and reconstructing a complete hyperspectral image. It specifically includes the following steps:

[0049] Step 4.1: Calculate the gradient through the adaptive moment estimation learning algorithm and update the weight parameters of and to minimize the loss function and iteratively optimize the fused imaging model in an unsupervised training manner.

[0050] Step 4.2: Set the number of training iterations to T times. When the number of iterations is greater than T, stop the iteration and at the same time output the fused imaging result with high reconstruction accuracy

[0051] such as Figure 3As shown, based on the same inventive concept, the present invention provides a hyperspectral fusion imaging device based on deep non-linear transformation tensor low-rank representation, comprising:

[0052] An encoded aperture spectral imaging system, configured to convert the incident light energy acquired from an observation scene into compressed measurement data;

[0053] A multispectral sensor, configured to convert the incident light energy acquired from an observation scene into multispectral image data;

[0054] A beam splitter, configured to capture the incident light energy of an observation scene and split it into two parts, which are respectively input into the encoded aperture spectral imaging system and the multispectral sensor;

[0055] A storage unit: configured to store the hyperspectral compression measurement - prior multispectral image data pairs acquired by the encoded aperture spectral imaging system and the multispectral sensor in an observation scene;

[0056] A calculation unit: configured to read the hyperspectral compression measurement - prior multispectral image data pairs in the storage unit as inputs, execute the aforementioned hyperspectral fusion imaging method based on deep non-linear transformation tensor low-rank representation, output the reconstructed high-precision complete hyperspectral image, and store the result in the storage unit.

[0057] In the following embodiments, in order to verify the implementation effects of the method and device proposed by the present invention in actual application scenarios, a data is randomly selected from the publicly available CAVE hyperspectral image dataset for experiments. The CAVE dataset provides hyperspectral images with a size of 512×512×28. In this embodiment, the encoded aperture spectral imaging system is used to acquire compressed measurements, as Figure 4 shown, and the spectral response function of a publicly available Nikon D700 camera is used to acquire the three-channel prior multispectral image, as Figure 5 shown. By storing the above data pairs on the storage unit module of the device provided by the present invention and executing the fusion imaging method provided by the present invention on the calculation unit of the device provided by the present invention, a hyperspectral fusion imaging result with a complete size of 512×512×28 is obtained, as Figure 6 shown. It can be clearly observed that the method proposed by the present invention can provide high-quality fusion imaging results, in which spatial information such as edge textures and spectral information of all bands are accurately reconstructed, verifying the effectiveness of the method proposed by the present invention.

[0058] The exemplary embodiments of the present invention have been described only by way of illustration. Without doubt, for those of ordinary skill in the art, the described embodiments can be adjusted in various different ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention. For the device embodiments, since they basically correspond to the method embodiments, the relevant parts related to the device can be referred to the partial description of the method embodiments. The parts related to the device in the above-described embodiments are only illustrative, and some or all of the modules can be selected according to actual needs to achieve the purpose of the present invention solution. Those of ordinary skill in the art can understand and implement the relevant content of the device embodiments without creative efforts.

Claims

1. A hyperspectral fusion imaging method based on low-rank representation of deep nonlinear transformation tensor, characterized in that: The following steps are involved: Step 1: Collect a priori multispectral image while collecting hyperspectral compression measurements in the observation scene, take both data as input at the same time, and represent the hyperspectral compression measurements and the priori multispectral image as a second-order matrix and a third-order tensor respectively; Step 2: Based on tensor representation and deep learning theory, a new deep nonlinear transformation tensor low-rank regularization for hyperspectral images is established. The regularization includes a deep low-rank transformation network and a tensor nuclear norm to fully characterize the global high-dimensional low-rank correlation of hyperspectral images in the deep nonlinear transformation domain. Step 3: Take the low-rank regularization term of the deep nonlinear transformation tensor as the objective function, model the compression imaging and spectral degradation processes as two data fidelity terms respectively, and construct a fusion imaging model and loss function; Step 4: Minimize the loss function through the adaptive moment estimation learning algorithm to optimize the fusion imaging model, fully integrate the information of the compressed measurement and the prior multispectral image, and reconstruct the complete hyperspectral image.

2. The hyperspectral fusion imaging method based on deep nonlinear transformation tensor low-rank representation according to claim 1 is characterized in that: In step 1, a priori multispectral image is collected while collecting a hyperspectral compression measurement in the observation scene, and the two data are simultaneously used as inputs, and the hyperspectral compression measurement and the priori multispectral image are represented as a second-order matrix and a third-order tensor, respectively, which specifically includes the following steps: Step 1.1: Represent the acquired hyperspectral compression measurements as a second-order matrix Among them, H and W represent the size of the spatial dimension of the hyperspectral image, and B represents the size of the spectral dimension; Step 1.2: Represent the acquired prior multispectral image as a third-order tensor Wherein, b is the number of channels of the prior multispectral image, and satisfies the condition b<<B.

3. The hyperspectral fusion imaging method based on deep nonlinear transformation tensor low-rank representation according to claim 2 is characterized in that: Based on tensor representation and deep learning theory, a new type of deep nonlinear transformation tensor low-rank regularization for hyperspectral images is established in step 2. The regularization includes a deep low-rank transformation network and a tensor nuclear norm to fully characterize the global high-dimensional low-rank correlation of the hyperspectral image in the deep nonlinear transformation domain. Specifically, the following steps are included: Step 2.1: Represent the complete hyperspectral image to be fused as a third-order tensor Step 2.2: Targeting the Hyperspectral Image Tensor Building a deep nonlinear low-rank transformation network The network consists of three parts: Represents the number of feature dimensions after transformation; the first part of the network contains k branches to extract the nonlinear transformation features of high-dimensional redundant hyperspectral image tensors at different scales, and each branch consists of the following two-layer network structure: (1) A 1×1 convolution and a LeakyReLU linear rectifier unit. This layer of network structure projects the hyperspectral image tensor along the spectral dimension to the high-dimensional feature space; (2) A 3×3 convolution and a LeakyReLU linear rectifier unit. This layer of network structure enhances the nonlinear representation of hyperspectral image features. The second part of the network is a splicing operator, which splices the k nonlinear transformation features obtained by the k branches along the spectral dimension. The spliced ​​features are The third part of the network repeats the following structure twice: 3×3 convolution and a LeakyReLU linear rectifier unit; this part of the network structure compresses the concatenated nonlinear transformation features into dimension; Step 2.3: Using this deep nonlinear low-rank transformation network The hyperspectral image tensor Projecting into a nonlinear transformation domain to obtain a deep nonlinear transformation representation of the hyperspectral image tensor The mathematical expression of this process is That is, the data-adaptive deep nonlinear network designed in step 2.2 Replace the fixed linear transformation used in the existing tensor singular value decomposition algebraic framework to enhance the compactness and robustness of the tensor low-rank representation in the transform domain; Step 2.4: Combine the tensor nuclear norm to construct a deep nonlinear transformation tensor low-rank regularization term. The mathematical expression of this new regularization term is That is, the hyperspectral image tensor The low-rank regularization of the deep nonlinear transformation tensor is computationally equivalent to its deep nonlinear transformation The nuclear norm of all i-th forward slice matrices under ; Step 2.5: Represent the tensor for the acquired deep nonlinear transformation Building a deep nonlinear inverse transform network The deep nonlinear low-rank representation of the hyperspectral image tensor is projected back to the original space; the deep nonlinear inverse transformation network consists of the following network structure: (1) (2) B 1×1 convolutions and a LeakyReLU linear rectifier unit; that is, through a deep nonlinear network The fixed linear inverse transform used by the tensor singular value decomposition algebraic framework is replaced to obtain the hyperspectral image tensor after low-rank approximation.

4. The hyperspectral fusion imaging method based on deep nonlinear transformation tensor low-rank representation according to claim 3 is characterized in that: Step 3: Use the low-rank regularization term of the deep nonlinear transformation tensor constructed in step 2 as the objective function, and model the compression imaging and spectral degradation processes as two data fidelity terms respectively to construct a fusion imaging model and loss function, which specifically includes the following steps: Step 3.1: Model the process of obtaining compressed measurements by compressing the hyperspectral image as the first data fidelity term, which is composed of the reconstructed fusion imaging result and the observed compression measurement X, whose mathematical expression is: The function Ψ(·): The vector represents the mathematical process of compressed imaging of the observed scene by the coded aperture spectral imaging system. The vectorized result of X representing the compressed measurement has a dimension of T = H × (W + B-1); Step 3.2: Model the process of hyperspectral image degradation to multispectral image in spectral dimension as the second data fidelity term, which is composed of the reconstructed fusion imaging result With prior multispectral images The mathematical expression is: The matrix It represents the spectral response function of degenerating a hyperspectral image with dimension B into a multispectral image with dimension b along the spectral dimension, and ×3 represents the modulo 3 product of the tensor and the matrix; Step 3.3: Combine the deep nonlinear transformation tensor low-rank objective function with the data fidelity term to build a fusion imaging model. The mathematical expression of the model is: Where ε1 and ε2 represent the noise perturbation intensity of the compression imaging and spectral degradation processes, respectively; Step 3.4: Convert the fusion imaging model into a composite loss function suitable for optimization using a learning algorithm Where α and β are trade-off parameters; the first part of the loss function is the low-rank objective function of the deep nonlinear transformation tensor of the fusion imaging model, which is mathematically expressed as The second part of the loss function is the first data fidelity term of the fusion imaging model, which is mathematically expressed as The third part of the loss function is the second data fidelity term of the fusion imaging model, which is mathematically expressed as 5. The hyperspectral fusion imaging method based on deep nonlinear transformation tensor low-rank representation according to claim 4 is characterized in that: Step 4 described in minimizing the loss function through the adaptive moment estimation learning algorithm to optimize the fusion imaging model, fully integrate the information of the compression measurement and the prior multispectral image, and reconstruct a complete hyperspectral image, specifically including the following steps: Step 4.1: Calculate the gradient through the adaptive moment estimation learning algorithm and update the deep nonlinear transformation network and The weight parameters are used to minimize the loss function Iteratively optimize the fusion imaging model in an unsupervised training manner; Step 4.2: Set the number of training iterations to T. When the number of iterations is greater than T, stop the iteration and output the fusion imaging result with high reconstruction accuracy.

6. A hyperspectral fusion imaging device based on low-rank representation of deep nonlinear transformation tensor, characterized in that: For implementing the method described in any one of claims 1 to 5, the device comprises: Coded aperture spectral imaging system, used to convert incident light energy acquired from the observed scene into compressed measurement data; A multispectral sensor is used to convert incident light energy acquired from an observed scene into multispectral image data; A spectroscope is used to capture the incident light energy of the observation scene and divide it into two parts, which are input into the coded aperture spectral imaging system and the multispectral sensor respectively; Storage unit: used to store the hyperspectral compression measurement and a priori multispectral image data pairs collected by the coded aperture spectral imaging system and the multispectral sensor in the observation scene; Computing unit: used to read the hyperspectral compression measurement conversion prior multispectral image data pair in the storage unit as input, execute the aforementioned hyperspectral fusion imaging method based on deep nonlinear transformation tensor low-rank representation, output the reconstructed high-precision complete hyperspectral image, and store the fusion imaging result in the storage unit.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of any method described in claims 1-5 are implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images