A hyperspectral fusion imaging method and device based on low-rank representation of deep nonlinear transform tensor

Abstract

This invention discloses a hyperspectral fusion imaging method and apparatus based on low-rank representation of deep nonlinear transform tensors. The method includes the following steps: simultaneously acquiring a prior multispectral image while performing hyperspectral compression measurements in the observation scene, using both data points as inputs; establishing a novel low-rank regularization of deep nonlinear transform tensors for hyperspectral images based on tensor representation and deep learning theory to fully characterize the global high-dimensional low-rank correlation of hyperspectral images in the deep nonlinear transform domain; using this regularization term as the objective function, and modeling the compression imaging and spectral degradation processes as two data fidelity terms respectively, constructing a fusion imaging model and a loss function; minimizing the loss function through a learning algorithm to optimize the model, and fusing and reconstructing a complete hyperspectral image. The novel method and apparatus proposed in this invention can achieve high-precision fusion imaging of the observation scene without manual intervention or explicit intermediate steps.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral intelligent imaging processing technology, specifically to a hyperspectral fusion imaging method and apparatus based on low-rank representation of deep nonlinear transform tensor. Background Technology

[0002] Hyperspectral imaging technology can image across hundreds of consecutive bands from visible to near-infrared, acquiring detailed spectral information of the observed scene. This spectral information enables accurate analysis and identification of observed ground features, achieving high-precision results in applications such as target detection, land cover classification, and change detection. However, traditional imaging systems typically require multiple exposures of the observed scene to acquire a complete hyperspectral data cube. This not only significantly sacrifices temporal resolution, making it difficult to capture spectral information from dynamic scenes, but also places enormous pressure on storage and transmission due to the large data volume of acquired hyperspectral images, severely limiting its application in important scenarios such as military and aerospace. Coordinated aperture snapshot spectral imaging technology acquires compressed hyperspectral measurements of the observed scene through dimensionality reduction, and then reconstructs a complete hyperspectral image from the compressed measurements using imaging algorithms. This achieves rapid acquisition of rich spectral information in a single exposure, while the compressed acquisition mode greatly reduces the storage and transmission costs of large-scale hyperspectral information.

[0003] Reconstructing a complete hyperspectral image from observed compressed measurements using imaging algorithms is a typical compressed sensing problem, characterized by high underdeterminism. Current common solutions include model optimization methods and deep learning methods. The former manually mines the intrinsic prior linear representations of the hyperspectral image, builds an optimization model, and iteratively solves it to complete the reconstruction process; the latter uses a data-driven approach, constructing a deep network to learn the nonlinear mapping relationship from compressed measurements to the complete image. However, the manual design of prior regularization in model optimization methods often lacks the ability to represent the high-dimensional structural correlations of complex hyperspectral images, resulting in insufficient reconstruction accuracy. Deep learning methods not only require large-scale, high-quality labeled data for training but also suffer from poor interpretability, limiting their application in real-world scenarios. Furthermore, because single-source compressed measurements themselves carry relatively limited information, the reconstruction equations are excessively underdetermined, making it difficult to achieve high-fidelity imaging of spatial details. Summary of the Invention

[0004] The purpose of this invention is to provide a hyperspectral fusion imaging method and device based on low-rank representation of deep nonlinear transform tensors. Addressing the problems of insufficient representational power of linear prior optimization models, poor interpretability of deep nonlinear networks and over-reliance on large-scale, high-quality label data in existing imaging algorithms, as well as the lack of usable information in single-source compressed imaging systems, this invention introduces a low-cost multispectral imaging sensor into the compressed imaging system, using the concept of fusion imaging. The acquired multispectral images are used as prior images, and a fusion imaging method that combines the advantages of model optimization methods and deep learning is designed. This achieves high-precision hyperspectral fusion imaging by fully integrating information from compressed measurements and prior multispectral images while deeply exploring the high-dimensional structural correlations of hyperspectral images.

[0005] To achieve the above objectives, the technical solution adopted by this invention is: a hyperspectral fusion imaging method based on low-rank representation of deep nonlinear transform tensor, comprising the following steps:

[0006] Step 1: While acquiring a hyperspectral compressed measurement in the observation scene, acquire a prior multispectral image. Use both data as inputs simultaneously, and represent the hyperspectral compressed measurement and the prior multispectral image as a second-order matrix and a third-order tensor, respectively.

[0007] Step 2: Based on tensor representation and deep learning theory, a novel deep nonlinear transform tensor low-rank regularization for hyperspectral images is established. This regularization includes a deep low-rank transform network and a tensor kernel norm to fully characterize the global high-dimensional low-rank correlation of hyperspectral images in the deep nonlinear transform domain.

[0008] Step 3: Use the low-rank regularization term of the depth nonlinear transformation tensor as the objective function, and model the compressed imaging and spectral degradation processes as two data fidelity terms respectively to construct the fusion imaging model and loss function;

[0009] Step 4: Minimize the loss function through the adaptive moment estimation learning algorithm to optimize the fusion imaging model, fully integrate the information from compressed measurements and prior multispectral images, and reconstruct the complete hyperspectral image.

[0010] The present invention also provides a hyperspectral fusion imaging device based on deep nonlinear transform tensor low-rank representation, for implementing the above-mentioned hyperspectral fusion imaging method based on deep nonlinear transform tensor low-rank representation, the device comprising:

[0011] Encoded aperture spectral imaging system, used to convert incident light energy acquired from an observation scene into compressed measurement data;

[0012] Multispectral sensors are used to convert incident light energy acquired from an observed scene into multispectral image data;

[0013] A beam splitter is used to capture the incident light energy of the observed scene and split it into two parts, which are then input into the coded aperture spectral imaging system and the multispectral sensor, respectively.

[0014] Storage unit: Used to store pairs of hyperspectral compressed measurement and prior multispectral image data acquired by the coded aperture spectral imaging system and multispectral sensors in the observation scene;

[0015] Computation unit: Used to read the hyperspectral compressed measurement-converted prior multispectral image data pairs in the storage unit as input, execute the aforementioned hyperspectral fusion imaging method based on the low-rank representation of the depth nonlinear transformation tensor, output the reconstructed high-precision complete hyperspectral image, and store the fusion imaging result in the storage unit.

[0016] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described above.

[0017] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the above-described method.

[0018] A computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described method.

[0019] Compared with the prior art, the beneficial effects of the present invention are: (1) The hyperspectral fusion imaging method based on deep nonlinear transform tensor low-rank representation provided by the present invention introduces prior multispectral image information into the compressed imaging system, and combines the advantages of two reconstruction modes, namely model optimization method and deep learning method, in the imaging algorithm, which greatly improves the shortcomings of the existing hyperspectral imaging system in terms of low reconstruction accuracy and narrow application range. (2) The hyperspectral fusion imaging method based on deep nonlinear transform tensor low-rank representation provided by the present invention improves the limitation of the existing tensor low-rank representation theory in dealing with hyperspectral fusion imaging problems by establishing a new deep nonlinear transform tensor low-rank regularization, which only considers the high-dimensional low-rank correlation of the image in a fixed linear transform domain. Through effective deep transform network design, a more compact low-rank representation of the hyperspectral image is obtained, which significantly improves the accuracy of the existing imaging method. (3) The hyperspectral fusion imaging method based on deep nonlinear transformation tensor low-rank representation provided by this invention improves upon the limitation of existing deep learning theories in dealing with hyperspectral fusion imaging problems by reasonably optimizing the objective function and unsupervised loss function design. This limitation requires large-scale labeled data to effectively learn the deep nonlinear features of hyperspectral images. At the same time, it maintains the advantage of strong interpretability of tensor low-rank representation theory and significantly improves the generalization of existing imaging methods in a data adaptive manner. (4) The hyperspectral fusion imaging device based on deep nonlinear transformation tensor low-rank representation provided by this invention couples the coded aperture spectral imaging system, multispectral sensor, storage unit and computing unit, which can effectively and conveniently realize hyperspectral fusion imaging and has significant application value in real-world hyperspectral imaging scenarios. Attached Figure Description

[0020] Figure 1 This is the overall flowchart of the hyperspectral fusion imaging method based on the low-rank representation of the deep nonlinear transform tensor of the present invention.

[0021] Figure 2 This is a structural diagram of the deep nonlinear low-rank transform network designed in the fusion imaging method proposed in this invention.

[0022] Figure 3 This is a schematic diagram of the structure of the hyperspectral fusion imaging device with low-rank representation of depth nonlinear transform tensor according to the present invention.

[0023] Figure 4 This is a schematic diagram of hyperspectral compression measurements acquired by the coded aperture spectral imaging system.

[0024] Figure 5 This is a schematic diagram of a priori multispectral images acquired by a multispectral sensor.

[0025] Figure 6 This is a high-precision fusion imaging result obtained by the present invention. Detailed Implementation

[0026] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0027] like Figure 1 As shown, the hyperspectral fusion imaging method based on low-rank representation of deep nonlinear transform tensor of the present invention includes the following steps:

[0028] Step 1: While acquiring a hyperspectral compressed measurement in the observation scene, acquire a prior multispectral image. Use both data as inputs simultaneously, and represent the hyperspectral compressed measurement and the prior multispectral image as a second-order matrix and a third-order tensor, respectively.

[0029] Step 2: Based on tensor representation and deep learning theory, a novel deep nonlinear transform tensor low-rank regularization for hyperspectral images is established. This regularization includes a deep low-rank transform network and a tensor kernel norm to fully characterize the global high-dimensional low-rank correlation of hyperspectral images in the deep nonlinear transform domain.

[0030] Step 3: Use the low-rank regularization term of the depth nonlinear transformation tensor as the objective function, and model the compressed imaging and spectral degradation processes as two data fidelity terms respectively to construct the fusion imaging model and loss function;

[0031] Step 4: Minimize the loss function through the adaptive moment estimation learning algorithm to optimize the fusion imaging model, fully integrate the information from compressed measurements and prior multispectral images, and reconstruct the complete hyperspectral image.

[0032] Specifically, step 1 involves acquiring a prior multispectral image while simultaneously performing a hyperspectral compressed measurement in the observation scenario. Both data points are used as input, and the hyperspectral compressed measurement and the prior multispectral image are represented as a second-order matrix and a third-order tensor, respectively. This includes the following steps:

[0033] Step 1.1: Represent the acquired hyperspectral compressed measurements as a second-order matrix. Where H and W represent the size of the spatial dimension of the hyperspectral image, and B represents the size of the spectral dimension.

[0034] Step 1.2: Represent the acquired prior multispectral images as third-order tensors Where b is the number of channels in the prior multispectral image, and the condition b << B is satisfied.

[0035] Specifically, step 2, based on tensor representation and deep learning theory, establishes a novel deep nonlinear transform tensor low-rank regularization for hyperspectral images. This regularization includes a deep low-rank transform network and a tensor kernel norm to fully characterize the global high-dimensional low-rank correlation of hyperspectral images in the deep nonlinear transform domain. The steps include:

[0036] Step 2.1: Represent the complete hyperspectral image acquired by the imaging to be fused as a third-order tensor.

[0037] Step 2.2: As Figure 2 As shown, for hyperspectral image tensors Construct a deep nonlinear low-rank transform network The network consists of three parts, among which This indicates the number of feature dimensions after transformation. The first part of the network contains k branches to extract the nonlinear transformation features of the high-dimensional redundant hyperspectral image tensor at different scales. Each branch consists of the following two-layer network structure: (1) This layer of network structure, consisting of a 1×1 convolution and a LeakyReLU linear rectified unit, projects the hyperspectral image tensor along the spectral dimension to a high-dimensional feature space, fully considering the correlation of the spectral dimension; (2) This network structure, consisting of a 3×3 convolution and a LeakyReLU linear rectified unit, further enhances the nonlinear representation of hyperspectral image features while fully considering spatial correlation. The second part of the network is a concatenation operator that concatenates the k nonlinear transformation features obtained from the k branches along the spectral dimension, resulting in the concatenated features. The third part of the network repeats the following structure twice: Each 3×3 convolution is followed by a LeakyReLU linear rectified unit; this part of the network structure compresses the spliced ​​nonlinear transformation features into... Dimensionality, and further enhance its nonlinear representation.

[0038] Step 2.3: Utilize this deep nonlinear low-rank transform network Hyperspectral image tensor Projecting the hyperspectral image tensor into a nonlinear transform domain yields a deep nonlinear transform representation. The mathematical expression for this process is: That is, through the data adaptive deep nonlinear network designed in step 2.2 Replace the fixed linear transformations used in the existing tensor singular value decomposition algebra framework, such as Fourier transform or discrete cosine transform, to enhance the compactness and robustness of the low-rank tensor representation in the transform domain.

[0039] Step 2.4: Construct a low-rank regularization term for the deep nonlinear transformation tensor by combining the tensor nuclear norm. The mathematical expression of this novel regularization term is: That is, hyperspectral image tensor The low-rank regularization of the deep nonlinear transform tensor is computationally equivalent to its application in the deep nonlinear transform tensor. The nuclear norm of all i-th positive slice matrices. This novel regularization term, while maintaining the algebraic structure of the tensor nuclear norm minimization method, further characterizes the global high-dimensional low-rank correlation of hyperspectral image tensors in the more representative deep nonlinear transform domain.

[0040] Step 2.5: For the obtained depth nonlinear transformation representation tensor Construct a deep nonlinear inverse transform network The deep nonlinear low-rank representation of the hyperspectral image tensor is projected back into the original space. This deep nonlinear inverse transform network consists of the following network structure: (1) (2) B 3×3 convolutions with a LeakyReLU linear rectified unit; (3) B 1×1 convolutions with a LeakyReLU linear rectified unit. That is, through a deep nonlinear network Replace the fixed linear inverse transform used in the tensor singular value decomposition algebraic framework, such as the inverse Fourier transform or the inverse discrete cosine transform, to obtain the low-rank approximation hyperspectral image tensor.

[0041] Specifically, step 3 uses the low-rank regularization term of the deep nonlinear transformation tensor constructed in step 2 as the objective function, and models the compressed imaging and spectral degradation processes as two data fidelity terms respectively, constructing a fusion imaging model and loss function, which includes the following steps:

[0042] Step 3.1: Model the process of acquiring compressed measurements through hyperspectral image compression imaging as the first data fidelity term, which is derived from the reconstructed fused imaging results. It consists of the compressed measurement X observed. Its mathematical expression is: Where the function Ψ(·): This represents the mathematical process by which an coded aperture spectral imaging system performs compressed imaging of an observed scene; the vector... The result of the compressed measurement X after vectorization has a dimension T = H × (W + B - 1).

[0043] Step 3.2: Model the process of hyperspectral image degrading to multispectral image in the spectral dimension as a second data fidelity term, which is derived from the reconstructed fused imaging result. With prior multispectral images Composition, its mathematical expression is: Where the matrix denoted as the spectral response function that degenerates a hyperspectral image of dimension B into a multispectral image of dimension b along the spectral dimension, and ×3 represents the modulo 3 product of the tensor and the matrix.

[0044] Step 3.3: Combining the low-rank objective function of the depth nonlinear transform tensor with the data fidelity term, a fusion imaging model is constructed. The mathematical expression of this model is:

[0045]

[0046] Where ε1 and ε2 represent the noise perturbation intensity of the compression imaging and spectral degradation processes, respectively. While maintaining the concise structure of the typical tensor rank minimization model, which is easy to optimize and solve, this model fully considers the global tensor low rank of hyperspectral images in a more compact and robust deep nonlinear transform domain, and fully integrates information from hyperspectral compression measurements and prior multispectral images to obtain accurate fused imaging results.

[0047] Step 3.4: Transform the fused imaging model into a composite loss function suitable for optimization using a learning algorithm. Where α and β are trade-off parameters. The first part of the loss function is the low-rank objective function of the depth nonlinear transform tensor of the fusion imaging model, which is mathematically expressed as follows: The second part of the loss function is the first data fidelity term of the fused imaging model, which is mathematically expressed as follows: The third part of the loss function is the second data fidelity term of the fused imaging model, which is mathematically expressed as follows:

[0048] Specifically, step 4 optimizes the fusion imaging model by minimizing the loss function using an adaptive moment estimation learning algorithm, fully fusing information from compressed measurements and prior multispectral images to reconstruct a complete hyperspectral image. This includes the following steps:

[0049] Step 4.1: Calculate the gradient using the adaptive moment estimation learning algorithm and update the deep nonlinear transformation network. and The weight parameters are used to minimize the loss function. The fusion imaging model is iteratively optimized using unsupervised training.

[0050] Step 4.2: Set the number of training iterations to T. Stop iterating when the number of iterations exceeds T, and output the fusion imaging result with high reconstruction accuracy.

[0051] like Figure 3 As shown, based on the same inventive concept, this invention proposes a hyperspectral fusion imaging device based on low-rank representation of depth nonlinear transform tensor, comprising:

[0052] Encoded aperture spectral imaging system, used to convert incident light energy acquired from an observation scene into compressed measurement data;

[0053] Multispectral sensors are used to convert incident light energy acquired from an observed scene into multispectral image data;

[0054] A beam splitter is used to capture the incident light energy of the observed scene and split it into two parts, which are then input into the coded aperture spectral imaging system and the multispectral sensor, respectively.

[0055] Storage unit: Used to store hyperspectral compressed measurement-prior multispectral image data pairs acquired by the coded aperture spectral imaging system and multispectral sensors in the observation scenario;

[0056] Computation unit: Used to read the hyperspectral compressed measurement-prior multispectral image data pair in the storage unit as input, execute the aforementioned hyperspectral fusion imaging method based on the low-rank representation of the depth nonlinear transform tensor, output the reconstructed high-precision complete hyperspectral image, and store the result in the storage unit.

[0057] The following embodiments, to verify the implementation effect of the proposed method and apparatus in practical application scenarios, randomly select a dataset from the publicly available CAVE hyperspectral image dataset for experimentation. The CAVE dataset provides hyperspectral images with a size of 512×512×28. This embodiment uses a coded aperture spectral imaging system to acquire compressed measurements, such as... Figure 4 As shown, a three-channel prior multispectral image was obtained using the publicly available spectral response function of the Nikon D700 camera, as shown. Figure 5 As shown. By storing the above data pairs into the storage unit module of the device provided by the present invention, and executing the fusion imaging method provided by the present invention on the computing unit of the device provided by the present invention, a hyperspectral fusion imaging result with a complete size of 512×512×28 is obtained, as shown. Figure 6 As shown, it can be clearly observed that the method proposed in this invention can provide high-quality fusion imaging results, in which spatial information such as edge texture and spectral information of all bands are accurately reconstructed, verifying the effectiveness of the method proposed in this invention.

[0058] The exemplary embodiments of the present invention have been described above by way of illustration only. Undoubtedly, those skilled in the art can adjust the described embodiments in various ways without departing from the spirit and scope of the invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention. For apparatus implementations, since they basically correspond to the method embodiments, the descriptions of the method embodiments are sufficient for the apparatus-related parts. The apparatus-related parts in the embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement the relevant content of this apparatus example without creative effort.

Claims

1. A hyperspectral fusion imaging method based on low-rank representation of deep nonlinear transform tensor, characterized in that, Includes the following steps: Step 1: While acquiring a hyperspectral compressed measurement in the observation scene, acquire a prior multispectral image. Use both data as inputs simultaneously, and represent the hyperspectral compressed measurement and the prior multispectral image as a second-order matrix and a third-order tensor, respectively. Step 2: Based on tensor representation and deep learning theory, a novel deep nonlinear transform tensor low-rank regularization for hyperspectral images is established. This regularization includes a deep low-rank transform network and a tensor kernel norm to fully characterize the global high-dimensional low-rank correlation of hyperspectral images in the deep nonlinear transform domain. Step 3: Use the low-rank regularization term of the depth nonlinear transformation tensor as the objective function, and model the compressed imaging and spectral degradation processes as two data fidelity terms respectively to construct the fusion imaging model and loss function; Step 4: Minimize the loss function through the adaptive moment estimation learning algorithm to optimize the fusion imaging model, fully integrate the information from compressed measurements and prior multispectral images, and reconstruct the complete hyperspectral image.

2. The hyperspectral fusion imaging method based on low-rank representation of deep nonlinear transform tensor according to claim 1, characterized in that: Step 1 describes acquiring a prior multispectral image simultaneously with a hyperspectral compressed measurement in the observation scenario. Both data are used as input, and the hyperspectral compressed measurement and the prior multispectral image are represented as a second-order matrix and a third-order tensor, respectively. This specifically includes the following steps: Step 1.1: Represent the acquired hyperspectral compressed measurements as a second-order matrix. Where H and W represent the size of the spatial dimension of the hyperspectral image, and B represents the size of the spectral dimension; Step 1.2: Represent the acquired prior multispectral images as third-order tensors Where b is the number of channels in the prior multispectral image, and the condition b << B is satisfied.

3. The hyperspectral fusion imaging method based on low-rank representation of deep nonlinear transform tensor according to claim 2, characterized in that: Step 2 describes the establishment of a novel deep nonlinear transform tensor low-rank regularization for hyperspectral images based on tensor representation and deep learning theory. This regularization includes a deep low-rank transform network and a tensor kernel norm to fully characterize the global high-dimensional low-rank correlation of hyperspectral images in the deep nonlinear transform domain. Specifically, it includes the following steps: Step 2.1: Represent the complete hyperspectral image acquired by the imaging to be fused as a third-order tensor. Step 2.2: For hyperspectral image tensors Construct a deep nonlinear low-rank transform network The network consists of three parts, among which The number of feature dimensions after transformation is indicated; the first part of the network contains k branches to extract the nonlinear transformation features of the high-dimensional redundant hyperspectral image tensor at different scales, and each branch consists of the following two-layer network structure: (1) This layer of network structure, consisting of a 1×1 convolution and a LeakyReLU linear rectified unit, projects the hyperspectral image tensor along the spectral dimension into a high-dimensional feature space; (2) This network structure, consisting of a 3×3 convolution and a LeakyReLU linear rectified unit, enhances the nonlinear representation of hyperspectral image features. The second part of the network is a concatenation operator that concatenates the k nonlinear transformation features obtained from the k branches along the spectral dimension, resulting in the concatenated features. The third part of the network repeats the following structure twice: Each 3×3 convolution is followed by a LeakyReLU linear rectified unit; this part of the network structure compresses the spliced ​​nonlinear transformation features into... dimension; Step 2.3: Utilize this deep nonlinear low-rank transform network Hyperspectral image tensor Projecting the hyperspectral image tensor into a nonlinear transform domain yields a deep nonlinear transform representation. The mathematical expression for this process is: That is, through the data adaptive deep nonlinear network designed in step 2.2 Replace the fixed linear transformation used in the existing tensor singular value decomposition algebra framework to enhance the compactness and robustness of the low-rank tensor representation in the transform domain; Step 2.4: Construct a low-rank regularization term for the deep nonlinear transformation tensor by combining the tensor nuclear norm. The mathematical expression of this novel regularization term is: That is, hyperspectral image tensor The low-rank regularization of the deep nonlinear transform tensor is computationally equivalent to its application in the deep nonlinear transform tensor. The nuclear norm of the i-th positive slice matrix; Step 2.5: For the obtained depth nonlinear transformation representation tensor Construct a deep nonlinear inverse transform network The deep nonlinear low-rank representation of the hyperspectral image tensor is projected back into the original space; this deep nonlinear inverse transform network consists of the following network structure: (1) (2) B 3×3 convolutions with a LeakyReLU linear rectified unit; that is, B 1×1 convolutions with a LeakyReLU linear rectified unit; that is, through a deep nonlinear network Replace the fixed linear inverse transform used in the tensor singular value decomposition algebra framework to obtain a low-rank approximation hyperspectral image tensor.

4. The hyperspectral fusion imaging method based on low-rank representation of deep nonlinear transform tensor according to claim 3, characterized in that: Step 3: Using the low-rank regularization term of the deep nonlinear transformation tensor constructed in Step 2 as the objective function, and modeling the compressed imaging and spectral degradation processes as two data fidelity terms respectively, a fusion imaging model and loss function are constructed, which specifically includes the following steps: Step 3.1: Model the process of acquiring compressed measurements through hyperspectral image compression imaging as the first data fidelity term, which is derived from the reconstructed fused imaging results. Composed of the observed compressed measurement X, its mathematical expression is: Where the function Ψ(·): This represents the mathematical process by which an coded aperture spectral imaging system performs compressed imaging of an observed scene; the vector... The result of the compressed measurement X after vectorization has a dimension T = H × (W + B - 1); Step 3.2: Model the process of hyperspectral image degrading to multispectral image in the spectral dimension as a second data fidelity term, which is derived from the reconstructed fused imaging result. With prior multispectral images Composition, its mathematical expression is: Where the matrix This represents the spectral response function that degenerates a hyperspectral image of dimension B into a multispectral image of dimension b along the spectral dimension, and ×3 represents the modulo 3 product of the tensor and the matrix. Step 3.3: Combining the low-rank objective function of the depth nonlinear transform tensor with the data fidelity term, a fusion imaging model is constructed. The mathematical expression of this model is: Where ε1 and ε2 represent the noise perturbation intensity of the compression imaging and spectral degradation processes, respectively; Step 3.4: Transform the fused imaging model into a composite loss function suitable for optimization using a learning algorithm. Where α and β are trade-off parameters; the first part of the loss function is the low-rank objective function of the depth nonlinear transform tensor of the fusion imaging model, which is mathematically expressed as follows: The second part of the loss function is the first data fidelity term of the fused imaging model, which is mathematically expressed as follows: The third part of the loss function is the second data fidelity term of the fused imaging model, which is mathematically expressed as follows:

5. The hyperspectral fusion imaging method based on low-rank representation of deep nonlinear transform tensor according to claim 4, characterized in that: Step 4, which involves minimizing the loss function using an adaptive moment estimation learning algorithm to optimize the fusion imaging model and fully integrate information from compressed measurements and prior multispectral images to reconstruct a complete hyperspectral image, specifically includes the following steps: Step 4.1: Calculate the gradient using the adaptive moment estimation learning algorithm and update the deep nonlinear transformation network. and The weight parameters are used to minimize the loss function. Iterative optimization of the fusion imaging model using unsupervised training; Step 4.2: Set the number of training iterations to T. Stop iterating when the number of iterations exceeds T, and output the fusion imaging result with high reconstruction accuracy.

6. A hyperspectral fusion imaging device based on low-rank representation of deep nonlinear transform tensor, characterized in that, The apparatus for implementing the method of any one of claims 1-5 comprises: Encoded aperture spectral imaging system, used to convert incident light energy acquired from an observation scene into compressed measurement data; Multispectral sensors are used to convert incident light energy acquired from an observed scene into multispectral image data; A beam splitter is used to capture the incident light energy of the observed scene and split it into two parts, which are then input into the coded aperture spectral imaging system and the multispectral sensor, respectively. Storage unit: Used to store pairs of hyperspectral compressed measurement and prior multispectral image data acquired by the coded aperture spectral imaging system and multispectral sensors in the observation scene; Computation unit: Used to read the hyperspectral compressed measurement-converted prior multispectral image data pairs in the storage unit as input, execute the aforementioned hyperspectral fusion imaging method based on the low-rank representation of the depth nonlinear transformation tensor, output the reconstructed high-precision complete hyperspectral image, and store the fusion imaging result in the storage unit.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-5.

Images