Spectral data cube reconstruction method and device based on compressed physical prior, equipment, storage medium and program product

Abstract

The application provides a spectral data cube reconstruction method and device based on compressed physical prior, equipment, storage medium and program product, relating to the technical field of spectral reconstruction, and the method comprises the following steps: obtaining a two-dimensional measurement image formed by encoding and modulation of a to-be-observed target through a spectral imaging chip; obtaining an original transmission spectrum matrix corresponding to each pixel in the spectral imaging chip; performing principal component analysis on the original transmission spectrum matrix in the spectral dimension, and extracting a principal component feature vector by a singular value decomposition method; projecting the original transmission spectrum matrix into an orthogonal subspace formed by the principal component feature vector to obtain a compressed physical prior vector; inputting the two-dimensional measurement image and the compressed physical prior vector into a deep unfolding neural network to output a high-dimensional spectral data cube of the to-be-observed target. The application can significantly improve the reconstruction efficiency of the spectral data cube.

Description

Technical Field

[0001] This invention relates to the field of spectral reconstruction technology, and in particular to a method, apparatus, device, storage medium, and program product for spectral data cube reconstruction based on compressed physical priors. Background Technology

[0002] Miniature spectral imaging technology, as an emerging technology capable of simultaneously acquiring the spatial morphology and spectral characteristics of a target, has been widely applied in various fields such as environmental monitoring, medical diagnosis, industrial inspection, and security monitoring due to its advantages of small size, high integration, and portability. The core logic of this technology is as follows: The target light signal incident on the chip is spectrally encoded and modulated using micro / nano structures or filter arrays integrated on the miniature spectral imaging chip. An image sensor then acquires the encoded and modulated two-dimensional measurement image. Finally, a specific algorithm processes the two-dimensional measurement image to reconstruct a spectral data cube containing both spatial and spectral dimensions. However, traditional spectral reconstruction algorithms typically use a complete high-dimensional transmission spectrum matrix in the spectral data cube calculation process, which results in a huge number of network parameters, thus affecting the reconstruction efficiency of the spectral data cube. Summary of the Invention

[0003] This invention provides a method, apparatus, device, storage medium, and program product for spectral data cube reconstruction based on compressed physical priors, in order to solve the technical problem that the large number of network parameters in traditional spectral reconstruction algorithms affects the reconstruction efficiency of spectral data cubes.

[0004] This invention provides a method for cube reconstruction of spectral data based on compressed physical priors, comprising:

[0005] Acquire a two-dimensional measurement image of the target object after it has been encoded and modulated by a spectral imaging chip;

[0006] Obtain the original transmission spectrum matrix corresponding to each pixel in the spectral imaging chip;

[0007] Principal component analysis was performed on the original transmission spectrum matrix in the spectral dimension, and the principal component eigenvectors were extracted by singular value decomposition.

[0008] The original transmission spectrum matrix is ​​projected onto the orthogonal subspace formed by the principal component eigenvectors to obtain the compressed physical prior vector;

[0009] The two-dimensional measurement image and the compressed physical prior vector are input into a deep unfolded neural network, which outputs a high-dimensional spectral data cube of the target to be observed.

[0010] According to the present invention, a method for reconstructing a cube of spectral data based on compressed physical priors includes performing principal component analysis on the original transmission spectrum matrix in the spectral dimension and extracting principal component eigenvectors through singular value decomposition, comprising:

[0011] The original transmission spectrum matrix is ​​expanded along the spatial pixel dimension to obtain a two-dimensional matrix, wherein each row vector of the two-dimensional matrix corresponds to the spectral transmission response data of the spatial pixel in the spectral imaging chip.

[0012] The two-dimensional matrix is ​​subjected to principal component analysis in the spectral dimension for decentralization preprocessing to obtain a decentralized matrix;

[0013] Calculate the covariance matrix of the spectral dimension based on the decentralized matrix;

[0014] The covariance matrix is ​​subjected to singular value decomposition to obtain the singular value matrix;

[0015] The eigenvectors in the singular value matrix are sorted in descending order according to their singular values ​​to obtain an eigenvector sequence, and the principal component eigenvectors are extracted from the eigenvector sequence.

[0016] According to the present invention, a method for reconstructing a spectral data cube based on compressed physical priors, before inputting the two-dimensional measurement image and the compressed physical prior vector into a deep unfolded neural network to output the high-dimensional spectral data cube of the target to be observed, further includes:

[0017] Construct an initial training sample set for a deep unfolded neural network, wherein the training sample set includes multiple sets of training samples, each set of training samples including two-dimensional measurement sample images, compressed physical prior sample vectors, and a real high-dimensional spectral data cube;

[0018] Randomly select target training samples from the training sample set;

[0019] The two-dimensional measurement sample image of the target training sample and the compressed physical prior sample vector are input into the deep unfolded neural network to obtain the predicted high-dimensional spectral data cube.

[0020] Based on the predicted high-dimensional spectral data cube and the real high-dimensional spectral data cube, calculate the spectral distribution constraint loss value;

[0021] Based on the spectral distribution constraint loss value, the initial deep unfolded neural network is trained to obtain a trained deep unfolded neural network.

[0022] According to the present invention, a method for reconstructing a spectral data cube based on compressed physical priors is provided. The deep unfolded neural network includes a pre-convolutional layer, a cascaded module, and an end convolutional layer. The step of inputting the two-dimensional measurement image and the compressed physical prior vector into the deep unfolded neural network to output a high-dimensional spectral data cube of the target object includes:

[0023] The two-dimensional measurement image and the compressed physical prior vector are input into a deep unfolded neural network. The two-dimensional measurement image and the compressed physical prior vector are convolved by the pre-convolutional layer to obtain the initial fusion features.

[0024] The target cascaded features are obtained by processing the two-dimensional measurement image, the compressed physical prior vector, and the initial fusion features through the cascaded module.

[0025] The target cascaded features are processed by the terminal convolutional layer to output a high-dimensional spectral data cube of the target to be observed.

[0026] According to the present invention, a method for cube reconstruction of spectral data based on compressed physical priors is provided. The cascaded module comprises M sequentially connected cascaded sub-modules. The step of processing the two-dimensional measurement image, the compressed physical prior vector, and the initial fusion features through the cascaded module to obtain target cascaded features includes:

[0027] The two-dimensional measurement image, the compressed physical prior vector, and the initial fusion feature are input into the cascaded module. The two-dimensional measurement image, the compressed physical prior vector, and the initial fusion feature are processed by the first cascaded submodule of the cascaded module to obtain the first intermediate feature.

[0028] For the i-th cascaded submodule, perform the following steps:

[0029] The i-th intermediate feature, the two-dimensional measurement image, and the compressed physical prior vector are processed by the i-th cascade submodule to output the i-th intermediate feature, where the value of i is in the range of 2≤i≤M.

[0030] After being processed sequentially by M cascaded sub-modules, the Mth intermediate feature output by the Mth cascaded sub-module is determined as the target cascaded feature.

[0031] According to the present invention, a method for reconstructing a cube of spectral data based on compressed physical prior is provided. The cascaded submodules include an information fusion unit and a convolutional network unit. The step of processing the (i-1)th intermediate feature output from the previous cascaded submodule, the two-dimensional measurement image, and the compressed physical prior vector through the i-th cascaded submodule to output the i-th intermediate feature includes:

[0032] The (i-1)th intermediate feature, the two-dimensional measurement image, and the compressed physical prior vector are input into the i-th cascaded submodule. The information fusion unit performs channel splicing and convolution fusion on the (i-1)th intermediate feature, the two-dimensional measurement image, and the compressed physical prior vector to obtain the i-th multi-source fusion feature.

[0033] The i-th multi-source fusion feature is extracted using the convolutional network unit, and the i-th intermediate feature is output.

[0034] The present invention also provides a spectral data cube reconstruction device based on compressed physical priors, comprising the following modules:

[0035] The acquisition module is used to acquire a two-dimensional measurement image of the target to be observed after it has been encoded and modulated by the spectral imaging chip;

[0036] The acquisition module is also used to acquire the original transmission spectrum matrix corresponding to each pixel in the spectral imaging chip;

[0037] The analysis module is used to perform principal component analysis on the original transmission spectrum matrix in the spectral dimension and extract the principal component eigenvectors by singular value decomposition.

[0038] The projection module is used to project the original transmission spectrum matrix onto the orthogonal subspace formed by the principal component eigenvectors to obtain a compressed physical prior vector.

[0039] The input module is used to input the two-dimensional measurement image and the compressed physical prior vector into the deep unfolded neural network and output the high-dimensional spectral data cube of the target to be observed.

[0040] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the spectral data cube reconstruction method based on compressed physical priors as described above.

[0041] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the spectral data cube reconstruction method based on compressed physical priors as described above.

[0042] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the spectral data cube reconstruction method based on compressed physical priors as described above.

[0043] This invention provides a method, apparatus, device, storage medium, and program product for reconstructing a spectral data cube based on compressed physical priors. The method involves: acquiring a two-dimensional measurement image of the target object after encoding and modulation by a spectral imaging chip; acquiring the original transmission spectrum matrix corresponding to each pixel in the spectral imaging chip; performing principal component analysis on the original transmission spectrum matrix in the spectral dimension and extracting principal component eigenvectors using singular value decomposition; projecting the original transmission spectrum matrix onto an orthogonal subspace formed by the principal component eigenvectors to obtain compressed physical prior vectors; and inputting the two-dimensional measurement image and the compressed physical prior vectors into a deep unfolded neural network to output a high-dimensional spectral data cube of the target object. This invention addresses the technical problem of traditional spectral reconstruction algorithms, where the large number of network parameters negatively impacts the reconstruction efficiency of spectral data cubes. Compared to existing technologies, this invention performs principal component analysis and singular value decomposition on the original transmission spectrum matrix to extract core principal component feature vectors and construct orthogonal subspaces. This projects and compresses the high-dimensional original transmission spectrum matrix into low-dimensional physical prior vectors, preserving the core physical mechanism information of spectral imaging while significantly eliminating redundant data. After the compressed physical prior vectors are input into a deep unfolded neural network along with the two-dimensional measurement image, the number of network parameters and memory usage are effectively reduced, avoiding the problem of linear growth in computational complexity with the number of spectral bands, and significantly improving the efficiency of spectral imaging. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0045] Figure 1 This is one of the flowcharts of the spectral data cube reconstruction method based on compressed physical priors provided by the present invention.

[0046] Figure 2 This is the second flowchart of the spectral data cube reconstruction method based on compressed physical priors provided by the present invention.

[0047] Figure 3 This is a schematic diagram of the structure of the deep unfolded neural network for the spectral data cube reconstruction method based on compressed physical priors provided by the present invention.

[0048] Figure 4 This is a schematic diagram of the spectral data cube reconstruction device based on compressed physical priors provided by the present invention.

[0049] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0051] The following is combined Figure 1 and Figure 2 The present invention describes a method for reconstructing spectral data cubes based on compressed physical priors. This method is applicable to any reconstruction of spectral data cubes based on compressed physical priors. The execution subject of this method can be an electronic device or a spectral data cube reconstruction device based on compressed physical priors installed in the electronic device. The spectral data cube reconstruction device based on compressed physical priors can be implemented by software, hardware, or a combination of both.

[0052] Figure 1 This is one of the flowcharts illustrating the spectral data cube reconstruction method based on compressed physical priors provided by this invention, such as... Figure 1 As shown, the method includes the following:

[0053] Step 101: Acquire a two-dimensional measurement image of the target to be observed after it has been encoded and modulated by a spectral imaging chip;

[0054] It should be noted that the two-dimensional measurement image is the imaging result formed after the light radiation signal emitted or reflected by the target is projected onto the spectral imaging chip. The chip then performs synchronous encoding and modulation processing on the spatial and spectral dimensions of the target's light signal according to preset encoding and modulation rules. This two-dimensional measurement image is not a traditional image that merely represents the spatial shape of the target. Instead, it integrates the spectral modulation information encoded by the chip. The gray value or signal intensity of each pixel is associated with the spectral characteristics and encoding mode of the corresponding location of the target, thus completely preserving the coupled information of the spatial distribution and spectral characteristics of the target.

[0055] Step 102: Obtain the original transmission spectrum matrix corresponding to each pixel in the spectral imaging chip;

[0056] It should be noted that the original transmission spectrum matrix refers to a real-number matrix that characterizes the intrinsic spectral transmission characteristics of each pixel unit in the spectral imaging chip. Its dimension corresponds to the spatial pixel scale of the chip and the preset number of detection spectral bands (usually expressed as R). H×W×C The original transmission spectrum matrix (where H and W represent the spatial pixel height and width of the chip, and C represents the number of detection spectral bands) contains the original transmittance value of a spatial pixel in the chip at a specific wavelength for incident light. It comprehensively records the spectral response of each pixel at different wavelengths and is a core physical parameter reflecting the optical performance of the spectral imaging chip. Furthermore, this matrix serves as the foundational physical prior data for subsequent principal component analysis of the spectral dimensions and the generation of compressed physical prior vectors. It typically requires optical calibration before chip delivery or on-site calibration in actual measurement scenarios to ensure it accurately reflects the intrinsic transmission performance of each pixel.

[0057] Step 103: Perform principal component analysis on the original transmission spectrum matrix in the spectral dimension, and extract the principal component eigenvectors by singular value decomposition.

[0058] It should be noted that the spectral dimension principal component analysis performed on the original transmission spectrum matrix is ​​to achieve dimensionality reduction of spectral features and removal of redundant information. Since the transmittance data corresponding to different wavelengths in the original transmission spectrum matrix have strong linear correlations, directly using them for subsequent calculations would increase computational complexity and easily introduce noise interference. Therefore, the singular value decomposition method is used to extract features of the matrix in the spectral dimension, which can map the high-dimensional spectral transmission features to the low-dimensional principal component space. The extracted principal component eigenvectors are the core basis vectors characterizing the pixel transmission characteristics of the spectral imaging chip. They are sorted from largest to smallest singular value, and the first few order principal component eigenvectors can cover most of the effective information of the original transmission spectrum matrix. This process can not only effectively reduce the computational load of subsequent data processing and improve the real-time performance of spectral demodulation, but also filter out noise and non-critical features in the matrix, and enhance the representation ability of effective spectral information.

[0059] Step 104: Project the original transmission spectrum matrix onto the orthogonal subspace formed by the principal component eigenvectors to obtain the compressed physical prior vector;

[0060] It should be noted that the principal component eigenvectors are selected from the core spectral features of the original transmission spectrum matrix using singular value decomposition, while the original transmission spectrum matrix to be projected fully carries the intrinsic transmission characteristics of each pixel in the spectral imaging chip. The essence of projection is to map the high-dimensional original transmission spectrum matrix to a low-dimensional orthogonal subspace. By leveraging the orthogonality of the subspace, the spectral information in the matrix is ​​filtered and reconstructed. This process eliminates noise components unrelated to the core transmission characteristics and redundant information caused by process deviations in the original transmission spectrum matrix, while retaining the essential physical features crucial for spectral demodulation. The compressed physical prior vector is a low-dimensional condensed form of the original transmission spectrum matrix after projection into the orthogonal subspace. It inherits the core physical prior information of the transmission characteristics of each pixel in the original matrix, significantly simplifies the data scale through dimensionality compression, and avoids cross-interference of feature information due to the characteristics of the orthogonal subspace. This vector will serve as the core physical constraint in the subsequent spectral demodulation process, enabling accurate inversion of the true spectral information of the observed target from the two-dimensional measurement image.

[0061] Step 105: Input the two-dimensional measurement image and the compressed physical prior vector into the deep unfolded neural network, and output the high-dimensional spectral data cube of the target to be observed.

[0062] It should be noted that deep unfolded neural networks are a novel network architecture that deeply integrates physical models with data-driven learning. They are trainable models composed of cascaded networks of M stages. Their core design logic simulates the iterative solution process of the spectral demodulation physical model, mapping each step of the traditional iterative optimization algorithm to a stage layer of the network. Each stage clearly corresponds to an iterative update in the physical solution. The signal flowing from the first stage to the Mth stage is equivalent to executing a complete iterative algorithm with adaptively optimized parameters, enabling the high-dimensional spectral data cube to be accurately reconstructed step by step and iteratively.

[0063] In practice, during the inference phase of a deep unfolded neural network, the network weights and input data can be converted into half-precision floating-point (FP16) format, and then convolution and matrix multiplication operations can be performed using the GPU's Tensor Cores or NPU hardware units, which can achieve approximately 3 times the inference frame rate (i.e., Mixed Precision Acceleration).

[0064] This invention acquires a two-dimensional measurement image of the target object after it has been encoded and modulated by a spectral imaging chip; acquires the original transmission spectrum matrix corresponding to each pixel in the spectral imaging chip; performs principal component analysis on the original transmission spectrum matrix in the spectral dimension and extracts principal component feature vectors using singular value decomposition; projects the original transmission spectrum matrix onto an orthogonal subspace formed by the principal component feature vectors to obtain compressed physical prior vectors; and inputs the two-dimensional measurement image and the compressed physical prior vectors into a deep unfolded neural network to output a high-dimensional spectral data cube of the target object. This invention addresses the technical problem of traditional spectral reconstruction algorithms, where the large number of network parameters negatively impacts the reconstruction efficiency of spectral data cubes. Compared to existing technologies, this invention performs principal component analysis and singular value decomposition on the original transmission spectrum matrix to extract core principal component feature vectors and construct orthogonal subspaces. This projects and compresses the high-dimensional original transmission spectrum matrix into low-dimensional physical prior vectors, preserving the core physical mechanism information of spectral imaging while significantly eliminating redundant data. After the compressed physical prior vectors are input into a deep unfolded neural network along with the two-dimensional measurement image, the number of network parameters and memory usage are effectively reduced, avoiding the problem of linear growth in computational complexity with the number of spectral bands, and significantly improving the efficiency of spectral imaging.

[0065] Based on any of the above embodiments, the step of performing principal component analysis on the original transmission spectrum matrix in the spectral dimension and extracting principal component eigenvectors using singular value decomposition includes:

[0066] The original transmission spectrum matrix is ​​expanded along the spatial pixel dimension to obtain a two-dimensional matrix, wherein each row vector of the two-dimensional matrix corresponds to the spectral transmission response data of the spatial pixel in the spectral imaging chip.

[0067] The two-dimensional matrix is ​​subjected to principal component analysis in the spectral dimension for decentralization preprocessing to obtain a decentralized matrix;

[0068] Calculate the covariance matrix of the spectral dimension based on the decentralized matrix;

[0069] The covariance matrix is ​​subjected to singular value decomposition to obtain the singular value matrix;

[0070] The eigenvectors in the singular value matrix are sorted in descending order according to their singular values ​​to obtain an eigenvector sequence, and the principal component eigenvectors are extracted from the eigenvector sequence.

[0071] It should be noted that the original transmission spectrum matrix (i.e., Φ∈R) H×W×C(H×W is the spatial pixel dimension, and C is the spectral dimension) is expanded into a two-dimensional matrix according to the spatial pixel dimension. This is to integrate the three-dimensional data of "space-spectrum" into the two-dimensional form of "spatial pixel-spectral response". At this time, each row vector of the matrix corresponds to the complete spectral transmission response data of a single spatial pixel, which is convenient for focusing on the feature analysis of the spectral dimension.

[0072] Understandably, performing spectral dimension decentering preprocessing on a two-dimensional matrix is ​​a standard pre-operation for principal component analysis. The purpose is to eliminate the interference of the mean of the transmission response data in the spectral dimension, so that the covariance matrix calculated subsequently can accurately characterize the linear correlation and fluctuation characteristics between transmission responses at different wavelengths.

[0073] It should be noted that the covariance matrix is ​​a C×C symmetric matrix calculated based on the decentralized matrix in the spectral dimension (where C is the number of spectral bands in the original transmission spectrum matrix). It is also the core statistical matrix for quantifying the correlation of spectral dimension features in principal component analysis. The rows and columns of this matrix correspond to different wavelength bands in spectral imaging. The (i,j)-th element in the matrix represents the covariance value between the transmission response data of the i-th wavelength and the j-th wavelength after decentralization. The magnitude of the covariance value reflects the degree of linear correlation between the transmission responses of the two wavelengths. The larger the value, the more consistent the trends of the two, and the more redundant information there is; the closer the value is to 0, the weaker the linear correlation between the two, and the more independent information it carries.

[0074] Understandably, performing singular value decomposition (SVD) on the covariance matrix decomposes the spectral covariance matrix into a combination of singular values ​​and eigenvectors, which is also the core computational step for extracting principal components in the spectral dimension. Since the covariance matrix is ​​a C×C symmetric matrix, there is a one-to-one correspondence between singular values ​​and eigenvalues ​​in the SVD results. Furthermore, the magnitude of the singular value directly reflects the contribution of the corresponding eigenvector to the spectral dimension features. Specifically, the larger the singular value, the higher the proportion of effective information from the spectral transmission response data carried by the corresponding eigenvector, and the stronger the representational ability of the overall spectral dimension features.

[0075] It should be noted that sorting the eigenvectors in the singular value matrix in descending order of their corresponding singular values ​​essentially rearranges the spectral dimension features according to their information contribution from high to low. In the resulting sequence of eigenvectors, the first eigenvectors correspond to the most representative core features of the spectral transmission response, while the later eigenvectors correspond to relatively minor features containing noise or redundant information. Extracting principal component eigenvectors from this sequence specifically involves selecting the first K eigenvectors as principal component eigenvectors (where K is much smaller than the spectral dimension C). This process achieves dimensionality reduction of the spectral dimension: by retaining the first K principal component eigenvectors with the highest information contribution, the core feature information of the spectral transmission response data is fully preserved; by discarding eigenvectors with low contribution, noise and redundant information in the spectral dimension are eliminated, significantly reducing the computational complexity of subsequent spectral feature analysis while ensuring the accuracy and effectiveness of feature representation.

[0076] The spectral data cube reconstruction method based on compressed physical priors provided in this invention expands the original transmission spectrum matrix into a two-dimensional matrix corresponding to the spectral transmission response of each pixel along the spatial pixel dimension. After spectral dimension decentralization preprocessing, the covariance matrix is ​​calculated and singular value decomposition is performed. Principal component feature vectors are extracted in descending order of singular values. While fully preserving the core features of the chip pixel transmission response, the high-dimensional prior is compressed. This simplifies the prior dimension, reduces the computational load of the subsequent model, strengthens the physical relevance of the compressed physical prior to avoid redundant information interference, and also helps the model fit the spectral imaging logic to fuse information. This effectively improves the spatial detail and spectral accuracy of high-dimensional spectral reconstruction, alleviates the spectral "smoothing" distortion of traditional methods, and improves the global matching degree between the reconstruction results and the real data.

[0077] Based on any of the above embodiments, before inputting the two-dimensional measurement image and the compressed physical prior vector into the deep unfolded neural network and outputting the high-dimensional spectral data cube of the target to be observed, the method further includes:

[0078] Construct an initial training sample set for a deep unfolded neural network, wherein the training sample set includes multiple sets of training samples, each set of training samples including two-dimensional measurement sample images, compressed physical prior sample vectors, and a real high-dimensional spectral data cube;

[0079] Randomly select target training samples from the training sample set;

[0080] The two-dimensional measurement sample image of the target training sample and the compressed physical prior sample vector are input into the deep unfolded neural network to obtain the predicted high-dimensional spectral data cube.

[0081] Based on the predicted high-dimensional spectral data cube and the real high-dimensional spectral data cube, calculate the spectral distribution constraint loss value;

[0082] Based on the spectral distribution constraint loss value, the initial deep unfolded neural network is trained to obtain a trained deep unfolded neural network.

[0083] It should be noted that the training sample set includes multiple sets of training samples. Each set of training samples includes two-dimensional measurement sample images corresponding to different observation targets, compressed physical prior sample vectors matching the observation target, and the real high-dimensional spectral data cube corresponding to the observation target.

[0084] It should be noted that the initial deep unfolded neural network refers to the deep unfolded neural network to be trained. In its training process, randomly selected target training samples are input into the initial deep unfolded neural network. The output predicted high-dimensional spectral data cube usually has significant deviations from the real high-dimensional spectral data cube, such as missing spatial details and distorted spectral curves. These deviations constitute the core basis for calculating the spectral distribution constraint loss value. Based on the backpropagation mechanism of the spectral distribution constraint loss value, the parameters of the deep unfolded neural network are adjusted in a targeted manner, so that it gradually adapts to the exclusive mapping logic from "two-dimensional measurement image + compressed physical prior vector" to high-dimensional spectral data cube. Finally, it realizes the transformation from a training framework with only general capabilities to a dedicated model that fits the needs of spectral data cube reconstruction tasks.

[0085] It should be noted that the Wasserstein-1 distance can be introduced to calculate the spectral distribution constraint loss value. This method aims to solve the "smoothing" problem of the spectral curve caused by the traditional MSE loss function. Specifically, it is achieved by quantizing the predicted spectral distribution Z. pred Compared with the true spectral distribution Z gt The difference between them is achieved by the following formula:

[0086]

[0087] In the formula, Z pred Z represents the spectral distribution corresponding to the predicted high-dimensional spectral data cube. gt λ represents the spectral distribution corresponding to the real high-dimensional spectral data cube. min λ represents the minimum wavelength of the spectrum of the target object to be observed. max F represents the maximum wavelength of the spectrum of the target to be observed. pred (λ) is Z pred The integral of (λ) over wavelength λ, F gt (λ) is Z gt (λ) is the integral of wavelength λ.

[0088] In practical implementation, the basic loss value for spectral reconstruction can be calculated based on the predicted high-dimensional spectral data cube and the actual high-dimensional spectral data cube. Then, this basic loss value and the spectral distribution constraint loss value are combined to jointly optimize the initial depth of the unfolded neural network. Specifically, the basic loss value for spectral reconstruction can be calculated using the mean squared error (MSE) method. Its core is to quantize the predicted high-dimensional spectral data cube (denoted as Z). pred The dimensions are H×W×C (formed by spatial pixels H×W and spectral dimension C) and the real high-dimensional spectral data cube (denoted as Z). gt Dimensions and Z pred Consistent overall deviation across all dimensions: First, locate the predicted value Z for each spatial-spectral pixel. pred (i,j,k) (i and j are spatial pixel coordinates, k is the spectral channel) and the true value Z gt For each point (i,j,k), calculate the squared difference between the predicted and actual values, sum the squared differences for all points, and finally divide by the total number of pixels (H×W×C) to obtain the result. The expression is as follows:

[0089]

[0090] The spectral data cube reconstruction method based on compressed physical priors provided in this invention constructs a training sample set containing two-dimensional measurement sample images, compressed physical prior sample vectors, and real high-dimensional spectral data cubes. It combines random sampling input to the model to obtain prediction results and a process of training the model based on dual-loss values. This approach leverages the "input-prior-label" matching relationship of the samples to train the model to fit the actual physical scenario of spectral imaging. Random sampling also enhances the model's generalization ability to different observation targets. Furthermore, the collaborative constraint of dual losses balances the global error of the reconstruction results with the fidelity of spectral details, enabling the trained deep unfolded neural network to more accurately fuse information. This effectively improves the reconstruction accuracy of high-dimensional spectral data, alleviates the spectral "smoothing" problem, and provides reliable model support for spectral reconstruction in real-world scenarios.

[0091] Figure 2 This is the second flowchart of the spectral data cube reconstruction method based on compressed physical priors provided by this invention, as shown below. Figure 2 As shown, the deep unfolded neural network includes a pre-convolutional layer, a cascaded module, and an end convolutional layer. Step 105 also includes steps 1051 to 1053:

[0092] Step 1051: Input the two-dimensional measurement image and the compressed physical prior vector into the deep unfolded neural network, and perform convolution operation on the two-dimensional measurement image and the compressed physical prior vector through the pre-convolutional layer to obtain the initial fusion features;

[0093] It should be noted that the deep unfolded neural network consists of three core functional modules: a pre-convolutional layer, a cascaded module, and an end convolutional layer, forming a three-level processing architecture of "front-end preprocessing - mid-level deep iterative optimization - end-level dimensionality restoration". The three modules are connected in sequence and complement each other, which not only avoids the problem of insufficient feature extraction of a single network structure, but also achieves accurate and controllable reconstruction of high-dimensional spectral data through modular design.

[0094] It should be noted that the two-dimensional measurement image, as low-dimensional spatial domain data, only contains the spatial structure information of the target to be observed (the dimension is usually H×W×1), while the compressed physical prior vector is obtained by dimensionality reduction of the original transmission spectrum matrix and carries the core spectral prior information of the target (the dimension is usually H×W×K, where K < the number of original spectral channels). Although the two have the same spatial dimension, their spectral dimensions are different. The pre-convolutional layer performs joint convolution operation through preset convolution kernel parameters (such as convolution kernel size, stride, and padding method). It is not a simple feature superposition, but a deep fusion of spatial structure information and spectral prior information (i.e., the two-dimensional measurement image and the compressed physical prior vector) based on the local perception mechanism of convolution. At the same time, it maps the two types of heterogeneous input data to a unified feature dimension space, effectively avoiding the problems of dimensional redundancy, information fragmentation and poor feature adaptability caused by direct splicing. Finally, the output is an initial fused feature that has both spatial structure attributes and spectral prior attributes.

[0095] Step 1052: Process the two-dimensional measurement image, the compressed physical prior vector, and the initial fusion features through the cascade module to obtain the target cascade features;

[0096] It should be noted that the core logic of the cascade module lies in the multi-source collaborative optimization of the spatial structure information of the two-dimensional measurement image, the spectral prior constraints of the compressed physical prior vector, and the initial fusion features output by the pre-convolutional layer. It relies on a hierarchical processing mechanism to break the information limitations of a single feature: the cascade module does not simply repeat the calculation of the input information, but is based on the natural complementarity of multi-source information. On the basis of fully preserving the coupled spatial-spectral correlation information in the initial fusion features, it continuously reuses the original spatial details of the two-dimensional measurement image and the core spectral laws of the compressed physical prior vector. By strengthening the feature representation accuracy layer by layer and dynamically correcting feature deviations, it achieves a step-by-step advancement from the initial fusion features to the high-discrimination, strongly correlated target cascade features. The target cascade features not only deeply integrate the key information advantages of the three types of inputs, but also have the deep feature expression capability to accurately adapt to the needs of high-dimensional spectral data cube reconstruction, effectively solving the technical pain points such as feature information loss and insufficient representation capability that are prone to occur in the process of mapping from low-dimensional input to high-dimensional output.

[0097] Step 1053: Process the target cascaded features through the terminal convolutional layer to output the high-dimensional spectral data cube of the target to be observed.

[0098] It should be noted that the core function of the terminal convolutional layer is to perform dimensional mapping on the target cascaded features (which have integrated spatial structure, spectral priors, and deep correlation information) output by the cascaded module. By adapting the parameter design to the number of high-dimensional spectral channels, the low-dimensional fused features are accurately converted into a high-dimensional spectral data cube with dimensions of H×W×C (C being the original number of spectral channels). This ensures both the fidelity of spatial details and the accuracy of spectral curves, and completes the final reconstruction loop from low-dimensional input to high-dimensional output, directly determining the overall reconstruction effect. Specifically, the terminal convolutional layer can overcome the dimensionality limitations of the low-dimensional feature space by designing pre-defined targeted parameters (such as the output channel dimension matching the number of high-dimensional spectral channels of the target and the convolutional kernel size that adapts to the spatial-spectral mapping). This allows the target cascaded features to be accurately transformed from the fused feature dimension to the full-spectral dimension. At the same time, the local correlation of convolutional operations ensures the integrity and accuracy of the spectral curve corresponding to each spatial pixel, avoiding spectral distortion or loss of spatial details during the dimensionality transformation process. The final output high-dimensional spectral data cube (usually with dimensions of H×W×C, where C is the original number of high-dimensional spectral channels) completely reproduces the spatial distribution and full-spectral information of the target to be observed.

[0099] The spectral data cube reconstruction method based on compressed physical priors provided in this invention employs a deep unfolded neural network architecture consisting of a "pre-convolutional layer + cascaded module + terminal convolutional layer". The pre-convolutional layer performs joint convolution operations on the two-dimensional measurement image and the compressed physical prior vector, deeply coupling the spatial structure information and spectral prior information carried by each, thus avoiding the information fragmentation problem caused by directly splicing heterogeneous data and effectively improving the correlation and dimensionality adaptability of the initial fused features. The cascaded module performs layer-by-layer collaborative processing of the aforementioned multi-source information, strengthening the deep correlation between spatial and spectral features. This method addresses the issue of insufficient feature representation when mapping low-dimensional input to high-dimensional output, further improving the accuracy and discriminative power of reconstructed features. The terminal convolutional layer, based on parameter design adapted to the number of high-dimensional spectral channels, accurately maps the target cascaded features into a high-dimensional spectral data cube, ensuring both the fidelity of spatial details and the accuracy of spectral curve reconstruction. Overall, this method reduces the cost of high-dimensional spectral data acquisition by compressing physical priors while balancing reconstruction efficiency and reconstruction quality in both spatial and spectral dimensions. It can efficiently and accurately reconstruct high-dimensional spectral data, adapting to various observation scenarios requiring high-dimensional spectral information.

[0100] Based on any of the above embodiments, the cascaded module includes M sequentially connected cascaded sub-modules; wherein, the step of processing the two-dimensional measurement image, the compressed physical prior vector, and the initial fusion features through the cascaded module to obtain the target cascaded features includes:

[0101] The two-dimensional measurement image, the compressed physical prior vector, and the initial fusion feature are input into the cascaded module. The two-dimensional measurement image, the compressed physical prior vector, and the initial fusion feature are processed by the first cascaded submodule of the cascaded module to obtain the first intermediate feature.

[0102] For the i-th cascaded submodule, perform the following steps:

[0103] The i-th intermediate feature, the two-dimensional measurement image, and the compressed physical prior vector are processed by the i-th cascade submodule to output the i-th intermediate feature, where the value of i is in the range of 2≤i≤M.

[0104] After being processed sequentially by M cascaded sub-modules, the Mth intermediate feature output by the Mth cascaded sub-module is determined as the target cascaded feature.

[0105] It should be noted that the core of the cascaded module is to achieve layer-by-layer feature optimization through the collaborative operation of multiple rounds of cascaded sub-modules. Specifically, the first cascaded sub-module integrates the two-dimensional measurement image, compressed physical prior vector, and initial fusion features to complete the initial coupling of multi-source information and output the first intermediate feature. Subsequently, the i-th cascaded sub-module reuses the intermediate features of the previous level and the two types of original input data (i.e., the two-dimensional measurement image and the compressed physical prior vector). Through continuous information interaction and feature processing, spatial-spectral correlation information is strengthened and feature bias is corrected in each level of processing, achieving a step-by-step refinement of feature representation. After layer-by-layer iteration of M cascaded sub-modules, the final output M-th level intermediate feature (i.e., the target cascaded feature) has deeply integrated the advantages of multi-source information and has a stronger high-dimensional spectral representation capability. This design not only avoids the problem of insufficient feature extraction in single-module processing, but also improves the stability and accuracy of features by reusing prior information in multiple rounds.

[0106] Understandably, the iterative design of the M cascaded sub-modules is not a simple hierarchical superposition, but rather a closed-loop mechanism of "previous-level feature feedback + original information reuse." This allows each processing level to further explore deep spatial-spectral correlations based on the results of previous optimizations, while continuously correcting feature biases with spectral constraints that compress physical prior vectors, thus avoiding feature drift. The value of M can be flexibly adjusted according to the actual reconstruction accuracy requirements, satisfying the high-efficiency processing needs of low-precision scenarios while also adapting to high-precision reconstruction scenarios by increasing the number of sub-modules, balancing flexibility and practicality. Furthermore, this layer-by-layer refined feature optimization mode can gradually filter redundant information and strengthen effective feature representations, continuously improving the purity and discriminative power of the target cascaded features. This not only provides more accurate feature support for the high-dimensional mapping of the final convolutional layer but also makes the processing logic of the entire deep unfolded neural network more interpretable, avoiding the black-box problem of traditional end-to-end models.

[0107] The spectral data cube reconstruction method based on compressed physical prior provided in this invention employs a cascaded module structure consisting of M sequentially connected sub-modules. The first cascaded sub-module integrates a two-dimensional measurement image, compressed physical prior vectors, and initial fusion features, achieving the initial deep coupling of multi-source information and outputting a first intermediate feature with both spatial and spectral attributes, thus realizing the initial accurate correlation of heterogeneous information. Subsequent cascaded sub-modules reuse the intermediate features from the previous stage and the two types of original input data, continuously strengthening the spatial-spectral relationship at each processing stage through hierarchical information interaction and feature processing. By associating information and correcting feature biases, a step-by-step refinement of feature representation is achieved, solving the problem of insufficient feature extraction that is prone to occur in single-module processing. After iterative processing by M cascaded sub-modules, the final target cascaded features deeply integrate the advantages of multi-source information and have stronger high-dimensional spectral representation capabilities. At the same time, by reusing and compressing physical prior information in multiple rounds, the stability and accuracy of the features are improved, providing high-quality core feature support for the accurate reconstruction of high-dimensional spectral data cubes in the terminal convolutional layer. Thus, while ensuring reconstruction efficiency, the spatial fidelity and spectral accuracy of high-dimensional spectral data are effectively improved.

[0108] Based on any of the above embodiments, the cascaded submodule includes an information fusion unit and a convolutional network unit; wherein, the step of processing the (i-1)th intermediate feature output by the previous cascaded submodule, the two-dimensional measurement image, and the compressed physical prior vector through the i-th cascaded submodule to output the i-th intermediate feature includes:

[0109] The (i-1)th intermediate feature, the two-dimensional measurement image, and the compressed physical prior vector are input into the i-th cascaded submodule. The information fusion unit performs channel splicing and convolution fusion on the (i-1)th intermediate feature, the two-dimensional measurement image, and the compressed physical prior vector to obtain the i-th multi-source fusion feature.

[0110] The i-th multi-source fusion feature is extracted using the convolutional network unit, and the i-th intermediate feature is output.

[0111] It should be noted that the information fusion unit includes a channel concatenation (Concat) subunit and a convolution (Conv) subunit. The specific processing flow is as follows: First, the channel concatenation subunit merges the (i-1)th intermediate feature, the two-dimensional measurement image, and the compressed physical prior vector in terms of channel dimension, integrating these heterogeneous data into the same feature space to form aggregated information. This breaks down the dimensional barriers of heterogeneous data and completes the initial aggregation of multi-source information. Then, the convolution subunit performs targeted convolution operations on the concatenated aggregated information. With the help of the local correlation characteristics of the convolution kernel, the pre-level optimization features, spatial structural details, and spectral prior constraints carried by these three types of inputs are deeply coupled, and finally, the i-th multi-source fusion feature with unified dimension and high information correlation is obtained. This processing logic of "first concatenation and aggregation, then convolution and coupling" not only ensures the complete coverage of the three types of input information, but also avoids the information fragmentation problem that is easy to occur when directly concatenating heterogeneous data. It provides high-quality basic feature support for the subsequent convolutional network unit to deeply extract and refine the fusion feature.

[0112] It should be noted that convolutional network units typically adopt an encoder-decoder network architecture (such as U-Net). The core of this architecture is to perform a two-way optimization of the i-th multi-source fusion feature, namely "deep feature mining + spatial detail completion" (such as denoising and feature refinement): In the encoding stage, through layer-by-layer convolution and downsampling operations, more abstract deep correlation information is extracted from the multi-source fusion feature, strengthening the inherent correspondence between spatial structure and spectral constraints; In the decoding stage, the spatial dimension of the feature is restored through upsampling, and the deep features in the encoding stage are fused with the shallow detail features in the decoding process across layers, while retaining the deep correlation information and completing the spatial pixel-level detail features.

[0113] Understandably, the i-th intermediate feature, compared to the input multi-source fusion feature, has more accurate spatial-spectral correlation attributes and higher feature purity and discriminability. This processing method avoids the spatial detail loss problem common in deep feature extraction and realizes the step-by-step upgrade of feature representation, so that the output of each cascaded sub-module can provide a better feature foundation for the next level of processing, continuously driving the entire cascaded module to iterate towards high-precision target cascaded features.

[0114] In specific implementations, such as Figure 3 As shown, dimensionality reduction is first performed on the high-dimensional data Φ to obtain a data of dimension R. H×W×K Φ sub (i.e., compressing the physical prior vector); then Φ sub The observed data y (i.e., the two-dimensional measurement image) is input into the pre-convolutional layer, and after convolution, the initial fused feature Z0 is output. Then, Z0 enters the cascaded sub-module of stage 1, and is processed by the information fusion unit and the convolutional network unit in sequence to obtain the intermediate feature Z1. The cascaded sub-modules from stage 2 to M have the same structure as stage 1, and iteratively optimize the intermediate feature of the previous stage to obtain Z. M Ultimately, Z M The input convolutional layer performs dimension restoration and outputs the reconstructed high-dimensional spectral data Z. pred (Dimension R) H×W×C ).

[0115] The spectral data cube reconstruction method based on compressed physical priors provided in this invention uses a combined architecture of cascaded sub-modules: an information fusion unit and a convolutional network unit. The information fusion unit can integrate three types of heterogeneous data, covering multi-source information while avoiding splicing and fragmentation, resulting in highly correlated fused features. The convolutional network unit refines these features, strengthening spatial-spectral correlation and preserving details. The single-module "fusion followed by refinement" combined with multi-level iterations makes the target cascaded features more accurate and stable, reducing the cost of high-dimensional spectral acquisition and improving the spatial fidelity and spectral accuracy of the reconstruction results.

[0116] The following examples, using specific scenarios, further illustrate the method for spectral data cube reconstruction based on compressed physical priors:

[0117] Example 1:

[0118] (1) For a spectral imaging system with a resolution of 1024×1024 and 101 bands.

[0119] (2) The size of the original transmission spectrum matrix Φ is 1024×1024×101, corresponding to a spatial size of 1024×1024. The wavelength range is 400~900nm, and the number of wavelength channels is 101. The size of the measured image y is 1024×1024, and the size of the target output spectrum image is 1024×1024×101.

[0120] (3) Calculate the covariance matrix of the 101-channel transmission spectrum Φ, extract the top 16 eigenvectors with the largest eigenvalues, and compress the transmission spectrum into a 1024×1024×16 feature map Φ. sub ∈R H×W×K .

[0121] (4) The convolutional neural network used is a 3-stage deep unfolding network. The input of each stage includes the estimated value of the previous stage, the original measurement map and the compressed physical features of 16 channels. The sub-module uses U-Net and the downsampling number is 2.

[0122] (5) The running speed on the RTX5090 graphics card is 65 frames per second, which meets the requirements of video-level reconstruction, and the reconstructed image has sharp edges and no obvious mosaic effect or false color.

[0123] The following describes the spectral data cube reconstruction apparatus based on compressed physical priors provided by this invention. The spectral data cube reconstruction apparatus described below corresponds to and can be referred to in conjunction with the spectral data cube reconstruction method based on compressed physical priors described above. Figure 4 As shown, the spectral data cube reconstruction device based on compressed physical priors includes:

[0124] The acquisition module 10 is used to acquire a two-dimensional measurement image of the target to be observed after it has been encoded and modulated by the spectral imaging chip;

[0125] The acquisition module 10 is also used to acquire the original transmission spectrum matrix corresponding to each pixel in the spectral imaging chip;

[0126] Analysis module 20 is used to perform principal component analysis on the original transmission spectrum matrix in the spectral dimension and extract the principal component eigenvectors by singular value decomposition.

[0127] Projection module 30 is used to project the original transmission spectrum matrix onto the orthogonal subspace formed by the principal component eigenvectors to obtain a compressed physical prior vector.

[0128] The input module 40 is used to input the two-dimensional measurement image and the compressed physical prior vector into the deep unfolded neural network and output the high-dimensional spectral data cube of the target to be observed.

[0129] Optionally, the analysis module 20 is used for:

[0130] The original transmission spectrum matrix is ​​expanded along the spatial pixel dimension to obtain a two-dimensional matrix, wherein each row vector of the two-dimensional matrix corresponds to the spectral transmission response data of the spatial pixel in the spectral imaging chip.

[0131] The two-dimensional matrix is ​​subjected to principal component analysis in the spectral dimension for decentralization preprocessing to obtain a decentralized matrix;

[0132] Calculate the covariance matrix of the spectral dimension based on the decentralized matrix;

[0133] The covariance matrix is ​​subjected to singular value decomposition to obtain the singular value matrix;

[0134] The eigenvectors in the singular value matrix are sorted in descending order according to their singular values ​​to obtain an eigenvector sequence, and the principal component eigenvectors are extracted from the eigenvector sequence.

[0135] Optionally, the input module 40 is used for:

[0136] Construct an initial training sample set for a deep unfolded neural network, wherein the training sample set includes multiple sets of training samples, each set of training samples including two-dimensional measurement sample images, compressed physical prior sample vectors, and a real high-dimensional spectral data cube;

[0137] Randomly select target training samples from the training sample set;

[0138] The two-dimensional measurement sample image of the target training sample and the compressed physical prior sample vector are input into the deep unfolded neural network to obtain the predicted high-dimensional spectral data cube.

[0139] Based on the predicted high-dimensional spectral data cube and the real high-dimensional spectral data cube, calculate the spectral distribution constraint loss value;

[0140] Based on the spectral distribution constraint loss value, the initial deep unfolded neural network is trained to obtain a trained deep unfolded neural network.

[0141] Optionally, the input module 40 is used for:

[0142] The two-dimensional measurement image and the compressed physical prior vector are input into a deep unfolded neural network. The two-dimensional measurement image and the compressed physical prior vector are convolved by the pre-convolutional layer to obtain the initial fusion features.

[0143] The target cascaded features are obtained by processing the two-dimensional measurement image, the compressed physical prior vector, and the initial fusion features through the cascaded module.

[0144] The target cascaded features are processed by the terminal convolutional layer to output a high-dimensional spectral data cube of the target to be observed.

[0145] Optionally, the input module 40 is used for:

[0146] The two-dimensional measurement image, the compressed physical prior vector, and the initial fusion feature are input into the cascaded module. The two-dimensional measurement image, the compressed physical prior vector, and the initial fusion feature are processed by the first cascaded submodule of the cascaded module to obtain the first intermediate feature.

[0147] For the i-th cascaded submodule, perform the following steps:

[0148] The i-th intermediate feature, the two-dimensional measurement image, and the compressed physical prior vector are processed by the i-th cascade submodule to output the i-th intermediate feature, where the value of i is in the range of 2≤i≤M.

[0149] After being processed sequentially by M cascaded sub-modules, the Mth intermediate feature output by the Mth cascaded sub-module is determined as the target cascaded feature.

[0150] Optionally, the input module 40 is used for:

[0151] The (i-1)th intermediate feature, the two-dimensional measurement image, and the compressed physical prior vector are input into the i-th cascaded submodule. The information fusion unit performs channel splicing and convolution fusion on the (i-1)th intermediate feature, the two-dimensional measurement image, and the compressed physical prior vector to obtain the i-th multi-source fusion feature.

[0152] The i-th multi-source fusion feature is extracted using the convolutional network unit, and the i-th intermediate feature is output.

[0153] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a spectral data cube reconstruction method based on compressed physical priors. This method includes: acquiring a two-dimensional measurement image of the target to be observed after being encoded and modulated by a spectral imaging chip; acquiring the original transmission spectrum matrix corresponding to each pixel in the spectral imaging chip; performing principal component analysis on the original transmission spectrum matrix in the spectral dimension and extracting principal component feature vectors by singular value decomposition; projecting the original transmission spectrum matrix onto an orthogonal subspace formed by the principal component feature vectors to obtain compressed physical prior vectors; and inputting the two-dimensional measurement image and the compressed physical prior vectors into a deep unfolded neural network to output a high-dimensional spectral data cube of the target to be observed.

[0154] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0155] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the spectral data cube reconstruction method based on compressed physical prior provided by the above methods. The method includes: acquiring a two-dimensional measurement image of the target to be observed after being encoded and modulated by a spectral imaging chip; acquiring the original transmission spectrum matrix corresponding to each pixel in the spectral imaging chip; performing principal component analysis on the original transmission spectrum matrix in the spectral dimension and extracting principal component feature vectors by singular value decomposition; projecting the original transmission spectrum matrix onto an orthogonal subspace formed by the principal component feature vectors to obtain compressed physical prior vectors; and inputting the two-dimensional measurement image and the compressed physical prior vectors into a deep unfolded neural network to output a high-dimensional spectral data cube of the target to be observed.

[0156] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the spectral data cube reconstruction method based on compressed physical priors provided by the above methods. The method includes: acquiring a two-dimensional measurement image of the target to be observed after being encoded and modulated by a spectral imaging chip; acquiring the original transmission spectrum matrix corresponding to each pixel in the spectral imaging chip; performing principal component analysis on the original transmission spectrum matrix in the spectral dimension and extracting principal component feature vectors by singular value decomposition; projecting the original transmission spectrum matrix onto an orthogonal subspace formed by the principal component feature vectors to obtain compressed physical prior vectors; and inputting the two-dimensional measurement image and the compressed physical prior vectors into a deep unfolded neural network to output a high-dimensional spectral data cube of the target to be observed.

[0157] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0158] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for cube reconstruction of spectral data based on compressed physical priors, characterized in that, include: Acquire a two-dimensional measurement image of the target object after it has been encoded and modulated by a spectral imaging chip; Obtain the original transmission spectrum matrix corresponding to each pixel in the spectral imaging chip; Principal component analysis was performed on the original transmission spectrum matrix in the spectral dimension, and the principal component eigenvectors were extracted by singular value decomposition. The original transmission spectrum matrix is ​​projected onto the orthogonal subspace formed by the principal component eigenvectors to obtain the compressed physical prior vector; The two-dimensional measurement image and the compressed physical prior vector are input into a deep unfolded neural network to output a high-dimensional spectral data cube of the target to be observed. The deep unfolded neural network includes a front convolutional layer, a cascaded module, and an end convolutional layer; The step of performing principal component analysis on the original transmission spectrum matrix in the spectral dimension, and extracting principal component eigenvectors using singular value decomposition, includes: The original transmission spectrum matrix is ​​expanded along the spatial pixel dimension to obtain a two-dimensional matrix, wherein each row vector of the two-dimensional matrix corresponds to the spectral transmission response data of the spatial pixel in the spectral imaging chip. The two-dimensional matrix is ​​subjected to principal component analysis in the spectral dimension for decentralization preprocessing to obtain a decentralized matrix; Calculate the covariance matrix of the spectral dimension based on the decentralized matrix; The covariance matrix is ​​subjected to singular value decomposition to obtain the singular value matrix; The eigenvectors in the singular value matrix are sorted in descending order according to their singular values ​​to obtain an eigenvector sequence. Principal component eigenvectors are then extracted from the eigenvector sequence. The descending order sorting is a rearrangement of the spectral dimension eigenvectors according to their information contribution from high to low.

2. The method for spectral data cube reconstruction based on compressed physical priors according to claim 1, characterized in that, Before inputting the two-dimensional measurement image and the compressed physical prior vector into the deep unfolded neural network to output the high-dimensional spectral data cube of the target to be observed, the method further includes: Construct an initial training sample set for a deep unfolded neural network, wherein the training sample set includes multiple sets of training samples, each set of training samples including two-dimensional measurement sample images, compressed physical prior sample vectors, and a real high-dimensional spectral data cube; Randomly select target training samples from the training sample set; The two-dimensional measurement sample image of the target training sample and the compressed physical prior sample vector are input into the deep unfolded neural network to obtain the predicted high-dimensional spectral data cube. Based on the predicted high-dimensional spectral data cube and the real high-dimensional spectral data cube, calculate the spectral distribution constraint loss value; Based on the spectral distribution constraint loss value, the initial deep unfolded neural network is trained to obtain a trained deep unfolded neural network.

3. The method for spectral data cube reconstruction based on compressed physical priors according to claim 1, characterized in that, The step of inputting the two-dimensional measurement image and the compressed physical prior vector into a deep unfolded neural network to output a high-dimensional spectral data cube of the target to be observed includes: The two-dimensional measurement image and the compressed physical prior vector are input into a deep unfolded neural network. The two-dimensional measurement image and the compressed physical prior vector are convolved by the pre-convolutional layer to obtain the initial fusion features. The target cascaded features are obtained by processing the two-dimensional measurement image, the compressed physical prior vector, and the initial fusion features through the cascaded module. The target cascaded features are processed by the terminal convolutional layer to output a high-dimensional spectral data cube of the target to be observed.

4. The method for spectral data cube reconstruction based on compressed physical priors according to claim 3, characterized in that, The cascaded module includes M sequentially connected cascaded sub-modules; wherein, the process of processing the two-dimensional measurement image, the compressed physical prior vector, and the initial fusion features through the cascaded module to obtain the target cascaded features includes: The two-dimensional measurement image, the compressed physical prior vector, and the initial fusion feature are input into the cascaded module. The two-dimensional measurement image, the compressed physical prior vector, and the initial fusion feature are processed by the first cascaded submodule of the cascaded module to obtain the first intermediate feature. For the i-th cascaded submodule, perform the following steps: The i-th intermediate feature, the two-dimensional measurement image, and the compressed physical prior vector are processed by the i-th cascade submodule to output the i-th intermediate feature, where the value of i is in the range of 2≤i≤M. After being processed sequentially by M cascaded sub-modules, the Mth intermediate feature output by the Mth cascaded sub-module is determined as the target cascaded feature.

5. The method for spectral data cube reconstruction based on compressed physical priors according to claim 4, characterized in that, The cascaded submodules include an information fusion unit and a convolutional network unit; wherein, the process of processing the (i-1)th intermediate feature output by the previous cascaded submodule, the two-dimensional measurement image, and the compressed physical prior vector through the i-th cascaded submodule to output the i-th intermediate feature includes: The (i-1)th intermediate feature, the two-dimensional measurement image, and the compressed physical prior vector are input into the i-th cascaded submodule. The information fusion unit performs channel splicing and convolution fusion on the (i-1)th intermediate feature, the two-dimensional measurement image, and the compressed physical prior vector to obtain the i-th multi-source fusion feature. The i-th multi-source fusion feature is extracted using the convolutional network unit, and the i-th intermediate feature is output.

6. A spectral data cube reconstruction device based on compressed physical priors, characterized in that, include: The acquisition module is used to acquire a two-dimensional measurement image of the target to be observed after it has been encoded and modulated by the spectral imaging chip; The acquisition module is also used to acquire the original transmission spectrum matrix corresponding to each pixel in the spectral imaging chip; The analysis module is used to perform principal component analysis on the original transmission spectrum matrix in the spectral dimension and extract the principal component eigenvectors by singular value decomposition. The projection module is used to project the original transmission spectrum matrix onto the orthogonal subspace formed by the principal component eigenvectors to obtain a compressed physical prior vector. The input module is used to input the two-dimensional measurement image and the compressed physical prior vector into the deep unfolded neural network and output the high-dimensional spectral data cube of the target to be observed; the deep unfolded neural network includes a pre-convolutional layer, a cascaded module and an end convolutional layer. The analysis module is used for: The original transmission spectrum matrix is ​​expanded along the spatial pixel dimension to obtain a two-dimensional matrix, wherein each row vector of the two-dimensional matrix corresponds to the spectral transmission response data of the spatial pixel in the spectral imaging chip. The two-dimensional matrix is ​​subjected to principal component analysis in the spectral dimension for decentralization preprocessing to obtain a decentralized matrix; Calculate the covariance matrix of the spectral dimension based on the decentralized matrix; The covariance matrix is ​​subjected to singular value decomposition to obtain the singular value matrix; The eigenvectors in the singular value matrix are sorted in descending order according to their singular values ​​to obtain an eigenvector sequence. Principal component eigenvectors are then extracted from the eigenvector sequence. The descending order sorting is a rearrangement of the spectral dimension eigenvectors according to their information contribution from high to low.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the spectral data cube reconstruction method based on compressed physical priors as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the spectral data cube reconstruction method based on compressed physical priors as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the spectral data cube reconstruction method based on compressed physical priors as described in any one of claims 1 to 5.

Images