Hyperspectral rapid imaging method and system based on spectral difference deep matrix decomposition

Abstract

The present invention discloses a hyperspectral rapid imaging method and system based on spectral difference deep matrix decomposition. For low signal-to-noise ratio data under extreme imaging conditions, high-quality image restoration is achieved through average spectrum map and spectral difference map structural modeling. First, hyperspectral data is quickly collected and spliced ​​to form a three-dimensional tensor, and a forward observation model under weak illumination conditions is constructed; it is decomposed into an average spectrum map, a spectral difference map and a noise term; the spectral difference map is rearranged into a two-dimensional low-rank matrix, and PCA decomposition is used to obtain the spatial and spectral principal component matrices; an unsupervised dual-path optimization network is innovatively constructed: a 2D jump network is used to optimize the spatial principal components to model structural information, and a 1D jump network is used to optimize the spectral principal components to fit detail changes; finally, the jointly trained network outputs the optimized spectral difference map, which is fused with the average spectrum map to complete the reconstruction. High-fidelity image reconstruction is achieved under single-frame, unsupervised conditions, significantly improving the reconstruction quality and robustness.

Description

Technical Field

[0001] The present invention belongs to the technical field of hyperspectral imaging, and in particular relates to a hyperspectral rapid imaging method and system based on spectral difference depth matrix decomposition. Background Art

[0002] Hyperspectral imaging technology collects object information across dozens to hundreds of continuous spectral bands, generating high-dimensional data and enabling precise identification of material composition, structure, and chemical properties. It is widely used in remote sensing, environmental protection, biomedicine, industrial testing, and security. Compared to traditional RGB imaging, its high spectral resolution can capture subtle spectral differences.

[0003] Currently, mainstream acquisition methods include spatial scanning (point-by-point / line scanning), spectral scanning (multi-band filtering), and snapshot imaging. Scanning imaging requires sequential acquisition of spatial or band information to reconstruct a three-dimensional data cube. To ensure imaging quality in low light or low-reflectivity conditions, long exposure times are often required. However, this leads to two core problems: first, slow imaging speed. Point-by-point or line-by-line scanning significantly prolongs the overall imaging time, making it difficult to meet the high frame rate and fast response requirements of dynamic scenes, real-time monitoring, or high-speed target detection. Second, significant noise interference occurs. Photon statistical noise, sensor noise, and ambient light interference are particularly severe in low-light conditions, resulting in distortion of spatial details and spectral characteristics, affecting subsequent quantitative analysis.

[0004] To increase speed, researchers are exploring ways to shorten integration time or reduce sampling, and using image restoration techniques to recover high-quality images from low-SNR data. Existing restoration techniques fall into three main categories: Traditional filtering methods (such as Savitzky-Golay and BM3D) are simple to implement and computationally fast, but they independently process spatial or spectral dimensions, ignoring coupling characteristics, which can easily lead to spectral line distortion and loss of detail. Model-driven methods (based on low-rank, sparse, and total variation priors) are theoretically robust and highly controllable, preserving structural information. However, they face challenges such as complex hyperparameter tuning, low iterative computational efficiency, and artifacts or detail degradation caused by a mismatch between model assumptions and actual noise. Deep learning methods (CNN / Transformer) offer significant restoration results, but rely heavily on large amounts of paired low-quality and high-quality training data (which is expensive to obtain), require large computational resources, and have weak scene generalization capabilities. Changes in the system or environment can easily lead to a decline in reconstruction quality.

[0005] The unsupervised / self-supervised methods that have emerged in recent years (such as Deep Image Prior) attempt to get rid of the dependence on paired data and only use the noisy image itself and the implicit prior of the network. They have advantages when data is scarce, but still face challenges such as low optimization efficiency (requiring hundreds or thousands of iterations), unstable restoration effects (blurred details, artifact generation), and lack of explicit structural prior constraints (such as low rank and spectral continuity). It is difficult to distinguish between effective signals and noise at extremely low signal-to-noise ratios, affecting spectral fidelity and spatial clarity.

[0006] In summary, while existing methods each have their own advantages, they struggle to simultaneously meet the requirements of high-speed imaging, high-fidelity reconstruction, and system portability in low-light environments. They also suffer from reliance on large amounts of training data, low optimization efficiency, weak generalization capabilities, and insufficient prior constraints. Therefore, there is an urgent need for a hyperspectral image restoration method that does not require paired training samples, can well express structural priors, and can be efficiently optimized. To address this, this paper proposes a hyperspectral fast imaging method based on spectral difference deep matrix decomposition, specifically designed for high-fidelity restoration of raw data acquired under short-exposure, high-noise conditions. Summary of the Invention

[0007] In response to the above technical problems, the present invention provides a hyperspectral rapid imaging method and system based on spectral difference depth matrix decomposition.

[0008] The technical solution adopted by the present invention to solve the technical problem is:

[0009] A hyperspectral rapid imaging method based on spectral difference depth matrix decomposition comprises the following steps:

[0010] S100: A hyperspectral imaging system uses a fast image acquisition strategy to acquire low signal-to-noise ratio hyperspectral data frames under extreme imaging conditions. After image stitching, these frames are combined into a complete three-dimensional hyperspectral image tensor in both time and space dimensions.

[0011] S200: Constructing an imaging forward observation model, modeling the three-dimensional hyperspectral image tensor as consisting of an average spectral image, a spectral difference image, and a noise term; performing averaging processing on each pixel of the three-dimensional spectral image in the spectral dimension to obtain an average spectral image, and constructing a spectral difference image based on the three-dimensional hyperspectral image tensor and the average spectral image;

[0012] S300: rearranging the three-dimensional spectral difference image into a two-dimensional matrix and approximating it into a product form of a low-rank matrix, and performing matrix decomposition using the PCA method to obtain a principal component matrix of the image part and a principal component matrix of the spectral part;

[0013] S400: Construct an unsupervised neural network and use a two-dimensional skip connection network to optimize the principal component matrix of the image part to model spatial structure information. Use a one-dimensional skip connection network to optimize the principal component matrix of the spectrum part to fit the changes in the principal components of the spectrum. The optimized principal component matrices of the image part and the optimized principal component matrices of the spectrum part are obtained respectively. The optimized spectral difference map is obtained by matrix product.

[0014] S500: Construct a loss function, train an unsupervised neural network based on the loss function and the optimized spectral difference image, obtain the trained unsupervised neural network when the preset training end condition is reached, obtain the final spectral difference image based on the unsupervised neural network, and complete hyperspectral image restoration based on the average spectral image and the final spectral difference image.

[0015] Preferably, S200 includes:

[0016] S210: Construct an imaging forward observation model and model the three-dimensional hyperspectral image tensor as an image composed of an average spectral image, a spectral difference image, and a noise term. Specifically:

[0017] ;

[0018] in, is a noisy hyperspectral image, is the average spectral image, is the spectral difference image, is the random noise term, and represent the height, width and number of bands of the acquired hyperspectral image respectively;

[0019] S220: Performing averaging processing on each pixel of the three-dimensional hyperspectral image in the spectral dimension to obtain an average spectral image, specifically:

[0020] ;

[0021] in, Respectively represent the height and width coordinate positions in space, Indicates the bands;

[0022] S230: Constructing a spectral difference image based on the three-dimensional hyperspectral image tensor and the average spectral image, specifically:

[0023] .

[0024] Preferably, S300 includes:

[0025] S310: 3D spectral difference image Rearrange into a two-dimensional spectral difference matrix , where each row corresponds to the spectral curve of a pixel point, and each column corresponds to a spectral band;

[0026] S320: Two-dimensional spectral difference matrix Approximately a low-rank matrix product form:

[0027] ;

[0028] in , is the set rank, indicating that the spectral difference has redundancy compressibility;

[0029] S330: Based on low-rank matrix modeling, the matrix Perform centralization processing, that is, subtract the mean of each column spectral dimension to obtain a zero-mean matrix, and perform eigenvalue decomposition of the centralized matrix using the PCA method to obtain the matrix and ,in is the principal component matrix of the image part, is the principal component matrix of the spectral part, before selecting principal components, ensuring that the rank of the decomposed matrix is .

[0030] Preferably, S400 includes:

[0031] S410: Decompose PCA Initialize tensors as input to the 2D skip connection network and the 1D skip connection network respectively;

[0032] S420: Using a two-dimensional skip connection network and a one-dimensional skip connection network to respectively Optimize and get and , and through matrix product, the optimized spectral difference map is obtained, specifically:

[0033] .

[0034] Preferably, the two-dimensional skip connection network adopts a symmetrical 5-level encoder-decoder architecture designed to optimize the spatial principal component matrix of the hyperspectral image;

[0035] The network input is the initialization space matrix of PCA decomposition, the number of channels is r, each channel corresponds to a principal component feature map, and the output size is r×H×W;

[0036] The encoder stage compresses features by downsampling at five levels: Encoder 1 performs a 3×3 convolution with a stride of 2, a 3×3 convolution with a stride of 1, and an activation layer on the input, reducing the output size to 16 × H / 2 × W / 2; Encoder 2 performs the same sequence of operations, with an output size of 32 × H / 4 × W / 4; Encoder 3 performs the same sequence of operations, with an output size of 64 × H / 8 × W / 8; Encoder 4 performs the same sequence of operations, with an output size of 128 × H / 16 × W / 16; Encoder 5 performs the same sequence of operations, and finally outputs deep features with an output size of 128 × H / 32 × W / 32.

[0037] The decoder stage gradually restores the spatial resolution of the shallow details of the corresponding encoder stage through upsampling jump connections: decoder 4 performs upsampling, 3×3 convolution with a stride of 1, 3×3 convolution with a stride of 1, and activation layer operations on the input in sequence, with an output size of 128×H / 16×W / 16; decoder 3 performs the same sequence of operations, with an output size of 64×H / 8×W / 8; decoder 2 performs the same sequence of operations, with an output size of 32×H / 4×W / 4; decoder 1 performs the same sequence of operations, with an output size of 16×H / 2×W / 2; and finally outputs the optimized spatial feature map, with the size restored to r×H×W.

[0038] Preferably, the one-dimensional skip connection network adopts a symmetrical 5-level encoder-decoder architecture designed to optimize the spectral principal component matrix of the hyperspectral image;

[0039] The network input is the initialized spectral matrix of PCA decomposition, with r channels and each row corresponding to a principal component vector of size r × B;

[0040] The encoder stage compresses the spectrum length by downsampling at five levels: Encoder 1 performs a 3×1 convolution with a stride of 2 and an activation layer on the input, reducing the output size to 16 × B / 2; Encoder 2 performs the same sequence of operations, reducing the output size to 32 × B / 4; Encoder 3 performs the same sequence of operations, reducing the output size to 64 × B / 8; Encoder 4 performs the same sequence of operations, reducing the output size to 128 × B / 16; Encoder 5 performs the same sequence of operations, reducing the output size to 128 × B / 32.

[0041] The decoder stage combines upsampling and shallow details injected by skip connections to gradually restore the spectral resolution: decoder 4 performs upsampling, 3×1 convolution with a stride of 1, and activation layer operations on the input in sequence, with an output size of 128×B / 16; decoder 3 performs the same sequence of operations, with an output size of 64×B / 8; decoder 2 performs the same sequence of operations, with an output size of 32×B / 4; decoder 1 performs the same sequence of operations, with an output size of 16×B / 2; and finally outputs the optimized spectral feature map, with the size restored to r×B.

[0042] Preferably, the loss function constructed in S500 is specifically:

[0043] ;

[0044] ;

[0045] ;

[0046] in, is the reconstruction error loss function, is the regularization term.

[0047] Preferably, S500 includes:

[0048] Based on the loss function and the optimized spectral difference graph, the Adam optimizer is used to update the parameters and iteratively train the neural network. The early stopping strategy is used during the training process. Once the verification error no longer decreases, the training is stopped to avoid overfitting and shorten the training time. After the training is completed, the final spectral difference graph is obtained. , based on the average spectral image and the final spectral difference image, the hyperspectral image restoration is completed:

[0049] ;

[0050] in is the final training round.

[0051] The hyperspectral rapid imaging system based on spectral difference deep matrix decomposition includes:

[0052] The hyperspectral image acquisition module is used to acquire low signal-to-noise ratio hyperspectral data frames under extreme imaging conditions using a fast image acquisition strategy through the hyperspectral imaging system. After image stitching, the data is combined into a complete three-dimensional hyperspectral image tensor in the time and space dimensions.

[0053] The imaging forward observation model construction module models the 3D hyperspectral image tensor as consisting of an average spectral image, a spectral difference image, and a noise term. Each pixel of the 3D spectral image is averaged in the spectral dimension to obtain an average spectral image. A spectral difference image is constructed based on the 3D hyperspectral image tensor and the average spectral image.

[0054] The matrix decomposition module is used to rearrange the three-dimensional spectral difference image into a two-dimensional matrix and approximate it to the product form of a low-rank matrix. The PCA method is used to complete the matrix decomposition to obtain the principal component matrix of the image part and the principal component matrix of the spectral part;

[0055] The unsupervised neural network construction module uses a two-dimensional skip connection network to optimize the principal component matrix of the image part to model the spatial structure information, and a one-dimensional skip connection network to optimize the principal component matrix of the spectrum part to fit the changes in the spectral principal components. The optimized principal component matrices of the image part and the optimized principal component matrices of the spectrum part are obtained respectively. The optimized spectral difference map is obtained through matrix product.

[0056] The hyperspectral image restoration module is used to construct a loss function and train an unsupervised neural network based on the loss function and the optimized spectral difference map. When the preset training end conditions are reached, the trained unsupervised neural network is obtained, and the final spectral difference map is obtained based on the unsupervised neural network. The hyperspectral image restoration is completed based on the average spectral image and the final spectral difference image.

[0057] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a hyperspectral rapid imaging method based on spectral difference depth matrix decomposition are implemented.

[0058] The aforementioned hyperspectral rapid imaging method and system based on spectral difference deep matrix decomposition is suitable for hyperspectral image restoration tasks under extreme imaging conditions such as low light and high throughput. Its core technologies include hyperspectral image acquisition, average spectral map construction, spectral difference map modeling, low-rank matrix decomposition initialization, unsupervised neural network optimization, and image reconstruction. This method achieves high-fidelity image reconstruction under single-frame, unsupervised conditions, demonstrating greater adaptability and imaging efficiency than existing technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a flow chart of a hyperspectral rapid imaging method based on spectral difference depth matrix decomposition in one embodiment of the present invention;

[0060] Figure 2 The figure is a schematic diagram of the principle of a hyperspectral rapid imaging method based on spectral difference depth matrix decomposition in one embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below with reference to the accompanying drawings.

[0062] In one embodiment, Figure 1 and Figure 2 As shown, a hyperspectral rapid imaging method based on spectral difference depth matrix decomposition comprises the following steps:

[0063] S100: A hyperspectral imaging system uses a fast image acquisition strategy to acquire low signal-to-noise ratio hyperspectral data frames under extreme imaging conditions. After image stitching, the data are combined into a complete three-dimensional hyperspectral image tensor in both time and space dimensions.

[0064] Specifically, in hyperspectral imaging systems, the image acquisition speed is often limited by the exposure time, especially when faced with dynamic scenes or large field of view continuous scanning tasks. In order to meet the needs of fast imaging, high temporal resolution and high-throughput data acquisition in these application scenarios, the present invention introduces a low-exposure, high-frame-rate image acquisition strategy in the imaging stage. By setting the exposure time of each frame to an extremely low value (such as 10 milliseconds or lower), the frame rate is significantly improved (up to tens to hundreds of frames per second), thereby acquiring more hyperspectral data frames per unit time. After image stitching, a complete three-dimensional hyperspectral image tensor is combined in the time dimension and the spatial dimension. in and Respectively represent the height, width and number of bands of the acquired hyperspectral image.

[0065] S200: Constructing an imaging forward observation model, modeling the three-dimensional hyperspectral image tensor as consisting of an average spectral image, a spectral difference image, and a noise term; performing averaging processing on each pixel of the three-dimensional spectral image in the spectral dimension to obtain an average spectral image, and constructing a spectral difference image based on the three-dimensional hyperspectral image tensor and the average spectral image;

[0066] Specifically, after completing rapid image acquisition, in order to effectively reconstruct the hyperspectral image, it is necessary to first systematically model the image degradation mechanisms under low exposure conditions. This step, by constructing an imaging forward observation model, explicitly characterizes the various types of degradation experienced by the original high-quality image during low signal-to-noise ratio acquisition, providing a theoretical foundation for subsequent spectral structure recovery and optimization.

[0067] Specifically, the S200 includes:

[0068] S210: Construct an imaging forward observation model and model the three-dimensional hyperspectral image tensor as an image composed of an average spectral image, a spectral difference image, and a noise term. Specifically:

[0069] ;

[0070] in, is a noisy hyperspectral image, is the average spectral image, is the spectral difference image, is a random noise term, and Respectively represent the height, width and number of bands of the acquired hyperspectral image.

[0071] Furthermore, The Mean Spectral Image (MSI) is the image obtained by averaging all spectral bands. This image reflects the overall spatial structure and core information of the scene, effectively filtering out high-frequency components that vary in spectral detail, thereby enhancing image stability and noise resistance. The mean image can be understood as the result of spectral projection of hyperspectral data and serves as the basic reference surface for subsequent differential calculations and modeling.

[0072] The spectral difference image is defined as the residual between the original image and the average spectrum. This difference image reflects the degree of deviation of each spectral channel from the main structure, that is, the fine-grained change of each pixel in the spectral dimension. Due to the significant spectral redundancy in hyperspectral images, the main details of the original image are often hidden in this difference information. Therefore, It can be regarded as an important structural component that carries details and reveals material differences, and restoring this part of information is the core goal of the recovery mechanism of the present invention.

[0073] represents the random noise term due to low exposure conditions, sensor noise, etc. Therefore, the present invention aims to solve the problem of how to use the low signal-to-noise ratio data when only low signal-to-noise ratio data is available. , restore high-quality hyperspectral images .

[0074] S220: To extract the basic structural information of the scene in the image and reduce random fluctuations caused by sensor thermal noise and low exposure, each pixel of the three-dimensional hyperspectral image is averaged in the spectral dimension to obtain an average spectral image. Specifically,

[0075] ;

[0076] in, Respectively represent the height and width coordinate positions in space, Indicates the bands;

[0077] This operation extracts the main structural contours of the image, which can effectively reduce noise interference and serve as a benchmark for subsequent modeling.

[0078] S230: Construct a spectral difference image based on the three-dimensional hyperspectral image tensor and the average spectral image to reveal the deviation of each spectral channel from the average value, specifically:

[0079] .

[0080] Difference map It reflects the details of spectral changes and is an important part of high-quality images.

[0081] S300: rearrange the three-dimensional spectral difference image into a two-dimensional matrix and approximate it to the product form of a low-rank matrix. Use the PCA method to complete matrix decomposition to obtain the principal component matrix of the image part and the principal component matrix of the spectral part.

[0082] In one embodiment, S300 includes:

[0083] S310: In order to facilitate the use of linear algebra tools such as matrix decomposition to process the difference image, the three-dimensional spectral difference image Rearrange into a two-dimensional spectral difference matrix , where each row corresponds to the spectral curve of a pixel point, and each column corresponds to a spectral band; this conversion form is conducive to unified representation and operation in subsequent low-rank modeling and deep network optimization.

[0084] S320: Hyperspectral data has a significant low-rank property in nature, that is, the spectral curves of most pixels in the image are highly correlated. Therefore, its spectral difference matrix also has a low-rank property. Approximately a low-rank matrix product form:

[0085] ;

[0086] in , is the set rank, indicating that the spectral difference has redundancy compressibility;

[0087] S330: Based on the low-rank matrix modeling, the singular value decomposition (SVD) method is used to further Decompose to extract its main features and reduce dimensionality. Perform centralization, that is, subtract the mean of each column spectral dimension to obtain a zero-mean matrix, and perform eigenvalue decomposition (or singular value decomposition) on the centralized matrix using the PCA method to obtain the matrix and ,in is the principal component matrix of the image part, is the principal component matrix of the spectral part, before selecting principal components, ensuring that the rank of the decomposed matrix is .

[0088] S400: Construct an unsupervised neural network and use a two-dimensional skip connection network to optimize the principal component matrix of the image part to model the spatial structure information. Use a one-dimensional skip connection network to optimize the principal component matrix of the spectrum part to fit the changes in the principal components of the spectrum. The optimized principal component matrix of the image part and the optimized principal component matrix of the spectrum part are obtained respectively. Through matrix product, the optimized spectral difference map is obtained.

[0089] In one embodiment, S400 includes:

[0090] S410: Decompose PCA Initialize tensors as input to the 2D skip connection network and the 1D skip connection network respectively;

[0091] S420: Using a two-dimensional skip connection network and a one-dimensional skip connection network to respectively Optimize and get and , and through matrix product, the optimized spectral difference map is obtained, specifically:

[0092] .

[0093] In one embodiment, a two-dimensional skip connection network adopts a symmetric 5-level encoder-decoder architecture designed to optimize the spatial principal component matrix of hyperspectral images;

[0094] The network input is the initialization space matrix of PCA decomposition, the number of channels is r, each channel corresponds to a principal component feature map, and the output size is r×H×W;

[0095] The encoder stage compresses features by downsampling at five levels: Encoder 1 performs a 3×3 convolution with a stride of 2, a 3×3 convolution with a stride of 1, and an activation layer on the input, reducing the output size to 16 × H / 2 × W / 2; Encoder 2 performs the same sequence of operations, with an output size of 32 × H / 4 × W / 4; Encoder 3 performs the same sequence of operations, with an output size of 64 × H / 8 × W / 8; Encoder 4 performs the same sequence of operations, with an output size of 128 × H / 16 × W / 16; Encoder 5 performs the same sequence of operations, and finally outputs deep features with an output size of 128 × H / 32 × W / 32.

[0096] The decoder stage gradually restores the spatial resolution of the shallow details of the corresponding encoder stage through upsampling jump connections: decoder 4 performs upsampling, 3×3 convolution with a stride of 1, 3×3 convolution with a stride of 1, and activation layer operations on the input in sequence, with an output size of 128×H / 16×W / 16; decoder 3 performs the same sequence of operations, with an output size of 64×H / 8×W / 8; decoder 2 performs the same sequence of operations, with an output size of 32×H / 4×W / 4; decoder 1 performs the same sequence of operations, with an output size of 16×H / 2×W / 2; and finally outputs the optimized spatial feature map, with the size restored to r×H×W.

[0097] Specifically, the two-dimensional skip connection network structure is shown in Table 1.

[0098] Table 1

[0099]

[0100] In one embodiment, a one-dimensional skip connection network adopts a symmetric 5-level encoder-decoder architecture designed to optimize the spectral principal component matrix of hyperspectral images;

[0101] The network input is the initialized spectral matrix of PCA decomposition, with r channels and each row corresponding to a principal component vector of size r × B;

[0102] The encoder stage compresses the spectrum length by downsampling at five levels: Encoder 1 performs a 3×1 convolution with a stride of 2 and an activation layer on the input, reducing the output size to 16 × B / 2; Encoder 2 performs the same sequence of operations, reducing the output size to 32 × B / 4; Encoder 3 performs the same sequence of operations, reducing the output size to 64 × B / 8; Encoder 4 performs the same sequence of operations, reducing the output size to 128 × B / 16; Encoder 5 performs the same sequence of operations, reducing the output size to 128 × B / 32.

[0103] The decoder stage combines upsampling and shallow details injected by skip connections to gradually restore the spectral resolution: decoder 4 performs upsampling, 3×1 convolution with a stride of 1, and activation layer operations on the input in sequence, with an output size of 128×B / 16; decoder 3 performs the same sequence of operations, with an output size of 64×B / 8; decoder 2 performs the same sequence of operations, with an output size of 32×B / 4; decoder 1 performs the same sequence of operations, with an output size of 16×B / 2; and finally outputs the optimized spectral feature map, with the size restored to r×B.

[0104] Specifically, the two-dimensional skip connection network structure is shown in Table 2.

[0105] Table 2

[0106]

[0107] S500: Construct a loss function, train an unsupervised neural network based on the loss function and the optimized spectral difference image, obtain the trained unsupervised neural network when the preset training end condition is reached, obtain the final spectral difference image based on the unsupervised neural network, and complete hyperspectral image restoration based on the average spectral image and the final spectral difference image.

[0108] In one embodiment, to achieve unsupervised hyperspectral image restoration, a loss function that does not require high-quality hyperspectral image supervision is defined. The loss function constructed in S500 is specifically:

[0109] ;

[0110] ;

[0111] ;

[0112] in, is the reconstruction error loss function, is the regularization term. The reconstruction error is used to measure the difference between the restored spectral image and the original measured image. The regularization term is used to prevent overfitting and improve the continuity of the optimized spectral difference image.

[0113] In one embodiment, S500 includes:

[0114] Based on the loss function and the optimized spectral difference graph, the Adam optimizer is used to update the parameters and iteratively train the neural network. The early stopping strategy is used during the training process. Once the verification error no longer decreases, the training is stopped to avoid overfitting and shorten the training time. After the training is completed, the final spectral difference graph is obtained. , based on the average spectral image and the final spectral difference image, the hyperspectral image restoration is completed:

[0115] ;

[0116] in is the final training round.

[0117] Compared with existing hyperspectral image restoration methods, the deep matrix decomposition restoration method based on spectral difference modeling proposed in this paper has the following significant advantages and beneficial effects:

[0118] 1) Use spectral difference to perform residual modeling to reduce reconstruction complexity

[0119] Traditional hyperspectral image restoration methods mostly perform modeling and optimization directly on raw 3D data, resulting in high processing overhead and slow convergence. However, this method transforms the complex 3D restoration problem into a low-rank matrix decomposition problem by extracting a spectral difference map, significantly reducing modeling and computational complexity. The mean spectral image (MSI) naturally has a high signal-to-noise ratio, effectively offloading the structural restoration task and thus making differential modeling more stable.

[0120] 2) Introducing a spatial-spectral joint jump network to improve recovery accuracy

[0121] This method employs a dual-branch neural network architecture: a 2D spatial skip connection network and a 1D spectral skip connection network to model spatial and spectral features, respectively. The network is then fused using low-rank constraints, effectively avoiding the spectral aliasing and detail blurring problems that plague existing methods when processing hyperspectral data. The skip structure preserves multi-scale information and exhibits strong resilience even in high-noise and weak-texture conditions.

[0122] 3) Strong adaptability, with end-to-end and plug-and-play features

[0123] This method, which relies entirely on an unsupervised deep learning framework and does not rely on extensive prior knowledge or external labels, can adapt to the noise characteristics of hyperspectral images from different sensors and scenarios, demonstrating excellent versatility and scalability. Its simple forward propagation structure supports deployment in embedded devices or online recovery systems, offering practical "plug-and-play" applications.

[0124] In one embodiment, a hyperspectral rapid imaging system based on spectral difference depth matrix decomposition is further provided, comprising:

[0125] The hyperspectral image acquisition module is used to acquire low signal-to-noise ratio hyperspectral data frames under extreme imaging conditions using a fast image acquisition strategy through the hyperspectral imaging system. After image stitching, the data is combined into a complete three-dimensional hyperspectral image tensor in the time and space dimensions.

[0126] The imaging forward observation model construction module models the 3D hyperspectral image tensor as consisting of an average spectral image, a spectral difference image, and a noise term. Each pixel of the 3D spectral image is averaged in the spectral dimension to obtain an average spectral image. A spectral difference image is constructed based on the 3D hyperspectral image tensor and the average spectral image.

[0127] The matrix decomposition module is used to rearrange the three-dimensional spectral difference image into a two-dimensional matrix and approximate it to the product form of a low-rank matrix. The PCA method is used to complete the matrix decomposition to obtain the principal component matrix of the image part and the principal component matrix of the spectral part;

[0128] The unsupervised neural network construction module uses a two-dimensional skip connection network to optimize the principal component matrix of the image part to model the spatial structure information, and a one-dimensional skip connection network to optimize the principal component matrix of the spectrum part to fit the changes in the spectral principal components. The optimized principal component matrices of the image part and the optimized principal component matrices of the spectrum part are obtained respectively. The optimized spectral difference map is obtained through matrix product.

[0129] The hyperspectral image restoration module is used to construct a loss function and train an unsupervised neural network based on the loss function and the optimized spectral difference map. When the preset training end conditions are reached, the trained unsupervised neural network is obtained, and the final spectral difference map is obtained based on the unsupervised neural network. The hyperspectral image restoration is completed based on the average spectral image and the final spectral difference image.

[0130] Regarding the specific definition of a hyperspectral rapid imaging system based on spectral difference depth matrix decomposition, please refer to the definition of a hyperspectral rapid imaging method based on spectral difference depth matrix decomposition above, which will not be repeated here. Each module in the above-mentioned hyperspectral rapid imaging system based on spectral difference depth matrix decomposition can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0131] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a hyperspectral rapid imaging method based on spectral difference depth matrix decomposition are implemented.

[0132] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0133] The above is a detailed introduction to a hyperspectral rapid imaging method and system based on spectral differential depth matrix decomposition provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core idea of ​​the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in several ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A hyperspectral rapid imaging method based on spectral difference deep matrix decomposition, characterized in that: The method comprises the following steps: S100: A hyperspectral imaging system uses a fast image acquisition strategy to acquire low signal-to-noise ratio hyperspectral data frames under extreme imaging conditions. After image stitching, these frames are combined into a complete three-dimensional hyperspectral image tensor in both time and space dimensions. S200: Constructing an imaging forward observation model, modeling the three-dimensional hyperspectral image tensor as consisting of an average spectral image, a spectral difference image, and a noise term; performing averaging processing on each pixel of the three-dimensional spectral image in the spectral dimension to obtain an average spectral image, and constructing a spectral difference image based on the three-dimensional hyperspectral image tensor and the average spectral image; S300: rearranging the three-dimensional spectral difference image into a two-dimensional matrix and approximating it into a product form of a low-rank matrix, and performing matrix decomposition using the PCA method to obtain a principal component matrix of the image part and a principal component matrix of the spectral part; S400: Construct an unsupervised neural network and use a two-dimensional skip connection network to optimize the principal component matrix of the image part to model spatial structure information. Use a one-dimensional skip connection network to optimize the principal component matrix of the spectrum part to fit the changes in the principal components of the spectrum. The optimized principal component matrices of the image part and the optimized principal component matrices of the spectrum part are obtained respectively. The optimized spectral difference map is obtained by matrix product. S500: Construct a loss function, train an unsupervised neural network based on the loss function and the optimized spectral difference image, obtain the trained unsupervised neural network when the preset training end condition is reached, obtain the final spectral difference image based on the unsupervised neural network, and complete hyperspectral image restoration based on the average spectral image and the final spectral difference image.

2. The method according to claim 1, characterized in that S200 includes: S210: Construct an imaging forward observation model and model the three-dimensional hyperspectral image tensor as an image composed of an average spectral image, a spectral difference image, and a noise term. Specifically: ； in, is a noisy hyperspectral image, is the average spectral image, is the spectral difference image, is a random noise term, and represent the height, width and number of bands of the acquired hyperspectral image respectively; S220: Performing averaging processing on each pixel of the three-dimensional hyperspectral image in the spectral dimension to obtain an average spectral image, specifically: ； in, Respectively represent the height and width coordinate positions in space, Indicates the bands; S230: Constructing a spectral difference image based on the three-dimensional hyperspectral image tensor and the average spectral image, specifically: 。 3. The method according to claim 2, characterized in that S300 includes: S310: 3D spectral difference image Rearrange into a two-dimensional spectral difference matrix , where each row corresponds to the spectral curve of a pixel point, and each column corresponds to a spectral band; S320: Two-dimensional spectral difference matrix Approximately a low-rank matrix product form: ； in , is the set rank, indicating that the spectral difference has redundancy compressibility; S330: Based on low-rank matrix modeling, the matrix Perform centralization processing, that is, subtract the mean of each column spectral dimension to obtain a zero-mean matrix, and perform eigenvalue decomposition of the centralized matrix using the PCA method to obtain the matrix and ,in is the principal component matrix of the image part, is the principal component matrix of the spectral part, before selecting principal components, ensuring that the rank of the decomposed matrix is .

4. The method according to claim 3, characterized in that S400 includes: S410: Decompose PCA Initialize tensors as input to the 2D skip connection network and the 1D skip connection network respectively; S420: Using a two-dimensional skip connection network and a one-dimensional skip connection network to respectively Optimize and get and , and through matrix product, the optimized spectral difference map is obtained, specifically: 。 5. The method according to claim 4, characterized in that A 2D skip connection network with a symmetric 5-level encoder-decoder architecture is designed to optimize the spatial principal component matrix of hyperspectral images. The network input is the initialization space matrix of PCA decomposition, the number of channels is r, each channel corresponds to a principal component feature map, and the output size is r×H×W; The encoder stage compresses features through 5 levels of downsampling: Encoder 1 performs a 3×3 convolution with a stride of 2, a 3×3 convolution with a stride of 1, and an activation layer operation on the input, reducing the output size to 16 × H / 2 × W / 2; Encoder 2 performs the same sequence of operations, and the output size is 32 × H / 4 × W / 4; Encoder 3 performs the same sequence of operations, and the output size is 64 × H / 8 × W / 8; Encoder 4 performs the same sequence of operations, and the output size is 128 × H / 16 × W / 16; Encoder 5 performs the same sequence of operations and finally outputs deep features with an output size of 128 × H / 32 × W / 32. The decoder stage gradually restores the spatial resolution of the shallow details of the corresponding encoder stage through upsampling jump connections: decoder 4 performs upsampling, 3×3 convolution with a stride of 1, 3×3 convolution with a stride of 1, and activation layer operations on the input in sequence, with an output size of 128×H / 16×W / 16; decoder 3 performs the same operation sequence, with an output size of 64×H / 8×W / 8; decoder 2 performs the same operation sequence, with an output size of 32×H / 4×W / 4; decoder 1 performs the same operation sequence, with an output size of 16×H / 2×W / 2; finally, the optimized spatial feature map is output, and the size is restored to r×H×W.

6. The method according to claim 5, characterized in that The one-dimensional skip connection network adopts a symmetric 5-level encoder-decoder architecture and is designed to optimize the spectral principal component matrix of hyperspectral images; The network input is the initialized spectral matrix of PCA decomposition, with r channels and each row corresponding to a principal component vector of size r × B; The encoder stage compresses the spectrum length by downsampling at five levels: Encoder 1 performs a 3×1 convolution with a stride of 2 and an activation layer on the input, reducing the output size to 16 × B / 2; Encoder 2 performs the same sequence of operations, reducing the output size to 32 × B / 4; Encoder 3 performs the same sequence of operations, reducing the output size to 64 × B / 8; Encoder 4 performs the same sequence of operations, reducing the output size to 128 × B / 16; Encoder 5 performs the same sequence of operations, reducing the output size to 128 × B / 32. The decoder stage combines upsampling and skip connections to gradually restore spectral resolution by injecting shallow details: Decoder 4 performs upsampling, 3×1 convolution with a stride of 1, and activation layer operations on the input, with an output size of 128×B / 16; Decoder 3 performs the same sequence of operations, with an output size of 64×B / 8; Decoder 2 performs the same sequence of operations, with an output size of 32×B / 4; Decoder 1 performs the same sequence of operations, with an output size of 16 × B / 2; it finally outputs the optimized spectral feature map, with the size restored to r × B.

7. The method according to claim 6, characterized in that The loss function constructed in S500 is specifically: ； ； ； in, is the reconstruction error loss function, is the regularization term.

8. The method according to claim 7, characterized in that S500 includes: Based on the loss function and the optimized spectral difference graph, the Adam optimizer is used to update the parameters and iteratively train the neural network. The early stopping strategy is used during the training process. Once the verification error no longer decreases, the training is stopped to avoid overfitting and shorten the training time. After the training is completed, the final spectral difference graph is obtained. , based on the average spectral image and the final spectral difference image, the hyperspectral image restoration is completed: ； in is the final training round.

9. A hyperspectral rapid imaging system based on spectral difference deep matrix decomposition, characterized in that: include: The hyperspectral image acquisition module is used to acquire low signal-to-noise ratio hyperspectral data frames under extreme imaging conditions using a fast image acquisition strategy through the hyperspectral imaging system. After image stitching, the data is combined into a complete three-dimensional hyperspectral image tensor in the time and space dimensions. The imaging forward observation model building module models the three-dimensional hyperspectral image tensor as consisting of an average spectral image, a spectral difference image, and a noise term; Each pixel of the three-dimensional spectral image is averaged in the spectral dimension to obtain an average spectral image, and a spectral difference image is constructed based on the three-dimensional hyperspectral image tensor and the average spectral image; The matrix decomposition module is used to rearrange the three-dimensional spectral difference image into a two-dimensional matrix and approximate it to the product form of a low-rank matrix. The PCA method is used to complete the matrix decomposition to obtain the principal component matrix of the image part and the principal component matrix of the spectral part; The unsupervised neural network construction module uses a two-dimensional skip connection network to optimize the principal component matrix of the image part to model the spatial structure information, and a one-dimensional skip connection network to optimize the principal component matrix of the spectrum part to fit the changes in the spectral principal components. The optimized principal component matrices of the image part and the optimized principal component matrices of the spectrum part are obtained respectively. The optimized spectral difference map is obtained through matrix product. The hyperspectral image restoration module is used to construct a loss function and train an unsupervised neural network based on the loss function and the optimized spectral difference map. When the preset training end conditions are reached, the trained unsupervised neural network is obtained, and the final spectral difference map is obtained based on the unsupervised neural network. The hyperspectral image restoration is completed based on the average spectral image and the final spectral difference image.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images