A hyperspectral compression sensing oriented image reconstruction method, system, device and medium

By integrating high- and low-frequency features through a spatial convolution modulation module and a high- and low-frequency domain feature integration module, the problems of spatial distortion and neglect of frequency domain features in existing methods are solved, and efficient hyperspectral image reconstruction is achieved.

CN119600145BActive Publication Date: 2025-10-21NAT UNIV OF DEFENSE TECH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202411774049.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-05
Publication Date
2025-10-21
Estimated Expiration
2044-12-05

AI Technical Summary

Technical Problem

Existing hyperspectral image reconstruction methods fail to effectively combine spatial and channel features and ignore image frequency domain features, resulting in low reconstruction quality and slow speed. Furthermore, existing methods suffer from spatial distortion in spectral image reconstruction.

Method used

A spatial convolution modulation module and a high- and low-frequency domain feature integration module are adopted, combined with frequency branching and residual enhancement branching, to integrate high- and low-frequency features through a trained reconstruction model, thereby enhancing the model's learning ability and reconstruction effect.

Benefits of technology

It improves the quality and speed of hyperspectral image reconstruction, alleviates spatial distortion problems, and enhances the model's learning ability and reconstruction effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119600145B_ABST
    Figure CN119600145B_ABST
Patent Text Reader

Abstract

The application discloses a hyperspectral compression sensing-oriented image reconstruction method, system, device and medium, and relates to the technical field of hyperspectral image reconstruction. The method comprises the following steps: acquiring a target hyperspectral image; pre-processing the target hyperspectral image to obtain measurement data for input of a matching model; inputting the measurement data into a trained reconstruction model for integration to obtain a reconstructed hyperspectral image; the reconstruction model comprises a spatial convolution modulation module, a high-low frequency domain feature integration module and a set optimization objective function; and the reconstruction model is determined by training according to a set iteration round, an initial learning rate and a loss function. The application can combine high-frequency features and low-frequency features by using a frequency branch and a residual enhancement branch, and enhance the model learning ability and the reconstruction effect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image reconstruction, and in particular to an image reconstruction method, system, device and medium for hyperspectral compressed sensing. Background Art

[0002] Each channel in a hyperspectral image captures information about a specific spectral wavelength in a real scene. More spectral bands can store richer information and scene features. Compressed sensing systems compress snapshot information into two-dimensional measurements and then use reconstruction algorithms to recover three-dimensional hyperspectral data from these two-dimensional measurements. Hyperspectral data compression and reconstruction are widely used in fields such as remote sensing, target tracking, and medicine, and hold significant scientific value and research potential.

[0003] In order to enhance the spectral image reconstruction effect, traditional model-based methods use manual priors such as sparsity and non-local similarity to regulate the reconstruction process. These methods require manual adjustment of parameters, resulting in poor generalization ability, low reconstruction quality and slow speed. Convolutional neural learning models have been used to capture the characteristics of hyperspectral images, greatly improving the reconstruction effect. However, existing methods do not combine spatial and channel features well. Due to masks, shifts and compression factors, as well as the inherent over-smoothing and information degradation in the network training process, some spatial distortions will occur in certain frequency bands. The methods fail to effectively focus on the problem of spatial information degradation in these spectral modeling processes. In addition, in spectral image reconstruction, both low-frequency and high-frequency information are crucial. Existing methods often ignore the learning of image frequency domain feature information. Summary of the Invention

[0004] The purpose of the present invention is to provide an image reconstruction method, system, device and medium for hyperspectral compressed sensing, which can combine high-frequency features with low-frequency features by using frequency branches and residual enhancement branches to enhance model learning ability and reconstruction effect.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] An image reconstruction method for hyperspectral compressed sensing, comprising:

[0007] Acquire target hyperspectral images;

[0008] Preprocessing the target hyperspectral image to obtain measurement data matching the model input;

[0009] The measurement data is input into a trained reconstruction model for integration to obtain a reconstructed hyperspectral image; the reconstruction model includes a spatial convolution modulation module, a high- and low-frequency domain feature integration module, and a set optimization objective function; the reconstruction model is trained and determined based on set iteration rounds, initialization learning rate, and loss function.

[0010] Optionally, preprocessing the target hyperspectral image to obtain measurement data for matching model input specifically includes:

[0011] Based on the target hyperspectral image, a randomly initialized mask M is used. The data of each spectral segment is encoded and mapped by the dot product of the M matrix. An offset operation is performed along the spectral channel. The offset step size is set to 2. The offset results are superimposed and reshaped to obtain the measurement data that matches the model input.

[0012] Optionally, the spatial convolution modulation module specifically includes:

[0013] Set the first branch P to a 1×1 convolutional layer plus a Sigmod activation function for feature extraction;

[0014] The second branch U is set to a 1×1 convolution layer plus a 3×3 depthwise convolution, and two ReLU activation functions are combined to generate the modulation signal;

[0015] Finally, the output matrices of the two branches are multiplied to obtain the modulated output feature X mod , X mod =(σ u2 (σ u1 (XW u1 )W u2 )⊙σ p (XW p ))W mod , σ(·) is the activation function of each branch, W u and W p is the corresponding branch weight matrix.

[0016] Optionally, the high- and low-frequency domain feature integration module specifically includes:

[0017] Use the frequency domain branch to map the features to the frequency domain through Fourier transform, concatenate the obtained imaginary part and real part, and then pass through two DoConv layers and activation layers to obtain the frequency domain output X1;

[0018] Use the residual branch to pass the features through the DoConv layer and the activation layer to obtain the residual output X2;

[0019] The module input features are further refined using two branch flows to obtain the output X f .

[0020] Optionally, the set optimization objective function is specifically:

[0021]

[0022] Among them, {f n} is the training set data, d nis the model output data, W Θ is the reconstruction model with training parameters Θ, M Φ is the mask perception matrix, R(Φ) is the physical constraint in optical coding, N is the number of training samples, μ is the physical constraint weight parameter, and L task is the mean square error loss function of the reconstructed image.

[0023] Optionally, the measurement data is input into a trained reconstruction model for integration to obtain a reconstructed hyperspectral image, specifically comprising:

[0024] Step (1): Initially use a 3×3 convolutional layer to map the input X to the feature space X';

[0025] Step (2): After expanding the mask dimension to match the model size, the mask map M' is obtained through two convolution paths of 1×1 and 5×5;

[0026] Step (3): The feature data X' is linearly mapped to calculate the corresponding key K, value V, and query Q information, and the attention is calculated using cross-covariance attention. K i is the key information of each attention head; where A i is the i-th attention score, τ is the learnable weight parameter, i is the index of each attention head, and T is the transposition operation;

[0027] Step (4): Multiply the value V by the mask M' to get V', connect each attention to weight and get the output through linear mapping Among them, W O is the mapping layer weight; where h is the number of attention heads;

[0028] Step (5): Use covariance self-attention to calculate the attention map X of the input feature X' A , and combined with the mask map M', weighted to the input feature output X c ;

[0029] Step (6): For the output X of the covariance attention module c , using spatial convolutional modulation modules to enhance spatial representation X mod ;

[0030] Step (7): For the output of the spatial convolution modulation module, use the residual Fourier module to integrate the high and low frequency domain features to output X f ;

[0031] Step (8): Take steps (3) to (7) as the process of the model basic module, stack the model basic module twice to form a core module, and after the data is passed through the core module to learn features, cache the data features and perform downsampling and dimensionality reduction operations on the data features;

[0032] Step (9): Repeat step (8) twice and pass the data through the feature extraction of the core module again;

[0033] Step (10): Upsample the data outputted in step (9), add the data features cached at the same size after learning the core model features twice in step (8) along the channel, and then perform the feature extraction step of the core module;

[0034] Step (11): Repeat step (10) twice, and finally use the transposed convolution layer to map the data to the real space X R , and added to the model input X to form a residual structure, and finally the reconstructed hyperspectral image X is obtained. out .

[0035] Optionally, the training process of the reconstruction model is:

[0036] Acquire training data; the training data is hyperspectral images of different scenes, and each hyperspectral image is randomly cropped, rotated, flipped and spliced, and the hyperspectral data after data enhancement is stacked, and the data is constructed in batches for output;

[0037] Input the training data into the initial network structure of the reconstruction model and specify the iteration round N epoch The training is performed for 200 times, the initial learning rate is set to 0.004, and the objective function is optimized until the loss converges to obtain the trained reconstruction model.

[0038] The present invention also provides an image reconstruction system for hyperspectral compressed sensing, comprising:

[0039] A data acquisition unit, used for acquiring target hyperspectral images;

[0040] a preprocessing unit, configured to preprocess the target hyperspectral image to obtain measurement data inputted by a matching model;

[0041] An image reconstruction unit is used to input the measurement data into a trained reconstruction model for integration to obtain a reconstructed hyperspectral image; the reconstruction model includes a spatial convolution modulation module, a high- and low-frequency domain feature integration module, and a set optimization objective function; the reconstruction model is trained and determined based on set iteration rounds, an initialization learning rate, and a loss function.

[0042] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the above-mentioned image reconstruction method for hyperspectral compressed sensing.

[0043] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned image reconstruction method for hyperspectral compressed sensing.

[0044] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0045] The present invention discloses an image reconstruction method, system, device, and medium for hyperspectral compressed sensing. The method comprises acquiring a target hyperspectral image; preprocessing the target hyperspectral image to obtain measurement data for matching model input; inputting the measurement data into a trained reconstruction model for integration to obtain a reconstructed hyperspectral image; the reconstruction model comprises a spatial convolution modulation module, a high- and low-frequency domain feature integration module, and a set optimization objective function; the reconstruction model is trained and determined based on set iteration rounds, an initialization learning rate, and a loss function. The present invention uses a spatial convolution modulation module to enhance spatial representation and introduces a residual fast Fourier transform module to integrate high- and low-frequency domain information. That is, frequency branches and residual enhancement branches are used to combine high-frequency features with low-frequency features, thereby enhancing the model's learning ability and reconstruction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 Schematic diagram of the process of the image reconstruction method for hyperspectral compressed sensing of the present invention;

[0048] Figure 2 The diagram is a diagram of the reconstructed network structure in this embodiment; wherein, part (a) is a schematic diagram of the overall network structure; part (b) is a schematic diagram of the core module structure;

[0049] Figure 3 : is a module structure diagram of this embodiment; wherein, part (a) is a schematic diagram of the spatial convolution modulation module structure; part (b) is a schematic diagram of the high and low frequency domain feature integration module structure;

[0050] Figure 4This is a visual comparison diagram of the input and output of the high- and low-frequency feature integration module in this embodiment;

[0051] Figure 5 Comparison chart of the reconstruction results of the test scene of this embodiment with other mainstream algorithms; among them, part (a) is a local detail picture; part (b) is an overall display picture. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] The purpose of the present invention is to provide an image reconstruction method, system, device and medium for hyperspectral compressed sensing, which can combine high-frequency features with low-frequency features by using frequency branches and residual enhancement branches to enhance model learning ability and reconstruction effect.

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] like Figure 1 As shown, the present invention provides an image reconstruction method for hyperspectral compressed sensing, comprising:

[0056] Step 100: Acquire a target hyperspectral image.

[0057] Step 200: Preprocess the target hyperspectral image to obtain measurement data for matching model input.

[0058] Step 300: The measurement data is input into a trained reconstruction model for integration to obtain a reconstructed hyperspectral image; the reconstruction model includes a spatial convolution modulation module, a high- and low-frequency domain feature integration module, and a set optimization objective function; the reconstruction model is trained and determined based on a set number of iterations, an initialization learning rate, and a loss function.

[0059] As a specific implementation method, the above steps are further described in detail.

[0060] Step 1: Obtain the hyperspectral image dataset CAVE as training data and preprocess it to perform image cropping and data augmentation to enhance the generalization ability of the model.

[0061] In step 1-1, for the acquired hyperspectral training data, randomly crop the data to a size of 256 × 256, use 28 channels, and perform rotation and flipping or downsampling and splicing operations.

[0062] In steps 1-2, the hyperspectral data after data enhancement are stacked and the data is constructed and output in batches, with the number of batches set to 5.

[0063] Step 2: Randomly initialize the mask, encode the compressed sensing spectral image, perform correlation mapping and offset, and obtain the measurement data.

[0064] In step 2-1, based on the preprocessed hyperspectral data, a randomly initialized mask M with a size of 256×256 is used. Each spectral segment data is encoded and mapped by dot multiplication with the M matrix, and an offset operation is performed along the spectral channel with an offset step of 2.

[0065] After superimposing the offset results in step 2-2, the Y size of the measured data is obtained to be 256×310.

[0066] Step 3: Reshape the measured data Y to match the input size of the reconstructed model.

[0067] In step 3-1, the measured data is offset to obtain data X, with an offset step size of 2, matching the reconstruction network input size, which is 256×256×28.

[0068] Step 4: Design the spatial convolution modulation module and the high and low frequency domain feature integration module, build the network structure, and set the optimization objective function.

[0069] Step 4-1 designs the spatial convolution modulation module and the high and low frequency domain feature integration module.

[0070] 1) Spatial convolution modulation module, such as Figure 3 As shown in part (a) of .

[0071] 1-1) The first branch P is a 1×1 convolutional layer with Sigmoid activation function for feature extraction;

[0072] 1-2) The second branch U is a 1×1 convolution layer plus a 3×3 depthwise convolution, combined with two ReLU activation functions to generate the modulation signal.

[0073] 1-3) Multiply the output matrices of the two branches to obtain the modulated output feature X mod , X mod =(σ u2 (σ u1 (XW u1 )W u2 )⊙σ p (XW p ))W mod,σ(·) is the activation function of each branch, W u and W p is the corresponding branch weight matrix.

[0074] 2) High and low frequency domain feature integration module, such as Figure 3 As shown in part (b) of .

[0075] 2-1) Use the frequency domain branch to map the features to the frequency domain through Fourier transform, concatenate the obtained imaginary part and real part, and then pass through two DoConv layers and an activation layer to obtain the frequency domain output X1.

[0076] 2-2) Use the residual branch to pass the features through the DoConv layer and the activation layer to obtain the residual output X2.

[0077] 2-3) The module input features are further refined using the above two branch flows to obtain the output X f .

[0078] Step 4-2 builds the network model structure. The network structure diagram is as follows Figure 2 shown.

[0079] 1) Initially, a 3×3 convolutional layer is used to map the input X to the feature space X'.

[0080] 2) After expanding the mask dimension to 256×256×28, the mask map M' is obtained through two convolution paths of 1×1 and 5×5.

[0081] 3) The feature data X' is linearly mapped to calculate the corresponding key K, value V, and query Q information, and the attention is obtained using cross-covariance attention calculation K i is the key information of each attention head.

[0082] 4) Multiply the value V by the mask M' to get V'. Connect each attention to weight and get the output through linear mapping Among them, W O is the mapping layer weight.

[0083] 5) Use covariance self-attention to calculate the attention map X of the input feature X' A , and combined with the mask map M', weighted to the input feature output X c .

[0084] 6) For the output X of the covariance attention module c , using spatial convolutional modulation modules to enhance spatial representation X mod ; The input and output visualization results of the spatial convolution modulation module are shown in Figure 4 Take the spectral band information of 2, 7, 11, and 16 as an example.

[0085] 7) For the output of the spatial convolution modulation module, the residual Fourier integration module is used to integrate the high and low frequency domain features to output X f .

[0086] 8) The above 3) to 7) processes show the basic module of the model. This module is stacked twice to form the core module. After the data is passed through the core module to learn the features, the data features are cached and subsequently downsampled and reduced in dimension.

[0087] 9) Repeat step 8) twice, and pass the data through the feature extraction of the core module again;

[0088] 10) The data outputted in 9) is upsampled and added along the channel with the data features cached at the same size after the first two core model feature learnings in 8), and then the feature extraction step of the core module is performed.

[0089] 11) Repeat step 10) twice, and finally use the transposed convolution layer to map the data to the real space X R , and added to the model input X to form a residual structure. Finally, the reconstructed hyperspectral image X is obtained. out , calculate the target loss.

[0090] Step 4-3 sets the corresponding objective function. The objective function is:

[0091]

[0092] Among them, {f n} is the training set data, d n is the model output data, W Θ is the reconstruction model with training parameters Θ, M φ is the mask perception matrix, R(φ) is the physical constraint in optical coding, N is the number of training samples, μ is the physical constraint weight parameter, and L task Is the mean square error loss function of the reconstructed image. MSE is used to calculate the loss between the reconstructed multispectral image and the original image.

[0093] Step 5: Train the reconstructed network to the specified rounds, until the training loss converges, and save the optimal model.

[0094] Step 5-1: Train the network model to the specified iteration number N epoch The training was performed 200 times with an initial learning rate of 0.004. The objective function was optimized until the loss converged, using Adam as the optimizer. The model was trained on an NVIDIA A800 GPU for approximately 33 hours.

[0095] Step 5-2 saves the optimal model and parameters.

[0096] Step 6: Use the hyperspectral test scene for compressed sensing and obtain the reconstructed hyperspectral image through the reconstruction model.

[0097] In step 6-1, 10 scenes in the KAIST dataset are used as test data to perform compressed sensing mapping to obtain measurement data.

[0098] Step 6-2, use the compressed measurement data to perform spectrum reconstruction through the reconstruction model.

[0099] In step 6-3, the final reconstructed hyperspectral image is obtained.

[0100] Compared with the existing mainstream hyperspectral image compression and reconstruction models: GAP-NET[2], HD-NET[3], MST[4], CST[5], BIRNAT[6], we use 10 scenes in the KAIST dataset as test data, and use peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) as performance indicators. The average results are shown in Table 1.

[0101] Table 1: Quantitative comparison results of the algorithm of the present invention

[0102] Model Params GFLOPs PSNR SSIM GAP-NET 4.27 78.58 33.26 0.917 HD-NET 2.37 154.76 34.97 0.943 MST 2.03 28.15 35.18 0.948 CST 3.0 40.10 36.12 0.957 BIRNAT 4.40 2122.66 37.58 0.960 The present invention 1.71 71.10 39.76 0.975

[0103] Figure 5 The hyperspectral reconstruction results of different existing mainstream algorithms and the method proposed in this patent are shown, including four bands: 462.0nm, 551.5nm, 549.5nm and 636.5nm. Figure 5 Part (a) is Figure 5 The details of the red frame in part (b) are shown.

[0104] Therefore, the present invention obtains compressed sensing measurement data of hyperspectral images based on the aperture coded computational imaging system, uses the covariance self-attention module to model the spectral channel features, uses the spatial convolution modulation module to enhance the spatial representation, alleviates the spatial degradation problem existing in the spatial spectrum domain modeling, introduces the residual fast Fourier transform module to integrate high and low frequency domain information, that is, uses the frequency branch and the residual enhancement branch to combine the high-frequency features with the low-frequency features, enhances the model learning ability and the effect of hyperspectral image reconstruction.

[0105] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0106] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. An image reconstruction method for hyperspectral compressed sensing, characterized in that: include: Acquire target hyperspectral images; Preprocessing the target hyperspectral image to obtain measurement data matching the model input; Inputting the measured data into a trained reconstruction model for integration to obtain a reconstructed hyperspectral image; The reconstruction model includes a spatial convolution modulation module, a high- and low-frequency domain feature integration module, and a set optimization objective function; the reconstruction model is trained and determined according to set iteration rounds, initialization learning rate, and loss function; The high- and low-frequency domain feature integration module specifically includes: Use the frequency domain branch to map the features to the frequency domain through Fourier transform, concatenate the obtained imaginary part and real part, and then pass through two DoConv layers and activation layers to obtain the frequency domain output X1; Use the residual branch to pass the features through the DoConv layer and the activation layer to obtain the residual output X2; The module input features are further refined using the above two branch flows to obtain the output X f .

2. The image reconstruction method for hyperspectral compressed sensing according to claim 1, characterized in that: Preprocessing the target hyperspectral image to obtain measurement data for matching model input specifically includes: Based on the target hyperspectral image, a randomly initialized mask M is used. The data of each spectral segment is encoded and mapped by the dot product of the M matrix. An offset operation is performed along the spectral channel. The offset step size is set to 2. The offset results are superimposed and reshaped to obtain the measurement data that matches the model input.

3. The image reconstruction method for hyperspectral compressed sensing according to claim 1, characterized in that: The spatial convolution modulation module specifically includes: Set the first branch P to a 1×1 convolutional layer plus a Sigmod activation function for feature extraction; The second branch U is set to a 1×1 convolution layer plus a 3×3 depthwise convolution, and two ReLU activation functions are combined to generate the modulation signal; Finally, the output matrices of the two branches are multiplied to obtain the modulated output feature X mod , X mod =(σ u2 (σ u1 (XW u1 )W u2 )⊙σ p (XW p ))W mod , σ(·) is the activation function of each branch, W u and W p is the corresponding branch weight matrix.

4. The image reconstruction method for hyperspectral compressed sensing according to claim 1, characterized in that: The optimization objective function is specifically: Among them, {f n } is the training set data, d n is the model output data, W Θ is the reconstruction model with training parameters Θ, M Φ is the mask perception matrix, R(Φ) is the physical constraint in optical coding, N is the number of training samples, μ is the physical constraint weight parameter, and L task is the mean square error loss function of the reconstructed image.

5. The image reconstruction method for hyperspectral compressed sensing according to claim 1, characterized in that: The measurement data is input into the trained reconstruction model for integration to obtain a reconstructed hyperspectral image, specifically including: Step (1): Initially use a 3×3 convolutional layer to map the input X to the feature space X'; Step (2): After expanding the mask dimension to match the model size, the mask map M' is obtained through two convolution paths of 1×1 and 5×5; Step (3): The feature data X' is linearly mapped to calculate the corresponding key K, value V, and query Q information, and the attention is calculated using cross-covariance attention. K i is the key information of each attention head; where A i is the i-th attention score, τ is the learnable weight parameter, i is the index of each attention head, and T is the transposition operation; Step (4): Multiply the value V by the mask M' to get V', connect each attention to weight and get the output through linear mapping Among them, W O is the mapping layer weight; where h is the number of attention heads; Step (5): Use covariance self-attention to calculate the attention map X of the input feature X' A , and combined with the mask map M', weighted to the input feature output X c ; Step (6): For the output X of the covariance attention module c , using spatial convolutional modulation modules to enhance spatial representation X mod ; Step (7): For the output of the spatial convolution modulation module, use the residual Fourier module to integrate the high and low frequency domain features to output X f ; Step (8): Take steps (3) to (7) as the process of the model basic module, stack the model basic module twice to form a core module, and after the data is passed through the core module to learn features, cache the data features and perform downsampling and dimensionality reduction operations on the data features; Step (9): Repeat step (8) twice and pass the data through the feature extraction of the core module again; Step (10): Upsample the data outputted in step (9), add the data features cached at the same size after learning the core model features twice in step (8) along the channel, and then perform the feature extraction step of the core module; Step (11): Repeat step (10) twice, and finally use the transposed convolution layer to map the data to the real space X R , and added to the model input X to form a residual structure, and finally the reconstructed hyperspectral image X is obtained. out .

6. The image reconstruction method for hyperspectral compressed sensing according to claim 1, characterized in that: The training process of the reconstruction model is: Acquire training data; the training data is hyperspectral images of different scenes, and each hyperspectral image is randomly cropped, rotated, flipped and spliced, and the hyperspectral data after data enhancement is stacked, and the data is constructed in batches for output; Input the training data into the initial network structure of the reconstruction model and specify the iteration round N epoch The training is performed for 200 times, the initial learning rate is set to 0.004, and the objective function is optimized until the loss converges to obtain the trained reconstruction model.

7. An image reconstruction system for hyperspectral compressed sensing, applying the method according to any one of claims 1 to 6, characterized in that: include: A data acquisition unit, used for acquiring target hyperspectral images; a preprocessing unit, configured to preprocess the target hyperspectral image to obtain measurement data inputted by a matching model; An image reconstruction unit is used to input the measurement data into a trained reconstruction model for integration to obtain a reconstructed hyperspectral image; the reconstruction model includes a spatial convolution modulation module, a high- and low-frequency domain feature integration module, and a set optimization objective function; the reconstruction model is trained and determined based on set iteration rounds, an initialization learning rate, and a loss function.

8. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the image reconstruction method for hyperspectral compressed sensing according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The device stores a computer program, which, when executed by a processor, implements the image reconstruction method for hyperspectral compressed sensing according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Hyperspectral image reconstruction method based on spatial-spectral double-prior frequency domain enhancement

    CN117764862A