Hyperspectral snapshot compressive sensing imaging method and system based on spatial-spectral inter priori decoupling model

Abstract

The application provides a hyperspectral snapshot compression imaging method and system based on a space-spectrum prior decoupling model, high-quality reconstruction is realized through decoupling optimization and a deep unfolding network, and the method comprises the following steps: constructing a training data set containing a compression measurement image and corresponding reconstructed spectrum; establishing a target function fusing space and spectrum priors and converting the target function into a constraint optimization; adopting a semi-quadratic splitting method to convert the constraint optimization into three sub-problems of linear reconstruction, space prior and spectrum prior; designing a deep unfolding network to alternately solve the sub-problems; the linear sub-problem is solved analytically, the space / spectrum sub-problems are respectively modeled by special networks to implicitly model the priors, and end-to-end reconstruction is realized; a hybrid loss function is adopted to optimize model parameters, and after training, the image can be reconstructed in real time. The space and spectrum priors are decoupled, space structure details and spectrum characteristics are respectively captured through independent network architectures, the mutual interference problem in the joint modeling of traditional methods is overcome, and high-quality spectrum image reconstruction is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, and particularly relates to a hyperspectral snapshot compressive sensing imaging method and system based on a spatial-spectral inter prior decoupling model. BACKGROUND

[0002] In recent years, snapshot compressive imaging (SCI) systems have been developed for real-time hyperspectral image recovery. Among them, the coded aperture snapshot spectral imaging (CASSI) system has shown outstanding effectiveness and efficiency. The CASSI system mainly consists of two stages: measurement and reconstruction. In the measurement stage, the scene is first coded by a specially designed coded aperture, then different wavelengths are spectrally dispersed by a dispersion device, and finally captured by a detection device to form a measurement image. In the reconstruction stage, the detected measurement data needs to be processed and a recovery model needs to be constructed to realize the reconstruction of the original hyperspectral scene. Since the measurement process is determined by the coded aperture and the dispersion system, selecting an appropriate reconstruction method is crucial for high-fidelity recovery of the original hyperspectral scene.

[0003] In recent years, the progress of deep learning has made significant achievements in hyperspectral image reconstruction, especially the deep unfolding method integrates deep neural networks into an iterative optimization framework, improving the reconstruction quality. However, existing deep unfolding methods mostly use a single network architecture as a denoiser for prior estimation, ignoring the inherent complexity of hyperspectral data. Hyperspectral images contain two different types of features: spatial features (including local texture and global structural similarity) and spectral features (distinguishing objects through their unique spectral characteristics). When a single network emphasizes global spatial relationships in modeling spatial priors, it may blur the spectral differences between objects, leading to an averaging effect in the spectral domain. Therefore, these single network architectures, which mainly focus on denoising optimization, are difficult to maintain both fine spatial details and accurate spectral features.

[0004] To solve the above problems, a hyperspectral snapshot compressive sensing imaging method based on a spatial-spectral inter prior decoupling model is proposed, which independently models spatial priors and spectral priors by designing different networks to achieve high-quality spectral image reconstruction. SUMMARY

[0005] To solve the above technical problems, the application provides a hyperspectral snapshot compressive sensing imaging method and system based on a spatial-spectral inter prior decoupling model.

[0006] The technical solution adopted by the application to solve its technical problems is:

[0007] The hyperspectral snapshot compressive sensing imaging method based on the spatial-spectral inter prior decoupling model comprises the following steps:

[0008] S100: In a specific scene, a plurality of compressed measurement images and corresponding spectral reconstruction images are shot to construct a training data set;

[0009] S200: A space-spectral prior decoupling model is built, and a target function containing a space prior and a spectral prior is constructed based on the space-spectral prior decoupling idea;

[0010] S300: Additional variables and constraint conditions are introduced to convert the target function into a constrained optimization problem, the constrained optimization problem is converted into an unconstrained form based on a semi-quadratic splitting method, and is decomposed into three sub-problems of linear reconstruction, space prior and spectral prior; the linear sub-problems are solved by alternating iteration, and the corresponding implicit regularization terms are modeled by using the space and spectral prior networks respectively, so as to construct a deep unfolding neural network, solve the space sub-problems and the spectral sub-problems, and realize end-to-end image reconstruction prediction;

[0011] S400: The training data set is input into the space-spectral prior decoupling model to construct a model training process, the optimization process is constrained based on a preset hybrid loss function, and when a preset training end condition is reached, a trained space-spectral prior decoupling model is obtained, and real-time image reconstruction is completed based on the trained space-spectral prior decoupling model.

[0012] Preferably, the target function containing the space prior and the spectral prior constructed in S200 is specifically:

[0013] ;

[0014] wherein, the target function is to minimize the current formula, and are weight coefficients, is a measurement image, is a hyperspectral image to be reconstructed, and Φ is a sensing matrix, and are a space prior and a spectral prior respectively.

[0015] Preferably, in S300, additional variables , are introduced, and constraint conditions are added to convert the target function into a constrained optimization problem, which is specifically:

[0016] ;

[0017] wherein, and are space auxiliary variables and spectral auxiliary variables introduced for independently representing the space prior and the spectral prior respectively.

[0018] Preferably, in S300, the constrained optimization problem is converted into an unconstrained optimization problem based on the semi-quadratic splitting method, specifically:

[0019] ;

[0020] in, and is the penalty parameter, Indicates that when the minimum value is achieved under the current conditions, the variable in the equation The value of .

[0021] Preferably, in S300, linear subproblems are solved by alternating iterations, and corresponding implicit regularization terms are modeled using spatial and spectral prior networks, respectively, thereby constructing a deep unfolded neural network to solve spatial subproblems and spectral subproblems, including:

[0022] S310: The obtained k-th round and , combined with weights 、 , sensor matrix and the transpose of the sensing matrix Solve the linear subproblem, specifically:

[0023] ;

[0024] in, , represents element-wise division, Indicates extracting the diagonal elements of the subsequent matrix, Represents the transpose of a matrix;

[0025] S322: What you will get Input to the designed spatial prior network Zhonglai Forecast , to absorb the spatial prior :

[0026] ;

[0027] S323: What you will get Input to the designed spectral prior network In, to predict , with the absorption spectrum prior :

[0028] .

[0029] Preferably, the spatial prior estimation module and the spectral prior estimation module are both constructed based on a three-layer U-shaped network architecture, which includes an encoder, a bottleneck layer and a decoder. The encoder part includes a 3×3 convolution layer, two downsampling modules and two prior stages; the bottleneck layer is located at the bottom of the U-shape and has a prior stage; the decoder part symmetrically includes two upsampling modules and two prior stages, and a 3×3 convolution layer. The final output of the decoder will be added to the input of the encoder as the final output; the feature map obtained after each upsampling module will be spliced ​​with the feature map obtained before the corresponding downsampling layer, and mapped using 1×1 convolution; each layer of the prior stage of the spatial prior estimation module is embedded in a spatial prior block, and each layer of the prior stage of the spectral prior estimation module is embedded in a spectral prior block.

[0030] Preferably, the spatial prior block includes layer normalization, global-local attention mechanism and forward propagation layer;

[0031] For the input feature X, the length, width, and number of bands correspond to H, W, and C respectively. in , firstly, layer normalization is performed to standardize the data distribution, and then the learnable parameters , mapped to query ,key Sum :

[0032] ;

[0033] These tensors are then split equally into two parts along the spectral dimension for the computation of the local and global attention branches respectively; for the first part processed using the local multi-head self-attention operation, the local query, key, and value tensors Directly divide into non-overlapping windows of length and width of M×M and convert the dimensions to , further divide the tensors of these window segments into h attention heads along the channel dimension, and get ; The local multi-head self-attention operation is expressed as:

[0034] ;

[0035] in is the position code representing the relative position within each window, is the dimension of each head, That is the result of dimensionally splicing h attention heads;

[0036] For the second part, global multi-head self-attention processing is used, the feature map is divided into M×M non-overlapping regions, and the dimension is converted to , global query, key, and value tensors after partitioning , further divide the tensors of these regions into h attention heads along the channel dimension, and get ; The local multi-head self-attention operation is expressed as:

[0037] ;

[0038] in, Indicates the relative position encoding between each region, represents the dimension of each head, represents the output of the global multi-head self-attention processing;

[0039] Will and Connect along the spectral dimension, then through 1×1 convolution to get the output of global-local attention , specifically:

[0040] ;

[0041] Finally, the forward propagation layer remaps the features and adjusts the feature distribution.

[0042] Preferably, the spectral prior block includes layer normalization, spectral-window attention mechanism and forward propagation layer;

[0043] For the input feature X, the length, width, and number of bands correspond to H, W, and C respectively. in , firstly, layer normalization is performed to standardize the data distribution, and then the learnable parameters , mapped to query ,key Sum , specifically:

[0044] ;

[0045] Windowed spectral MSA first divides the input features into non-overlapping windows of size M×M. Within each window, attention between spectral channels is calculated to capture local spectral correlations. Reshape into , in order to perform multi-head attention calculation, these tensors are divided into h attention heads along the last dimension. The multi-head attention calculation is as follows:

[0046] ;

[0047] in, represents the sequential encoding of spectral channels, is the dimension of each head, represents the learnable parameters of 1×1 convolution, denotes concatenation along the last dimension, for spectral-window attention output results;

[0048] Finally, the forward propagation layer remaps the features, adjusting the feature distribution.

[0049] Preferably, the preset hybrid loss function in S400 is specifically:

[0050]

[0051]

[0052]

[0053] wherein β is a balance parameter, is a converged loss function, is a prior exchange loss function, T(·) and G(·) represent a spatial prior network and a spectral prior network respectively, and k is the number of unfolded stages, is an actually labeled reconstructed sample, and det represents a gradient truncation operation.

[0054] The hyperspectral snapshot compressive sensing imaging system based on the spatial-spectral inter-prior decoupling model comprises a data set construction module, a target function construction module, an image reconstruction prediction module, and a model training and imaging module.

[0055] The data set construction module is used for shooting a plurality of compressed measurement images and corresponding spectral reconstruction images under a specific scene, and constructing a training data set.

[0056] The target function construction module is used for building a spatial-spectral inter-prior decoupling model, and constructing a target function containing a spatial prior and a spectral prior based on the spatial-spectral inter-prior decoupling idea.

[0057] The image reconstruction prediction module is used for introducing an additional variable, converting the target function into a constraint optimization problem, converting the constraint optimization problem into an unconstrained form based on a semi-quadratic splitting method, and decomposing the constraint optimization problem into three sub-problems of linear reconstruction, spatial prior, and spectral prior. The linear sub-problems are solved by alternating iteration, and the corresponding implicit regularization terms are modeled by using the spatial and spectral prior networks, so as to construct a deep unfolded neural network, solve the spatial and spectral sub-problems, and realize end-to-end image reconstruction prediction.

[0058] ​​​The model training and imaging module is configured to input a training data set into a spatial-spectral prior decoupling model construction model training process, constrain an optimization process based on a preset hybrid loss function, and obtain a trained spatial-spectral prior decoupling model when a preset training end condition is reached, and complete real-time image reconstruction based on the trained spatial-spectral prior decoupling model.

[0059] The hyperspectral snapshot compressive sensing imaging method and system based on the spatial-spectral prior decoupling model reconstruct the hyperspectral image reconstruction as a prior absorption problem, independently model the spatial and spectral priors using a special network architecture, can simultaneously maintain the spatial details and spectral characteristics of the image, and significantly improve the quality of the hyperspectral image reconstruction. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 The flowchart of the hyperspectral snapshot compressive sensing imaging method based on the spatial-spectral prior decoupling model in an embodiment of the present application is shown in FIG. 1.

[0061] Figure 2 The overall architecture schematic diagram of the hyperspectral snapshot compressive sensing imaging method based on the spatial-spectral prior decoupling model in an embodiment of the present application is shown in FIG. 2.

[0062] Figure 3 The structure schematic diagram of the prior estimation module in an embodiment of the present application is shown in FIG. 3.

[0063] Figure 4 The structure schematic diagram of the spatial prior block in an embodiment of the present application is shown in FIG. 4.

[0064] Figure 5 The structure schematic diagram of the spectral prior block in an embodiment of the present application is shown in FIG. 5.

[0065] Figure 6 The structure schematic diagram of the window spectral multi-head self-attention mechanism in the spatial prior block in an embodiment of the present application is shown in FIG. 6.

[0066] Figure 7 The structure schematic diagram of the window spectral multi-head self-attention mechanism in the spectral prior block in an embodiment of the present application is shown in FIG. 7.

[0067] Figure 8 The flowchart of the specific implementation of the method proposed in the present application is shown in FIG. 8. DETAILED DESCRIPTION

[0068] In order to enable those skilled in the art to better understand the technical solutions of the present application, the present application will be further described in detail below with reference to the drawings.

[0069] In one embodiment, as shown in FIG. 1, the hyperspectral snapshot compressive sensing imaging method based on the spatial-spectral prior decoupling model comprises the following steps: Figure 1 The method comprises the following steps:

[0070] S100: In a specific scene, a plurality of compressed measurement images and corresponding spectral reconstruction images are shot to construct a training data set; in the embodiment, the specific scene is selected as an indoor scene, and the CASSI system is used to collect compressed measurement images under standard indoor lighting conditions, and at the same time, a traditional hyperspectral camera (such as a scanning hyperspectral imager) is used to collect high-precision spectral images of the same scene as the true label; the collected data includes different indoor objects, materials and lighting conditions to ensure the diversity of the training data; the size of the data set is 500 pairs of images, of which 400 pairs are used for training and 100 pairs are used for verification;

[0071] S200: A spatial-spectral prior decoupling model is built, and a target function containing spatial and spectral priors is constructed based on the spatial-spectral prior decoupling idea;

[0072] S300: An additional variable is introduced to convert the target function into a constrained optimization problem, and based on the semi-quadratic splitting method, the constrained optimization problem is converted into an unconstrained form and decomposed into three sub-problems of linear reconstruction, spatial prior and spectral prior; the linear sub-problems are solved by alternating iteration, and the spatial and spectral prior networks are used to model the corresponding implicit regularization terms, thereby constructing a deep unfolding neural network to solve the spatial and spectral sub-problems and realize end-to-end image reconstruction prediction;

[0073] S400: The training data set is input into the spatial-spectral prior decoupling model to construct a model training process, and based on the pre-set hybrid loss function, the optimization process is constrained, and when the pre-set training end condition is reached, the trained spatial-spectral prior decoupling model is obtained, and real-time image reconstruction is completed based on the trained spatial-spectral prior decoupling model.

[0074] The above hyperspectral snapshot compressive sensing imaging method based on the spatial-spectral prior decoupling model independently models the spatial and spectral priors by designing different networks to achieve high-quality spectral image reconstruction.

[0075] In one embodiment, the target function containing spatial and spectral priors constructed in S200 is specifically:

[0076] ;

[0077] wherein, the target function is to minimize the current formula, and indicate weight coefficients, indicates a measurement image, indicates a hyperspectral image to be reconstructed, and Φ indicates a sensing matrix, and indicate spatial and spectral priors, respectively.

[0078] In one embodiment, additional variables are introduced in S300 、 and constraints are added, and the objective function is converted into a constrained optimization problem, specifically:

[0079] ;

[0080] wherein and are spatial auxiliary variables and spectral auxiliary variables introduced for independently representing spatial prior and spectral prior, respectively.

[0081] In one embodiment, the constrained optimization problem in S300 is converted into an unconstrained optimization problem based on a semi-quadratic splitting method, specifically:

[0082] ;

[0083] wherein and are penalty parameters, represents the value of variable in the equation when the minimum value is obtained under the current condition.

[0084] In one embodiment, as shown in Figure 2 , the linear sub-problems are solved by alternating iteration in S300, and the corresponding implicit regularization terms are modeled by using spatial and spectral prior networks, respectively, so as to build a deep unfolding neural network to solve the spatial sub-problem and the spectral sub-problem, including:

[0085] S310: by obtaining the kth round of and , the linear sub-problem is solved in combination with weights 、 , a sensing matrix and a transpose of the sensing matrix , specifically:

[0086] ;

[0087] wherein , represents an element-wise level division, represents extracting diagonal elements of a subsequent matrix, represents a transpose of a matrix;

[0088] S322: input the obtained into the designed spatial prior network to predict , so as to absorb the spatial prior term :

[0089] ;

[0090] S323: What you will get Input to the designed spectral prior network In, to predict , with the absorption spectrum prior :

[0091] .

[0092] In one embodiment, Figure 3 As shown in the figure, the spatial prior estimation module and the spectral prior estimation module are both built based on a three-layer U-shaped network architecture. The three-layer U-shaped network architecture includes an encoder, a bottleneck layer and a decoder. The encoder part contains a 3×3 convolution layer, two downsampling modules and two prior stages; the bottleneck layer is located at the bottom of the U-shape and has a prior stage; the decoder part symmetrically contains two upsampling modules and two prior stages, and a 3×3 convolution layer. The final output of the decoder will be added to the input of the encoder as the final output; the feature map obtained after each upsampling module will be spliced ​​with the feature map obtained before the corresponding downsampling layer, and mapped using 1×1 convolution; each layer of the prior stage of the spatial prior estimation module is embedded in a spatial prior block, and each layer of the prior stage of the spectral prior estimation module is embedded in a spectral prior block.

[0093] Specifically, the basic prior estimation network proposed in the present invention adopts a three-layer U-shaped structure as the basic architecture, such as Figure 3 To independently estimate spatial and spectral priors, two independent prior estimation networks are implemented: the spatial prior estimation network In its prior stage, the spatial prior block is used, and the prior estimation network Use a spectral prior block such as Figure 4 and 5 As shown in Figure 2, both blocks share the same three-component structure, including layer normalization, domain-specific attention mechanism, and forward propagation layer. For the attention mechanism, the spatial prior uses global-local attention, while the spectral prior uses spectral-window attention.

[0094] In one embodiment, Figure 4 As shown in , the spatial prior block includes layer normalization, global-local attention mechanism and forward propagation layer; further, as Figure 6 As shown in Figure 2, the global-local attention mechanism divides the input features into two equal parts along the spectral dimension: one part is processed by local multi-head self-attention, and the other part is processed by global multi-head self-attention. This design can capture both local details and global structural information. The specific implementation is as follows:

[0095] For the input feature X, the length, width, and number of bands correspond to H, W, and C respectively. in , firstly, layer normalization is performed to standardize the data distribution, and then the learnable parameters , mapped to query ,key Sum :

[0096] ;

[0097] These tensors are then split equally into two parts along the spectral dimension for the computation of the local and global attention branches respectively; for the first part processed using the local multi-head self-attention operation, the local query, key, and value tensors Directly divide into non-overlapping windows of length and width of M×M and convert the dimensions to , further divide the tensors of these window segments into h attention heads along the channel dimension, and get ; The local multi-head self-attention operation is expressed as:

[0098] ;

[0099] in is the position code representing the relative position within each window, is the dimension of each head, That is the result of dimensionally splicing h attention heads;

[0100] For the second part, global multi-head self-attention processing is used, the feature map is divided into M×M non-overlapping regions, and the dimension is converted to , global query, key, and value tensors after partitioning , further divide the tensors of these regions into h attention heads along the channel dimension, and get ; The local multi-head self-attention operation is expressed as:

[0101] ;

[0102] in, Indicates the relative position encoding between each region, represents the dimension of each head, represents the output of the global multi-head self-attention processing;

[0103] Will and Connect along the spectral dimension, then through 1×1 convolution to get the output of global-local attention , specifically:

[0104] ;

[0105] Finally, the forward propagation layer remaps the features and adjusts the feature distribution.

[0106] In one embodiment, Figure 5 As shown in , the spectral prior block includes layer normalization, spectral-window attention mechanism, and forward propagation layer. Furthermore, since different objects have different spectral reflectance characteristics, calculating the spectral channel similarity on the global image will produce an averaging effect and fail to preserve the reflectance characteristics of the object at the characteristic wavelength. Therefore, a windowed spectral multi-head self-attention method is proposed to process spectral similarity within a local spatial window, as shown in Figure 7 As shown, the specific implementation is as follows:

[0107] For the input feature X, the length, width, and number of bands correspond to H, W, and C respectively. in , firstly, layer normalization is performed to standardize the data distribution, and then the learnable parameters , mapped to query ,key Sum , specifically:

[0108] ;

[0109] Windowed spectral MSA first divides the input features into non-overlapping windows of size M×M. Within each window, attention between spectral channels is calculated to capture local spectral correlations. Reshape into , in order to perform multi-head attention calculation, these tensors are divided into h attention heads along the last dimension. The multi-head attention calculation is as follows:

[0110] ;

[0111] in, represents the sequential encoding of spectral channels, is the dimension of each head, represents the learnable parameters of 1×1 convolution, Indicates splicing along the last dimension, Output result for spectral-window attention;

[0112] Finally, the forward propagation layer remaps the features and adjusts the feature distribution.

[0113] Further, if Figure 8 As shown, the measured value y and the sensor matrix ,parameter , and after initialization and , input into the process of deep expansion problem construction, retaining the predicted values ​​of each stage in k stages , , , and build a loss function for training. The training rounds are 300 rounds, using all the stored , update the network weights through the loss function.

[0114] In one embodiment, the hybrid loss function preset in S400 is specifically:

[0115] ;

[0116] ;

[0117] ;

[0118] Where β is the equilibrium parameter, is the convergence loss function, is the prior exchange loss function, T(·) and G(·) represent the spatial prior network and the spectral prior network respectively, k is the number of stages of expansion, is the actual labeled reconstructed sample, and det represents the gradient truncation operation.

[0119] The Adam optimizer is used in the training process, the initial learning rate is set to 1e-4, the total number of training rounds is 300 epochs, the batch size is 16, and the weight coefficient is Set to 0.1.

[0120] The hyperspectral snapshot compressed sensing imaging method based on the spatial-spectral prior decoupling model has the following beneficial effects:

[0121] (1) A novel spatial-spectral prior decoupling framework is proposed, which reformulates hyperspectral image reconstruction as a prior absorption problem and uses a dedicated network architecture to independently model spatial and spectral priors, thus better achieving feature reconstruction of spectral images.

[0122] (2) Two special attention mechanisms are designed: a windowed semi-split spatial multi-head self-attention mechanism and a windowed spectral multi-head self-attention mechanism, which are used to capture the spatial correlation of hyperspectral images and maintain spectral features, respectively. They potentially utilize the internal self-similarity of hyperspectral images to make the reconstructed spectral images more consistent with the characteristics of the actual captured spectral images.

[0123] (3) A hybrid loss function combining convergence constraints and cross-prior interactions is designed to ensure accurate prior fusion and a stable reconstruction process. Ultimately, the established model is able to better reconstruct hyperspectral images.

[0124] In one embodiment, a hyperspectral snapshot compressive sensing imaging system based on a spatial-spectral inter-prior decoupling model is also provided, comprising a data set construction module, a target function construction module, an image reconstruction prediction module, and a model training and imaging module;

[0125] The data set construction module is configured to take a plurality of compressive measurement images and corresponding spectral reconstruction images under a specific scene, and construct a training data set.

[0126] The target function construction module is configured to build a spatial-spectral inter-prior decoupling model, and construct a target function containing spatial prior and spectral prior based on the spatial-spectral inter-prior decoupling idea.

[0127] The image reconstruction prediction module is configured to introduce an additional variable, convert the target function into a constrained optimization problem, convert the constrained optimization problem into an unconstrained form based on a semi-quadratic splitting method, and decompose it into three sub-problems of linear reconstruction, spatial prior, and spectral prior. The linear sub-problem is solved by alternating iteration, and the spatial and spectral prior networks are used to model the corresponding implicit regularization terms, thereby constructing a deep unfolding neural network to solve the spatial and spectral sub-problems and realize end-to-end image reconstruction prediction.

[0128] The model training and imaging module is configured to input the training data set into the spatial-spectral inter-prior decoupling model to construct a model training process, constrain the optimization process based on a preset hybrid loss function, and obtain a trained spatial-spectral inter-prior decoupling model when a preset training end condition is reached, and complete real-time image reconstruction based on the trained spatial-spectral inter-prior decoupling model.

[0129] The specific limitations of the hyperspectral snapshot compressive sensing imaging system based on the spatial-spectral inter-prior decoupling model can be referred to the limitations of the hyperspectral snapshot compressive sensing imaging method based on the spatial-spectral inter-prior decoupling model in the above, which will not be repeated here. Each module in the above hyperspectral snapshot compressive sensing imaging system based on the spatial-spectral inter-prior decoupling model can be realized by software, hardware, and their combinations, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0130] The above describes in detail the hyperspectral snapshot compressive sensing imaging method and system based on the spatial-spectral inter-prior decoupling model provided by the present application. The principles and implementation manners of the present application are described by using specific examples, and the above description of the examples is only used to help understand the core idea of the present application. It should be pointed out that, for those skilled in the art, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A hyperspectral snapshot compressed sensing imaging method based on a spatial-spectral prior decoupling model, characterized in that: The method comprises the following steps: S100: Take several compressed measurement images and corresponding spectral reconstruction images in a specific scene to build a training dataset; S200: Build a spatial-spectral prior decoupling model. Based on the idea of ​​spatial-spectral prior decoupling, construct an objective function that includes spatial and spectral priors. S300: Introducing additional variables, converting the objective function into a constrained optimization problem, converting the constrained optimization problem into an unconstrained form based on the semi-quadratic splitting method, and decomposing it into three sub-problems: linear reconstruction, spatial prior, and spectral prior. Solving the linear sub-problems through alternating iterations, and using the spatial and spectral prior networks to model the corresponding implicit regularization terms, thereby constructing a deep unfolded neural network to solve the spatial sub-problems and spectral sub-problems, and realizing end-to-end image reconstruction prediction. Both the spatial prior network and the spectral prior network are built based on a three-layer U-shaped network architecture, which includes an encoder, a bottleneck layer, and a decoder. The encoder The decoder part consists of a 3×3 convolution layer, two downsampling modules and two prior stages; the bottleneck layer is located at the bottom of the U-shape and has a prior stage; the decoder part symmetrically consists of two upsampling modules and two prior stages, and a 3×3 convolution layer. The output of the decoder will be added to the input of the encoder as the final output; the feature map obtained after each upsampling module will be spliced ​​with the feature map obtained before the corresponding downsampling layer and mapped using 1×1 convolution; each layer of the prior stage of the spatial prior network is embedded in a spatial prior block, and each layer of the prior stage of the spectral prior estimation module is embedded in a spectral prior block; S400: Input the training data set into the spatial-spectral prior decoupling model to construct a model training process. Based on the preset mixed loss function constraint optimization process, when the preset training end condition is reached, the trained spatial-spectral prior decoupling model is obtained, and image reconstruction is completed in real time based on the trained spatial-spectral prior decoupling model.

2. The method according to claim 1, characterized in that The objective function including spatial prior and spectral prior is constructed in S200 as follows: ； in, Indicates that the objective function is to minimize the current formula, and represents the weight coefficient, represents the measurement image, represents the hyperspectral image to be reconstructed, Φ represents the sensing matrix, and denote the spatial prior and spectral prior, respectively.

3. The method according to claim 2, characterized in that Introducing additional variables in S300 、 , and after adding constraints, the objective function is converted into a constrained optimization problem, specifically: ； in, and They are spatial auxiliary variables and spectral auxiliary variables introduced to independently represent spatial priors and spectral priors respectively.

4. The method according to claim 3, characterized in that In S300, the constrained optimization problem is converted into an unconstrained optimization problem based on the semi-quadratic splitting method, specifically: ； in, and is the penalty parameter, Indicates that when the minimum value is achieved under the current conditions, the variable in the equation The value of .

5. The method according to claim 4, characterized in that S300 solves linear subproblems through alternating iterations and uses spatial and spectral prior networks to model the corresponding implicit regularization terms, thereby constructing a deep unfolded neural network to solve spatial and spectral subproblems, including: S310: The obtained k-th round and , combined with weights 、 , sensor matrix and the transpose of the sensing matrix Solve the linear subproblem, specifically: ； in, , represents element-wise division, Indicates extracting the diagonal elements of the subsequent matrix, Represents the transpose of a matrix; S322: What you will get Input to the designed spatial prior network Zhonglai Forecast , to absorb the spatial prior : ； S323: What you will get Input to the designed spectral prior network In, to predict , with the absorption spectrum prior : 。 6. The method according to claim 5, characterized in that The spatial prior block includes layer normalization, global-local attention mechanism, and forward propagation layer; For the input feature X, the length, width, and number of bands correspond to H, W, and C respectively. in , firstly, layer normalization is performed to standardize the data distribution, and then the learnable parameters , mapped to query ,key Sum : ； These tensors are then split equally into two parts along the spectral dimension for the computation of the local and global attention branches respectively; for the first part processed using the local multi-head self-attention operation, the local query, key, and value tensors Directly divide into non-overlapping windows of length and width of M×M and convert the dimensions to , further divide the tensors of these window segments into h attention heads along the channel dimension, and get ; The local multi-head self-attention operation is expressed as: ； in is the position code representing the relative position within each window, is the dimension of each head, That is the result of dimensionally splicing h attention heads; For the second part, global multi-head self-attention processing is used, the feature map is divided into M×M non-overlapping regions, and the dimension is converted to , global query, key, and value tensors after partitioning , further divide the tensors of these regions into h attention heads along the channel dimension, and get ; The local multi-head self-attention operation is expressed as: ； in, Indicates the relative position encoding between each region, represents the dimension of each head, represents the output of the global multi-head self-attention processing; Will and Connect along the spectral dimension, then through 1×1 convolution to get the output of global-local attention , specifically: ； Finally, the forward propagation layer remaps the features and adjusts the feature distribution.

7. The method according to claim 6, characterized in that The spectral prior block includes layer normalization, spectral-window attention mechanism, and forward propagation layer; For the input feature X, the length, width, and number of bands correspond to H, W, and C respectively. in , firstly, layer normalization is performed to standardize the data distribution, and then the learnable parameters , mapped to query ,key Sum , specifically: ； Windowed spectral MSA first divides the input features into non-overlapping windows of size M×M. Within each window, attention between spectral channels is calculated to capture local spectral correlations. Reshape into , in order to perform multi-head attention calculation, these tensors are divided into h attention heads along the last dimension. The multi-head attention calculation is as follows: ； in, represents the sequential encoding of spectral channels, is the dimension of each head, represents the learnable parameters of 1×1 convolution, Indicates splicing along the last dimension, Output result for spectral-window attention; Finally, the forward propagation layer remaps the features and adjusts the feature distribution.

8. The method according to claim 7, characterized in that The hybrid loss function preset in S400 is specifically: ； ； ； Where β is the equilibrium parameter, is the convergence loss function, is the prior exchange loss function, T(·) and G(·) represent the spatial prior network and the spectral prior network respectively, k is the number of stages of expansion, is the actual labeled reconstructed sample, and det represents the gradient truncation operation.

9. Hyperspectral snapshot compressed sensing imaging system based on spatial-spectral prior decoupling model, characterized by: It includes a dataset construction module, an objective function construction module, an image reconstruction prediction module, and a model training and imaging module; The dataset construction module is used to capture several compressed measurement images and corresponding spectral reconstruction images in a specific scene to construct a training dataset; The objective function construction module is used to build a spatial-spectral prior decoupling model. Based on the idea of ​​spatial-spectral prior decoupling, it constructs an objective function that includes spatial and spectral priors. The image reconstruction prediction module is used to introduce additional variables, convert the objective function into a constrained optimization problem, convert the constrained optimization problem into an unconstrained form based on the semi-quadratic splitting method, and decompose it into three sub-problems: linear reconstruction, spatial prior, and spectral prior. The linear sub-problems are solved by alternating iterations, and the corresponding implicit regularization terms are modeled using spatial and spectral prior networks respectively, thereby constructing a deep unfolded neural network to solve the spatial and spectral sub-problems and achieve end-to-end image reconstruction prediction. Both the spatial and spectral prior networks are built based on a three-layer U-shaped network architecture, which includes an encoder, a bottleneck layer, and a decoder. The encoder part contains a 3×3 convolutional layer, two downsampling modules, and two prior stages. The bottleneck layer is located at the bottom of the U-shape and has a prior stage. The decoder symmetrically consists of two upsampling modules, two prior stages, and a 3×3 convolutional layer. The decoder output is added to the encoder input as the final output. The feature map obtained after each upsampling module is concatenated with the feature map obtained before the corresponding downsampling layer and mapped using 1×1 convolution. The spatial prior network has a spatial prior block embedded in each layer of the prior stage, and the spectral prior estimation module has a spectral prior block embedded in each layer of the prior stage. The model training and imaging module is used to input the training data set into the spatial-spectral prior decoupling model to construct the model training process. Based on the preset mixed loss function constrained optimization process, when the preset training end conditions are reached, the trained spatial-spectral prior decoupling model is obtained, and image reconstruction is completed in real time based on the trained spatial-spectral prior decoupling model.

Images