Method and system for reconstructing hyperspectrum from RGB image

Abstract

The invention provides a method and a system for reconstructing a hyperspectrum from an RGB image. The method comprises the following steps: acquiring a three-channel RGB image; performing embedding and channel expansion on the RGB image by using a convolutional neural network to obtain an initial feature map; inputting the initial feature map into an encoder, respectively extracting channel dimension features and space dimension features by utilizing VisionTransform and VisionMama in the initial feature map, fusing the channel dimension features and the space dimension features, and respectively inputting the channel dimension features and the space dimension features into jump connection and separable convolution to obtain feature maps of different levels; inputting the feature map into a decoder to obtain a final feature map; mapping the final feature map by using a convolutional neural network, and adding the final feature map with an initial feature map before the final feature map is input into an encoder to obtain a hyperspectral image in a compressed form; and splitting the hyperspectral image in the compressed form according to the number of channels, and then performing color mapping to finally obtain a reconstructed hyperspectral image. According to the method, the precision and generalization performance of reconstructing the hyperspectrum from the RGB image can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral reconstruction, and in particular to a method and system for reconstructing a hyperspectral spectrum from an RGB image. Background Art

[0002] Hyperspectral imaging is a technology that acquires images by capturing the spectral information of an object at multiple continuous wavelengths. Hyperspectral images can provide spectral information beyond the visible range of the human eye, so that the properties, composition and state of objects can be analyzed and identified in more detail. The fine spectral information of hyperspectral images can significantly improve the accuracy of feature classification and recognition, and is widely used in target recognition, environmental monitoring and resource surveys. However, hyperspectral imaging equipment is usually expensive, resulting in high data acquisition costs, which limits its widespread use in certain fields. In addition, due to the inherent physical properties of hyperspectral imaging sensors, hyperspectral images always have complex noise, low signal-to-noise ratio and low spatial resolution, which greatly restricts the accuracy improvement of high-level remote sensing tasks. Therefore, how to further improve the quality of hyperspectral images and reduce the acquisition cost has become an urgent problem to be solved.

[0003] Snapshot compression imaging systems and computational reconstruction algorithms are effective means of reconstructing three-dimensional hyperspectral cubes. However, these methods still rely on expensive hardware equipment. As a cost-effective alternative, spectral reconstruction technology has attracted attention. Spectral reconstruction technology uses color information and known spectral features in RGB images to estimate the spectral response curve of each pixel to generate a hyperspectral spectrum. Current spectral reconstruction methods can be roughly divided into sparse dictionary learning and deep learning-based methods.

[0004] At present, RGB image spectral reconstruction based on deep learning has achieved certain research results. However, since the reconstruction of high-dimensional spectral data from three-channel RGB images is an ill-posed mathematical inverse problem, the difficulty in solving it still exists, and the reconstructed spectral image still faces problems such as loss of structural details and low spectral reconstruction accuracy. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a method and system for reconstructing hyperspectral images from RGB images, which can fully explore the key features in spectral images through Transformer and Mamba, and improve the accuracy, robustness and generalization of spectral reconstruction.

[0006] To achieve the above-mentioned object of the invention, the first aspect of the present invention provides a method for reconstructing a hyperspectral spectrum from an RGB image, the method comprising: S11, obtaining a three-channel RGB image; S12, using a convolutional neural network to embed and expand the channels of the three-channel RGB image, expanding the number of channels of the three-channel RGB image to the required number of hyperspectral wavelengths N, and obtaining an initial feature map with N channels; S13, inputting the initial feature map in step S12 into an encoder to obtain feature maps of different levels, wherein the encoder is composed of n stacked encoder layers, each layer is composed of a multi-dimensional feature extraction module and a separable convolution; S14, inputting the feature maps of different levels in step S13 into the decoder for image reconstruction to obtain a final feature map with N channels, wherein the decoder is composed of n stacked encoders, and each layer is composed of a multi-dimensional feature extraction module and deconvolution; S15, mapping the final feature map in step S14 using a convolutional neural network, and adding it to the initial feature map before inputting the encoder to obtain a hyperspectral image in a compressed form; S16, constructing a reconstructed hyperspectral model, splitting and color mapping the compressed hyperspectral image in step S15 according to the number of channels, and obtaining a reconstructed hyperspectral image.

[0007] Furthermore, in step S12, the three-channel RGB image is expanded to obtain an initial feature map with N channels, which specifically includes: S21, linearly scaling the image to [0, 1] to achieve data normalization; S22, using a 3×3 convolutional layer to expand the number of channels of the image from 3 to N channels; S23. Obtain an initial feature map with N channels.

[0008] Furthermore, in step S13, the initial feature map is processed using an encoder composed of a multi-dimensional feature extraction module and a separable convolution, specifically: S31, using VisionTransformer in the multi-dimensional extraction module to extract channel dimension features in the initial feature map; S32, using VisionMamba in the multi-dimensional extraction module to extract spatial dimension features in the initial feature map; S33, fusing the spatial dimension features and channel dimension features of the initial feature map in steps S31 and S32 to obtain a fused feature map; S34. Input the fused feature maps into separable convolution and jump connection respectively to obtain feature maps of different levels.

[0009] Furthermore, the VisionTransformer consists of a channel multi-head self-attention module and a feedforward network, and the VisionMamba consists of a normalization layer, a linear mapping layer, a depth-separable convolution, and a two-dimensional selective scanning module.

[0010] Furthermore, in step S31, the channel dimension features in the initial feature map are extracted using VisionTransformer, specifically including: S41. By calculating the multi-head self-attention and feedforward neural network for different channels of the initial feature map, the expression is:

[0011] in, is the feature map input into VisionTransformer, represents the element-wise addition operation, MSA() represents the channel multi-head self-attention layer and FN() represents the feed-forward network; S42, capturing the global dependency between different channels and aggregating cross-channel information to obtain a hyperspectral encoding feature map in the channel dimension; S43, the calculation expression of the multi-head self-attention is:

[0012] in , , They are Query, Key, and Value in the self-attention mechanism. is the feature map input into VisionTransformer, , , are three learnable matrices, and , and Multiply to get the corresponding , , , and then along the channel dimension , , Decompose into N heads; is the Key of the jth head, T is the matrix transpose operation, is the j-th head Query, is the result of the j-th head matrix calculation, is the value of the jth head. Since the spectral density varies significantly with wavelength, a learnable parameter is used , to weight and Matrix calculation between to achieve attention Finally, the results of N head matrix calculations are connected and position coding is added to obtain the hyperspectral coding feature map of the channel dimension.

[0013] Further, in step S32, VisionMamba is used to extract the spatial dimension features in the initial feature map, specifically including: S51, by selectively scanning the state space sequence model to focus on different parts of the initial feature map, aggregating information across image patches; S52, processing image block embedding through a cross-scanning mechanism, flattening the two-dimensional image into a one-dimensional sequence and scanning along four different directions to capture the global dependency in the initial feature map; S53, merging all one-dimensional sequences and restoring them to the original two-dimensional structure to obtain a hyperspectral encoding feature map of the spatial dimension.

[0014] Furthermore, in step S34, the fused feature maps are respectively input into separable convolution and skip connection to obtain feature maps of different levels, specifically including: S61, use the concat operation to connect and fuse the spatial dimension features and the channel dimension features in the channel dimension, and use a 1×1 convolution layer to halve the number of channels of the fused feature map; S62, inputting the fused feature map into a separable convolution block, using the separable convolution to double the number of channels of the fused feature map, and reducing the length and width of the fused feature map by half, to obtain feature maps of different levels; S63. Input the fused feature map to the skip connection for fusion with the feature map in the corresponding layer of the decoder.

[0015] Furthermore, in step S14, the feature maps of different levels are input into the decoder for image reconstruction to obtain a final feature map with N channels, which specifically includes: S71, using deconvolution to perform upsampling to restore the size and number of channels of feature maps at different levels; S72, fusing the restored feature maps of different levels with the feature maps fused in the corresponding skip connection; S73, inputting the result outputted from step S72 into a multi-dimensional feature extraction module for further extracting channel features and spatial features of the image; S74. Repeat steps S71 to S73 in the decoder, reconstruct the output feature map to the same size as before inputting the encoder, and obtain a final feature map with N channels.

[0016] Furthermore, the construction and reconstruction of the hyperspectral model is specifically as follows: S81, obtaining training data, wherein the training data is a data set of 1000 RGB image-hyperspectral image pairs, and dividing the data set into a training set, a validation set, and a test set; S82, constructing a training model of the encoder, decoder, and front and back convolutional layers, inputting the training set in step S81 into the training model, and training using a mean square error loss function, the expression of the mean square error loss function is:

[0017] Where n is the number of pixels in the image, is the original hyperspectral image, To reconstruct hyperspectral images; S83. After each iteration, the validation set is used for validation, and the trained training model is determined as the hyperspectral reconstruction model.

[0018] A second aspect of the present invention provides a system for reconstructing a hyperspectral spectrum from an RGB image, the system comprising: Image acquisition module: used to acquire three-channel RGB images; Image channel expansion module: used to embed and expand the channels of three-channel RGB images through convolutional neural networks, expand the number of channels of three-channel RGB images to the required number of hyperspectral wavelengths N, and obtain an initial feature map with N channels; Feature encoding module: used to input the initial feature map into the encoder to obtain feature maps of different levels; Skip connection module: used to connect feature maps of different levels in the encoder to the decoder to reduce information loss caused by downsampling; Image decoding and reconstruction module: used to input feature maps of different levels into the decoder for image reconstruction to obtain the final feature map with N channels; Hyperspectral mapping module: used to map the final feature map using a convolutional neural network and add it to the initial feature map before inputting the encoder to obtain a compressed hyperspectral image; Hyperspectral splitting module: used to build a reconstructed hyperspectral model, split and color map the compressed hyperspectral image according to the number of channels, and obtain the reconstructed hyperspectral image.

[0019] Compared with the prior art, the present invention has the following beneficial effects: The present invention discloses a method and system for reconstructing a hyperspectral image from an RGB image. By combining Transformer with Mamba, channel dimension information and spatial dimension information of the image are extracted and fused in sequence, thereby deeply mining the contextual features between image channels, learning the internal connections of limited channel features, and adopting an encoder-decoder structure to fully fuse multi-level information to achieve high-quality reconstruction of hyperspectral images from RGB. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 It is a schematic flow chart of a method for reconstructing a hyperspectral spectrum from an RGB image provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The enumerated embodiments are only used to explain the present invention and are not used to limit the scope of the present invention.

[0023] Reference Figure 1 In a first aspect, this embodiment provides a method for reconstructing a hyperspectral spectrum from an RGB image, the method comprising: S11. Obtain a three-channel RGB image.

[0024] S12. Use a convolutional neural network to embed and expand the channels of the three-channel RGB image, expand the number of channels of the three-channel RGB image to the required number of hyperspectral wavelengths N, and obtain an initial feature map with N channels.

[0025] In step S12, channel expansion is performed on the three-channel RGB image to obtain an initial feature map with N channels, which specifically includes: S21. Linearly scale the image to [0, 1] to achieve data normalization.

[0026] S22. Use a 3×3 convolutional layer to expand the number of channels of the image from 3 to N channels.

[0027] S23. Obtain an initial feature map with N channels.

[0028] S13. Input the initial feature map in step S12 into an encoder to obtain feature maps of different levels, wherein the encoder is composed of n stacked encoder layers, each layer of which is composed of a multi-dimensional feature extraction module and a separable convolution.

[0029] The initial feature map is input into an encoder consisting of a multi-dimensional feature extraction module and separable convolution. The encoder consists of n stacked encoder layers (n=2), each of which consists of a multi-dimensional feature extraction module and a separable convolution. There are two branches in the multi-dimensional feature extraction module, namely the Transformer branch and the Mamba branch.

[0030] The Transformer branch consists of a channel multi-head self-attention module and a feedforward network. Assuming the input is X, its output is defined as follows:

[0031] in, represents the element-wise addition operation. MSA() represents the channel multi-head self-attention layer and FN() represents the feed-forward network.

[0032] The Mamba branch consists of a normalization layer, a linear mapping layer, a depth-wise separable convolution, and a 2D selective scanning module. Assuming the input is X, the output is defined as follows:

[0033] Where Lin() represents the linear mapping layer, LN() represents layer normalization, SS2D() represents the two-dimensional selective scanning module, SiLU() represents the SiLU activation function, DWConv() represents the depthwise separable convolution, and ⊙ represents the Hadamard product.

[0034] VisionTransformer in the Transformer branch and VisionMamba in the Mamba branch are used to extract channel features and spatial features and fuse them. and spatial characteristics After that, we first use the concat operation to connect the two dimensional features in the channel dimension, and then use a 1×1 convolution layer to halve the number of feature map channels to obtain the fusion of the two ( + ) after the feature map. The two are fused ( + The feature map after ) can be input into the separable convolution for downsampling, that is, the number of channels of the fused feature map increases and the size becomes smaller, so as to obtain feature maps of different levels. The fused feature map is input into the jump connection to connect the corresponding layers of the encoder and decoder to reduce the information loss caused by downsampling.

[0035] In step S13, the initial feature map is processed using an encoder composed of a multi-dimensional feature extraction module and a separable convolution, specifically: S31. Use the VisionTransformer in the multi-dimensional extraction module to extract the channel dimension features in the initial feature map, where the VisionTransformer consists of a channel multi-head self-attention module and a feedforward network.

[0036] S32, extracting spatial dimension features in the initial feature map using VisionMamba in the multi-dimensional extraction module. The VisionMamba is composed of a normalization layer, a linear mapping layer, a depth-separable convolution and a two-dimensional selective scanning module.

[0037] S33, fusing the spatial dimension features and channel dimension features of the initial feature map in steps S31 and S32 to obtain a fused feature map.

[0038] S34. Input the fused feature maps into separable convolution and jump connection respectively to obtain feature maps of different levels.

[0039] In step S31, the channel dimension features in the initial feature map are extracted using VisionTransformer, specifically including: S41. By calculating the multi-head self-attention and feedforward neural network for different channels of the initial feature map, the expression is:

[0040] in, is the feature map input into VisionTransformer, represents the element-wise addition operation, MSA() represents the channel multi-head self-attention layer and FN() represents the feed-forward network.

[0041] S42, capture the global dependencies between different channels and aggregate cross-channel information to obtain a hyperspectral encoding feature map in the channel dimension.

[0042] S43, the calculation expression of the multi-head self-attention is:

[0043] in , , They are Query, Key, and Value in the self-attention mechanism. is the feature map input into VisionTransformer, , , are three learnable matrices, and , and Multiply to get the corresponding , , , and then along the channel dimension , , Decompose into N heads; is the Key of the jth head, T is the matrix transpose operation, is the j-th head Query, is the result of the j-th head matrix calculation, is the value of the jth head. Since the spectral density varies significantly with wavelength, a learnable parameter is used , to weight and Matrix calculation between to achieve attention Finally, the results of N head matrix calculations are connected and position coding is added to obtain the hyperspectral coding feature map of the channel dimension.

[0044] In step S32, VisionMamba is used to extract the spatial dimension features in the initial feature map, specifically including: S51. Focusing on different parts of the initial feature map through the selective scanning state space sequence model, aggregating information across image patches. The selective scanning state space sequence model enables each element in the sequence to interact with any previously scanned sample, giving the network a good global receptive field.

[0045] S52, processing the image block embedding through the cross scanning mechanism, flattening the two-dimensional image into a one-dimensional sequence and scanning along four different directions to capture the global dependencies in the initial feature map. Using the cross scanning mechanism, the two-dimensional image is flattened into a one-dimensional sequence and scanned along four different directions, thereby effectively capturing the global context of the initial feature map.

[0046] S53, merging all one-dimensional sequences and restoring them to the original two-dimensional structure to obtain a hyperspectral coding feature map of the spatial dimension. Finally, merging all sequences and restoring them to the original two-dimensional structure to obtain a hyperspectral coding feature map of the spatial dimension.

[0047] In step S34, the fused feature maps are respectively input into separable convolution and skip connection to obtain feature maps of different levels, specifically including: S61. Use the concat operation to connect and fuse the spatial dimension features and the channel dimension features in the channel dimension, and use a 1×1 convolution layer to halve the number of channels of the fused feature map.

[0048] S62. Input the fused feature map into a separable convolution block, use the separable convolution to double the number of channels of the fused feature map, and reduce the length and width of the fused feature map by half, so as to achieve downsampling and obtain feature maps of different levels.

[0049] S63. Input the fused feature map to the skip connection for fusion with the feature map in the corresponding layer of the decoder, thereby reducing the information loss caused by downsampling.

[0050] S14, input the feature maps of different levels in step S13 into the decoder for image reconstruction to obtain a final feature map with N channels, wherein the decoder is composed of n stacked encoders, and each layer is composed of a multi-dimensional feature extraction module and deconvolution. First, deconvolution is used to upsample the feature map to restore the feature map, that is, the number of channels of the feature map is reduced and the size is increased. Then, the multi-dimensional feature extraction module is used to further extract the spatial dimension features and channel dimension features. Finally, the two dimensional features are fused and fused with the feature map in the jump connection. After passing through multiple deconvolution and multi-dimensional feature extraction modules in the decoder, the final feature map with N channels is obtained.

[0051] In step S14, the feature maps of different levels are input into the decoder for image reconstruction to obtain a final feature map with N channels, which specifically includes: S71. Use deconvolution to perform upsampling to restore the size and number of channels of feature maps at different levels.

[0052] S72: Fuse the restored feature maps of different levels with the feature maps fused in the corresponding jump connection.

[0053] S73. Input the result outputted from step S72 into a multi-dimensional feature extraction module for further extracting channel features and spatial features of the image.

[0054] S74. Repeat steps S71 to S73 in the decoder, reconstruct the output feature map to the same size as before inputting the encoder, and obtain a final feature map with N channels.

[0055] S15, using a convolutional neural network to map the final feature map in step S14, and add it to the initial feature map before inputting the encoder to obtain a compressed hyperspectral image. The compressed hyperspectral image is an image with N channels, where each channel represents a wavelength.

[0056] S16, constructing a reconstructed hyperspectral model, splitting the compressed hyperspectral image in step S15 according to the number of channels and performing color mapping to obtain a reconstructed hyperspectral image. Constructing a reconstructed hyperspectral model, splitting the compressed hyperspectral image according to the number of channels, and then performing color mapping to finally obtain a reconstructed hyperspectral image, the reconstructed hyperspectral image has a total of N images, including N wavelengths.

[0057] The construction and reconstruction of the hyperspectral model is specifically as follows: S81. Obtain training data, wherein the training data is 1000 RGB image-hyperspectral image pairs. The training data is a data set provided by the NTIRE2022 spectral reconstruction challenge, which contains 1000 RGB image-hyperspectral image pairs. The data set is divided into a training set, a validation set, and a test set in a ratio of 18:1:1.

[0058] S82, constructing a training model of the encoder, decoder, and front and rear convolutional layers, inputting the training set in step S82 into the training model, and training using a mean square error loss function, wherein the expression of the mean square error loss function is:

[0059] Where n is the number of pixels in the image, is the original hyperspectral image, To reconstruct hyperspectral images; S83. After each iteration, the validation set is used for validation, and the trained training model is determined as the hyperspectral reconstruction model.

[0060] A second aspect of the present embodiment provides a system for reconstructing a hyperspectral spectrum from an RGB image, the system comprising: Image acquisition module: used to acquire three-channel RGB images; Image channel expansion module: used to embed and expand the channels of three-channel RGB images through convolutional neural networks, expand the number of channels of three-channel RGB images to the required number of hyperspectral wavelengths N, and obtain an initial feature map with N channels; Feature encoding module: used to input the initial feature map into the encoder to obtain feature maps of different levels; Skip connection module: used to connect feature maps of different levels in the encoder to the decoder to reduce information loss caused by downsampling; Image decoding and reconstruction module: used to input feature maps of different levels into the decoder for image reconstruction to obtain the final feature map with N channels; Hyperspectral mapping module: used to map the final feature map using a convolutional neural network and add it to the initial feature map before inputting the encoder to obtain a compressed hyperspectral image; Hyperspectral splitting module: used to build a reconstructed hyperspectral model, split and color map the compressed hyperspectral image according to the number of channels, and obtain the reconstructed hyperspectral image.

[0061] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for reconstructing a hyperspectral spectrum from an RGB image, characterized in that: The method comprises: S11, obtaining a three-channel RGB image; S12, using a convolutional neural network to embed and expand the channels of the three-channel RGB image, expanding the number of channels of the three-channel RGB image to the required number of hyperspectral wavelengths N, and obtaining an initial feature map with N channels; S13, inputting the initial feature map in step S12 into an encoder to obtain feature maps of different levels, wherein the encoder is composed of n stacked encoder layers, each layer is composed of a multi-dimensional feature extraction module and a separable convolution; S14, inputting the feature maps of different levels in step S13 into the decoder for image reconstruction to obtain a final feature map with N channels, wherein the decoder is composed of n stacked encoders, and each layer is composed of a multi-dimensional feature extraction module and deconvolution; S15, mapping the final feature map in step S14 using a convolutional neural network, and adding it to the initial feature map before inputting the encoder to obtain a hyperspectral image in a compressed form; S16, constructing a reconstructed hyperspectral model, splitting and color mapping the compressed hyperspectral image in step S15 according to the number of channels, and obtaining a reconstructed hyperspectral image.

2. The method for reconstructing a hyperspectral spectrum from an RGB image according to claim 1, characterized in that: In step S12, channel expansion is performed on the three-channel RGB image to obtain an initial feature map with N channels, which specifically includes: S21, linearly scaling the image to [0, 1] to achieve data normalization; S22, using a 3×3 convolutional layer to expand the number of channels of the image from 3 to N channels; S23. Obtain an initial feature map with N channels.

3. The method for reconstructing a hyperspectral spectrum from an RGB image according to claim 1, characterized in that: In step S13, the initial feature map is processed using an encoder composed of a multi-dimensional feature extraction module and a separable convolution, specifically: S31, using VisionTransformer in the multi-dimensional extraction module to extract channel dimension features in the initial feature map; S32, using VisionMamba in the multi-dimensional extraction module to extract spatial dimension features in the initial feature map; S33, fusing the spatial dimension features and channel dimension features of the initial feature map in steps S31 and S32 to obtain a fused feature map; S34. Input the fused feature maps into separable convolution and jump connection respectively to obtain feature maps of different levels.

4. The method for reconstructing a hyperspectral spectrum from an RGB image according to claim 3, characterized in that: The VisionTransformer consists of a channel multi-head self-attention module and a feed-forward network, and the VisionMamba consists of a normalization layer, a linear mapping layer, a depth-wise separable convolution, and a two-dimensional selective scanning module.

5. The method for reconstructing a hyperspectral spectrum from an RGB image according to claim 4, characterized in that: In step S31, the channel dimension features in the initial feature map are extracted using VisionTransformer, specifically including: S41. By calculating the multi-head self-attention and feedforward neural network for different channels of the initial feature map, the expression is: in, is the feature map input into VisionTransformer, represents the element-wise addition operation, MSA() represents the channel multi-head self-attention layer and FN() represents the feed-forward network; S42, capturing the global dependency between different channels and aggregating cross-channel information to obtain a hyperspectral encoding feature map in the channel dimension; S43, the calculation expression of the multi-head self-attention is: in , , They are Query, Key, and Value in the self-attention mechanism. is the feature map input into VisionTransformer, , , are three learnable matrices, and , and Multiply to get the corresponding , , , and then along the channel dimension , , Decompose into N heads; is the Key of the jth head, T is the matrix transpose operation, is the j-th head Query, is the result of the j-th head matrix calculation, is the value of the jth head. Since the spectral density varies significantly with wavelength, a learnable parameter is used , to weight and Matrix calculation between to achieve attention Finally, the results of N head matrix calculations are connected and position coding is added to obtain the hyperspectral coding feature map of the channel dimension.

6. The method for reconstructing a hyperspectral spectrum from an RGB image according to claim 4, characterized in that: In step S32, VisionMamba is used to extract the spatial dimension features in the initial feature map, specifically including: S51, by selectively scanning the state space sequence model to focus on different parts of the initial feature map, aggregating information across image patches; S52, processing image block embedding through a cross-scanning mechanism, flattening the two-dimensional image into a one-dimensional sequence and scanning along four different directions to capture the global dependency in the initial feature map; S53, merging all one-dimensional sequences and restoring them to the original two-dimensional structure to obtain a hyperspectral encoding feature map of the spatial dimension.

7. The method for reconstructing a hyperspectral spectrum from an RGB image according to claim 3, characterized in that: In step S34, the fused feature maps are respectively input into separable convolution and skip connection to obtain feature maps of different levels, specifically including: S61, use the concat operation to connect and fuse the spatial dimension features and the channel dimension features in the channel dimension, and use a 1×1 convolution layer to halve the number of channels of the fused feature map; S62, inputting the fused feature map into a separable convolution block, using the separable convolution to double the number of channels of the fused feature map, and reducing the length and width of the fused feature map by half, to obtain feature maps of different levels; S63. Input the fused feature map to the skip connection for fusion with the feature map in the corresponding layer of the decoder.

8. The method for reconstructing a hyperspectral spectrum from an RGB image according to claim 1, characterized in that: In step S14, the feature maps of different levels are input into the decoder for image reconstruction to obtain a final feature map with N channels, which specifically includes: S71, using deconvolution to perform upsampling to restore the size and number of channels of feature maps at different levels; S72, fusing the restored feature maps of different levels with the feature maps fused in the corresponding jump connection; S73, inputting the result outputted from step S72 into a multi-dimensional feature extraction module for further extracting channel features and spatial features of the image; S74. Repeat steps S71 to S73 in the decoder, reconstruct the output feature map to the same size as before inputting the encoder, and obtain a final feature map with N channels.

9. The method for reconstructing a hyperspectral spectrum from an RGB image according to claim 1, characterized in that: The construction and reconstruction of the hyperspectral model is specifically as follows: S81, obtaining training data, wherein the training data is a data set of 1000 RGB image-hyperspectral image pairs, and dividing the data set into a training set, a validation set, and a test set; S82, constructing a training model of the encoder, decoder, and front and back convolutional layers, inputting the training set in step S81 into the training model, and training using a mean square error loss function, the expression of the mean square error loss function is: Where n is the number of pixels in the image, is the original hyperspectral image, To reconstruct hyperspectral images; S83. After each iteration, the validation set is used for validation, and the trained training model is determined as the hyperspectral reconstruction model.

10. A system for reconstructing a hyperspectral spectrum from an RGB image, characterized in that: The system comprises: Image acquisition module: used to acquire three-channel RGB images; Image channel expansion module: used to embed and expand the channels of three-channel RGB images through convolutional neural networks, expand the number of channels of three-channel RGB images to the required number of hyperspectral wavelengths N, and obtain an initial feature map with N channels; Feature encoding module: used to input the initial feature map into the encoder to obtain feature maps of different levels; Skip connection module: used to connect feature maps of different levels in the encoder to the decoder to reduce information loss caused by downsampling; Image decoding and reconstruction module: used to input feature maps of different levels into the decoder for image reconstruction to obtain the final feature map with N channels; Hyperspectral mapping module: used to map the final feature map using a convolutional neural network and add it to the initial feature map before inputting the encoder to obtain a compressed hyperspectral image; Hyperspectral splitting module: used to build a reconstructed hyperspectral model, split and color map the compressed hyperspectral image according to the number of channels, and obtain the reconstructed hyperspectral image.

Images