An end-side intelligent computing spectral imaging system and method for planetary exploration

Abstract

The application discloses a kind of end side intelligent computing spectral imaging systems and methods for planetary exploration, system includes RGB image input layer, feature embedding layer, encoder module, bottleneck feature enhancement module, decoder module and output mapping layer;Encoder and decoder are all by cascaded dual-domain collaborative processing module and channel pruning module constitute;First, RGB image is mapped as high-dimensional feature, multi-scale feature is extracted layer by layer and information compression is carried out in the encoding stage, deep feature fusion and enhancement are carried out in the bottleneck stage, spatial resolution is gradually recovered and shallow information is fused in the decoding stage, finally output hyperspectral data cube, realize from low-dimensional RGB to high-dimensional spectral space mapping;The application meets the strict restrictions of volume, power consumption and computing resources of planetary exploration platform under the condition, realizes high-precision, high-efficiency spectral information reconstruction, to improve the ability of planetary surface detection and scientific analysis.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral image processing technology, and more specifically to an edge-side intelligent computational spectral imaging system for planetary exploration.

[0002] This invention also relates to an end-side intelligent computational spectral imaging method for planetary exploration. Background Technology

[0003] In planetary exploration missions, a detailed analysis of the composition of planetary surface materials, mineral distribution, and geological structure is one of the important scientific objectives. Compared with traditional RGB imaging, hyperspectral imaging can acquire spectral information of target objects in a continuous and dense spectral band range. Each pixel contains complete spectral features, thereby enabling more accurate material identification and environmental analysis. It has important application value in planetary science exploration, remote sensing analysis, and resource exploration.

[0004] However, traditional hyperspectral imaging systems typically rely on pushbroom or swing-broom scanning methods to acquire a complete cube of spectral data by scanning line by line or point by point. These systems often require complex optical structures and sophisticated mechanical scanning devices, demanding high platform stability, and suffer from slow acquisition speeds, large equipment size, and high power consumption. For resource-constrained planetary exploration platforms, payload mass, size, and energy are all strictly limited, making it difficult to directly mount traditional hyperspectral imaging equipment.

[0005] To address the low efficiency of traditional scanning hyperspectral imaging systems, researchers have proposed snapshot hyperspectral imaging technology. This technology can acquire complete spectral data in a single exposure, offering higher acquisition efficiency compared to scanning systems. However, such systems typically involve a trade-off between spatial and spectral resolution, and their high hardware costs and complex system structure still make them difficult to meet the demands of miniaturized, low-power planetary exploration platforms.

[0006] Therefore, a new research direction has emerged in recent years: reconstructing hyperspectral images from ordinary RGB images. This method leverages the advantages of RGB sensors, such as low cost, small size, and fast acquisition speed, to recover high-dimensional spectral data from low-dimensional RGB information through algorithms. This allows for the acquisition of hyperspectral information without the need for complex optical systems, providing a promising solution for lightweight remote sensing and planetary exploration.

[0007] The existing technical solutions are as follows: In computational spectral reconstruction research from RGB to hyperspectral images, existing techniques have evolved from traditional methods to deep learning methods. Early studies mainly employed traditional methods such as sparse representation and dictionary learning, constructing a spectral dictionary and solving for sparse coefficients using RGB images to reconstruct hyperspectral data. These methods rely on manually designed prior features, which have limited expressive power, weak adaptability to complex scenes, and certain deficiencies in reconstruction accuracy and generalization ability.

[0008] With the development of deep learning technology, researchers have proposed spectral reconstruction methods based on convolutional neural networks, establishing a nonlinear mapping relationship between RGB images and hyperspectral data through end-to-end training. Subsequently, research further introduced residual structures, multi-scale feature fusion, and attention mechanisms to enhance the model's ability to model spatial structure and spectral correlations. In recent years, spectral reconstruction methods based on the Transformer architecture have also emerged, further improving reconstruction performance through global feature modeling and long-distance dependency modeling. However, the aforementioned high-precision spectral reconstruction models typically rely on deep network structures and a large number of parameters, making real-time operation difficult on embedded platforms used in planetary exploration.

[0009] To address the computational complexity issue, some studies have begun exploring lightweight spectral reconstruction models. Some methods improve attention mechanisms to reduce computational overhead while maintaining a certain level of reconstruction accuracy. Furthermore, some research has attempted to introduce structures such as state-space models to reduce complexity and improve efficiency. Although these methods have improved computational efficiency to some extent, they still have shortcomings, such as insufficient utilization of the frequency domain structural features of hyperspectral data or channel redundancy during multi-scale feature fusion, thus affecting the balance between spectral reconstruction accuracy and efficiency.

[0010] The disadvantages of existing technologies are as follows: (1) The system hardware is complex, costly and difficult to deploy in a lightweight manner. Traditional hyperspectral imaging systems have complex optical imaging structures, large equipment size, heavy weight, and high power consumption. They also have strict requirements for platform stability, making them difficult to apply directly to scenarios such as planetary exploration that are highly sensitive to payload and energy consumption.

[0011] (2) High-precision computational spectral imaging methods have high computational overhead and poor real-time performance. Existing high-precision computational spectral imaging methods based on deep learning typically employ complex models such as deep network structures, attention mechanisms, or Transformers. While these methods can improve reconstruction accuracy, they generally suffer from problems such as large parameter scale, high computational cost, and slow inference speed, making it difficult to achieve real-time operation on embedded or edge computing platforms.

[0012] (3) The lightweight method lacks precision and it is difficult to balance efficiency and spectral fidelity. Some lightweight computational spectral imaging methods reduce computational complexity by simplifying network structures, but they often ignore the key characteristics of hyperspectral data in terms of frequency domain information and channel redundancy, resulting in insufficient reconstruction results in terms of spectral consistency and detail representation, making it difficult to achieve a balance between high accuracy and high efficiency.

[0013] In summary, while existing technologies can reconstruct RGB to hyperspectral data, they generally suffer from high model complexity, difficulty in deployment on resource-constrained platforms, or insufficient accuracy in the reconstruction results of lightweight models. Therefore, there is an urgent need to propose a lightweight computational spectral imaging method that is more suitable for resource-constrained platforms such as planetary exploration. Summary of the Invention

[0014] The purpose of this invention is to address the aforementioned problems by providing an edge-side intelligent computational spectral imaging system and method for planetary exploration. This system achieves high-precision and high-efficiency spectral information reconstruction while meeting the strict limitations of planetary exploration platforms on size, power consumption, and computational resources. This enhances the capabilities of planetary surface exploration and scientific analysis, and solves the problem of existing technologies struggling to balance lightweight design and high performance.

[0015] The technical solution adopted in this invention is as follows: An edge-side intelligent computational spectral imaging system for planetary exploration, the system comprising: Image receiving module, used to receive RGB images; The feature embedding layer module is used to map the RGB image into high-dimensional features; The encoder module is used to extract multi-scale features from the mapped RGB image and compress the information. The bottleneck feature enhancement module is used to receive the data output by the encoder module and perform deep feature fusion and enhancement. The decoder module receives the fused and enhanced features and recovers the spatial resolution fusion shallow layer information, and outputs hyperspectral image data through the output mapping layer module.

[0016] Furthermore, both the encoder module and the decoder module include a cascaded dual-domain collaborative processing module and a channel pruning module; The dual-domain collaborative processing module is used to enhance the input features, wherein local structural information is extracted in the spatial domain and global features are modeled and enhanced in the frequency domain. The dual-domain collaborative processing module first performs pre-convolutional layer and batch normalization processing unit to perform preliminary feature shaping and distribution normalization on the input features, so as to improve the stability and expressive power of subsequent feature extraction. Furthermore, the dual-domain collaborative processing module adopts a dual-branch design, with one branch used for spatial domain feature extraction. It captures local structure and contextual information through multi-scale convolution operations. The multi-scale convolution operations include combinations of dilated convolution kernels with different dilation rates, and combine batch normalization and activation functions for nonlinear mapping to improve the ability to model complex spatial textures. Another branch maps features to the frequency domain, selectively optimizing different frequency components through frequency domain transformation each time to achieve progressive optimization. The frequency domain transformation uses Discrete Fast Fourier Transform and combines spectral banding and mask selection in the frequency domain to optimize special frequency components. After frequency domain processing, the features are restored to the spatial domain through inverse transformation, and spatial feature enhancement is performed by combining convolution operations and skip connections.

[0017] After completing the dual-branch feature extraction, the two are concatenated, and then channel attention is used to enhance the channels, full-scale depth convolution operation, and batch normalization and activation function to further model the fused features; and weighted skip connections are combined to enhance cross-scale information interaction capabilities and improve feature expression accuracy.

[0018] The channel pruning module is used to evaluate and filter the importance of feature channels and remove redundant features.

[0019] Furthermore, the channel pruning module includes a multi-angle pooling analysis unit, a correlation position encoding unit, and a channel saliency score calculation unit, which are used to extract global information from the feature map, obtain statistical features and perform correlation position encoding, then calculate the saliency score of each feature channel, and adaptively retain the feature channels according to the score and remove redundant information.

[0020] The multi-angle pooling analysis unit includes global average pooling and global max pooling operations; the correlation position encoding unit is used to model the correlation relationship between channels; the channel saliency score calculation unit generates channel weights through a fully connected layer and outputs a saliency score in combination with a normalization function.

[0021] This invention also provides an edge-side intelligent computational spectral imaging method for planetary exploration, employing an edge-side intelligent computational spectral imaging system for planetary exploration, the method comprising: Feature embedding: Receives an RGB image and maps the low-dimensional three-channel image into a high-dimensional feature representation; Feature encoding involves taking the mapped high-dimensional features and inputting them, and then using an encoder module with multiple stacked coding units to perform multi-scale feature extraction and information compression. Bottleneck feature enhancement involves globally modeling and enhancing deep features after feature encoding is completed. Feature decoding involves receiving fused and enhanced features through a decoder module with stacked multi-layer decoding units and restoring shallow information of spatial resolution fusion. The output mapping layer module outputs a complete hyperspectral image cube, achieving efficient reconstruction from RGB image to hyperspectral image.

[0022] Furthermore, in the feature encoding, in each encoding unit, the input features are first enhanced by a dual-domain collaborative processing module, which extracts local structural information in the spatial domain and models and enhances global features in the frequency domain. Then, the channel pruning module evaluates and filters the feature channels to remove redundant features.

[0023] Furthermore, in the bottleneck feature enhancement, a dual-domain collaborative processing module is used to perform global modeling and enhancement of deep features.

[0024] Furthermore, in the feature decoding, in each decoding unit, the features of the corresponding layer in the encoding stage are first fused by skip connections to supplement spatial detail information and alleviate information loss; then, the dimensionality of the fused features is compressed by the channel pruning module to control the model complexity; then, the features are further optimized and reconstructed by the dual-domain collaborative processing module. In the last decoding unit, the channel pruning operation is no longer performed, and the decoded high-dimensional features are converted into target hyperspectral image data through the output mapping layer.

[0025] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. Significantly reduces model complexity and computational overhead, improving deployment efficiency. To address the issues of large parameter count and high computational complexity in existing high-precision computational spectral imaging models represented by Transformer or deep networks, this invention introduces a dual-domain collaborative processing and channel compression mechanism to significantly reduce the number of model parameters and computational load while ensuring reconstruction performance. This enables efficient operation on resource-constrained platforms such as planetary exploration and provides stronger engineering deployment capabilities.

[0026] 2. Achieving higher spectral reconstruction accuracy under lightweight conditions Unlike traditional lightweight methods that suffer from decreased accuracy, this invention employs a dual-domain collaborative processing mechanism to finely optimize features in the frequency domain and efficiently fuse them with features in the spatial domain, thereby effectively improving the ability to restore reconstruction details. This makes the reconstruction results closer to the real data in terms of spatial structure and spectral curves, thus improving the overall reconstruction stability and accuracy. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the structure of an end-side intelligent computational spectral imaging system for planetary exploration according to the present invention; Figure 2 This is a flowchart illustrating an edge-side intelligent computational spectral imaging method for planetary exploration according to the present invention. Figure 3 This is a test result diagram of an end-side intelligent computational spectral imaging method for planetary exploration according to the present invention. Detailed Implementation

[0028] The present invention will now be described in detail with reference to the accompanying drawings.

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0030] Example This embodiment provides an edge-side intelligent computational spectral imaging system for planetary exploration, such as... Figure 1 As shown, the system adopts an encoder-decoder structure, which includes an RGB image input layer, a feature embedding layer, an encoder module, a bottleneck feature enhancement module, a decoder module, and an output mapping layer.

[0031] The encoder and decoder are both composed of cascaded dual-domain collaborative processing modules and channel pruning modules. In this embodiment, the system first maps the RGB image into high-dimensional features through a feature embedding layer. The encoder module extracts multi-scale features layer by layer and compresses information during the encoding stage. The bottleneck feature enhancement module performs deep feature fusion and enhancement at the bottleneck stage. The decoder module and the output mapping layer gradually restore the spatial resolution and fuse shallow information during the decoding stage. Finally, a hyperspectral data cube is output, realizing the mapping from low-dimensional RGB to high-dimensional spectral space.

[0032] In this embodiment, the dual-domain collaborative processing module serves as the main processing unit of the system. Its structure employs a dual-branch design: one branch extracts spatial domain features by capturing local structure and contextual information through multi-scale convolution operations; the other branch maps features to the frequency domain, decomposing and selectively optimizing different frequency components through frequency domain transformation. Its working principle leverages the frequency domain's ability to express global structural information, finely modulating spectral details, and then mapping the enhanced features back to the spatial domain through inverse transformation. This is then efficiently fused with the spatial branch features, thereby achieving collaborative optimization of spatial and frequency domain information and improving the accuracy and stability of spectral reconstruction.

[0033] To address the issue of high model complexity, this embodiment incorporates a channel pruning module. Structurally, this module includes a feature multi-angle pooling analysis unit, a correlation location encoding unit, and a channel saliency score calculation unit. Its working principle is as follows: First, global information is extracted from the feature map to obtain statistical features and perform correlation location encoding. Then, the saliency score for each channel is calculated. Finally, channels are adaptively retained based on their scores, removing redundant information and retaining only key features that significantly contribute to the computational spectral imaging. This approach significantly reduces the number of parameters and computational load while ensuring that the model's expressive power is not significantly weakened, thus achieving a balance between lightweight design and high performance.

[0034] This embodiment also provides an edge-side intelligent computational spectral imaging method for planetary exploration, such as... Figure 2 As shown, the details are as follows: First, the system receives an RGB image input; Feature embedding maps a low-dimensional three-channel image into a high-dimensional feature representation through a feature embedding layer, providing a basic feature expression for subsequent computation of spectral imaging; Feature encoding begins after feature embedding. During feature encoding, the encoder is stacked in an N-layer structure to achieve progressive abstraction and compression of features. In each encoding unit, the input features are first enhanced by a dual-domain collaborative processing module. On the one hand, local structural information is extracted in the spatial domain, and on the other hand, global features are modeled and enhanced in the frequency domain, thereby improving the ability of features to express spectral information. Subsequently, the channel pruning module evaluates and filters the importance of feature channels, removes redundant features, and reduces computational complexity. In this embodiment, the progressive frequency domain optimization module performs segmented processing and importance screening of features in the frequency domain, strengthens key frequency components, and combines inverse transform to restore to the spatial domain, thereby achieving fine optimization of the frequency domain structure and improving the consistency and detail expression ability in the reconstruction process. Bottleneck feature enhancement: After completing the encoding stage, the system enters the bottleneck stage. In this stage, a further dual-domain collaborative processing module is used to globally model and enhance deep features in order to improve the overall spectral reconstruction capability and the stability of feature expression. Feature decoding, after bottleneck feature enhancement, enters the decoding stage, and its overall process is relatively symmetrical to the encoding stage. In each decoding unit, features from the corresponding layer of the encoding stage are first fused through skip connections to supplement spatial detail information and mitigate information loss. Then, the dimensionality of the fused features is compressed by the channel pruning module to control model complexity. After that, the features are further optimized and reconstructed by the dual-domain collaborative processing module. The decoding process is stacked in an M-layer structure, but in the last decoding unit, channel pruning is not performed to ensure that the output mapping stage has sufficient feature information. Finally, the decoded high-dimensional features are converted into target hyperspectral data through the output mapping layer, and a complete hyperspectral image cube is output, realizing efficient reconstruction from RGB image to hyperspectral image.

[0035] In summary, the intelligent computational spectral imaging system and method proposed in this invention for planetary exploration, by utilizing RGB imaging combined with efficient computational spectral imaging algorithms, significantly reduces the system's dependence on hardware size, power consumption, and computing resources while ensuring the accuracy of computational spectral imaging, thus possessing significant engineering application value. In the field of planetary exploration, this method can replace traditional hyperspectral imaging equipment, enabling rovers to achieve detailed analysis of planetary surface mineral composition, water ice distribution, and environmental characteristics under limited payload and computing power conditions, and supports real-time or near-real-time processing on embedded platforms, thereby significantly improving the scientific exploration capabilities and mission efficiency of deep space exploration missions.

[0036] Meanwhile, this invention also has broad application prospects in resource-constrained scenarios such as UAV remote sensing, environmental monitoring, and industrial inspection. By replacing complex optical systems with algorithms, hyperspectral information acquisition can be achieved on the basis of existing RGB imaging equipment, reducing system costs and improving data utilization value. In addition, this method can promote the development of hyperspectral technology towards lightweighting, miniaturization, and popularization, making it easier to integrate into various smart devices and edge computing platforms, and has good industrial promotion potential and application prospects.

[0037] The method proposed in this invention was tested on the CAVE open-source dataset for spectral reconstruction, and the error map results are as follows: Figure 3 As shown: Intuitively, compared to other methods, the error map corresponding to this method is darker and bluer, which means that the root mean square error (RMSE) of the reconstruction result of this method is lower.

[0038] This article uses specific embodiments to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. An edge-side intelligent computational spectral imaging system for planetary exploration, characterized in that, The system includes: Image receiving module, used to receive RGB images; The feature embedding layer module is used to map low-dimensional three-channel RGB images into high-dimensional feature representations; The encoder module enters the encoding stage after passing through the feature embedding layer module. During the feature encoding process, the encoder is stacked in an N-layer structure to complete the progressive abstraction and compression of features. In each encoding unit, the input features are first enhanced by the dual-domain collaborative processing module, and then the importance of the feature channels is evaluated and filtered by the channel pruning module. The bottleneck feature enhancement module is used to receive the data output by the encoder module and perform deep feature fusion and enhancement. The decoder module, after bottleneck feature enhancement, enters the decoding stage, and its overall process is symmetrical to that of the encoding module. In each decoding unit, features from the corresponding layer of the encoding stage are first fused through skip connections. Then, the dimensionality of the fused features is compressed by the channel pruning module, and the features are further optimized and reconstructed by the dual-domain collaborative processing module. The decoding process is stacked in an M-layer structure. In the last decoding unit, channel pruning is no longer performed. Finally, the decoded high-dimensional features are converted into target hyperspectral data through the output mapping layer, and the complete hyperspectral image cube is output, completing the reconstruction of the hyperspectral image.

2. The end-side intelligent computational spectral imaging system for planetary exploration according to claim 1, characterized in that, Both the encoder module and the decoder module include a cascaded dual-domain collaborative processing module and a channel pruning module. The dual-domain collaborative processing module is used to enhance the input features, wherein local structural information is extracted in the spatial domain and global features are modeled and enhanced in the frequency domain. The channel pruning module is used to evaluate and filter the importance of feature channels and remove redundant features.

3. The edge-side intelligent computational spectral imaging system for planetary exploration according to claim 2, characterized in that, The dual-domain collaborative processing module adopts a dual-branch design. One branch is used for spatial domain feature extraction, which captures local structure and context information through multi-scale convolution operations. The other branch maps the features to the frequency domain, and decomposes and selectively optimizes different frequency components through frequency domain transformation.

4. The edge-side intelligent computational spectral imaging system for planetary exploration according to claim 2, characterized in that, The channel pruning module includes a feature multi-angle pooling analysis unit, a correlation position encoding unit, and a channel saliency score calculation unit. It is used to extract global information from the feature map, obtain statistical features and perform correlation position encoding, obtain the saliency score of each feature channel, and adaptively retain feature channels according to the score and remove redundant information.

5. An edge-side intelligent computational spectral imaging method for planetary exploration, employing the edge-side intelligent computational spectral imaging system for planetary exploration as described in any one of claims 1 to 4, characterized in that, The method includes: Feature embedding: Receives an RGB image and maps the low-dimensional three-channel image into a high-dimensional feature representation; Feature encoding begins after passing through the feature embedding layer module and enters the encoding stage. During feature encoding, the encoder is stacked in an N-layer structure to complete the progressive abstraction and compression of features. In each encoding unit, the input features are first enhanced by the dual-domain collaborative processing module, and then the importance of the feature channels is evaluated and filtered by the channel pruning module. Bottleneck feature enhancement involves globally modeling and enhancing deep features after feature encoding is completed. Feature decoding, after bottleneck feature enhancement, enters the decoding stage. Its overall process is symmetrical to the encoding module. Specifically, it first fuses the features of the corresponding layer in the encoding stage through skip connections, then compresses the dimensionality of the fused features through the channel pruning module, and then further optimizes and reconstructs the features through the dual-domain collaborative processing module to achieve efficient reconstruction from RGB image to hyperspectral image.

6. The edge-side intelligent computational spectral imaging method for planetary exploration according to claim 5, characterized in that, In the feature encoding, at each encoding unit, the input features are first enhanced by a dual-domain collaborative processing module. On the one hand, local structural information is extracted in the spatial domain, and on the other hand, global features are modeled and enhanced in the frequency domain. Then, the importance of feature channels is evaluated and filtered by a channel pruning module to remove redundant features.

7. The edge-side intelligent computational spectral imaging method for planetary exploration according to claim 5, characterized in that, In the bottleneck feature enhancement, a dual-domain collaborative processing module is used to perform global modeling and enhancement of deep features.

8. The edge-side intelligent computational spectral imaging method for planetary exploration according to claim 5, characterized in that, The feature decoding process includes stacked M-layer decoding units. In each decoding unit, features from the corresponding layer of the encoding stage are first fused by skip connections to supplement spatial detail information and mitigate information loss. Then, the fused features are dimensionally compressed by a channel pruning module to control model complexity. After that, the features are further optimized and reconstructed by a dual-domain collaborative processing module. In the last decoding unit, channel pruning is no longer performed, and the decoded high-dimensional features are converted into target hyperspectral image data through an output mapping layer.

Images