A dual-mode manifold hyperspectral imaging system and method for deep space exploration

Abstract

The application discloses a dual-mode manifold hyperspectral imaging system and method for deep space exploration, wherein a data acquisition module is used for acquiring planetary surface compressed observation data, an encoding mask, an initial hyperspectral estimation and a high-resolution RGB image; a linear manifold projection module receives the planetary surface compressed observation data, the encoding mask and the initial hyperspectral estimation value and outputs intermediate hyperspectral features; an RGB multi-attention feature extraction module receives the high-resolution RGB image and outputs a multi-scale RGB feature pyramid; a guided spectral space module receives the intermediate hyperspectral features and the multi-scale RGB feature pyramid for depth regularization and outputs updated hyperspectral estimation values, and final planetary hyperspectral data is obtained through output mapping; and the application solves the problems of spatial blur, spectral aliasing and artifact residue in existing planetary hyperspectral compression imaging by constructing a deep multi-level cross-modal attention and multi-scale fusion mechanism.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a dual-mode manifold hyperspectral imaging system for deep space exploration.

[0002] This invention also relates to a dual-mode manifold hyperspectral imaging method for deep space exploration. Background Technology

[0003] In the field of planetary deep space exploration and remote sensing, acquiring high-resolution spatial topography and high-fidelity spectral information of planetary surfaces is crucial for mineral composition inversion, surface material identification, and geological evolution analysis. In recent years, deep neural networks have made breakthrough progress in image restoration, compressed sensing reconstruction, and spectral estimation, prompting the rapid development of multimodal fusion hyperspectral reconstruction technology. Compared with traditional optimization methods that rely on physical models or manual priors, data-driven deep learning systems can automatically learn feature associations and imaging degradation patterns, exhibiting superior anti-interference performance and scene transfer potential in real physical environments.

[0004] In traditional hyperspectral data acquisition schemes, single-camera coded aperture snapshot spectral imaging systems employ coded aperture and spectral dispersion structures to compress and map three-dimensional hyperspectral information onto a two-dimensional detector, thereby achieving efficient snapshot imaging. However, this two-dimensional coding and integral imaging mechanism inevitably leads to degradation problems such as spatial structure destruction and severe spectral aliasing. In extreme and complex application scenarios such as planetary exploration, hyperspectral compression imaging systems relying solely on a single mode face key challenges such as insufficient recovery of spatial details, texture blurring, and poor perception quality, making it difficult to accurately reconstruct complete geological details and spectral responses.

[0005] Complementing this, high-resolution RGB images possess fine textures, rich edges, and strong visual perceptual consistency. However, due to the extremely limited spectral bands acquired by RGB images, retrieving hyperspectral data solely from RGB images is prone to spectral distortion and material identification bias, failing to meet the requirements for accurate classification of planetary minerals. Therefore, to simultaneously meet the dual demands of planetary exploration for extremely high spatial resolution and spectral accuracy, the use of dual-camera systems to simultaneously acquire auxiliary high-resolution RGB images and CASSI compressed measurement data has become an important trend in this field. However, how to deeply integrate RGB and CASSI measurements, fully leverage their complementary advantages in spatial representation and spectral expression, and solve the problems of spatial information loss and spectral aliasing, is the core technical challenge in the current field of deep space hyperspectral remote sensing and exploration.

[0006] Currently, in the field of cross-modal fusion hyperspectral image reconstruction combining RGB images and the CASSI system, existing technical solutions are mainly divided into traditional prior-based methods and data-driven methods based on deep networks. However, when these existing solutions are applied to complex and extreme planetary surface exploration missions, they all reveal significant technical bottlenecks and limitations. Traditional methods mainly rely on manually designed prior knowledge for hyperspectral reconstruction. In practical applications, such methods often exhibit large-area error accumulation and clutter signals at the texture edges and high-frequency details of the image. This indicates that relying solely on manually designed priors is insufficient to fully decouple complex spectral aliasing. When faced with complex craters or rock texture variations on planetary surfaces, traditional methods cannot accurately recover spectral information and also lose a large amount of spatial details, producing irregular noise spots.

[0007] In recent years, a series of single-camera reconstruction schemes based on deep unfolded networks have emerged. Although these single-camera reconstruction methods have significantly reduced the overall error level, they struggle to decouple accurate spatial structures from a single compressed measurement when faced with extreme texture variations, leading to edge artifacts and local spectral distortion. At specific edge bands, these schemes still exhibit noticeable noise and local red artifacts in their reconstruction results. When processing fine and dense textures, these methods show significant spatial blurring, with lines in localized areas appearing broken or adhered, and the background exhibiting artifacts of varying degrees.

[0008] To compensate for the limitations of single cameras, existing technologies attempt to incorporate high-resolution RGB images as an auxiliary method. However, existing RGB and CASSI fusion methods typically employ a simplistic fusion mechanism. These existing schemes often use simple feature stitching or shallow summation strategies for modal fusion, lacking effective deep cross-modal interactions. Consequently, they struggle to fully utilize the high signal-to-noise ratio of RGB to suppress spectral aliasing. This shallow fusion is prone to losing details during feature propagation and pays insufficient attention to multi-scale degradation compensation, resulting in artifacts and spectral distortion in the reconstruction results. Furthermore, existing fusion techniques have significant shortcomings in multi-scale texture enhancement and spectral consistency maintenance, preventing the full realization of the structure-guided effect of RGB images. In high-fidelity spectral inversion tasks involving planetary materials, this can easily lead to distortion of the spectral features used for mineral composition identification.

[0009] The disadvantages of existing technologies are as follows: (1) Severe spectral aliasing and spatial degradation The two-dimensional coding and integral imaging mechanism of the traditional CASSI system inevitably destroys the spatial structure; in planetary exploration, it is difficult to reconstruct complete geological details and accurate spectra by relying solely on compressed observations, resulting in blurred surface morphology and low spatial resolution.

[0010] (2) The cross-modal fusion mechanism is simple and lacks interaction. Existing dual-camera fusion methods mostly employ simple feature stitching or shallow summation strategies, lacking effective deep cross-modal interaction. When faced with complex planetary surface environments, such shallow fusion cannot fully utilize the high signal-to-noise ratio of RGB images to effectively suppress spectral aliasing and channel interference, easily causing spectral distortion and material identification errors.

[0011] (3) Multi-scale features are easily lost and artifacts remain. Existing technologies are prone to losing details during feature propagation and do not pay enough attention to multi-scale degradation compensation. This results in poor multi-scale texture enhancement and spectral consistency maintenance in the final reconstructed planetary exploration data when faced with extreme lighting contrasts or complex textures, and obvious spatial artifacts and edge blurring still exist in the reconstruction results. Summary of the Invention

[0012] The purpose of this invention is to address the aforementioned problems by providing a dual-mode manifold hyperspectral imaging system and method for deep space exploration. By constructing a deep multi-level cross-modal attention and multi-scale fusion mechanism, this invention fully leverages the advantages of high-resolution RGB images in terms of space priors, effectively solving the problems of spatial blurring, spectral aliasing, and artifact residue in existing planetary hyperspectral compression imaging, and achieving high-quality planetary surface hyperspectral data inversion.

[0013] The technical solution adopted in this invention is as follows: A dual-mode manifold hyperspectral imaging system for deep space exploration, the system comprising: The data acquisition module is used to acquire compressed observation data of the planetary surface, a coding mask, and a high-resolution RGB image spatially aligned with the compressed observation data of the planetary surface, and to generate an initial hyperspectral estimate based on the compressed observation data of the planetary surface and the coding mask. The linear manifold projection module is used to receive compressed observation data of planetary surface, encoding mask and initial hyperspectral estimate from the data acquisition module, and update the initial hyperspectral estimate based on the forward physical projection residual, channel-normalized back projection and relaxation factor to ensure measurement consistency, and output intermediate hyperspectral features. The RGB multi-attention feature extraction module is used to receive high-resolution RGB images from the data acquisition module, extract global, spectral and texture information of the high-resolution RGB images, fuse them, and output a multi-scale RGB feature pyramid. The guided spectral space module is used to receive the intermediate hyperspectral features and the corresponding scale RGB guided features in the multi-scale RGB guided feature pyramid, perform cross-modal depth regularization, output the updated hyperspectral estimate, and obtain the final planetary hyperspectral data through the output mapping.

[0014] Furthermore, the data acquisition module includes an coded aperture snapshot spectral imaging unit and an RGB imaging unit. The coded aperture snapshot spectral imaging unit is used to acquire compressed observation data of the planetary surface and provide an coded mask, while the RGB imaging unit is used to acquire high-resolution RGB images.

[0015] Furthermore, in the linear manifold projection module, after receiving the compressed observation data of the planetary surface, the coded mask, and the initial hyperspectral estimate from the data acquisition module, a forward sensing matrix is ​​constructed based on the coded mask. Calculate the forward projection of the current hyperspectral estimate onto the observation domain. and the residual between it and the compressed observation data y The residuals are arranged according to Perform channel-wise normalization and then back-projection. Recast to the hyperspectral domain, combined with relaxation factors Update the current hyperspectral estimate to satisfy the measurement equation. The linear manifold projection outputs intermediate hyperspectral features with physical reliability.

[0016] Furthermore, the RGB multi-attention feature extraction module specifically includes a linear attention module, a spectral gradient attention module, and a spatial gradient attention module. The linear attention module, the spectral gradient attention module, and the spatial gradient attention module form a three-branch parallel structure to extract global, spectral, and texture information from high-resolution RGB images.

[0017] Furthermore, the linear attention module extracts the global long-range spatial dependence of the planetary surface within the global receptive field by approximating the rotational position using a kernel function; The spectral gradient attention module performs first-order difference on adjacent channels of the RGB embedded features to obtain channel gradient responses, and the spatial gradient attention module performs first-order difference on the RGB embedded features in the horizontal and vertical directions to obtain spatial gradient responses. Based on the channel gradient responses and spatial gradient responses, the high-frequency feature expression of edge texture and spectral evolution trends is enhanced to obtain a multi-scale RGB guided feature pyramid.

[0018] Furthermore, the guided spectral spatial module specifically adopts a dual-stream input architecture and cascades the guided spectral module and the convolutional spatial module; Hyperspectral features and RGB features are initially aligned and fused in the spatial dimension through deep convolution; wherein, the hyperspectral features are the intermediate hyperspectral features output by the linear manifold projection module, and the RGB features are the RGB guiding features corresponding to the current scale in the multi-scale RGB guiding feature pyramid; The guided spectral module employs a cross-modal window attention mechanism, using high-resolution RGB texture features as query parameters and intermediate hyperspectral features as key and value parameters for cross-modal attention calculation, thus aggregating spectral information. The convolutional spatial module dynamically filters background redundant noise in deep space exploration through a gating mechanism composed of large-kernel deep convolutions, and adaptively modulates the activation state of local micro-textures through serial spatial attention, completing deep feature dealiasing and detail recovery, and outputting updated hyperspectral estimates.

[0019] This invention also provides a dual-mode manifold hyperspectral imaging method for deep space exploration, employing a dual-mode manifold hyperspectral imaging system for deep space exploration, the method comprising: Data acquisition involves acquiring compressed observation data of the planetary surface and obtaining a coded mask through a coded aperture snapshot spectral imaging unit, acquiring a high-resolution RGB image spatially aligned with the compressed observation data through an RGB imaging unit, and generating an initial hyperspectral estimate based on the compressed observation data and the coded mask. The linear manifold projection receives the compressed observation data of the planetary surface, the encoding mask and the initial hyperspectral estimate, and updates the initial hyperspectral estimate based on the forward physical projection residual, the channel-normalized back projection and the relaxation factor to ensure measurement consistency, and outputs intermediate hyperspectral features. RGB multi-attention feature extraction: The high-resolution RGB image is input into the RGB multi-attention feature extraction module, which extracts the global spatial dependency, channel gradient response and spatial texture edge information of the high-resolution RGB image and fuses them to output a multi-scale RGB guided feature pyramid; For spectral reconstruction, the intermediate hyperspectral features and the corresponding RGB guiding features in the multi-scale RGB guiding feature pyramid are input into the guiding spectral space module for cross-modal deep regularization, and the updated hyperspectral estimates are output. The final planetary hyperspectral data are obtained through output mapping.

[0020] Furthermore, in the linear manifold projection, after receiving the compressed observation data of the planetary surface, the coded mask, and the initial hyperspectral estimate from the data acquisition module, a forward sensing matrix is ​​constructed based on the coded mask. Calculate the forward projection of the current hyperspectral estimate onto the observation domain. and the residual between it and the compressed observation data y The residuals are arranged according to Perform channel-wise normalization and then back-projection. Recast to the hyperspectral domain, combined with relaxation factors Update the current hyperspectral estimate to satisfy the measurement equation. The linear manifold projection outputs intermediate hyperspectral features with physical reliability.

[0021] Furthermore, the RGB multi-attention feature extraction specifically includes: The high-resolution RGB image is input to the RGB multi-attention feature extraction module. The data stream is divided into three paths for parallel processing. One path extracts global long-distance spatial dependence through the linear attention submodule. The other two paths extract channel gradient response, high-frequency texture edge and spatial gradient response through the spectral gradient attention and spatial gradient attention submodules, respectively. The three features are fused to obtain a multi-scale RGB guided feature pyramid.

[0022] Furthermore, in the spectral reconstruction, the intermediate hyperspectral features are first subjected to deep convolution and then fused with the corresponding scale RGB guiding features after spatial alignment. The fused feature stream enters the guiding spectral module, which uses a cross-modal window attention mechanism to generate a query matrix with high-resolution RGB texture features and a key matrix and value matrix with hyperspectral features. The hyperspectral features are then recalibrated and spectral information is aggregated. Subsequently, the aggregated spectral information is sent to the convolutional spatial module, which uses a gating mechanism composed of large kernel deep convolution to dynamically filter background redundant noise in deep space exploration and uses spatial attention to adaptively modulate the activation state of local micro-textures to complete deep feature dealiasing and detail restoration, and outputs the updated hyperspectral estimate for this stage.

[0023] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: (1) Significantly improved the reconstruction resolution and texture edge fidelity of planetary surface space details. This invention designs an RGB multi-attention feature extraction module and a guided spectral spatial module; it explicitly extracts high-frequency spatial differences from high-resolution RGB images through spatial gradient attention branches, and uses a gating mechanism composed of large kernel deep convolution in the guided spectral spatial module to perform deep cross-modal interaction with spatial attention; this invention overcomes the spatial degradation and blurring problems caused by the two-dimensional encoding of the existing CASSI system; when observing the geologically complex planetary surface, it can accurately restore the fine and dense rock textures, fault lines and crater geometric edges, and suppress the blocky effect, local fracture and adhesion artifacts common in the prior art, with visual fidelity and spatial resolution superior to the prior art.

[0024] (2) It significantly suppressed spectral aliasing, ensuring high spectral fidelity for planetary mineral identification. This invention addresses the shortcomings of existing shallow fusion methods that struggle to decouple spectral aliasing. It innovatively introduces a spectral gradient attention calculation channel response and employs a cross-modal window attention mechanism in the guided spectral spatial module. This mechanism uses RGB textures with clear physical boundaries as "queries" to deduplicat hyperspectral features. This forced use of a reliable spatial structure to guide the aggregation of spectral information fundamentally corrects local spectral distortions. The reconstructed pixel spectral density curve closely approximates the actual surface reflectance, significantly reducing spectral angle mapping errors and mitigating the risk of spectral distortion and misjudgment caused by algorithms in planetary mineral composition inversion and material classification.

[0025] (3) It achieves a coordinated balance between reconstruction accuracy, model lightweighting, and computational efficiency, which is highly compatible with the computing power limitations of deep space exploration platforms. This invention addresses the problem that the computational complexity of traditional Transformers increases quadratically with the input size. It introduces a linear attention module with kernel function approximation and rotation position encoding into the RGB auxiliary branch, compressing the computational complexity of long-distance spatial dependencies to [amount missing]. The lightweight model of this invention achieves or surpasses the reconstruction results of existing high-complexity models with a relatively small number of parameters and floating-point operations, enabling the model to run efficiently in the global receptive field. It also improves the parameter utilization efficiency of the algorithm, making it suitable for deployment on planetary probes or Mars rovers where computing resources and power consumption are strictly limited.

[0026] (4) It has extremely strong physical reliability and adaptability to extreme scenarios. This invention employs a phased, iterative reconstruction framework, embedding a linear manifold projection module. This module, through channel-normalized backprojection, forces the estimates of the deep network at each step back onto a manifold that satisfies the physical observation equations, thus avoiding the risks of physically infeasible solutions and error accumulation inherent in purely data-driven networks. Even in extreme degradation scenarios such as strong light contrasts and large areas of shadow on planetary surfaces, this invention maintains extremely high reconstruction stability and robustness, reducing irregular noise specks and divergence. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the structure of a dual-mode manifold hyperspectral imaging system for deep space exploration according to the present invention; Figure 2 This is a flowchart of a dual-mode manifold hyperspectral imaging method for deep space exploration according to the present invention; Figure 3 The images shown are the color guide image, coded aperture mask, compressed observation image, reconstruction result, ground truth image, and mean absolute error plot of Sample 1 in this embodiment of the invention. Figure 4The following are the color guide image, coded aperture mask, compressed observation image, reconstruction result, ground truth image and mean absolute error plot of sample 2 in this embodiment of the invention; Figure 5 The images shown are the color guide image, coded aperture mask, compressed observation image, reconstruction result, ground truth image, and mean absolute error diagram of sample 3 in this embodiment of the invention. Figure 6 This is the single-band reconstruction result of sample 1 in the 466.0 nm band in this embodiment of the invention; Figure 7 This is the single-band reconstruction result of sample 2 in the 466.0 nm band in this embodiment of the invention; Figure 8 This is the single-band reconstruction result of sample 3 in the 466.0 nm band in the embodiment of the present invention. Detailed Implementation

[0028] The present invention will now be described in detail with reference to the accompanying drawings.

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0030] Example This embodiment provides a dual-mode manifold hyperspectral imaging system for deep space exploration, such as... Figure 1 As shown, the hyperspectral reconstruction model provided in this embodiment adopts a phased, alternating iterative reconstruction framework, including a data acquisition module, a linear manifold projection module, an RGB multi-attention feature extraction module, and a guiding spectral space module as a depth regularizer. In each iteration stage, the system first calculates the forward physical projection residual through the linear manifold projection module and performs channel-normalized back projection. Under physical constraints, the current hyperspectral estimate (initial hyperspectral estimate in the initial stage) is pulled back to the linear manifold that satisfies the measurement equation to ensure the consistency of the observation data. Subsequently, in the depth regularization stage, the RGB auxiliary branch is used to provide multi-scale structure guidance, and depth denoising and detail restoration are completed in the guiding spectral space module. The two are updated alternately to gradually approximate the real planetary hyperspectral data.

[0031] In this embodiment, Figure 1 The parameters are expressed as follows: This indicates compressed observation data. This represents the forward sensing matrix corresponding to the coded mask. Indicates the encoding operator used for initialization. This represents the initial hyperspectral estimate. This represents the hyperspectral estimate from the previous iteration. This represents the hyperspectral estimate after the k-th stage update. Denotes the relaxation factor in stage k. and These represent the forward projection operator and the back projection operator, respectively. , These represent the image height and width, respectively. This indicates a feature concatenation operation. , , This represents the RGB guiding features at different scales output by the RGB multi-attention module. This represents the intermediate latent variables or feature maps that guide the output of the spectral space module.

[0032] Specifically, the functions of each module are as follows: In this embodiment, the data acquisition module includes an coded aperture snapshot spectral imaging unit and an RGB imaging unit. The coded aperture snapshot spectral imaging unit is used to acquire compressed observation data of the planetary surface and provide a coded mask. The RGB imaging unit is used to synchronously or register and acquire high-resolution RGB images spatially aligned with the compressed observation data. The initial hyperspectral estimate is obtained by initializing the compressed observation data and the coded mask.

[0033] In this embodiment, the linear manifold projection module receives compressed observation data y and the forward sensing matrix corresponding to the encoding mask. and the current stage of hyperspectral estimates First calculate the forward projection. The residual between the compressed observation data y , and then As a normalization term, it is normalized by channel, and then... Perform back projection, combined with relaxation factor The updated item is obtained, and the update format is as follows: This pulls the current estimate back to a linear manifold that satisfies the physical observation equations. This intermediate physical state is then input into the guiding spectral space module for deep dealiasing, such as... Figure 1 The bias shown is adaptively generated by a lightweight convolutional network based on the features of the current stage, and is used to adjust the update intensity.

[0034] In this embodiment, the RGB multi-attention module, designed for the extremely complex geological structure and lighting conditions of planetary surfaces, employs a three-branch parallel feature extraction architecture that simultaneously extracts global spatial dependence, spectral evolution patterns, and high-frequency edge details from high-resolution RGB images. As an auxiliary processing unit for extracting prior guidance information in this embodiment, it includes a linear attention module, a spectral gradient attention module, and a spatial gradient attention module. Its working principle involves extracting physically meaningful textures and spectral cues from multi-scale RGB images. The linear attention module, through kernel function approximation and rotational position encoding, captures long-distance spatial dependence of the vast planetary surface within the global receptive field while maintaining low computational complexity at the linear level. Of the other two gradient branches, one extracts channel responses and uncovers physical evolution patterns by calculating the first-order difference between adjacent bands; the other generates a spatial mask by calculating horizontal and vertical differences, explicitly enhancing the high-frequency feature expression of planetary material edge textures and spectral evolution trends. Finally, the three features are fused to construct a multi-scale feature pyramid, which is then injected into the main network level by level, providing accurate cross-modal guidance for hyperspectral reconstruction.

[0035] In this embodiment, the guided spectral spatial module addresses the issues of insufficient deep fusion of cross-modal features and blurred edges. Structurally, it adopts a dual-stream input architecture and cascades the guided spectral module and the convolutional spatial module. Its working principle is as follows: First, deep convolution is used to achieve preliminary alignment and fusion of hyperspectral features and RGB features in the spatial dimension. The hyperspectral features are intermediate hyperspectral features output by the linear manifold projection module or features obtained by dimensional embedding of the hyperspectral estimate updated in the previous iteration. The RGB features are corresponding-scale RGB guided features output by the RGB multi-attention feature extraction module. Subsequently, in the guided spectral module, a cross-modal window attention mechanism is used to use high-resolution RGB texture features as queries and hyperspectral features as keys and values. Clear surface textures are used to aggregate spectral information to maintain the consistency of mineral spectra. Finally, in the convolutional spatial module, a gating mechanism composed of large-kernel deep convolution is used to dynamically filter background redundant noise in deep space exploration, and spatial attention is used to adaptively modulate the activation state of local micro-textures, effectively suppressing the blurring defects induced by cross-domain feature transformations and achieving high-fidelity collaborative restoration of spatial morphology and hyperspectral information.

[0036] like Figure 2 As shown in the figure, this embodiment also provides a dual-mode manifold hyperspectral imaging method for deep space exploration, the specific process of which is as follows: Data acquisition begins with the system receiving compressed observation data and coded masks of the planetary surface via the coded aperture snapshot spectral imaging unit, and spatially aligned high-resolution RGB images via the RGB imaging unit. Initial hyperspectral estimates are then generated from the compressed observation data and coded masks. Subsequently, the system enters an iterative reconstruction phase, which continuously executes a preset number of iterations. Each iteration includes the following: RGB multi-attention feature extraction: High-resolution RGB images are input into the RGB multi-attention extraction module, and the data stream is divided into three paths for parallel processing: one path extracts global long-range spatial dependence through the linear attention sub-module, and the other two paths extract channel gradient response, high-frequency texture edge and spatial gradient response through the spectral gradient attention and spatial gradient attention sub-modules, respectively. After the three features are fused, a multi-scale RGB guided feature pyramid is constructed to provide cross-modal guidance information for subsequent reconstruction. For linear manifold projection, compressed observation data, coded masks, and initial or previous stage hyperspectral estimates are input into the linear manifold projection module. Within this module, the residuals are first calculated using forward physical projection, and then... The residuals are normalized by channel and then processed. Back-projecting back to the hyperspectral domain, and combining the relaxation factor, the current hyperspectral estimate is pulled back to a linear manifold that satisfies the physical observation equation, and the intermediate hyperspectral features with physical reliability are output. Spectral reconstruction involves integrating physically constrained intermediate hyperspectral features with corresponding RGB guided features from the multi-scale RGB guided feature pyramid into a guided spectral space module for depth regularization. Within this module, the intermediate hyperspectral features are first spatially aligned and fused with the corresponding RGB guided features through deep convolution. The fused feature stream then enters the guided spectral module, where a cross-modal window attention mechanism is used to generate a query matrix from high-resolution RGB texture features and a key and value matrix from hyperspectral features. The hyperspectral features are then recalibrated and spectral information is aggregated. Subsequently, the data stream is fed into the convolutional space module, where a gating mechanism composed of large-kernel deep convolutions dynamically filters background redundancy noise from deep space exploration. Spatial attention is used to adaptively modulate the activation state of local micro-textures, completing deep feature dealiasing and detail restoration, and outputting the updated hyperspectral estimate for this stage. Finally, after a set number of iterations, the model ends the alternating feature update process and generates high-fidelity planetary hyperspectral data through the final output mapping stage, achieving accurate inversion from two-dimensional compressed observations and RGB images to high-dimensional spectral space.

[0037] In summary, the advantages of the system of the present invention are as follows: (1) Constructing a multi-level RGB-guided spatial spectral feature sensing mechanism By guiding the spectral spatial module and utilizing high-resolution RGB images as additional observations, clear prior knowledge of planetary surface structures is supplemented to compressed measurements. This mechanism guides the CASSI branch to dynamically focus on key geological structural regions, significantly enhancing the response to edges and textures and addressing the problem of missing spatial details.

[0038] (2) Introduce an RGB multi-attention feature extraction module to mine deep priors. This module uses linear attention to capture long-range spatial dependencies of RGB images within a global receptive field and computes spatial differences and spectral gradients in parallel. Based on this cross-modal gradient prior, it explicitly enhances the feature representation of edge textures and spectral evolution trends, effectively suppresses the propagation of spectral estimation errors and aliasing errors, and ensures the spectral consistency of planetary mineral inversion.

[0039] (3) Combining linear manifold projection with multi-scale detail reconstruction strategy In the iterative reconstruction framework with alternating execution, linear manifold projection is used to perform measurement consistency updates under physical constraints to ensure the physical reliability of the data. At the same time, multi-scale detail reconstruction and convolutional spatial modules are combined to enhance cross-level feature transfer and retain high-frequency features, effectively alleviate spatial blurring and effectively suppress local artifacts.

[0040] In summary, this invention aims to fully leverage the spatial prior advantages of RGB images through the aforementioned multi-level cross-modal guidance and deep feature interaction design, overcoming the spectral aliasing and spatial degradation bottlenecks of traditional hyperspectral imaging, thereby achieving high-quality data reconstruction with both high spatial resolution and high spectral fidelity in planetary deep space exploration missions.

[0041] (4) Simulation verification results and advantages analysis To further verify the effectiveness of the dual-mode manifold hyperspectral imaging system and method described in this invention, this embodiment runs a test script in a test environment on a remote server, performs joint processing on high-resolution color images, coded aperture masks, and compressed observation data, and outputs reconstructed hyperspectral results, true value comparison results, and error maps. The simulation results are shown below.

[0042] Table 1. Hyperspectral reconstruction quality indicators for 10 test samples.

[0043] like Figures 3-8As shown in Table 1, the reconstruction results output by this invention maintain high consistency with the ground truth image in terms of subject edges, color block boundaries, and local texture locations. The average absolute error is mainly concentrated in bright edges and local strong texture areas, with an overall low error level. In 10 test samples, the average peak signal-to-noise ratio reached 48.78 dB, the average structural similarity reached 0.9966, the average root mean square error was 0.003674, and the average spectral angle was 1.9940 degrees. These results demonstrate that this invention, through the synergistic effect of color guiding features, linear manifold projection, and guided spectral spatial modules, can restore a clearer spatial structure and more stable spectral information while maintaining the physical consistency of compressed observations, exhibiting advantages in suppressing spatial blur, spectral aliasing, and local artifacts. Specifically, Figure 3 (a) in the image is the color guide image of sample 1. Figure 3 (b) in the image is the coded aperture mask for sample 1. Figure 3 (c) in the image is a compressed observation image of sample 1. Figure 3 (d) in the figure represents the reconstruction result of sample 1. Figure 3 In the image, (e) represents the ground truth image of sample 1. Figure 3 (f) in the figure represents the mean absolute error plot of sample 1; Figure 4 (a) in the image is the color guide image of sample 2. Figure 4 (b) in the image is the coded aperture mask for sample 2. Figure 4 (c) in the image is a compressed observation image of sample 2. Figure 4 (d) in the figure represents the reconstruction result of sample 2. Figure 4 In the image, (e) represents the ground truth image of sample 2. Figure 4 (f) in the figure represents the mean absolute error of sample 2; Figure 5 (a) in the image is the color guide image of sample 3. Figure 5 (b) in the figure is the coded aperture mask of sample 3. Figure 5 (c) in the image is a compressed observation image of sample 3. Figure 5 (d) in the figure represents the reconstruction result of sample 3. Figure 5 In the image, (e) is the ground truth image of sample 3. Figure 5 (f) in the figure is the mean absolute error plot of sample 3.

[0044] This article uses specific embodiments to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A dual-mode manifold hyperspectral imaging system for deep space exploration, characterized in that, The system includes: The data acquisition module is used to acquire compressed observation data of the planetary surface, a coding mask, and a high-resolution RGB image spatially aligned with the compressed observation data of the planetary surface, and to generate an initial hyperspectral estimate based on the compressed observation data of the planetary surface and the coding mask. The linear manifold projection module is used to receive compressed observation data of planetary surface, encoding mask and initial hyperspectral estimate from the data acquisition module, and update the initial hyperspectral estimate based on the forward physical projection residual, channel-normalized back projection and relaxation factor to ensure measurement consistency, and output intermediate hyperspectral features. The RGB multi-attention feature extraction module is used to receive high-resolution RGB images from the data acquisition module, extract global, spectral and texture information of the high-resolution RGB images, fuse them, and output a multi-scale RGB feature pyramid. The guided spectral space module is used to receive the intermediate hyperspectral features and the corresponding scale RGB guided features in the multi-scale RGB guided feature pyramid, perform cross-modal depth regularization, output the updated hyperspectral estimate, and obtain the final planetary hyperspectral data through the output mapping.

2. The dual-mode manifold hyperspectral imaging system for deep space exploration according to claim 1, characterized in that, The data acquisition module includes an coded aperture snapshot spectral imaging unit and an RGB imaging unit. The coded aperture snapshot spectral imaging unit is used to acquire compressed observation data of the planetary surface and provide an coded mask, while the RGB imaging unit is used to acquire high-resolution RGB images.

3. The dual-mode manifold hyperspectral imaging system for deep space exploration according to claim 1, characterized in that, In the linear manifold projection module, after receiving compressed planetary surface observation data, coded masks, and initial hyperspectral estimates from the data acquisition module, a forward sensing matrix is ​​constructed based on the coded mask. Calculate the forward projection of the current hyperspectral estimate onto the observation domain. and the residual between it and the compressed observation data y The residuals are arranged according to Perform channel-wise normalization and then back-projection. Recast to the hyperspectral domain, combined with relaxation factor Update the current hyperspectral estimate to satisfy the measurement equation. The linear manifold projection outputs intermediate hyperspectral features with physical reliability.

4. A dual-mode manifold hyperspectral imaging system for deep space exploration according to claim 1, characterized in that, The RGB multi-attention feature extraction module specifically includes a linear attention module, a spectral gradient attention module, and a spatial gradient attention module. The linear attention module, the spectral gradient attention module, and the spatial gradient attention module form a three-branch parallel structure to extract global, spectral, and texture information from high-resolution RGB images.

5. A dual-mode manifold hyperspectral imaging system for deep space exploration according to claim 4, characterized in that, The linear attention module extracts the global long-range spatial dependence of the planetary surface within the global receptive field by approximating the kernel function and encoding the rotational position. The spectral gradient attention module performs first-order difference on adjacent channels of the RGB embedded features to obtain channel gradient responses, and the spatial gradient attention module performs first-order difference on the RGB embedded features in the horizontal and vertical directions to obtain spatial gradient responses. Based on the channel gradient responses and spatial gradient responses, the high-frequency feature expression of edge texture and spectral evolution trends is enhanced to obtain a multi-scale RGB guided feature pyramid.

6. A dual-mode manifold hyperspectral imaging system for deep space exploration according to claim 1, characterized in that, The guided spectral spatial module specifically adopts a dual-stream input architecture and is cascaded with the guided spectral module and the convolutional spatial module; Hyperspectral features and RGB features are initially aligned and fused in the spatial dimension through deep convolution; wherein, the hyperspectral features are the intermediate hyperspectral features output by the linear manifold projection module, and the RGB features are the RGB guiding features corresponding to the current scale in the multi-scale RGB guiding feature pyramid; The guided spectral module employs a cross-modal window attention mechanism, using high-resolution RGB texture features as query parameters and intermediate hyperspectral features as key and value parameters for cross-modal attention calculation, thus aggregating spectral information. The convolutional spatial module dynamically filters background redundant noise in deep space exploration through a gating mechanism composed of large-kernel deep convolutions, and adaptively modulates the activation state of local micro-textures through serial spatial attention, completing deep feature dealiasing and detail recovery, and outputting updated hyperspectral estimates.

7. A dual-mode manifold hyperspectral imaging method for deep space exploration, employing the dual-mode manifold hyperspectral imaging system for deep space exploration as described in any one of claims 1 to 6, characterized in that, The method includes: Data acquisition involves acquiring compressed observation data of the planetary surface and obtaining a coded mask through a coded aperture snapshot spectral imaging unit, acquiring a high-resolution RGB image spatially aligned with the compressed observation data through an RGB imaging unit, and generating an initial hyperspectral estimate based on the compressed observation data and the coded mask. Linear manifold projection receives compressed observation data of the planetary surface, coded mask and initial hyperspectral estimate, and updates the initial hyperspectral estimate based on forward physical projection residual, channel-normalized back projection and relaxation factor to ensure measurement consistency, and outputs intermediate hyperspectral features. RGB multi-attention feature extraction: The high-resolution RGB image is input into the RGB multi-attention feature extraction module, which extracts the global spatial dependency, channel gradient response and spatial texture edge information of the high-resolution RGB image and fuses them to output a multi-scale RGB guided feature pyramid; For spectral reconstruction, the intermediate hyperspectral features and the corresponding RGB guiding features in the multi-scale RGB guiding feature pyramid are input into the guiding spectral space module for cross-modal deep regularization, and the updated hyperspectral estimates are output. The final planetary hyperspectral data are obtained through output mapping.

8. A dual-mode manifold hyperspectral imaging method for deep space exploration according to claim 7, characterized in that, In the linear manifold projection, after receiving the compressed observation data of the planetary surface, the coded mask, and the initial hyperspectral estimate from the data acquisition module, a forward sensing matrix is ​​constructed based on the coded mask. Calculate the forward projection of the current hyperspectral estimate onto the observation domain. and the residual between it and the compressed observation data y The residuals are arranged according to Perform channel-wise normalization and then back-projection. Recast to the hyperspectral domain, combined with relaxation factor Update the current hyperspectral estimate to satisfy the measurement equation. The linear manifold projection outputs intermediate hyperspectral features with physical reliability.

9. A dual-mode manifold hyperspectral imaging method for deep space exploration according to claim 7, characterized in that, The RGB multi-attention feature extraction specifically includes: The high-resolution RGB image is input to the RGB multi-attention feature extraction module. The data stream is divided into three paths for parallel processing. One path extracts global long-distance spatial dependence through the linear attention submodule. The other two paths extract channel gradient response, high-frequency texture edge and spatial gradient response through the spectral gradient attention and spatial gradient attention submodules, respectively. The three features are fused to obtain a multi-scale RGB guided feature pyramid.

10. A dual-mode manifold hyperspectral imaging method for deep space exploration according to claim 7, characterized in that, In the spectral reconstruction, the intermediate hyperspectral features are first subjected to deep convolution and then fused with the corresponding scale RGB guiding features after spatial alignment. The fused feature stream enters the guiding spectral module, which uses a cross-modal window attention mechanism to generate a query matrix with high-resolution RGB texture features and a key matrix and value matrix with hyperspectral features. The hyperspectral features are then recalibrated and spectral information is aggregated. Subsequently, the aggregated spectral information is sent to the convolutional spatial module, which uses a gating mechanism composed of large kernel deep convolution to dynamically filter background redundant noise in deep space exploration and uses spatial attention to adaptively modulate the activation state of local micro-textures to complete deep feature dealiasing and detail restoration, and outputs the updated hyperspectral estimate for this stage.

Images