Planetary surface geology multi-modal exploration system and method

Abstract

The present application discloses a planetary surface geological multi-modal exploration system and method, comprising an interaction module, a feature decoupling module and a feature guided attention Mamba module; the interaction module is used for obtaining multi-modal features of hyperspectral images and LiDAR images and performing calibration; the feature decoupling module is used for separating the multi-modal features into cross-modal shared consistency features and each modal specific feature; the feature guided attention Mamba module is used for taking the cross-modal shared consistency features as global semantic prior, generating an attention weight matrix to perform directional screening and noise suppression on the decoupled each modal specific feature, and performing aggregation of the multi-modal features through the Mamba module; the present application solves the problems of modal information imbalance, feature coupling interference and lack of fusion guidance by constructing a feature calibration, feature decoupling and guided fusion whole-process integrated fusion mechanism, thereby realizing high-precision and high-robustness classification of planetary surface geological categories.

Description

Technical Field

[0001] This invention relates to the field of image classification and processing technology, and more specifically to a multimodal exploration system for planetary surface geology.

[0002] This invention also relates to a multimodal exploration method for planetary surface geology. Background Technology

[0003] Hyperspectral imaging technology can simultaneously acquire spatial geometry and spectral radiation information of the Earth's surface across multiple continuous spectral bands from ultraviolet to mid-infrared, accurately reflecting the inherent spectral characteristics and spatial distribution patterns of surface materials. This technology has become the cornerstone of Earth observation and is widely used in deep space observation, planetary exploration, and surface geological exploration. In surface geological exploration missions, Hyperspectral Image Classification (HSIC) aims to classify each pixel on the Earth's surface into its corresponding land cover category based on differences in spectral characteristics.

[0004] However, due to the complexity of planetary environments, significant differences in illumination and topography, and the common phenomenon of heterospectral similarities and heterogeneous similarities in hyperspectral data, using only hyperspectral images for surface geological exploration often cannot meet practical needs. To alleviate these problems, lidar (LiDAR) data provides complementary elevation information, which can effectively distinguish between features that are spectrally similar but have different elevations. Therefore, the synergistic fusion of hyperspectral images and lidar data has become a key strategy for improving the accuracy of feature classification in complex scenarios. The fusion of the two for image classification has become a key technical means to achieve accurate identification of planetary surface geological bodies, analysis of geological structures, and classification of material types, and has been widely used in deep space exploration missions such as those to the Moon, Mars, and asteroids.

[0005] Early fusion classification methods were based on traditional classifiers such as support vector machines and random forests. Feature-level fusion classifies data by directly splicing spectral features from hyperspectral data with elevation features from LiDAR. While this method is simple in principle, it does not fully consider the heterogeneity of multimodal features and is prone to decreased classification accuracy due to imbalanced feature dimensions and excessive redundant information. Decision-level fusion first classifies data from each single modality separately and then fuses the classification results from each modality through voting, weighting, or other methods. Although this method reduces the difficulty of directly fusing multimodal features, it does not achieve deep interaction of underlying features and cannot fully explore the complementary correlations of multimodal data. In complex geological scenarios on planetary surfaces, the classification accuracy is significantly reduced.

[0006] In recent years, deep learning, with its powerful capabilities in nonlinear feature extraction, complex pattern modeling, and heterogeneous information fusion, has become the mainstream research direction in multimodal remote sensing image fusion and classification, effectively overcoming the technical limitations of traditional methods. Deep learning-based multimodal fusion classification methods can deeply mine and interactively fuse low-level features from hyperspectral and LiDAR data, fully leveraging their complementary advantages and significantly improving the accuracy of planetary surface geological classification. However, problems such as information density imbalance, feature coupling interference, scarce labeled samples, and limited computational resources in multimodal remote sensing data of planetary surfaces still pose significant challenges to deep learning methods. Currently, several algorithms based on convolutional neural networks, Transformers, and state-space models have made significant progress in addressing these issues.

[0007] The existing technical solutions are as follows: Existing deep learning-based multimodal remote sensing image fusion and classification methods can be mainly divided into three categories: 1) Fusion classification method based on convolutional neural networks This type of method was the earliest deep learning approach applied to planetary surface hyperspectral-LiDAR data processing. Its core relies on the local feature extraction advantages of convolutional neural networks. It designs multi-branch convolutional network structures to address the heterogeneity of multimodal data, extracting features from different modalities separately. Then, it fuses multimodal features through feature concatenation, element-wise addition, and channel fusion, ultimately using a classifier to identify geological categories. A typical implementation of this method is CoupledCNN, which introduces a post-parameter sharing mechanism on top of the traditional dual-branch CNN. This allows the convolutional layers of the hyperspectral and LiDAR branches to share some weights during the feature fusion stage, improving the mutual learning ability of multimodal features and reducing the number of model parameters. This type of method has a simple model structure, high computational efficiency, and is easy to deploy on embedded hardware of planetary probes. Furthermore, the local convolutional characteristics of CNNs can accurately capture subtle spatial features of planetary surface geological bodies, making it suitable for small-scale, high-resolution planetary exploration data processing.

[0008] 2) Transformer-based fusion classification method This method, based on the self-attention and cross-attention mechanisms of Transformer, solves the problem that CNNs struggle to capture long-range spatial dependencies. Its core is to model cross-modal associations between hyperspectral and LiDAR data through an attention mechanism, achieving more refined multimodal feature interaction and fusion. It is suitable for identifying large-scale, continuously distributed geological structures on planetary surfaces, such as impact crater zones and rift valley clusters. Typical examples of this method include hierarchical vision Transformer and ExViT. ExViT, through parallel Transformer branches, combines separable convolutions and cross-modal attention to achieve effective interaction between hyperspectral spatial features and LiDAR spatial features. Transformer-based methods can effectively capture long-range spatial dependencies and cross-modal feature associations of planetary surface geological bodies, exhibiting robustness to spectral variability and higher classification accuracy than traditional CNN schemes. They are suitable for identifying complex, large-scale geological structures on planetary surfaces.

[0009] 3) Fusion classification method based on state-space model State-space models are an emerging multimodal remote sensing fusion scheme in recent years. They retain the advantages of linear time inference and long-sequence modeling, solving the problems of high computational complexity and large number of parameters in Transformers, while preserving long-range feature modeling capabilities. They are a new generation of deep learning solutions adapted to planetary exploration scenarios with limited computing power, and are also the existing solution most similar to the core technology of this invention. As an emerging network structure, typical methods include MSFMamba and HLMamba. This method combines long-sequence modeling capabilities with high computational efficiency. The linear inference complexity significantly reduces the computational and parameter count of the model. It can capture long-range dependencies of large-scale geological structures on planetary surfaces and adapt to the embedded computing power limitations of planetary probes, making it the mainstream development direction for current multimodal remote sensing data fusion of planetary surfaces.

[0010] In the existing research framework, the technological development directions for the fusion optimization of multimodal data are mainly divided into two core branches: one type of method focuses on the refined extraction and enhancement of single-modal features. By designing dedicated feature extraction networks and enhancement modules, it deeply mines and optimizes the spectral-spatial joint features of hyperspectral data and the elevation features of LiDAR data, respectively, to improve the discriminative and representational capabilities of single-modal features. Then, it integrates multimodal features through simple splicing, element-wise fusion, and other methods, thereby consolidating the feature foundation for multimodal fusion. The other type of method focuses on the interactive enhancement and construction of cross-modal features. It abandons the idea of ​​simple fusion after single-modal feature extraction. By introducing mechanisms such as cross-modal attention, feature interaction gating, and intermodal information transfer, it enables hyperspectral and LiDAR features to achieve deep interaction, mutual guidance, and weighted optimization during the extraction process. It directly mines the complementary correlations and intrinsic coupling laws between heterogeneous modal features, thereby improving the effectiveness and synergy of multimodal fusion features.

[0011] In summary, existing solutions have all achieved the fusion of multimodal data and the identification of planetary surface geological categories, but none of them have fundamentally solved the core problems of modal information imbalance and feature coupling interference in the fusion of planetary multimodal remote sensing data. These problems have become key factors restricting their classification accuracy in complex planetary surface scenarios, and are also the technical problems that this invention needs to overcome.

[0012] Existing multimodal fusion classification methods for planetary surface geological exploration, regardless of whether they focus on enhancing single-modal feature extraction or strengthening cross-modal feature interaction, have failed to fundamentally solve the three core problems of information imbalance, noise interference, and inefficiency in multimodal data fusion. This results in fusion classification accuracy and engineering adaptability that are insufficient to meet the exploration needs of complex planetary geological scenarios. Therefore, the specific problems with existing technologies are as follows: (1) The modal information is naturally imbalanced, resulting in a significant modality dominance problem. Existing methods do not actively calibrate the information density of hyperspectral and LiDAR data. Because hyperspectral data has higher dimensionality and information density, the model will naturally bias towards fitting hyperspectral features during training and optimization, resulting in a phenomenon where high-information modes suppress low-information modes. This leads to a severe suppression of the representation of LiDAR elevation and geometric features. This problem directly causes a significant decrease in the classification accuracy of geological bodies on the planetary surface that rely on elevation information for differentiation, such as gullies and dry valleys, rift valleys and joint zones. These are the core categories that need to be identified in planetary geological exploration, and existing methods cannot achieve accurate identification of such geological bodies.

[0013] (2) Feature coupling interference is severe, and effective decoupling between shared and private features has not been achieved. Most existing methods directly splice and fuse features after independently extracting each modality feature, which easily leads to indiscriminate coupling of cross-modal features. Redundant shared information will cover up the exclusive discriminative features of each modality. The elevation features of LiDAR and the spectral features of hyperspectral data cannot be effectively utilized, losing the core complementary value of multimodal data and making it difficult to cope with the core exploration difficulties of different objects with the same spectrum and different spectra of the same object on the planetary surface.

[0014] (3) Cross-modal fusion lacks effective guidance, and the discriminative power and task relevance of private features are insufficient. Existing cross-modal interaction methods mostly adopt undirected feature interaction mechanisms such as bidirectional cross-attention and simple feature summation, without establishing global semantic guidance and constraints on private features. This results in a large amount of noise information unrelated to the geological classification task being mixed into the decoupled modal private features, and the focusing direction of private features of different modalities is inconsistent in the semantic space, making it impossible to form a collaborative discriminative ability. At the same time, although some methods introduce sequence modeling or attention mechanisms, they do not use shared semantics as a global prior to guide the optimization and aggregation of private features, resulting in weak task relevance of fused features and difficulty in adapting to the complex planetary exploration environment with low signal-to-noise ratio. Summary of the Invention

[0015] The purpose of this invention is to address the aforementioned problems by providing a multimodal exploration system and method for planetary surface geology. By constructing an integrated fusion mechanism that integrates feature calibration, feature decoupling, and guided fusion, this invention solves the problems of modal information imbalance, feature coupling interference, and lack of fusion guidance, thereby achieving high-precision and robust classification of planetary surface geology.

[0016] The technical solution adopted in this invention is as follows: A multimodal exploration system for planetary surface geology, comprising: The interactive module is used to acquire and calibrate multimodal features of hyperspectral and LiDAR images; The feature decoupling module is used to separate the multimodal features into cross-modal shared consistent features and modality-specific features; The Feature-Guided Attention Mamba module is used to take cross-modal shared consistent features as global semantic priors, generate attention weight matrices to perform targeted screening and noise suppression of decoupled modality-specific features, aggregate and fuse multimodal features through the Mamba module, and feed the obtained fused features into the classifier to complete classification.

[0017] Furthermore, in the interaction module, based on the principle of adaptive prompting learning and feature calibration, internal prompts and cross prompts are generated, and the contribution weight of multimodal features is adjusted by a learning scaling factor. The multimodal features extracted by the dual-branch Mamba are subjected to intra-layer, inter-layer, and cross-modal feature interactions, dynamically balancing the information density of different modal features, eliminating the initial feature distribution differences, and obtaining calibrated multimodal features.

[0018] Furthermore, in the feature decoupling module, the calibrated multimodal features are explicitly decoupled into cross-modal shared consistent features and modality-specific features through a shared encoder and a private encoder, and the decoupling process is constrained and optimized through self-reconstruction and cross-reconstruction loss constraints.

[0019] This invention also provides a multimodal exploration method for planetary surface geology, employing the aforementioned multimodal exploration system for planetary surface geology. The method specifically includes: Image preprocessing: Acquire hyperspectral and LiDAR images and perform preprocessing. The dynamic feature prompting interaction processes the obtained hyperspectral and LiDAR images through the interactive module to obtain multimodal features and perform calibration; Feature mode decoupling involves explicitly decoupling the calibrated multimodal features through a feature decoupling module to obtain cross-modal shared consistent features and modality-specific features. Feature guidance and fusion uses cross-modal shared consistent features as global semantic priors to generate attention weight matrices for targeted screening and noise suppression of decoupled modality-specific features. The long sequence modeling capability of the feature-guided attention Mamba module is used to complete the deep aggregation and multimodal fusion of modality-specific features. The resulting fused features are then fed into the classifier to complete classification.

[0020] Furthermore, the image preprocessing specifically includes segmenting the hyperspectral image and LiDAR image into paired pixel patches.

[0021] Furthermore, in the dynamic feature prompting interaction, the acquired pixel patches are input into the interaction module. Based on the principle of adaptive prompting learning and feature calibration, internal prompts and cross prompts are generated, and the contribution weight of multimodal features is adjusted by a learning scaling factor. The features extracted by the dual-branch Mamba are subjected to intra-layer, inter-layer, and cross-modal feature interactions. The information density of different modal features is dynamically balanced, the differences in the initial feature distribution are eliminated, and the balanced learning and deep extraction of multimodal features are completed.

[0022] Furthermore, in the feature mode decoupling, the calibrated multimodal features are explicitly decoupled into cross-modal shared consistent features and modality-specific features through a shared encoder and a private encoder. At the same time, optimization is performed through self-reconstruction and cross-reconstruction loss constraints, specifically including feature information integrity optimization, cross-modal shared consistent feature and modality-specific feature overlap optimization, and cross-modal shared feature semantic consistency optimization.

[0023] Furthermore, in the feature guidance and fusion process, cross-modal shared consistent features and modality-specific features interact through the feature-guided attention Mamba module. The cross-modal shared consistent features are used to generate attention scores, which serve as global semantic priors to guide the deep extraction of modality-specific features, thereby enabling each modality-specific feature to surround the task-related parts while suppressing noise.

[0024] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: (1) Alleviating the problem of modal information imbalance and achieving balanced learning of multimodal features: This invention uses a dynamic feature prompting interaction strategy to dynamically calibrate the density of multimodal information, which improves the problem of large differences in the initial information distribution of different modes to a certain extent, avoids the suppression of LiDAR features by hyperspectral features, and enables balanced learning and representation of spectral features and elevation features.

[0025] (2) Solve the problem of feature coupling interference and realize explicit decoupling of shared and private features: This invention realizes the non-overlapping and lossless explicit decoupling of features through the encoder-decoder structure of the feature modality decoupling module and the self-reconstruction-cross reconstruction dual loss constraint scheme. This design solves the problem of feature coupling interference, allowing the private discrimination features of hyperspectral and LiDAR to be fully preserved, and maximizing the complementary value of multimodal data.

[0026] (3) Solve the problem of private feature noise and realize targeted fusion: The system of the present invention uses the feature-guided attention Mamba module to take the shared features as semantic priors and generate attention scores to guide the extraction of modality-specific features, thereby improving the problem of inconsistent focusing direction of modality-specific features in semantic space; enabling modality-specific features to achieve unified semantic focus under global semantic constraints, greatly improving the task relevance and discriminativeness of fused features, and significantly enhancing the noise resistance of the model. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the structure of the planetary surface geology multimodal exploration system of the present invention; Figure 2 This is a flowchart of the multimodal exploration method for planetary surface geology of the present invention; Figure 3This is a structural diagram of the feature-guided attention Mamba module in the planetary surface geology multimodal exploration system of this invention; Figure 4 The table shows the quantitative analysis of the method of this invention and existing methods; Figure 5 This is a visual comparison of the method of the present invention and existing methods. Detailed Implementation

[0028] The present invention will now be described in detail with reference to the accompanying drawings.

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0030] Example This embodiment provides a multimodal exploration system for planetary surface geology. Addressing the technical challenges of multimodal remote sensing image fusion and classification in planetary surface geological exploration, it proposes a full-process multimodal fusion and classification framework that integrates dynamic feature calibration, feature mode decoupling, and feature-guided fusion. Figure 1 As shown, the overall system architecture is a serially progressive modular structure, consisting of a dynamic feature prompting interaction module, a feature modality decoupling module, a feature-guided attention Mamba module, and a classifier head. Each module is interconnected, with unidirectional data transmission and complementary functions. Through the action logic of calibration, decoupling, guided fusion, and classification, it achieves dynamic balance of multimodal information density, explicit separation of shared and private features, semantic guidance optimization of private features, and efficient aggregation of multimodal features, ultimately completing the accurate classification of planetary surface geological categories. The specific implementation method and advantages of this system are as follows: (1) Design a dynamic feature prompting interaction mechanism. By generating intra-modal and inter-modal adaptive prompts, actively calibrate the information density of data from different modalities, eliminate initial feature distribution differences, and avoid the suppression of low-information modalities by high-information modalities. Specifically: The Dynamic Feature Prompt Interaction (DFPI) module, based on the principles of adaptive prompt learning and feature calibration, generates internal and cross-prompts and adjusts their feature contribution weights through a learning scaling factor. It performs intra-layer, inter-layer, and cross-modal feature interactions on features extracted by the dual-branch Mamba, dynamically balancing the information density of features from different modalities, eliminating initial feature distribution differences, avoiding the suppression of low-information modalities by high-information modalities, and achieving balanced learning and deep extraction of multimodal features.

[0031] (2) Construct a Feature Modality Decoupling (FMD) module. Through encoder and reconstruction loss constraints, multimodal features are explicitly separated into cross-modal shared consistent features and modality-specific features, thereby reducing the masking and interference of shared features on private features. Specifically: By using the structure of shared encoders and private encoders, the calibrated multimodal features are explicitly decoupled into cross-modal shared consistent features and modality-specific features. At the same time, through self-reconstruction and cross-reconstruction loss constraints, it is ensured that no feature information is lost during the decoupling process, there is no overlap between shared and private features, and the semantics of cross-modal shared features are consistent, providing input for subsequent processing to alleviate the feature coupling interference problem.

[0032] (3) Develop feature-guided attention Mamba module FGAM, such as Figure 3 As shown, using cross-modal shared consistency features as a global prior, explicit attention guidance and fine-grained optimization are applied to the decoupled modality-specific features (i.e., the features specific to each modality) to suppress irrelevant noise in the private features. Simultaneously, Mamba's linear time series modeling capabilities are utilized to achieve efficient aggregation of multimodal features, specifically: The decoupled cross-modal shared features are used as global semantic priors. Attention scores are generated to screen the decoupled modal private features for task relevance and suppress noise. Then, Mamba's long sequence modeling capabilities are combined to achieve deep aggregation and multimodal fusion of private features. This allows the fused features to retain the unique discriminative features of each modality while having unified global semantic constraints, thereby improving the discriminativeness and task relevance of the features.

[0033] After being processed by the three modules mentioned above, the resulting fused features are fed into the classifier. The modules are connected in sequence to form a complete classification system. The calibrated features output by the DFPI module are input into the FMD module for feature decoupling, resulting in shared features and private features of different modalities. These two types of features are fed into the FGAM module, which uses the shared features as a global semantic prior to guide the extraction of private features. Through these three core modules, the present invention can ultimately achieve robust feature representation and high-precision land cover classification.

[0034] This embodiment also provides a multimodal exploration method for planetary surface geology. From multimodal data input to geological category output, the actions of each module and stage are executed sequentially according to a fixed time sequence. Data flows unidirectionally, features are optimized step by step, and there is no manual intervention. The overall action relationship follows the core logic of data input → preprocessing → feature calibration and extraction → feature decoupling → feature guidance and fusion → classification determination. Figure 2 As shown, it specifically includes: First, the input hyperspectral and LiDAR images are divided into paired pixel patches. Then, the original patches are processed by the DFPI module to generate calibrated features for different modalities. Next, they are input into the FMD module for explicit decoupling to obtain cross-modal shared consistency features and modality-specific features. Finally, the shared and private features interact in the FGAM module. The shared features are used to generate attention scores, which serve as global semantic priors to guide the deep extraction of private features, enabling the private features to surround the task-relevant parts while suppressing noise.

[0035] In summary, the entire framework of this invention's system and method revolves around maximizing the complementary value of multimodal data. Through modular functional design and sequential, progressive action logic, it alleviates the core pain points of existing technologies, achieving accurate and robust classification of planetary surface geological categories. Furthermore, leveraging Mamba's linear computational complexity and lightweight design ensures the engineering adaptability of the technical solution, as detailed below: (1) Adaptive calibration mechanism for multimodal information density in dynamic feature-based interaction: Addressing the inherent imbalance of modal information and the suppression of low-information modes by high-information modes in existing technologies, a two-layer prompting interaction strategy is designed. Combined with adaptive weighting of learning scaling factors, dynamic and data-driven calibration of information density of different modal features is achieved. This breaks through the limitations of existing technologies such as "direct fusion without calibration" and "manual weight setting calibration". It ensures that the representation of LiDAR elevation structure features is no longer suppressed by HSI spectral features, and ensures that geological bodies on the planetary surface that depend on elevation features can be learned in a balanced manner. This is the core innovation for solving the modality dominance problem.

[0036] (2) Feature mode explicit decoupling architecture based on reconstruction loss: To address the issues of feature coupling interference and the indiscriminate mixing of shared and private features in existing technologies, an orthogonal decoupling structure of four feature encoders and a single feature decoder is designed. Through the functional orthogonal design of the shared encoder and the private encoder, explicit decoupling of cross-modal shared consistent features and modal-specific private features is achieved without overlap or information loss. Simultaneously, a dual reconstruction loss constraint mechanism is incorporated to ensure both the integrity of single-modal information and the semantic consistency of cross-modal shared features. This overcomes the limitation of existing technologies where shared information masks private information, fully preserving the core complementary features of multimodal data, and represents a core innovation in solving feature coupling interference.

[0037] (3) A feature-guided attention modulation fusion method with shared semantics as the global prior: To address the pain points of high noise levels and low task relevance of private features, a targeted fusion mechanism is designed, which involves shared feature guidance, attention modulation, and state space model sequence modeling. Cross-modal shared features are used as global semantic priors, and an attention weight matrix is ​​generated to perform targeted screening and noise suppression of decoupled modal private features. Then, the state space model is combined to realize long-distance dependency modeling and aggregation of private features. Private features from different modalities are collaboratively fused under global semantic constraints, which greatly improves the discriminativeness and task relevance of the fused features. This is the core innovation for solving the problem of inefficient fusion.

[0038] In summary, the planetary surface geology multimodal exploration system proposed in this invention can robustly classify surface materials under complex surface conditions by efficiently utilizing the complementarity of different modal data, demonstrating broad application potential and promotional value. This system can be deployed onboard embedded hardware of planetary probes or in ground receiving station data processing systems, adaptable to various planetary geological exploration missions such as those on the Moon, Mars, and asteroids. For core geological bodies with spectrally homogeneous but morphologically heterogeneous structures, such as Martian gullies and dry valleys, lunar basaltic weathered layers and freshly exposed layers, and asteroid impact crater ridges and bedding hills, the system achieves accurate classification and boundary delineation by fusing HSI spectral composition features with LiDAR elevation features. The system maintains stable classification performance even under complex imaging conditions, significantly improving the intelligent processing and autonomous identification capabilities of multimodal data in deep space exploration missions.

[0039] Beyond planetary surface geological exploration, the multimodal remote sensing image fusion and classification technology of this invention can be directly transferred to hyperspectral and LiDAR multi-source remote sensing scenarios on the Earth's surface, solving the problem of distinguishing between spectrally homogeneous but heterogeneous materials. It is suitable for fields such as Earth resource exploration, ecological environment monitoring, and land planning, while also leveraging its advantages of high precision and robustness, and is compatible with various remote sensing platforms such as UAVs, satellites, and airborne systems. The core application scenario of this invention is surface geological exploration of planets such as the Moon, Mars, and asteroids, and it is a key technical solution for achieving accurate interpretation of multimodal remote sensing data in the field of deep space exploration. Simultaneously, the technology can be seamlessly extended to various multi-source remote sensing scenarios on Earth, providing high-precision fusion and classification technical support for fields such as Earth resource exploration, ecological environment monitoring, and land planning, forming a comprehensive application system covering deep space and near-Earth.

[0040] Verification Example Experiments were conducted using the open-source Houston 2013 dataset to verify the advantages of this invention. This dataset is a highly representative benchmark dataset in the field of remote sensing, constructed based on complex terrain scenes, and possesses strong value for algorithm validation and model generalization evaluation. The dataset accurately classifies 15 typical land features, covering diverse land surface types such as vegetation, buildings, roads, and bare land. Its annotation system is standardized and rigorous, the total sample size is sufficient, and the class distribution is relatively balanced, effectively avoiding classification bias caused by class imbalance. Furthermore, this dataset includes HSI and LiDAR data, making it suitable for validating the method proposed in this invention. The quantitative and visualization comparison results of this method with other state-of-the-art methods are shown below. Figure 4 and Figure 5 As shown.

[0041] like Figure 4 In the quantitative analysis table shown, red represents the optimal result and blue represents the second-best result. The table shows that the method of this invention achieved the highest classification accuracy and obtained both optimal and second-best results across the largest number of categories. Figure 5 As shown in the visualization results, (a) is a pseudo-color image, (b) is a ground truth image, and (c) is the prediction result image of the method of this invention. Compared with other more advanced methods, such as Figure 5 In (d) to (l), the method of the present invention is closest to the truth map, and the classification effect of multiple categories is significantly improved.

[0042] This article uses specific embodiments to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A multi-modal exploration system for planetary surface geology, characterized in that, The system includes: The interaction module is used to acquire and calibrate the multimodal features of hyperspectral and LiDAR images. Specifically, based on the principle of adaptive cue learning and feature calibration, it generates internal and cross-cues, and adjusts the contribution weight of multimodal features through a learning scaling factor. It performs intra-layer, inter-layer, and cross-modal feature interactions on the extracted multimodal features, dynamically balances the information density of different modal features, eliminates the differences in the initial feature distribution, and obtains the calibrated multimodal features. The feature decoupling module is used to separate the multimodal features into cross-modal shared consistent features and modality-specific features; The Feature-Guided Attention Mamba module is used to facilitate the interaction between cross-modal shared consistent features and modality-specific features. It generates attention scores using cross-modal shared consistent features, which serve as global semantic priors to guide the deep extraction of modality-specific features. This ensures that the modality-specific features focus on the task-relevant parts while suppressing noise. The Mamba module aggregates and fuses multimodal features, and the resulting fused features are fed into the classifier to complete the classification.

2. The planetary surface geology multimodal exploration system according to claim 1, characterized in that, In the feature decoupling module, the calibrated multimodal features are explicitly decoupled into cross-modal shared consistent features and modality-specific features through a shared encoder and a private encoder, and the decoupling process is constrained and optimized through self-reconstruction and cross-reconstruction loss constraints.

3. A multimodal exploration method for planetary surface geology, employing the multimodal exploration system for planetary surface geology as described in any one of claims 1-2, characterized in that, The method specifically includes: Image preprocessing: Acquire hyperspectral and LiDAR images and perform preprocessing. The dynamic feature-based interactive system processes the obtained hyperspectral and LiDAR images through an interactive module to acquire and calibrate multimodal features. Specifically, based on the principles of adaptive cue learning and feature calibration, internal and cross-cues are generated, and the contribution weights of multimodal features are adjusted through a learned scaling factor. Intra-layer, inter-layer, and cross-modal feature interactions are performed on the extracted multimodal features to dynamically balance the information density of different modal features, eliminate the differences in initial feature distribution, and obtain calibrated multimodal features. Feature mode decoupling involves explicitly decoupling the calibrated multimodal features through a feature decoupling module to obtain cross-modal shared consistent features and modality-specific features. Feature guidance and fusion involves the interaction between cross-modal shared consistent features and modality-specific features through the feature-guided attention Mamba module. The cross-modal shared consistent features are used to generate attention scores, which serve as global semantic priors to guide the deep extraction of modality-specific features. This ensures that the modality-specific features focus on the task-related parts while suppressing noise. The long sequence modeling capability of the Mamba module is used to complete the deep aggregation and multimodal fusion of the modality-specific features. The resulting fused features are then fed into the classifier to complete the classification.

4. The multi-modal exploration method for planetary surface geology according to claim 3, characterized in that, The image preprocessing specifically includes: dividing the hyperspectral image and LiDAR image into blocks, which are then divided into paired pixel patches.

5. The multimodal exploration method for planetary surface geology according to claim 4, characterized in that, In the dynamic feature prompting interaction, the acquired pixel patches are input into the interaction module. Based on the principle of adaptive prompting learning and feature calibration, internal prompts and cross prompts are generated. The contribution weight of multimodal features is adjusted by a learning scaling factor. Intra-layer, inter-layer, and cross-modal feature interactions are performed on the extracted multimodal features to dynamically balance the information density of different modal features, eliminate the differences in the initial feature distribution, and complete the balanced learning and deep extraction of multimodal features.

6. The multimodal exploration method for planetary surface geology according to claim 5, characterized in that, In the feature mode decoupling process, the calibrated multimodal features are explicitly decoupled into cross-modal shared consistent features and modality-specific features through a shared encoder and a private encoder. Simultaneously, optimization is performed through self-reconstruction and cross-reconstruction loss constraints, specifically including feature information integrity optimization, cross-modal shared consistent feature and modality-specific feature overlap optimization, and cross-modal shared feature semantic consistency optimization.

Images