A multi-modal remote sensing image classification method under an arbitrary modality missing condition

By constructing a multimodal remote sensing image classification model, the problem of missing modalities was solved, and stable and high-performance classification under extreme conditions was achieved. The model is adaptable to arbitrary modal combinations, and its scalability and robustness to small sample classes are improved.

CN122023948BActive Publication Date: 2026-06-23NANJING UNIV OF INFORMATION SCI & TECH
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
NANJING UNIV OF INFORMATION SCI & TECH
Filing Date
2026-04-13
Publication Date
2026-06-23

AI Technical Summary

Technical Problem

Existing multimodal remote sensing image classification models are ill-equipped to handle modality loss caused by extreme weather or specific time periods. Furthermore, existing methods are computationally intensive, complex to train, and lack good scalability and flexibility when dealing with modality loss.

Method used

A multimodal remote sensing image classification method is designed under arbitrary modality loss conditions. By constructing encoders for hyperspectral, synthetic aperture radar and digital surface model images, and combining them with shared feature extraction and generation modules, a one-stage pre-training and two-stage fine-tuning approach is adopted. Attention mechanism and logic-guided gating fusion module are used for feature reconstruction and fusion.

Benefits of technology

It achieves stable high-performance classification even with arbitrary modal missing conditions, and has good scalability and versatility. It performs particularly well on small sample classes and imbalanced data, and the classification results are closer to the true labels. The classification of edge regions and dense regions is more refined.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122023948B_ABST
    Figure CN122023948B_ABST
Patent Text Reader

Abstract

The application discloses a kind of multi-modal remote sensing image classification methods under the condition of arbitrary modal absence, comprising: obtaining multi-modal remote sensing image data, which contains hyperspectral image and other modal images, and ground object type label;For hyperspectral image, construct hyperspectral image encoder, for the image of other modal, construct other modal image encoder, extract the real specific feature of each modal image;And construct modal shared feature encoder, extract the shared feature of each modal image;Real specific feature loss and shared feature alignment loss are constructed;Generate missing modal and generate specific feature;Total loss of one-stage pre-training is constructed, and one-stage pre-training is carried out;Two-stage fine-tuning is carried out, and total classification feature is constructed;After classifier, obtain the ground object classification of multi-modal remote sensing image;The method of the application has high flexibility, can adapt to the absence of any number of modal, and has higher ground object classification accuracy under the condition of modal absence.
Need to check novelty before this filing date? Find Prior Art