The invention is applicable to the technical field of
remote sensing, and provides a
remote sensing multi-
modal reasoning method based on a
hybrid expert mechanism, which comprises the following steps of: constructing a
hybrid expert enhanced
geographic space perception visual language basic model; constructing a
remote sensing data set for model training, which comprises two stages: in the stage I, forming a basic
data set by aggregating existing remote
sensing data sets across tasks, and in the
stage II, improving the diversity of the
data set by adopting a data enhancement technology; a double-stage training strategy is adopted, pre-training initialization and sparse architecture optimization are combined, and meanwhile, a total
loss function containing autoregression loss and auxiliary loss is designed to
train the model. According to the method, task requirements are adapted through a mixed expert mechanism, multi-
modal features are fused, a sequence long-range dependency relationship is captured, performance is optimized, a diversified remote
sensing data set is constructed, and a model is promoted to distinguish the global situation and details. According to the method, the performance of the remote sensing
perception task is improved, and the powerful performance of the reasoning task is maintained.