A method and system for intelligent autonomy of a multimodal fusion core network based on images and text.

Abstract

This invention provides an intelligent autonomous method and system for a multimodal graph-text fusion core network, relating to the fields of mobile communication networks and artificial intelligence. Based on a spaceborne platform, this method constructs a unified multimodal fusion model to receive natural language instructions and a dynamic network topology graph, extracting graph structure features and text semantic features respectively. It utilizes a cross-attention mechanism to achieve bidirectional feature fusion between graph and text modalities, evaluates the dominance of each modality, and adaptively adjusts the configuration of self-attention and cross-modal attention heads. It combines MoE dynamic routing to select the target output head to generate task instructions. After secure verification in an isolated environment by a deterministic execution engine, it is executed through an atomic operation execution layer. Furthermore, a hierarchical federated learning framework is introduced to aggregate the parameter increments of various satellite models on the ground, achieving multi-satellite collaborative evolution. It also realizes graph-text joint inference, low-power multi-task processing, secure and reliable execution, and continuous model evolution, significantly improving the level of intelligence, reliability, and security.

Images