A collaborative method and system that supports heterogeneous modal models and is scalable

Abstract

This invention provides a collaborative method and system that supports modal model heterogeneity and scalability, comprising: end-to-end training of a collaborative network of multiple base agents, constructing a shared feature space, and performing collaborative mutual detection; transforming the features of new heterogeneous agents into the shared feature space using offline training and knowledge transfer, and adding extended collaboration for mutual detection. This invention simultaneously solves the problems of modality, model heterogeneity, and scalability; supports sharing heterogeneous information at the feature level, improving the benefits of multi-agent collaboration; new agent types are merged into the existing collaborative framework, requiring only offline training of the feature encoders of the new agents, significantly reducing training costs, and exhibiting high scalability for new agent types; significantly reducing usage costs, and possessing advanced and accurate detection performance. This invention can collect only limited collaborative data, reuse data using offline training, and infinitely expand the heterogeneous agent types supported by the collaborative framework.

Images