The application discloses a kind of computing and communication efficiency cross-
modal model collaborative training method and
system, training method includes the following steps: using the
hybrid network structure of fusion local
perception and global
semantic association, the model size is compressed by combining structure reparameterization technology to construct light cross-
modal basic
model architecture;Constructing a minimalist feature
adaptation module, adding a minimalist feature
adaptation module at the output end of the cross-
modal basic
model architecture, by gradually reducing its parameter quantity to a minimalist linear transformation form, local feature differentiation tuning is realized, while reducing the amount of communication transmission data;In local training, by comparing the feature distribution difference of public reference
data set and local data, the distribution distance
loss function is minimized, to promote the feature space of each participant to gradually align;
Server side maintains global feature
adaptation module parameters, each participant uploads updates after fine-tuning the module based on local data, and the
global model is optimized by weighted average.