A cross-modal model training method based on variational guided image local information routing

Abstract

The application discloses a cross-modal model training method based on variational guided image local information routing and belongs to the technical field of image classification. The method comprises the following steps: in the first step, a combined zero sample learning data set is acquired and pretreated; in the second step, a cross-modal model based on variational guided image local information routing is constructed; in the third step, the model is trained and fitted, and the model is evaluated; and in the fourth step, the model is evaluated and deployed. Without changing the original training target and loss function, the visual word is discriminatively guided and sequentially injected into the cross-modal process, the denoised visual feature is added to the text feature in the multiple paths, the ideal embedding for the combination recognition is formed, and the generalization ability of the model in the combined zero sample task is improved.

Images