The invention provides a multi-
modal neural
machine translation method for scene self-adaption based on visual fantasy guidance, and a scene self-adaption
translation function is realized by fusing different
modal features and continuously reducing differences among modals by utilizing visual fantasy as guidance. According to the method, an asymmetric alternative updating strategy is introduced, so that the
modal gap is reduced, and the characterization capability of the multi-
modal data is improved. And finally, establishing a cross-modal adaptive
fusion mechanism to effectively fuse the two modals. According to the method, the multi-modal representation learning ability of the model in a complex and changeable image-text environment can be effectively improved, and it is ensured that the model can still maintain high
machine translation performance even in a
noisy data or
plain text scene in the reasoning stage. According to the method, the optimal
score superior to that of other prior art is achieved on most sets, it is proved that the method shows outstanding stability and consistency in various scenes, especially in a
noise environment, and the
machine translation performance is powerfully improved.