Model training method, machine translation method and related devices and equipment

Abstract

The embodiment of the invention discloses a neural network model training method, device and equipment and a medium. The method comprises the steps: acquiring a training sample set comprising trainingsamples and standard label vectors corresponding to the training samples; inputting the training sample into a neural network model comprising a plurality of attention networks; performing nonlineartransformation on the respective output vectors of the attention networks through the neural network model to obtain feature fusion vectors corresponding to the attention networks; and obtaining a neural network model, outputting a prediction label vector according to the feature fusion vector, and adjusting model parameters of the neural network model according to a comparison result of the prediction label vector and a standard label vector until a convergence condition is met, thereby obtaining a target neural network model. The output vectors of all the attention networks are fused in a nonlinear transformation mode, so that the output vectors of all the attention networks are fully interacted, a feature fusion feature vector with more information amount is generated, and the final output representation effect is ensured to be better.

Description

technical field [0001] The present application relates to the field of computer technology, and in particular to a neural network model training method, a machine translation method, a neural network model training device, a machine translation device, equipment, and a computer-readable storage medium. Background technique [0002] In recent years, the attention mechanism (Attention Mechanism) has been widely used in various tasks of deep learning-based natural language processing (Netural Language Processing, NLP), such as machine translation, intelligent question answering, speech recognition and other tasks. [0003] At present, the multi-headed attention mechanism is widely used. The so-called multi-headed attention mechanism refers to learning different features through multiple attention networks, that is, capturing relevant information on different subspaces by computing multiple times. [0004] However, the current model based on the multi-head attention mechanism tr...

AI Technical Summary

Problems solved by technology

[0004] However, the current model based on the multi-head attention mechanism treats each subspace independently during the training process, and does not pay attention to the correlation between each subspace. Therefore, the learning representation ability of the current model based on the multi-head attention mechanism is still relatively weak.

Images