A method and apparatus for processing a sequence model

Abstract

The embodiment of the invention discloses a method and apparatus for processing a sequence model, which are used for improving the task execution effect of the sequence model. The method comprises thefollowing steps of: obtaining a source end sequence from a source end database and inputting the source end sequence to an encoding end of a sequence model, wherein the encoding end comprises a self-attention encoder and a timing encoder; encoding The source sequence by a timing encoder to obtain a first encoding result, wherein the first encoding result comprises timing information obtained by using the source sequence to perform timing modeling; And performing encoding processing on the source sequence by a self-attention encoder to obtain a second encoding result; Obtaining a target end sequence from a target end database, and inputting the target end sequence, a first coding result and a second coding result to a decoding end of a sequence model; using A decoding end to perform decoding processing on a target end sequence, a first encoding result and a second encoding result, and outputting a decoding result obtained after the decoding processing.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a sequence model processing method and device. Background technique [0002] The sequence model can be used to realize various tasks according to the input sequence data, and the sequence model can be realized based on the self-attention neural network (Self-Attention Network, SAN). The sequence model may be a neural machine translation model. For example, there is a Transformer model in the prior art. The Transformer model is based on the aforementioned self-attention neural network and is formed by stacking multiple layers of self-attention neural networks. [0003] Compared with the neural machine translation model based on Recurrent Neural Network (RNN) in the prior art, the Transformer model uses a self-attention neural network instead of a recurrent neural network to model sequence dependencies. While RNN operates sequentially using loops (that is, the output of each ste...

AI Technical Summary

Problems solved by technology

[0004] Since the sequence model in the prior art only relies on SAN to calculate the dependency relationship of sequence elements, there is a problem of poor task realization effect when using this sequence model

For example, the Transformer model in the prior art only relies on SAN to calculate the dependency relationship of sequence elements, which will affect the translation quality, usually affecting the translation quality, including translation errors, translation omissions, etc.

Images