A method and apparatus for processing a sequence model

Abstract

The embodiment of the invention discloses a method and apparatus for processing a sequence model, which are used for improving the task execution effect of the sequence model. The method comprises thefollowing steps of: obtaining a source end sequence from a source end database and inputting the source end sequence to an encoding end of a sequence model, wherein the encoding end comprises a self-attention encoder and a timing encoder; encoding The source sequence by a timing encoder to obtain a first encoding result, wherein the first encoding result comprises timing information obtained by using the source sequence to perform timing modeling; And performing encoding processing on the source sequence by a self-attention encoder to obtain a second encoding result; Obtaining a target end sequence from a target end database, and inputting the target end sequence, a first coding result and a second coding result to a decoding end of a sequence model; using A decoding end to perform decoding processing on a target end sequence, a first encoding result and a second encoding result, and outputting a decoding result obtained after the decoding processing.

Description

technical field

[0001] The invention relates to the field of computer technology, in particular to a sequence model processing method and device. Background technique

[0002] The sequence model can be used to realize various tasks according to the input sequence data, and the sequence model can be realized based on the self-attention neural network (Self-Attention Network, SAN). The sequence model may be a neural machine translation model. For example, there is a Transformer model in the prior art. The Transformer model is based on the aforementioned self-attention neural network and is formed by stacking multiple layers of self-attention neural networks.

[0003] Compared with the neural machine translation model based on Recurrent Neural Network (RNN) in the prior art, the Transformer model uses a self-attention neural network instead of a recurrent neural network to model sequence dependencies. While RNN operates sequentially using loops (that is, the output of each ste...

Images