Neural network training method, apparatus, computer device and storage medium

Abstract

The present application relates to a neural network training method, an apparatus, a computer readable storage medium and a computer device. The method includes: acquiring a training sample set, wherein each training sample in the training sample set has a corresponding standard tag; the training samples in the training sample set being inputted into the neural network model, wherein the neural network model includes a plurality of attention networks; the training samples being respectively mapped to a plurality of different subspaces, wherein each subspace includes a corresponding request vector sequence, a key vector sequence and a value vector sequence; the neural network model being used to calculate the spatial difference between each subspace; the output similarity being calculated according to the output of the neural network model and the standard label corresponding to each training sample; according to the spatial difference degree and output similarity degree, the parametersof neural network model being adjusted until the convergence condition is satisfied, and the target neural network model being obtained. The scheme provided by the present application can improve theaccuracy of the output result of the neural network model.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to a neural network training method, device, computer equipment and storage medium. Background technique [0002] Attention Mechanism (Attention Mechanism) is a method for building a model for the dependency relationship between the hidden state of the encoder and the decoder in the neural network. The attention mechanism is widely used in deep learning-based natural language processing (NLP, Natural Language Processing) in each task. [0003] At present, the ability of the attention network in the neural network model based on the attention mechanism to capture feature information is weak, resulting in low accuracy of the output results of the neural network model based on the attention mechanism. Contents of the invention [0004] Based on this, it is necessary to address the above technical problems and provide a neural network training method, device, computer equ...

AI Technical Summary

Problems solved by technology

[0003] At present, the ability of the attention network in the neural network model based on the attention mechanism to capture feature information is weak, resulting in low accuracy of the output results of the neural network model based on the attention mechanism.

Images