Data processing method, data processing device, electronic equipment and storage medium

Abstract

The invention provides a data processing method, a data processing device and electronic equipment. The method comprises the following steps: acquiring a fine-tuning training sample set corresponding to a target interaction scene, wherein the fine-tuning training sample set comprises fine-tuning training samples and intention labels corresponding to the fine-tuning training samples; performing semantic coding on the fine-tuning training sample through a first module in an intention recognition model, and performing intention prediction on a semantic coding result through a second module in the intention recognition model to obtain an intention prediction result corresponding to the fine-tuning training sample, the first module is obtained by performing unsupervised pre-training based on historical dialogue data in an associated scene of a target interaction scene and a single-head attention mechanism. And performing fine-tuning training on the second module based on the intention label of the fine-tuning training sample and the intention prediction result of the fine-tuning training sample. According to the method, the training of the intention recognition model can be quickly completed, the dependence on the labeled sample is reduced, and the method is particularly suitable for application scenes of cold start and small sample support.

Description

technical field [0001] The present application belongs to the technical field of artificial intelligence, and in particular, relates to a data processing method, a data processing device, an electronic device and a storage medium. Background technique [0002] The core of human-computer interaction is that the machine recognizes the user's intention to output accurate feedback information for the user's intention. Machine recognition of user intent is usually implemented based on language models. The structures of several commonly used language models are extremely complex. Taking the mainstream converter-based Bidirectional Encoder Representations from Transformers (Bert) model as an example, the The various functional layers in the language model can be as high as hundreds, and the parameters can reach hundreds of millions of orders of magnitude, which results in a huge amount of computation for such language models and relatively low training efficiency. [0003] Therefo...

AI Technical Summary

Problems solved by technology

Machine recognition of users' intentions is usually based on language models. The structures of several commonly used language models are extremely complex. Take the mainstream Bidirectional Encoder Representations from Transformers (Bert) model based on transformers as an example. Type The various functional layers in the language model can be as high as hundreds, and the parameters have reached the order of hundreds of millions, which leads to a huge amount of calculation for this type of language model and relatively low training efficiency.

Images