Training method and device of knowledge pre-training model and electronic equipment

Abstract

The invention discloses a training method and device of a knowledge pre-training model and electronic equipment, and relates to the technical field of voice, natural language processing and deep learning. According to the specific implementation scheme, a training text is acquired, the training text comprises a structured knowledge text and a corresponding article, and the structured knowledge text comprises a head node, a tail node and a relationship between the head node and the tail node; and the knowledge pre-training model is trained to be trained according to the training text. Accordingto the method, the knowledge pre-training model to be trained can learn common knowledge and rich semantic knowledge at the same time, joint training of the common knowledge and the semantic knowledge can be achieved, and a training entity does not need to be embedded into the knowledge pre-training model to be trained; the performance gain of the knowledge pre-training model is not limited by the embedding quality of the training entity, the knowledge pre-training model can obtain rich context information from articles in the training text, dynamic adjustment can be carried out, and the flexibility is high.

Description

technical field [0001] The present disclosure relates to the technical fields of language, natural language processing, and deep learning in the field of computer technology, and in particular to a training method, device, electronic equipment, storage medium, and computer program product of a knowledge pre-training model. Background technique [0002] At present, most of the models do not have common sense reasoning ability. For example, if the question is "using what can use ink to copy a document on paper", the answer includes pen, copier, carbon paper, notebook, and people can correctly choose the answer of copier based on common sense, However, due to the high co-occurrence frequency of carbon paper with the carbon and paper in the question, the model is likely to choose the answer of carbon paper, causing the model to output wrong results. The model training method in the related art cannot realize the joint training of common sense learning and semantic learning, and ...

AI Technical Summary

Problems solved by technology

[0002]Currently, most of the models are not capable of commonsense reasoning, for example, if the question is "with what can use ink to make a copy of a document on paper", the answer includes pen, photocopier , carbon paper, notebook, people can correctly choose the answer of the copier based on common sense, however, due to the high frequency of co-occurrence of carbon paper and the carbon and paper in the question, the model is likely to choose the answer of the carbon paper, resulting in the wrong result of the model output

The model training method in the related art cannot realize the joint training of common sense learning and semantic learning, and the model gain is limited by the quality of the sample, and often needs to retrain the model, which is less flexible

Images