A cross-linguistic knowledge graph question answering method based on a translation model

Abstract

This invention provides a cross-language knowledge graph question answering method based on a translation model, comprising the following steps: Step 1: To answer the input question text, a global pointer network model is constructed based on the characteristics of the given training set and a pre-trained BERT model; Step 2: Based on Step 1, a key entity is received as input, and all possible answer paths are output. These paths satisfy the requirements of the baseline rules, including the transformation between knowledge graphs related to the key entity; Step 3: The original input query and corresponding paths proposed in Step 2 are received, and the most matching answer path is output. This invention solves the problem of cross-language joint application, achieving a balance between text and cross-language aspects, improving the training stability of the model, and increasing the accuracy of text matching, thus achieving a balance between text parsing and cross-language joint application.

Description

Technical Field

[0001] This invention relates to the field of natural language intelligent question answering technology, specifically to a cross-language knowledge graph question answering method based on a translation model. Background Technology

[0002] With the continuous development of NLP, various BERT models based on BERT_base have emerged, such as MacBert, DeBerta, and ALBert. These different BERT models are suitable for different tasks in NLP. In cross-lingual knowledge graph question answering tasks, pre-trained multilingual BERT formats have been widely used. However, current multilingual knowledge graph question answering focuses more on the natural language parsing of text sentences by multilingual BERT models, and does not pay much attention to the joint application of cross-lingual models in the question-answering path generation process. This results in no significant improvement in the performance of cross-lingual knowledge graph question answering across various BERT models, and the generated paths may not contain genuine question-answering paths.

[0003] Therefore, it is necessary to design a cross-language knowledge graph question answering method based on a translation model to solve the above-mentioned problems. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a cross-language knowledge graph question answering method based on a translation model, which solves the problems mentioned in the background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A cross-language knowledge graph question answering method based on a translation model includes the following steps:

[0007] Step 1: To answer the input question text, it is necessary to identify the key entities at the beginning of the answer path, treating it as a NER task in mixed language (Chinese, English, and French) text with only one category. Based on the characteristics of the given training set, a global pointer network model is built on the basis of the pre-trained BERT model.

[0008] Step 2: Based on Step 1, receive a key entity as input and output all possible answer paths that satisfy the baseline rules, including transformations between knowledge graphs related to the key entity;

[0009] The entity at the recognition point is regarded as the beginning of the answer path. There is a conversion of entities in different languages ​​during the path generation process. In order to solve this cross-language knowledge graph conversion, a Transformer-based translation model is built based on the English-Chinese subset of WMT17 (Machine Translation Workshop 2017) as the training set. If the translated name does not match the entity in the knowledge graph, all entity names are recorded in Elasticsearch. The target entity is recalled through Elasticsearch. In this process, the conversion between Simplified Chinese and Traditional Chinese is applied.

[0010] Step 3: Receive the original input query and corresponding path from Step 2, and output the most matching answer path;

[0011] The model takes as input a text concatenating the original input query and a candidate path (starting entity and predicate following the predicate), and outputs a matching score. This model is a RoBERTa-large-based classification model, with a lower-level model connected to a Dense+Sigmoid algorithm, ultimately outputting a score for a path. The path with the highest matching score is selected as the final prediction, and the positive samples in the training set are the original training set. Simultaneously, for each sample in the original training set, negative samples are constructed using some incorrect answer paths generated in step two. The ratio of positive to negative samples is 1:3, and the process iterates twice, adding indistinguishable negative samples to the training set.

[0012] Furthermore, the specific steps in step one are as follows:

[0013] S1. The training set is constructed in the form of (text, entities);

[0014] S2. The upper-layer model uses the Bert-base model, and the lower-layer model uses a global pointer network. The model is trained and predicted with all entity categories as one class, and finally the correct entity is identified.

[0015] S3, Bert-base, is a pre-trained model based on 104 languages ​​from Wikipedia, used as an encoder, with the output of Bert as the input to the lower-level model.

[0016] Furthermore, the specific steps in step two are as follows:

[0017] S1. Train a translation model on a transformer based on the English-Chinese subset of WMT17 (Machine Translation Workshop 2017);

[0018] S2. Generate the next hop for the entities identified in step one based on the triplet knowledge base. Then, translate the new entities based on the translation model and recall them based on ElasticSearch. Use the translated entities and the untranslated entities as the new starting entities to generate the next hop.

[0019] S3. After generating all candidate paths, use them as input for the next stage.

[0020] Furthermore, the specific steps in step three are as follows:

[0021] S1. Construct a training set in the form of (text + first starting entity + predicate). Positive samples are the original dataset, and negative samples are sampled from all candidate paths in a ratio of 1:3.

[0022] S2. The upper-level model uses Roberta-large, and the lower-level model uses Dense+Sigmoid. A logical value is output for all paths, which is used as the score for each path.

[0023] S3. The trained model makes predictions on the training set, adds samples that are difficult to distinguish to the training set, and iteratively generates a new dataset.

[0024] S4. Use all the paths in step two as the test set, and take the path with the highest score for the same entity as the final answer.

[0025] This invention provides a cross-language knowledge graph question answering method based on a translation model. Compared with existing technologies, it has the following advantages:

[0026] 1. This invention solves the problem of cross-language joint application, achieving both text and cross-language compatibility. Compared with existing technologies, it is more suitable for the field of cross-language knowledge graph question answering, with higher accuracy and more stable model training.

[0027] 2. This invention improves the training stability of the model and the accuracy of text matching. Compared with existing technologies, it achieves both text parsing and cross-language collaboration. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Example

[0030] To address the technical issues in the background section, the following cross-language knowledge graph question answering method based on a translation model is proposed:

[0031] This invention provides a cross-language knowledge graph question answering method based on a translation model, comprising the following steps:

[0032] Step 1: To answer the input question text, it is necessary to identify the key entities at the beginning of the answer path, treating it as a NER task in mixed language (Chinese, English, and French) text with only one category. Based on the characteristics of the given training set, a global pointer network model is built on the basis of the pre-trained BERT model.

[0033] The specific steps in step one are as follows:

[0034] S1. The training set is constructed in the form of (text, entities);

[0035] S2. The upper-layer model uses the Bert-base model, and the lower-layer model uses a global pointer network. The model is trained and predicted with all entity categories as one class, and finally the correct entity is identified.

[0036] S3, Bert-base, is a pre-trained model based on 104 languages ​​from Wikipedia, used as an encoder, with the output of Bert as the input to the lower-level model.

[0037] Step 2: Based on Step 1, receive a key entity as input and output all possible answer paths that satisfy the baseline rules, including transformations between knowledge graphs related to the key entity;

[0038] The entity at the recognition point is regarded as the beginning of the answer path. There is a conversion of entities in different languages ​​during the path generation process. In order to solve this cross-language knowledge graph conversion, a Transformer-based translation model is built based on the English-Chinese subset of WMT17 (Machine Translation Workshop 2017) as the training set. If the translated name does not match the entity in the knowledge graph, all entity names are recorded in Elasticsearch. The target entity is recalled through Elasticsearch. In this process, the conversion between Simplified Chinese and Traditional Chinese is applied.

[0039] The specific steps in step two are as follows:

[0040] S1. Train a translation model on a transformer based on the English-Chinese subset of WMT17 (Machine Translation Workshop 2017);

[0041] S2. Generate the next hop for the entities identified in step one based on the triplet knowledge base. Then, translate the new entities based on the translation model and recall them based on ElasticSearch. Use the translated entities and the untranslated entities as the new starting entities to generate the next hop.

[0042] S3. After generating all candidate paths, use them as input for the next stage.

[0043] Step 3: Receive the original input query and corresponding path from Step 2, and output the most matching answer path;

[0044] The model takes as input a text concatenating the original input query and a candidate path (starting entity and predicate following the predicate), and outputs a matching score. This model is a RoBERTa-large-based classification model, with a lower-level model connected to a Dense+Sigmoid algorithm, ultimately outputting a score for a path. The path with the highest matching score is selected as the final prediction, and the positive samples in the training set are the original training set. Simultaneously, for each sample in the original training set, negative samples are constructed using some incorrect answer paths generated in step two. The ratio of positive to negative samples is 1:3, and the process iterates twice, adding indistinguishable negative samples to the training set.

[0045] The specific steps in step three are as follows:

[0046] S1. Construct a training set in the form of (text + first starting entity + predicate). Positive samples are the original dataset, and negative samples are sampled from all candidate paths in a ratio of 1:3.

[0047] S2. The upper-level model uses Roberta-large, and the lower-level model uses Dense+Sigmoid. A logical value is output for all paths, which is used as the score for each path.

[0048] S3. The trained model makes predictions on the training set, adds samples that are difficult to distinguish to the training set, and iteratively generates a new dataset.

[0049] S4. Use all the paths in step two as the test set, and take the path with the highest score for the same entity as the final answer.

[0050] In summary, this invention solves the problem of cross-language joint application, achieving both text and cross-language compatibility, while improving the training stability of the model and the accuracy of text matching, thus achieving both text parsing and cross-language joint application.

[0051] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0052] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cross-language knowledge graph question answering method based on a translation model, characterized in that: Includes the following steps: Step 1: To answer the input question text, it is necessary to identify the key entities at the beginning of the answer path. Treat it as a NER task in a mixed Chinese, English and French text with only one category. Based on the characteristics of the given training set, a global pointer network model is built on the basis of the pre-trained BERT model. Step 2: Based on Step 1, receive a key entity as input and output all possible answer paths that satisfy the baseline rules, including transformations between knowledge graphs related to the key entity; Step 3: Receive the original input query and corresponding path from Step 2, and output the most matching answer path; The specific steps in step one are as follows: S1. The training set is constructed in the form of (text, entities); S2. The upper-layer model uses the Bert-base model, and the lower-layer model uses a global pointer network. The model is trained and predicted with all entity categories as one class, and finally the correct entity is identified. S3 and Bert-base are pre-trained models trained on 104 languages ​​from Wikipedia, which are used as encoders and the output of Bert is used as the input of the lower-level models. The specific steps in step two are as follows: S1. Train a translation model on a transformer based on the English-Chinese subset of WMT17 (Machine Translation Workshop 2017); S2. Generate the next hop for the entities identified in step one based on the triplet knowledge base. Then, translate the new entities based on the translation model and recall them based on ElasticSearch. Use the translated entities and the untranslated entities as the new starting entities to generate the next hop. S3. After generating all candidate paths, use them as input for the next stage; The specific steps in step three are as follows: S1. Construct a training set in the form of (text + first starting entity + predicate). Positive samples are the original dataset, and negative samples are sampled from all candidate paths in a ratio of 1:

3. S2. The upper-level model uses Roberta-large, and the lower-level model uses Dense + Sigmoid. A logical value is output for all paths, which is used as the score for each path. S3. The trained model makes predictions on the training set, adds samples that are difficult to distinguish to the training set, and iteratively generates a new dataset. S4. Use all the paths in step two as the test set, and take the path with the highest score for the same entity as the final answer.