A Joint Extraction Method of Entity Relationships Based on Automatic Question Answering

通过基于自动问答的实体关系联合抽取方法,解决了现有技术中跨句和多元关系识别困难的问题,实现了识别未预先定义关系类型的能力,提高了实体关系抽取的准确性和适用性。

CN113836281BActive Publication Date: 2024-11-12NAT UNIV OF DEFENSE TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111068697.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-13
Publication Date
2024-11-12
Estimated Expiration
2041-09-13

AI Technical Summary

Technical Problem

The existing entity relationship extraction model has difficulties in cross-sentence relationship recognition and multiple relationship recognition, and predefined relationship types cannot recognize unpredefined relationship types.

Method used

The joint extraction method of entity relationships based on automatic question-and-answer is adopted, and the question templates are extracted through data preprocessing, entity and relationships are designed, and question-and-answer pairs are generated using the pre-trained language model, and joint training is carried out to output triplets to identify cross-sentences and multiple relationships.

Benefits of technology

Effectively identify cross-sentence relationships and multiple relationships, identify unpredefined entity relationship types, and improve the accuracy and applicability of entity relationship extraction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113836281B_ABST
    Figure CN113836281B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for joint extraction of entity relations based on automatic question-answering, which belongs to the technical field of information extraction and natural language processing. The method includes preprocessing data; designing entity extraction question templates and generating entity extraction question-answer pairs; extracting entities based on a first automatic question-answering model of a pre-trained language model; combining the identified entities with the relationship extraction question template designed according to the characteristics of the data set, and generating relationship extraction question-answer pairs in an unsupervised manner; extracting relations based on a second automatic question-answering model of a pre-trained language model; jointly training the first automatic question-answering model and the second automatic question-answering model; and converting the relationship extraction question-answer pairs and the output tail entity into triples. The present invention converts entity relationship extraction into a question-answering task, which can effectively solve the problem that cross-sentence relations and multi-relationships are difficult to identify in entity relationship extraction, and can identify entity relationship types that are not pre-defined in the training set.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of information extraction and natural language processing, and specifically relates to a method for joint extraction of entity relationships based on automatic question answering. Background Art

[0002] In the era of big data, how to quickly and efficiently extract valuable information from massive unstructured or semi-structured data has become an important issue facing people, and information extraction technology has emerged. Entity extraction and relationship extraction, as key steps in information extraction, have received widespread attention from academia and industry in recent years. Entity extraction, also known as named entity recognition, is the task of identifying entities with specific meanings in text, such as names of people, organizations, institution names, geographic locations, etc. Relationship extraction, that is, after identifying key entities, determines the semantic relationship between entities. As an important part of natural language processing, entity relationship extraction can quickly extract effective semantic information from unstructured text, providing strong support for downstream tasks such as text summarization, machine translation, and knowledge graph construction.

[0003] Entity relationship extraction is often treated as a sequence labeling task, facing the following challenges: 1) Lack of high-quality manually labeled data. Training deep neural network models for entity relationship extraction requires a large amount of labeled data. Although manual labeling is of high quality, it is time-consuming and labor-intensive. Automatic labeling methods such as template matching save time costs but introduce noise. 2) Cross-sentence entity relationships are difficult to identify. Existing entity relationship extraction models are mainly aimed at the sentence level, that is, identifying entities in a sentence and the relationships between them. In real scenarios, entity relationships may appear across sentences, making judgment more difficult. 3) Pre-defined relationship types do not conform to application scenarios. When labeling training data, several entity types and relationship types are pre-defined. After training on labeled data, existing models can only recognize pre-defined entity relationship types, but cannot recognize types that exist in real scenarios but are not pre-defined. 4) Difficulty in identifying multiple relationships. Existing entity relationship extraction models can only recognize a single relationship. If multiple relationships appear in a sentence and the entities overlap, it is difficult to accurately identify them. Summary of the invention

[0004] Technical problem: In response to the above problems, the present invention proposes a method for joint extraction of entity relationships based on automatic question answering. The method for joint extraction of entity relationships based on automatic question answering is more in line with the general process of human information extraction, and effectively solves problems such as cross-sentence relationship recognition and multi-party relationship recognition.

[0005] Technical solution: The present invention provides a method for joint extraction of entity relationships based on automatic question answering, comprising:

[0006] Preprocess the data, including at least word segmentation and part-of-speech tagging;

[0007] Design entity extraction question templates based on the characteristics of the dataset, and generate entity extraction question-answer pairs based on the templates;

[0008] Input the entity extraction question and the given text into the first automatic question answering model based on the pre-trained language model, and output the entity;

[0009] Combining the identified entities with the relation extraction question template designed according to the characteristics of the dataset, an unsupervised approach is used to generate relation extraction question-answer pairs.

[0010] Input the relation extraction question and the given text into the second automatic question answering model based on the pre-trained language model, and output the tail entity;

[0011] Jointly training the first automatic question answering model and the second automatic question answering model;

[0012] The relation extraction question-answer pair and the output tail entity are converted into triples, and a triple consisting of subject, predicate, and object is output, corresponding to the head entity, relation, and tail entity respectively.

[0013] Furthermore, the word segmentation method is: dividing a continuous character sequence into a word sequence according to a specification;

[0014] The method of part-of-speech tagging is: tagging each word in the word segmentation result with a correct part-of-speech.

[0015] Furthermore, the method of designing an entity extraction question template according to the characteristics of the data set and generating an entity extraction question-answer pair according to the template includes:

[0016] By analyzing the data set, different questions are designed for different types of head entities to form an entity extraction question template;

[0017] When a piece of input text is given, all head entity types are traversed to ask questions. If a head entity of this type exists in the text, the head entity is marked as the answer to the question. If a head entity of this type does not exist, the answer is marked as None, and an entity extraction question-answer pair is constructed.

[0018] Furthermore, the method for constructing the first automatic question-answering model based on the pre-trained language model includes:

[0019] Concatenate the input text and question and embed them into code;

[0020] Input the results of text and question embedding encoding into the pre-trained language model;

[0021] Fine-tune the output layer of the pre-trained language model.

[0022] Furthermore, the method of splicing the input text and the question is: given a question and text Where N q and N c Represent the number of words in the question and text respectively. The question and text are concatenated with special symbols [CLS] and [SEP]. The input format is [CLS]Q[SEP]C[SEP];

[0023] The embedding coding method is: taking the sum of the word vector, the fragment vector and the position vector of each word in the sequence [CLS]Q[SEP]C[SEP] as the embedding code.

[0024] Furthermore, the pre-trained language model is a BERT model.

[0025] Furthermore, the method for fine-tuning the output layer of the pre-trained language model is:

[0026] The output of the BERT model is input into the fully connected layer, followed by Softmax.

[0027] The positions with the highest probability of the starting position and the ending position are obtained, and the sequence from the starting position to the ending position is used as the output of the predicted answer.

[0028] Furthermore, it also includes: training a first automatic answering model based on the pre-trained model, and the training method is: by minimizing the probability sum of the negative logarithms of the start and end positions, the formula is as follows:

[0029]

[0030] Where N is the number of examples in the training set, and is the start and end position of the true tag of example i, The predicted answer starts at The probability of The predicted answer ends at The probability of , θ represents the set of all trainable weights in the model;

[0031] Give the confidence score vector p for the start and end positions of the predicted answer b and p e Concatenate a trainable bias t respectively, and the new confidence score vector is expressed as Where N S Indicates the number of words in a sentence. ";" means splicing by row;

[0032] The probability of no answer is expressed as: Where a represents the answer, Indicates no answer; when When the probability of is greater than the score of the best answer fragment, then the question has no answer.

[0033] Furthermore, the method of combining the identified entities with the relation extraction question template designed according to the characteristics of the data set to generate relation extraction question-answer pairs in an unsupervised manner is:

[0034] By analyzing the data set, different questions are designed for different types of head entities and relations to form a relation extraction template;

[0035] For each possible relationship of the entity identified by entity extraction, questions are asked according to the question template, and "xx" in the question template is replaced with the entity identified by entity extraction. If the entity has such a relationship in the text, the tail entity is used as the answer to the question. Otherwise, if such a relationship does not exist, the answer to the question is None. This method is used to loop through all samples to construct relationship extraction question and answer pairs.

[0036] Furthermore, the second automatic question answering model based on the pre-trained language model and the first automatic question answering model based on the pre-trained language model adopt the same model, and the two models share parameters during the training process.

[0037] Furthermore, the method for jointly training the first automatic question answering model and the second automatic question answering model is:

[0038] Using the objective function: L = (1-λ)L entity +λL relation Two automatic question answering models are trained, where L entity and L relation They represent the negative logarithmic probabilities of the start and end positions of entity extraction and relationship extraction respectively, and λ∈[0,1] is a parameter for adjusting the weights of the two objective functions of entity extraction and relationship extraction.

[0039] Compared with the prior art, the present invention has the following advantages: through data preprocessing, question-and-answer based entity extraction, question-and-answer based relationship extraction, triple conversion and other steps, the entity relationship extraction task is converted into an automatic question-and-answer task, which effectively gives play to the advantages of the question-and-answer model, can solve the problem of cross-sentence relations and multi-party relations being difficult to identify in entity relationship extraction, and is more in line with the general process of information extraction by humans, and can identify entity relationship types that are not pre-defined in the training set. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The figure is a flowchart of a method for joint extraction of entity relationships based on automatic question answering in an embodiment of the present invention.

[0041] Figure 2 Entity (relationship) extraction model diagram of automatic question answering in an embodiment of the present invention;

[0042] Figure 3 It is a flow chart of an algorithm for unsupervised generation of relation extraction question-answer pairs in an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of an application example of the present invention. DETAILED DESCRIPTION

[0044] Figure 1 The flowchart of the entity relationship joint extraction method based on automatic question answering in an embodiment of the present invention is shown. Figure 1 As shown, the method includes:

[0045] S100: Preprocess the data, including at least word segmentation and part-of-speech tagging. Specifically, other preprocessing operations can be added according to the data situation, such as removing stop words. The word segmentation method is to divide the continuous word sequence into word sequences according to the specification; the part-of-speech tagging method is to mark each word in the word segmentation result with a correct part-of-speech; the method of removing stop words is to delete meaningless words such as auxiliary words and modal particles in the text according to the stop word list. For example, the input text is: Zhang Sisi, the wife of Zhang Sansan, an official of country K, designed a pair of "silk indoor slippers" for Li Liuliu, the daughter of Li Wuwu, an official of country A. Jieba is used for word segmentation, and the result after word segmentation is: K country / official / Zhang Sansan / wife / Zhang Sisi / designed / a / pair / of / " / silk / indoor slippers / " / for / Li Liuliu, the daughter of / Li Wuwu, an official of country A. The results after word segmentation are tagged with the HanLP tool, and the results are: K country (ns) / official (n) / Zhang Sansan (nr) / wife (n) / Zhang Sisi (nr) / for (p) / A country (ns) / official (n) / Li Wuwu (nr) / of (u) / female (n) / Li Liuliu (nrf) / designed (vn) / made (ul) / a pair (m) / "(x) / silk (n) / indoor slippers (n) / "(x) / . (x)

[0046] S200: Design an entity extraction question template according to the characteristics of the data set, and generate an entity extraction question-answer pair according to the template. In one embodiment of the present invention, a dataset for relational extraction of the polymorphic information extraction task in the 2021 Language and Intelligent Technology Competition is used. The goal of this task is to extract all subject-predicate-object triples that meet the pattern constraints for a given natural language sentence according to a predefined set of patterns, and the sentence includes multiple relations. The training set includes 171,293 pieces of data, of which the input natural language sentences are up to 300 words long, and there are long-distance cross-sentence relationships. The entity types in the training set include 15 categories, including places, TV variety shows, songs, countries, administrative regions, institutions, attractions, historical figures, enterprises, characters, books, online novels, academic disciplines, film and television works, and entertainment figures. The question templates designed for these 15 types of subjects are shown in Table 1.

[0047] Table 1 Entity extraction question generation template

[0048]

[0049] The method of generating entity extraction question-answer pairs based on the designed template is: when a piece of input text is given, all subject types are traversed to ask questions. If an entity of this type exists in the text, the head entity is marked as the answer to the question. If the head entity of this type does not exist, the answer is marked as None. This method is used to construct entity extraction question-answer pairs. For example, the input text is: Zhang Sisi, the wife of Zhang Sansan, an official of country K, designed a pair of "silk indoor slippers" for Li Liuliu, the daughter of Li Wuwu, an official of country A. The constructed entity extraction question-answer pairs are: (Which place is mentioned in the article? None), (What is the TV variety show mentioned in the article? None), (What is the song mentioned in the article? None), (What country is mentioned in the article? Country K; Country A), (What is the administrative region mentioned in the article? None), (What is the organization mentioned in the article? None), (Which scenic spot is mentioned in the article? None), (Who is the historical figure mentioned in the article? None), (What is the company mentioned in the article? None), (Who is the person mentioned in the article? Zhang Sansan; Zhang Sisi; Li Wuwu; Li Liuliu), (What is the book mentioned in the article? None), (What is the online novel mentioned in the article? None), (What is the subject mentioned in the article? None), (What is the film and television work mentioned in the article? None), (Who is the entertainment figure mentioned in the article? None).

[0050] S300: Input the entity extraction question and the given text into the first automatic question answering model based on the pre-trained language model, and output the entity. The question answering model based on the pre-trained language model includes embedding encoding input, pre-training, and output fine-tuning that maps words in a high-dimensional discrete space to low-dimensional continuous space vectors.

[0051] In an embodiment of the present invention, the main body of the automatic question answering model based on the pre-trained language model is the BERT model trained for Chinese published by Google, that is, the pre-trained language model is the BERT model. Figure 2 ,The process of building the first automatic question answering model based on the pre-trained language model includes the following steps:

[0052] S310: concatenate the input text and the question, and embed the text and the question into a code. Specifically, given a question Q = {q1, q2, ..., q Nq} and text C = {c1,c2,…,c Nc}, where N q and N c Represent the number of words in the question and text respectively. The question and text are concatenated with special symbols [CLS] and [SEP]. The input format is [CLS]Q[SEP]C[SEP]. The sum of the word vector, segment vector and position vector of each word in the sequence [CLS]Q[SEP]C[SEP] is used as the embedding code.

[0053] S320: Input the results of text and question embedding coding into the pre-trained language model. The pre-trained language model adopts the BERT model trained for Chinese published by Google, that is, the pre-trained language model is the BERT model. The embedding coding in step S310 is input into the BERT model pre-trained for Chinese, where the segment vector is used to distinguish between questions and texts, and the position vector encodes the position of each word in the text. The BERT model is a multi-layer bidirectional Transformer structure. In the pre-training stage, it effectively encodes context information through two tasks: a masked language model and next sentence prediction, so as to learn grammar, syntax and other information.

[0054] S330: Fine-tune the output layer of the pre-trained language model. In order for the BERT model to complete the question-answering task, its output layer needs to be fine-tuned. In the question-answering task, the popular way to predict the answer is to predict the start and end positions of the answer in the text, so the output result of the BERT model is input into the fully connected layer, and then Softmax is performed to find the positions with the highest probability of the start and end positions, and the sequence from the start position to the end position is used as the output of the predicted answer.

[0055] In an embodiment of the present invention, when the automatic question answering model based on the pre-trained language model is trained, the goal of the training is to minimize the sum of the negative logarithmic probabilities of the start and end positions, that is:

[0056]

[0057] Where N is the number of examples in the training set, and is the start and end position of the true tag of example i, The predicted answer starts at The probability of The predicted answer ends at , θ represents the set of all trainable weights in the model.

[0058] In order for the automatic question answering model to determine whether the question has an answer, a confidence score vector p is given to the start and end positions of the predicted answer b and p e Concatenate a trainable bias t respectively, and the new confidence score vector is expressed as Where N S Indicates the number of words in a sentence. “;” means splicing by row.

[0059] The probability of no answer is expressed as: Where a represents the answer, Indicates no answer; when When the probability of is greater than the score of the best answer fragment, then the question has no answer.

[0060] S400: Combine the identified entities and the relation extraction question template designed according to the characteristics of the data set, and generate a relation extraction question-answer pair in an unsupervised manner. The design of the relation extraction question template is to analyze the data set, design different questions for different types of head entities and relations, and form a relation extraction template. For example, when the head entity type is a person and the relationship type is ancestral home, the question is "Where is the ancestral home of person xx?".

[0061] For example, in an example of the present invention, a relationship extraction question template is designed according to the types of subject, relationship, and object, as shown in Table 2, with a total of 54 types of relationships, among which multiple questions are asked for multiple relationships. For example, the relationship "box office" involves both box office value and region, so the two questions "What is the box office of xx film and television work?" and "Where is xx film and television work released?" are asked respectively.

[0062] Table 2 Relation extraction question generation template

[0063]

[0064]

[0065] Combine the entities identified by entity extraction and the relation extraction problem template, according to Figure 3The algorithm flow shown in the figure generates relation extraction question-answer pairs in an unsupervised manner. To generate relation extraction question-answer pairs in an unsupervised manner, questions are asked about the possible relationships of each entity identified by entity extraction according to the question template, and "xx" in the question template is replaced with the entity identified by entity extraction. If the entity has such a relationship in the text, the tail entity is used as the answer to the question. Otherwise, if such a relationship does not exist, the answer to the question is None. This method is used to loop through all samples to construct relation extraction question-answer pairs.

[0066] For example, the input text is: Zhang Sisi, wife of Zhang Sansan, an official of country K, designed a pair of "silk indoor slippers" for Li Liuliu, daughter of Li Wuwu, an official of country A. The extracted entities include countries: country K, country A, and characters: Zhang Sansan, Zhang Sisi, Li Wuwu, and Li Liuliu. For country-type entities, there may be three relationships: official language, capital, and official. The relationship extraction questions are: "What is the official language of country xx?", "What is the capital of country xx?", "Who is the official of country xx?", replace "xx" in the question template with the entity identified in the entity extraction, and the final constructed relationship extraction question-answer pairs are: (What is the official language of country K? None), (What is the capital of country K? None), (Who is the official of country K? Zhang Sansan), (What is the official language of country A? None), (What is the capital of country A? None), (Who is the official of country A? Li Wuwu). When constructing the relationship extraction question-answer pairs, the subject type is retained, which can provide additional auxiliary information for the subsequent model to identify the tail entity.

[0067] S500: Input the relation extraction question and the given text into the second automatic question answering model based on the pre-trained language model, and output the tail entity. In an embodiment of the present invention, the first automatic question answering model based on the pre-trained language model and the second automatic question answering model based on the pre-trained language model use the same question answering model and share parameters during the training process.

[0068] S600: jointly training the first automatic question answering model and the second automatic question answering model; in an embodiment of the present invention, the joint training is performed using the following objective function:

[0069] L=(1-λ)L entity +λL relation

[0070] Among them, L entity and L relation They represent the negative logarithmic probabilities of the start and end positions of entity extraction and relationship extraction respectively, and λ∈[0,1] is a parameter for adjusting the weights of the two objective functions of entity extraction and relationship extraction.

[0071] S700: Convert the relation-extracted question-answer pair and the output tail entity into triples, and output triples consisting of a subject, a predicate, and an object, which correspond to the head entity, the relation, and the tail entity, respectively. Figure 4 As shown in the figure, the input text is: Zhang Sisi, wife of Zhang Sansan, an official of country K, designed a pair of "silk indoor slippers" for Li Liuliu, daughter of Li Wuwu, an official of country A. Through two rounds of question and answer extraction, the final triples obtained are: (country K, official, Zhang Sansan), (Zhang Sansan, wife, Zhang Sisi), (Zhang Sisi, husband, Zhang Sansan), (country A, official, Li Wuwu), (Li Wuwu, daughter, Li Liuliu), (Li Liuliu, father, Li Wuwu). The extracted triples can be used to construct the knowledge graph.

[0072] The present invention transforms entity relationship extraction into a question-answering task, effectively leveraging the advantages of a mature question-answering model, and more naturally jointly training entity extraction and relationship extraction, thus avoiding the error transmission caused by the pipeline approach. At the same time, the present invention is more in line with the general process of information extraction by humans, and can solve the problems of cross-sentence relationships and multi-dimensional relationships being difficult to identify in entity relationship extraction.

[0073] The above embodiments are only preferred implementation modes of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and equivalent substitutions without departing from the principles of the present invention. These technical solutions after improvements and equivalent substitutions to the claims of the present invention all fall within the protection scope of the present invention.

Claims

1. A method for joint extraction of entity relations based on automatic question answering, characterized in that: include: Preprocess the data, including at least word segmentation and part-of-speech tagging; It also includes the removal of stop words, which is done by deleting auxiliary words and modal particles in the text according to the stop word list; Design entity extraction question templates based on the characteristics of the dataset, and generate entity extraction question-answer pairs based on the templates; The method of designing an entity extraction question template according to the characteristics of a data set and generating an entity extraction question-answer pair according to the template includes: analyzing the data set, designing different questions for different types of head entities, and forming an entity extraction question template; when a section of input text is given, traversing all types of head entities and asking questions, if the type of head entity exists in the text, marking the head entity as the answer to the question, and if the type of head entity does not exist, marking the answer as None, and constructing an entity extraction question-answer pair; Inputting the entity extraction question and the given text into the first automatic question answering model based on the pre-trained language model, and outputting the entity; the question answering model based on the pre-trained language model includes embedding encoding input, pre-training and output fine-tuning of mapping words in a high-dimensional discrete space to low-dimensional continuous space vectors; The method for constructing the first automatic question-answering model based on the pre-trained language model includes: Concatenate the input text and question and embed them into code; Input the results of text and question embedding encoding into the pre-trained language model; Fine-tune the output layer of the pre-trained language model; The method of splicing the input text and the question is as follows: given a question and text Where N q and N c Represent the number of words in the question and text respectively. The question and text are concatenated with special symbols [CLS] and [SEP]. The input format is [CLS]Q[SEP]C[SEP]; The embedding coding method is: the sum of the word vector, the segment vector and the position vector of each word in the sequence [CLS]Q[SEP]C[SEP] is used as the embedding code; The pre-trained language model is a BERT model; The method for fine-tuning the output layer of the pre-trained language model is: The output of the BERT model is input into the fully connected layer, followed by Softmax. Find the positions with the highest probability of the starting position and the ending position, and use the sequence from the starting position to the ending position as the output of the predicted answer; The method further includes: training a first automatic question-answering model based on the pre-trained model, wherein the training method is: By minimizing the probability sum of the negative logarithms of the start and end positions, the formula is as follows: Where N is the number of examples in the training set, and is the start and end position of the true tag of example i, The predicted answer starts at The probability of The predicted answer ends at The probability of , θ represents the set of all trainable weights in the model; Give the confidence score vector p for the start and end positions of the predicted answer b and p e Concatenate a trainable bias t respectively, and the new confidence score vector is expressed as Where N S Indicates the number of words in a sentence. ";" means splicing by row; The probability of no answer is expressed as: Where a represents the answer, Indicates no answer; when When the probability of is greater than the score of the best answer fragment, then the question has no answer; Combine the identified entities with the relation extraction question template designed according to the characteristics of the data set, and use an unsupervised method to generate relation extraction question-answer pairs: by analyzing the data set, design different questions for different types of head entities and relations to form a relation extraction template; ask questions about the possible relationship of each entity identified by entity extraction according to the question template, and replace "xx" in the question template with the entity identified by entity extraction. If the entity has such a relationship in the text, the tail entity is used as the answer to the question. Otherwise, if there is no such relationship, the answer to the question is None. This method is used to loop through all samples to construct relation extraction question-answer pairs; Input the relation extraction question and the given text into the second automatic question answering model based on the pre-trained language model, and output the tail entity; Jointly training the first automatic question answering model and the second automatic question answering model; The relation extraction question-answer pair and the output tail entity are converted into triples, and a triple consisting of subject, predicate, and object is output, corresponding to the head entity, relation, and tail entity respectively.

2. The entity relationship joint extraction method based on automatic question answering according to claim 1 is characterized in that: The word segmentation method is: dividing a continuous character sequence into a word sequence according to a specification; The method of part-of-speech tagging is: tagging each word in the word segmentation result with a correct part-of-speech.

3. The entity relationship joint extraction method based on automatic question answering according to claim 1 is characterized in that: The second automatic question answering model based on the pre-trained language model and the first automatic question answering model based on the pre-trained language model use the same model, and the two models share parameters during the training process.

4. The entity relationship joint extraction method based on automatic question answering according to claim 3 is characterized in that: The method for jointly training the first automatic question answering model and the second automatic question answering model is: Using the objective function: L = (1-λ)L entity +λL relation Two automatic question answering models are trained, where L entity and L relation They represent the negative logarithmic probabilities of the start and end positions of entity extraction and relationship extraction respectively, and λ∈[0,1] is a parameter for adjusting the weights of the two objective functions of entity extraction and relationship extraction.