Anaphora resolution method and system for dominant pronouns

Abstract

The invention provides an anaphora resolution method for dominant pronouns. The method comprises the following steps: S1, obtaining a to-be-processed text paragraph and splitting the to-be-processed text paragraph into a plurality of clauses; s2, judging whether the paragraphs of the to-be-processed text have an anaphora phenomenon or not, if so, taking the clause with a photograph as a current text, and selecting one or more clauses adjacent to each current text as corresponding historical texts, and executing the following steps on each current text and the corresponding historical text: S21, obtaining a candidate antecedent set of the photograph in the current text; s22, splicing the current text and the historical text corresponding to the current text to obtain an original text, and obtaining a replacement text set corresponding to the original text; s23, calculating semantic similarity between the original text and all corresponding replacement texts by adopting a pre-training language model; and S24, based on the semantic similarity calculated in the step S23, determining an anaphora object corresponding to the photograph in the original text.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, in particular to a reference resolution technology in the field of natural language processing, and more particularly to a reference resolution method and system for explicit pronouns. Background Art

[0002] As a common language phenomenon, reference is widely present in natural language expressions, such as: "Xiao Ming is the sports commissioner of Class 5, Grade 1. He participates in the school's spring sports meeting every year." In the example sentence, the pronoun is "he", and the object referred to by the pronoun "he" is "Xiao Ming". Although the reference phenomenon can avoid the problems of bloated sentences and redundancy caused by the repeated appearance of the same word; but this omission also increases the difficulty of natural language processing. For example, in natural language processing tasks such as information extraction, event graphs, automatic question answering, and machine reading comprehension, there are often a large number of reference relationships in the processed text, and the same pronoun may refer to different entities, and different pronouns may refer to the same entity. At this time, if the object referred to by the pronoun cannot be accurately determined, it will affect the understanding and analysis of the text, and then affect the effect and accuracy of the natural language processing task.

[0003] In order to solve the above problems, some researchers have proposed to resolve the reference of the text. Reference resolution is the process of determining the correspondence between the anaphora (the reference is called the anaphora, in English, Mention or Anaphor) and the antecedent (the object referred to is called the antecedent, in English, Antecedent). Reference resolution is generally divided into two types: explicit pronoun resolution and zero pronoun resolution. There are generally clear pronouns in explicit pronoun resolution, such as "she", "he", "it" and other pronouns. In zero pronoun resolution, there are generally no clear pronouns, such as: "Xiao Ming ate an apple and felt it was very sweet." There is no clear pronoun in the sentence, but the analysis of the sentence expression shows that there should be a pronoun "he" referring to "Xiao Ming" before "very sweet", and this pronoun is omitted.

[0004] In the prior art, the decomposition methods based on heuristic linguistic rules, the decomposition methods based on statistical models or the decomposition methods based on machine learning (or deep learning) models are mainly used to perform reference decomposition on texts. Although the above methods can be used to perform reference decomposition on texts, the effect of each type of method in performing reference decomposition is not good. Among them, the existing decomposition methods based on heuristic linguistic rules require the manual construction of a large number of rules based on experience, which is not only time-consuming and labor-intensive, but also the rules constructed based on experience can only perform reference decomposition on some texts with specific language structures or language rules. The existing decomposition methods based on statistical models can only solve some common problems with certain rules, and lack adaptability to some texts with special contexts. The existing resolution method based on machine learning (or deep learning) model extracts the semantic features of the anaphora and all candidate antecedents, and then sends the two semantic features to the neural network, and classifies the referential relationship between the anaphora and each candidate antecedent according to the output of the neural network to achieve reference resolution. Although the existing resolution method based on machine learning (or deep learning) model performs reference resolution by extracting the semantic features of the anaphora and the candidate antecedents, this method does not fully utilize the semantic features of the entire text, making it difficult to effectively identify the semantic relationship between the candidate antecedents and the anaphora, resulting in poor reference resolution effect. Summary of the invention

[0005] Therefore, the object of the present invention is to overcome the defects of the prior art and provide a method for resolving references of explicit pronouns and a system for resolving references of explicit pronouns.

[0006] The objectives of the present invention are achieved through the following technical solutions.

[0007] According to a first aspect of the present invention, a method for reference resolution of explicit pronouns is provided, the method comprising: S1, obtaining a text paragraph to be processed and splitting it into a plurality of clauses; S2, judging whether there is a reference phenomenon in the text paragraph to be processed, and when there is a reference phenomenon, taking the clause with the anaphora as a current text, and selecting the first one or more clauses adjacent to each current text as its corresponding historical text, and performing the following steps on each current text and its corresponding historical text: S21, obtaining a set of candidate antecedents of the anaphora in the current text, wherein the set of candidate antecedents is obtained by performing entity extraction on the current text and its corresponding historical text; S22, splicing the current text and its corresponding historical text to obtain an original text; The method comprises the following steps: obtaining an original text, and replacing the anaphora in the original text with the candidate antecedent set to obtain a replacement text set corresponding to the original text, wherein each candidate antecedent in the candidate antecedent set is used to replace the anaphora in the original text once to obtain a replacement text; S23, extracting semantic features from the original text and its corresponding replacement text set using a pre-trained language model, and calculating the semantic similarity between the original text and all its corresponding replacement texts based on the extracted semantic features; S24, determining the referent corresponding to the anaphora in the original text based on the semantic similarity calculated in step S23, wherein the candidate antecedent in the replacement text having the highest semantic similarity with the original text is used as the referent of the anaphora in the original text.

[0008] In some embodiments of the present invention, the step S21 includes: S211, extracting an anaphora from the current text based on a pre-constructed set of quasi-pronouns; S212, concatenating the text before the anaphora in the current text with its corresponding historical text to obtain a concatenated text, and performing entity extraction on the concatenated text using an entity extraction model to obtain all candidate antecedents of the current text; S213, screening candidate antecedents that meet the rules from all candidate antecedents obtained in step S212 according to preset screening rules.

[0009] In some embodiments of the present invention, the preset screening rule is: selecting candidate antecedents of the same type as the anaphora type according to the anaphora type.

[0010] In some embodiments of the present invention, the pre-trained language model is any one of the Chinese-Roberta-WWM-EXT model, the Bert-Base-Chinese model and the Chinese-BERT-WWM-EXT model.

[0011] In some embodiments of the present invention, in step S23, the pre-trained language model is trained by contrastive learning and then used to extract semantic features from the original text obtained in step S22 and its corresponding replacement text set.

[0012] In some embodiments of the present invention, in step S23, the pre-trained language model is trained by contrastive learning through the following steps: obtaining a training set, the training set including multiple groups of training samples, wherein each group of training samples includes an original text and its corresponding positive example text and negative example text; using the training set to train the pre-trained language model multiple times, wherein each training calculates the loss based on the contrastive learning training objective, and fine-tunes the parameters of the pre-trained language model according to the loss.

[0013] In some embodiments of the present invention, the contrastive learning training objective is:

[0014]

[0015] in, represents the loss, h i The feature vector representing the original text i, Represents the feature vector of the positive example text corresponding to the original text i, Represents the feature vector of the positive example text corresponding to the original text j, represents the feature vector of the counterexample text corresponding to the original text j, sim(·) represents the cosine function, N represents the number of training samples, and τ represents the temperature coefficient.

[0016] According to a second aspect of the present invention, there is provided a reference resolution system for explicit pronouns, the system comprising: a data acquisition module for acquiring a text paragraph to be processed; a data processing module for performing reference resolution on the text paragraph to be processed using the method described in the first aspect of the present invention.

[0017] In some embodiments of the present invention, the data processing module includes a text splitting submodule, a reference detection submodule, an entity extraction submodule, a text splicing submodule, a semantic similarity calculation submodule and a reference object output submodule, wherein: the text splitting submodule is used to split the text paragraph to be processed into multiple sentences; the reference detection submodule is used to determine whether there is a reference phenomenon in the text paragraph to be processed. When there is a reference phenomenon, the sentence with the reference is taken as a current text, and the previous one or more sentences adjacent to each current text are selected as its corresponding historical text; the entity extraction submodule is used to obtain a set of candidate antecedents of the reference in each current text, wherein the candidate antecedent set is obtained by performing entity extraction on each current text and its corresponding historical text; the text splicing submodule is used to splice each current text with its corresponding historical text The original text is obtained, and the candidate antecedent set of the anaphora in each current text is used to replace the anaphora in the original text to obtain a replacement text set corresponding to each original text, wherein each candidate antecedent in the candidate antecedent set is used to replace the anaphora in the original text once to obtain a replacement text; the semantic similarity calculation submodule is used to extract semantic features from each original text and its corresponding replacement text set using a pre-trained language model, and calculate the semantic similarity between each original text and all the corresponding replacement texts based on the extracted semantic features; the referent output submodule is used to determine the referent corresponding to the anaphora in each original text based on the semantic similarity calculated by the semantic similarity calculation submodule, wherein the candidate antecedent in the replacement text with the highest semantic similarity to the original text is used as the referent of the anaphora in the original text.

[0018] Compared with the prior art, the advantages of the present invention are: (1) by obtaining a set of candidate antecedents of the anaphora, irrelevant candidate antecedents can be eliminated, thereby reducing useless calculations and improving the efficiency and accuracy of coreference resolution; (2) a pre-trained language model is used to calculate the semantic similarity of the original text and all its corresponding replacement texts to determine the antecedent of the anaphora, which not only enables coreference resolution of texts in different contexts, but also makes full use of the semantic features between the original text and the replacement text, so that the semantic relationship between the candidate antecedent and the anaphora can be effectively identified, thereby improving the effect of coreference resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0020] Figure 1 A schematic diagram of a process flow of a reference resolution method according to an embodiment of the present invention;

[0021] Figure 2 Schematic diagram of a reference resolution system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail by specific embodiments below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0023] As mentioned in the background technology, although the reference resolution methods proposed in the prior art can resolve references in texts, the effect of each type of solution in reference resolution is not good. Among them, when using the existing resolution method based on heuristic linguistic rules to resolve references in texts, it is necessary to manually construct a large number of rules based on experience, which is not only time-consuming and labor-intensive, but can only resolve references for some texts with specific language structures or language rules, and has poor generalization; when using the existing resolution method based on statistical models to resolve references in texts, it can only solve some common problems with certain rules, and lacks adaptability to texts in some special contexts; when using the existing resolution method based on machine learning (or deep learning) models to resolve references in texts, there is no distinction between the reference types of anaphora, and no full use of the semantic features of the entire text, making it difficult to effectively identify the semantic relationship between the candidate antecedent and the anaphora, resulting in poor reference resolution.

[0024] In order to solve the above problems, the inventors proposed a scheme to replace the anaphora in the text with candidate antecedents to obtain replacement text, and determine the antecedent of the anaphora by calculating the semantic similarity between the replacement text and the original text. In this scheme, the coreference resolution process is decomposed into six steps: data preprocessing, coreference detection, entity extraction, text splicing, semantic similarity calculation and coreference output. Among them, data preprocessing is to split the text paragraph to be processed into multiple sentences; coreference detection is to perform anaphora detection on the split sentences to determine whether there is a coreference phenomenon in the text paragraph to be processed; entity extraction is to extract the candidate antecedent set of the anaphora using an entity extraction model, so as to more accurately extract the candidate antecedent set corresponding to the anaphora, so as to find the corresponding correct candidate antecedent faster in the subsequent steps; text splicing is to replace the original text with the anaphora with each candidate antecedent in the candidate antecedent set to obtain a replacement text set; semantic similarity calculation is to use a pre-trained language model to calculate the semantic similarity between each replacement text in the replacement text set and the original text. The semantic similarity between texts is determined based on the semantic similarity between the original text and the replacement text, so that the coreference resolution does not rely on a large number of experience-based rules, but the semantic similarity between the original text and the replacement text is calculated through a pre-trained language model to determine the antecedent of the anaphora. This not only can resolve the coreference of texts in different contexts, but also can make full use of the semantic features between the original text and the replacement text, so that the semantic relationship between the candidate antecedent and the anaphora can be effectively identified; the coreference output is to select the replacement text with the highest semantic similarity to the original text, and use the candidate antecedent in the replacement text as the coreference output of the anaphora in the original text; through the above steps, the computational efficiency and accuracy of coreference resolution can be improved.

[0025] In order to better understand the present invention, the scheme of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0026] According to one embodiment of the present invention, Figure 1As shown, the present invention provides a method for reference resolution of explicit pronouns, the method comprising: S1, obtaining a text paragraph to be processed and splitting it into multiple sentences; S2, judging whether there is a reference phenomenon in the text paragraph to be processed, and when there is a reference phenomenon, taking the sentence with the anaphora as a current text, and selecting the previous one or more sentences adjacent to each current text as its corresponding historical text, and performing the following steps on each current text and its corresponding historical text: S21, obtaining a set of candidate antecedents of the anaphora in the current text, wherein the set of candidate antecedents is obtained by performing entity extraction on the current text and its corresponding historical text; S22, splicing the current text and its corresponding historical text to obtain an original text This, and use the candidate antecedent set to replace the anaphora in the original text to obtain a replacement text set corresponding to the original text, wherein each candidate antecedent in the candidate antecedent set is used to replace the anaphora in the original text once to obtain a replacement text; S23, use the pre-trained language model to extract semantic features of the original text and its corresponding replacement text set, and calculate the semantic similarity between the original text and all its corresponding replacement texts based on the extracted semantic features; S24, determine the referent corresponding to the anaphora in the original text based on the semantic similarity calculated in step S23, wherein the candidate antecedent in the replacement text with the highest semantic similarity to the original text is used as the referent of the anaphora in the original text.

[0027] In order to better understand the present invention, the coreference resolution method of the present invention is described in detail from the aspects of data preprocessing, coreference detection, entity extraction, text splicing, semantic similarity calculation and coreference output.

[0028] 1. Data Preprocessing

[0029] The text paragraph to be processed may contain multiple clauses, and each clause may contain anaphora. If the anaphora in the entire text paragraph to be processed is subjected to anaphora resolution, the anaphora may not correctly correspond to the object it refers to due to too many clauses, thereby affecting the effect of anaphora resolution. Therefore, the text paragraph to be processed may be split into different clauses for processing. According to one embodiment of the present invention, the text paragraph to be processed is split into multiple clauses using periods and semicolons as separators to obtain a clause set.

[0030] 2. Reference Detection

[0031] After the text paragraph to be processed is split, it is necessary to perform reference detection to determine whether there is a reference phenomenon in the text paragraph to be processed. When there is a reference phenomenon, subsequent operations (steps between entity extraction and reference output) are performed on the text paragraph to be processed to achieve reference resolution; when there is no reference phenomenon, there is no need to perform reference resolution on the text paragraph to be processed.

[0032] Among them, the process of performing reference detection on the text paragraph to be processed is: traversing each sentence in the sentence set based on the pre-constructed class pronoun set to detect whether each sentence contains an anaphora, and taking the sentence with the anaphora as a current text, and selecting the first one or more sentences adjacent to each current text as its corresponding historical text. It should be noted that the selection of historical text is determined according to the actual situation. If too few sentences are selected, there may be no candidate antecedents. If too many sentences are selected, there may be too many candidate antecedents, which affects the efficiency of reference resolution. According to an example of the present invention, the first two sentences adjacent to each current text are selected as its corresponding historical text. It should also be noted that the class pronoun set can be a set, and different text types all use this class pronoun set. However, in order to better achieve reference resolution, class pronoun sets corresponding to different text types can be constructed. That is, the class pronoun set can be constructed for different text types, and different text types correspond to different class pronoun sets. According to an example of the present invention, the pre-constructed class pronoun set includes 100 words. In order to understand the types of words included in the pronoun-like set, take the news text as an example to illustrate the words (pronouns, pronoun-like or pronoun-like phrases) that may be included in the pronoun-like set corresponding to it. In the news text, the entity types included are generally people, institutions and places. For example, "Xiao Ming visited Shandong University during the summer vacation", in this sentence, "Xiao Ming" is a person, and "Shandong University" is a place; another example is "Xiao Hong visited Shangri-La during the summer vacation", in this sentence, "Xiao Hong" is a person, and "Shangri-La" is a place. According to the entity types that may be included in the news class, the corresponding pronoun-like set is constructed, and the set includes words related to people, institutions and places, among which the words related to people include "he", "she", "they", "them", "we", "you", "the official", "spokesperson", etc.; the words related to institutions include "the organization", "the institution", "the base", "the department", "the company", "the school", "this organization", "this institution", "this base", "this department", etc.; the words related to places include "here", "here", "the country", "the region", "this country", "this region", etc. Although the entity types included in news texts are generally people, organizations and places, there are also some words without specific reference types in news texts, such as "this", "that", "these", "those", etc. Therefore, the set of pronouns corresponding to news texts should also include such words without specific reference types, that is, the set of pronouns corresponding to news texts includes four types of words: people, organizations, places and others.

[0033] 3. Entity Extraction

[0034] After performing reference detection on the text paragraph to be processed, it can be determined whether there is a reference phenomenon therein. If there is a reference phenomenon, it means that there is an anaphora in the text paragraph to be processed, and it is necessary to perform reference resolution to determine the object referred to by the anaphora. The present invention proposes to determine all possible objects (antecedents) referred to by the anaphora by means of entity extraction. Since there may be multiple anaphora in the text paragraph to be processed, and each anaphora corresponds to a candidate antecedent set, in order to accurately find the candidate antecedent set corresponding to each anaphora, it is necessary to perform entity extraction on each current text containing the anaphora and the historical text corresponding to the current text to obtain the candidate antecedent set of the anaphora in each current text.

[0035] In order to better understand the entity extraction process, a current text and its corresponding historical text are used as examples to illustrate the entity extraction process. According to an embodiment of the present invention, a candidate antecedent set of an anaphora in the current text is obtained through steps P1-P3, and each step is described in detail below.

[0036] In step P1, anaphora are extracted from the current text based on a pre-constructed set of pronoun-like words.

[0037] In step P2, the text before the anaphora in the current text is spliced ​​with its corresponding historical text to obtain a spliced ​​text, and an entity extraction model is used to perform entity extraction on the spliced ​​text to obtain all candidate antecedents of the current text. According to one embodiment of the present invention, the entity extraction model adopts any one of an entity extraction model based on BERT+BiLSTM+CRF, an entity extraction model based on BiLSTM+CRF, an entity extraction model based on BERT+CRF, an entity extraction model based on CNN+BiLSTM+CRF, and an entity extraction model based on Lattice+CRF. It should be noted that the selected entity extraction model can perform entity extraction on the text paragraph to be processed, and the present invention does not make specific restrictions on the entity extraction model. According to one embodiment of the present invention, in step P2, Jieba can also be used to perform entity extraction on the spliced ​​text through part-of-speech recognition to obtain all candidate antecedents of the current text. In order to better understand the process of entity extraction from text by the entity extraction model, the entity extraction model based on BERT+BiLSTM+CRF is taken as an example to illustrate the entity extraction process. In the entity extraction process, the BERT+BiLSTM+CRF entity extraction model uses sequence labeling to classify each word in the text. BERT extracts text features; BiLSTM classifies each word by extracting contextual semantic features to obtain the label of each word; CRF is the label constraint relationship, extracts the global optimal sequence, and finally obtains the entity classification result of each word in the text. Taking the sentence "At the meeting that day, Wang Wu said that he was very happy to attend the meeting organized by the Science and Technology Association today" as an example, the entity extraction model based on BERT+BiLSTM+CRF is used to extract its entities: First, the entity extraction model based on BERT+BiLSTM+CRF is used to process the sentence to obtain the label set S1 corresponding to each word, S1 = ['in\O','out\O','seat\O','when\O','day\O','of\O','meeting\O','discussion\O',' The above-mentioned Wang Wu said that today we are very happy to be able to participate in the meeting of scientific and technological associations.\O']; Then traverse the tag set S1, identify the words with tags "B-PER" (start position of a person), "I-PER" (middle or end position of a person), "B-ORG" (start position of an organization), and "I-ORG" (middle or end position of an organization), and merge adjacent words in the example sentence whose tags conform to the specification norm into words, obtaining a set of word, tag information, and position information (entity set) S2 = ['Wang Wu\PER\10', 'Science and Technology Association\ORG\25']. Here, the specification norm is stipulated as follows: If the longest consecutive adjacent tags conform to 1 "B-PER" + m "I-PER" (m is an integer greater than or equal to 0), they are merged into the person tag PER; if the longest consecutive adjacent tags conform to 1 "B-ORG" + m "I-ORG" (m is an integer greater than or equal to 0), they are merged into the organization tag ORG. It should be noted that the entity extraction model based on BERT+BiLSTM+CRF is a pre-trained model. Schematically, its training data volume is 100,000 Chinese text data (80,000 for training, 10,000 for validation, and 10,000 for testing). Among them, the training data adopts the BIO annotation method, and the data tags include B-PER, I-PER, B-ORG, I-ORG, B-LOC, I-LOC, and O. "B-PER" represents the start position of a person, "I-PER" represents the middle or end position of a person, "B-ORG" represents the start position of an organization, "I-ORG" represents the middle or end position of an organization, "B-LOC" represents the start position of a location, "I-LOC" represents the middle or end position of a location, and "O" represents non-entity information.

[0038] In step P3, candidate antecedents that meet the rules are screened from all candidate antecedents obtained in step P2 according to the preset screening rules. According to one embodiment of the present invention, the preset screening rules are: select candidate antecedents of the same type according to the type of the anaphora. For example, if the anaphora is a word related to a person, then the candidate antecedents related to the person type are screened to obtain a candidate antecedent set; if the anaphora is a word related to an organization, then the candidate antecedents related to the organization type are screened to obtain a candidate antecedent set; if the anaphora is a word related to a place, then the candidate antecedents related to the place are screened to obtain a candidate antecedent set. It should be noted that the anaphora may not have a specific reference type. In this case, the candidate antecedents of all types extracted are used to form the candidate antecedent set. It should be noted that the purpose of using the above-mentioned screening rules is to improve the accuracy of reference resolution. If the anaphora is a word related to a person, the antecedent of the anaphora cannot be an institution or a place. If the institution or place is also used as a candidate antecedent, not only unnecessary calculations are added to the reference resolution, but also the accuracy of the reference resolution may be reduced. Through steps P1-P3, the object (candidate antecedent) that the anaphora may refer to can be quickly determined. In the process of entity extraction, since the existing resolution method does not distinguish the reference type of the anaphora, the candidate antecedents obtained are numerous and complex, resulting in some unnecessary calculations in the reference resolution process. Based on this, the present invention uses the reference type of the anaphora to determine the candidate antecedent set, that is, the candidate antecedents in the candidate antecedent set are all words of the same type as the anaphora, which can not only reduce unnecessary calculations, but also improve the accuracy of reference resolution.

[0039] 4. Text splicing

[0040] After entity extraction for each current text and its corresponding historical text, a set of candidate antecedents for the anaphora in each current text can be obtained. However, in order to select the correct antecedent from the candidate antecedent set of the anaphora in each current text, text splicing and semantic similarity calculation are required. Among them, text splicing refers to splicing each current text with its corresponding historical text to obtain the original text, and using the candidate antecedent set of the anaphora in each current text to replace the anaphora in its original text to obtain a replacement text set corresponding to each original text, wherein each candidate antecedent in the candidate antecedent set is used to replace the anaphora in the original text once to obtain a replacement text.

[0041] In order to better understand the process of text splicing, take "On the spot, he said that the Palace Museum is a great cultural heritage of mankind." as the current text, and "On December 12, the Shandong delegation arrived at Beijing South Station by train, and the chairman of the association, Zhang San, was there to greet them. The next day, the chairman of the delegation, Li Si, was invited to visit the Palace Museum." as an example of historical text to illustrate the process of text splicing. Among them, the anaphora in the current text is "he", and the candidate antecedent set of the anaphora "he" is ["Zhang San", "Li Si"]. Perform text splicing on the current text and its corresponding historical text: that is, first splice the current text with its corresponding historical text to obtain the original text "On December 12, the Shandong delegation arrived at Beijing South Railway Station by train, and the chairman of the association, Zhang San, was there to greet them. The next day, the chairman of the delegation, Li Si, was invited to visit the Palace Museum. On the spot, he said that the Palace Museum is a great cultural heritage of mankind."; then use the candidate antecedent set to replace the anaphora in the original text to obtain the replacement text set ["On December 12, the Shandong delegation arrived at Beijing South Railway Station by train, and the chairman of the association, Zhang San, was there to greet them. The next day, the chairman of the delegation, Li Si, was invited to visit the Palace Museum. On the spot, Zhang San said that the Palace Museum is a great cultural heritage of mankind.", "On December 12, the Shandong delegation arrived at Beijing South Railway Station by train, and the chairman of the association, Zhang San, was there to greet them. The next day, the chairman of the delegation, Li Si, was invited to visit the Palace Museum. On the spot, Li Si said that the Palace Museum is a great cultural heritage of mankind."]

[0042] 5. Semantic Similarity Calculation

[0043] Semantic similarity refers to the degree of semantic similarity between two texts. A high semantic similarity indicates that the meanings expressed by the two texts are similar, while a low semantic similarity indicates that the meanings expressed by the two texts are not similar. Based on this characteristic, the present invention proposes to calculate the semantic similarity of each original text and all the replacement texts corresponding thereto, so as to determine the replacement text closest in meaning to the original text based on the calculated semantic similarity, and then determine the antecedent corresponding to the anaphora in each original text to achieve the coreference resolution of the text paragraph to be processed. By calculating the semantic similarity between the original text and all the replacement texts corresponding thereto, the object (antecedent) referred to by the anaphora can be determined based on the calculation result of the semantic similarity, and the semantic features of the entire text can be fully utilized in the calculation process of the semantic similarity, that is, the semantic features between the original text and the replacement text can be fully utilized, which helps to improve the effect of coreference resolution.

[0044] The calculation process of semantic similarity is illustrated by taking the calculation of the semantic similarity between an original text and all its corresponding replacement texts as an example. The process is: using a pre-trained language model to extract semantic features from the original text and its corresponding replacement text set, and calculating the semantic similarity between the original text and all its corresponding replacement texts based on the extracted semantic features.

[0045] According to one embodiment of the present invention, the pre-trained language model is trained by contrastive learning and then used to extract semantic features from the original text and its corresponding replacement text set. It should be noted that contrastive learning methods in natural language processing include EsimCSE, ConSERT, unsupervised SimCSE and supervised SimCSE. Any contrastive learning method can be used to train the pre-trained language model, and the present invention does not limit this. The embodiment of the present invention uses the basic idea of ​​contrastive learning "pulling similar samples closer and pushing dissimilar samples away" to train the pre-trained language model, which enables the trained pre-trained language model to more accurately identify the semantic relationship between the anaphora and the candidate antecedent, which is beneficial to improving the accuracy of coreference resolution.

[0046] According to one embodiment of the present invention, the pre-trained language model is trained by contrastive learning through the following steps: obtaining a training set, the training set includes multiple groups of training samples, wherein each group of training samples includes an original text and its corresponding positive text and negative text; the pre-trained language model is trained multiple times using the training set, wherein each training calculates the loss based on the contrastive learning training objective, and fine-tunes the parameters of the pre-trained language model according to the loss. It should be noted that the original text in each group of training samples is a text including an anaphora (pronoun), and a label "origin" is also set on it; the corresponding positive text is a text in which the anaphora in the original text is replaced by a correct candidate antecedent, and the positive text is also set with a label "entailment"; the corresponding negative text is a text in which the anaphora in the original text is replaced by any incorrect candidate antecedent, and the negative text is also set with a label "contradiction". It should also be noted that the contrastive learning method used in the embodiment is supervised SimCSE. According to one embodiment of the present invention, the contrastive learning training objective is:

[0047]

[0048] in, represents the loss, h i The feature vector representing the original text i, Represents the feature vector of the positive example text corresponding to the original text i, Represents the feature vector of the positive example text corresponding to the original text j, represents the feature vector of the counter-example text corresponding to the original text j, sim(·) represents the cosine function, N represents the number of training samples, and τ represents the temperature coefficient. It should be noted that during the training process, the training set uses a self-labeled data set. Multiple sets of training samples in the training set can be obtained by performing data preprocessing, reference detection, entity extraction and text splicing on different text paragraphs. Take "On December 12, the Shandong delegation arrived at Beijing South Railway Station by train, and the chairman of the association, Li Si, was there to greet them. The next day, the chairman of the delegation, Zhang San, was invited to visit the Palace Museum. On the spot, he said that the Palace Museum is a great cultural heritage of mankind." as an example sentence to explain how to obtain training samples. First, the example sentence is divided into three clauses. The first clause is: "On December 12, the Shandong delegation arrived at Beijing South Railway Station by train, and the association chairman Li Si was there to greet them." The second clause is: "The next day, the chairman of the delegation Zhang San was invited to visit the Palace Museum." The third clause is: "At the scene, he said that the Palace Museum is a great cultural heritage of mankind." Then, the example sentence is subjected to reference detection and the current text and its corresponding historical text are determined. Among them, the current text is: "At the scene, he said that the Palace Museum is a great cultural heritage of mankind." The anaphora is "He", the historical text is: "On December 12, the Shandong delegation arrived at Beijing South Railway Station by train, and the association chairman Li Si was there to greet them. The next day, the chairman of the delegation Zhang San was invited to visit the Palace Museum." Then entity extraction is performed on the current text and the historical text to obtain all candidate antecedents of the anaphora: ["Shandong delegation", "Beijing South Railway Station", "Li Si", "Zhang San"], because the anaphora is a character type, the candidate antecedents of the character type are extracted to obtain the candidate antecedent set: ["Li Si", "Zhang San"], among which "Zhang San" is the correct candidate antecedent The candidate antecedent is "Li Si" which is a wrong candidate antecedent. Then all the candidate antecedents are used to replace the anaphora in the original sentence, and a positive example text and a negative example text are obtained. The positive example sample is: "On December 12, the Shandong delegation arrived at Beijing South Railway Station by train, and the chairman of the association, Li Si, was there to greet them. The next day, the chairman of the delegation, Zhang San, was invited to visit the Palace Museum. At the scene, Zhang San said that the Palace Museum is a great cultural heritage of mankind.", and the negative example text is: "On December 12, the Shandong delegation arrived at Beijing South Railway Station by train, and the chairman of the association, Li Si, was there to greet them. Welcome. The next day, the chairman of the delegation, Zhang San, was invited to visit the Palace Museum. On the spot, Li Si said that the Palace Museum is a great cultural heritage of mankind. "; Finally, the original example sentence, a positive example text and a negative example text are combined to obtain a set of training samples, the training samples are: {Original text (labeled as "origin"): "On December 12, the Shandong delegation arrived at Beijing South Railway Station by train, and the chairman of the association, Li Si, was there to welcome them. The next day, the chairman of the delegation, Zhang San, was invited to visit the Palace Museum. On the spot, he said that the Palace Museum is a great cultural heritage of mankind.", Positive example text (labeled as "entailment"): "On December 12, the Shandong delegation arrived at Beijing South Railway Station by train, and the chairman of the association, Li Si, was there to greet them. The next day, the chairman of the delegation, Zhang San, was invited to visit the Palace Museum. On the spot, Zhang San said that the Palace Museum is a great cultural heritage of mankind.", Negative example text (labeled as "contradiction"): "On December 12, the Shandong delegation arrived at Beijing South Railway Station by train, and the chairman of the association, Li Si, was there to greet them. The next day, the chairman of the delegation, Zhang San, was invited to visit the Palace Museum. On the spot, Li Si said that the Palace Museum is a great cultural heritage of mankind."}.

[0049] 6. Referent Output

[0050] After calculating the semantic similarity between each original text and all its corresponding replacement texts, the candidate antecedent in the replacement text with the highest semantic similarity to each original text is used as the referent of the anaphora in the original text, and the replacement text with the highest semantic similarity is used as the processing text to achieve reference resolution of the text paragraph to be processed.

[0051] Compared with the resolution method proposed in the prior art, the resolution method described in the above embodiment has a better reference resolution effect and can be configured in a reference resolution system. Based on this, the present invention also proposes a reference resolution system for explicit pronouns.

[0052] According to one embodiment of the present invention, Figure 2 As shown, the present invention provides a reference resolution system for explicit pronouns, the system comprising: a data acquisition module for acquiring a text paragraph to be processed; a data processing module for performing reference resolution on the text paragraph to be processed using the method described in the above embodiment.

[0053] According to one embodiment of the present invention, the data processing module includes a text segmentation submodule, a reference detection submodule, an entity extraction submodule, a text splicing submodule, a semantic similarity calculation submodule and a reference object output submodule, wherein: the text segmentation submodule is used to split the text paragraph to be processed into multiple sentences; the reference detection submodule is used to determine whether there is a reference phenomenon in the text paragraph to be processed. When there is a reference phenomenon, the sentence with the anaphora is taken as a current text, and the previous one or more sentences adjacent to each current text are selected as its corresponding historical text; the entity extraction submodule is used to obtain a set of candidate antecedents of the anaphora in each current text, wherein the candidate antecedent set is obtained by performing entity extraction on each current text and its corresponding historical text; the text splicing submodule is used to splice each current text with its corresponding historical text The original text is obtained, and the candidate antecedent set of the anaphora in each current text is used to replace the anaphora in the original text to obtain a replacement text set corresponding to each original text, wherein each candidate antecedent in the candidate antecedent set is used to replace the anaphora in the original text once to obtain a replacement text; the semantic similarity calculation submodule is used to extract semantic features from each original text and its corresponding replacement text set using a pre-trained language model, and calculate the semantic similarity between each original text and all the corresponding replacement texts based on the extracted semantic features; the referent output submodule is used to determine the referent corresponding to the anaphora in each original text based on the semantic similarity calculated by the semantic similarity calculation submodule, wherein the candidate antecedent in the replacement text with the highest semantic similarity to the original text is used as the referent of the anaphora in the original text.

[0054] The beneficial effects of the present invention are as follows: (1) by obtaining a set of candidate antecedents of the anaphora, irrelevant candidate antecedents can be eliminated, thereby reducing useless calculations and improving the efficiency and accuracy of coreference resolution; (2) a pre-trained language model is used to calculate the semantic similarity of the original text and all its corresponding replacement texts to determine the antecedent of the anaphora, which not only enables coreference resolution of texts in different contexts, but also makes full use of the semantic features between the original text and the replacement text, so that the semantic relationship between the candidate antecedent and the anaphora can be effectively identified, thereby improving the effect of coreference resolution.

[0055] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0056] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0057] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0058] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for resolving reference of explicit pronouns, characterized in that: The method comprises: S1. Obtain the text paragraph to be processed and split it into multiple sentences; S2. Determine whether there is a reference phenomenon in the text paragraph to be processed. If there is a reference phenomenon, take the sentence with the anaphora as a current text, select the previous one or more sentences adjacent to each current text as its corresponding historical text, and perform the following steps on each current text and its corresponding historical text: S21, obtaining a set of candidate antecedents of the anaphora in the current text, wherein the set of candidate antecedents is obtained by performing entity extraction on the current text and its corresponding historical text; S22, concatenating the current text with its corresponding historical text to obtain an original text, and replacing the anaphora in the original text with the candidate antecedent set to obtain a replacement text set corresponding to the original text, wherein each candidate antecedent in the candidate antecedent set is used to replace the anaphora in the original text once to obtain a replacement text; S23, using a pre-trained language model to extract semantic features from the original text and its corresponding replacement text set, and calculating the semantic similarity between the original text and all its corresponding replacement texts based on the extracted semantic features; S24, determining the referent corresponding to the anaphora in the original text based on the semantic similarity calculated in step S23, wherein the candidate antecedent in the replacement text having the highest semantic similarity to the original text is used as the referent of the anaphora in the original text.

2. The method according to claim 1, characterized in that The step S21 comprises: S211, extracting anaphora from the current text based on a pre-constructed set of quasi-pronouns; S212, concatenating the text before the anaphora in the current text with its corresponding historical text to obtain a concatenated text, and performing entity extraction on the concatenated text using an entity extraction model to obtain all candidate antecedents of the current text; S213 , screening candidate antecedents that meet the rules from all candidate antecedents obtained in step S212 according to a preset screening rule.

3. The method according to claim 2, characterized in that The preset screening rules are: According to the type of anaphora, select candidate antecedents of the same type.

4. The method according to claim 1, characterized in that: The pre-trained language model is any one of the Chinese-Roberta-WWM-EXT model, the Bert-Base-Chinese model and the Chinese-BERT-WWM-EXT model.

5. The method according to claim 4, characterized in that In the step S23, the pre-trained language model is trained by contrastive learning and then used to extract semantic features from the original text obtained in the step S22 and its corresponding replacement text set.

6. The method according to claim 5, characterized in that In step S23, the pre-trained language model is trained by using a contrastive learning method through the following steps: Obtaining a training set, wherein the training set includes multiple groups of training samples, wherein each group of training samples includes an original text and its corresponding positive example text and negative example text; The pre-trained language model is trained multiple times using the training set, wherein each training calculates a loss based on a contrastive learning training objective, and fine-tunes parameters of the pre-trained language model according to the loss.

7. The method according to claim 6, characterized in that The contrastive learning training objectives are: in, represents the loss, h i The feature vector representing the original text i, Represents the feature vector of the positive example text corresponding to the original text i, Represents the feature vector of the positive example text corresponding to the original text j, represents the feature vector of the counterexample text corresponding to the original text j, sim(·) represents the cosine function, N represents the number of training samples, and τ represents the temperature coefficient.

8. A reference resolution system for explicit pronouns, characterized in that: The system comprises: A data acquisition module, used to acquire text paragraphs to be processed; A data processing module is used to perform reference resolution on the text paragraph to be processed using the method described in any one of claims 1 to 7.

9. The system according to claim 8, characterized in that The data processing module includes a text segmentation submodule, a reference detection submodule, an entity extraction submodule, a text splicing submodule, a semantic similarity calculation submodule and a reference object output submodule, wherein: The text splitting submodule is used to split the to-be-processed text paragraph into a plurality of sentences; The reference detection submodule is used to determine whether the to-be-processed text paragraph has a reference phenomenon. If a reference phenomenon exists, the clause with the anaphora is taken as a current text, and one or more clauses adjacent to each current text are selected as the corresponding historical text; The entity extraction submodule is used to obtain a set of candidate antecedents of the anaphora in each current text, wherein the candidate antecedents are obtained by performing entity extraction on each current text and its corresponding historical text; The text concatenation submodule is used to concatenate each current text with its corresponding historical text to obtain an original text, and to use a candidate antecedent set of the anaphora in each current text to replace the anaphora in the original text to obtain a replacement text set corresponding to each original text, wherein each candidate antecedent in the candidate antecedent set is used to replace the anaphora in the original text once to obtain a replacement text; The semantic similarity calculation submodule is used to extract semantic features from each original text and its corresponding replacement text set using a pre-trained language model, and calculate the semantic similarity between each original text and all its corresponding replacement texts based on the extracted semantic features; The referent output submodule is used to determine the referent corresponding to each anaphora in the original text based on the semantic similarity calculated by the semantic similarity calculation submodule, wherein the candidate antecedent in the replacement text with the highest semantic similarity to the original text is used as the referent of the anaphora in the original text.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of any method described in claims 1-7.

11. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the electronic device to implement the steps of the method as described in any one of claims 1-7.

Images