Improved deployment method and system for multi-feature fusion entity co-reference resolution model

Abstract

The invention provides an improved deployment method and system for a multi-feature fusion entity co-reference resolution model. The method comprises the following steps: S1, constructing a multi-feature fusion entity co-reference resolution model; the multi-feature fusion entity co-reference resolution model comprises a word embedding layer, a feature extraction layer, a to-be-resolved item trimming layer and an output layer; and S2, processing an input text by using different layers of the multi-feature fusion entity co-reference resolution model, and deploying the multi-feature fusion entity co-reference resolution model. According to the end-to-end-based multi-feature fusion entity co-reference resolution model provided by the invention, by effectively capturing semantic information and contexts of words, potential co-reference relationships in documents are deeply mined, and a machine is helped to more efficiently and accurately understand and analyze complex texts.

Description

Technical Field

[0001] The present invention relates to the field of coreference resolution, and in particular to a deployment method and system for an improved multi-feature fusion entity coreference resolution model. Background Art

[0002] Coreference resolution is a core task in the field of natural language processing. It uses statistics, machine learning and related computing technologies to identify coreference chains in texts and analyze the objects referred to by nouns, pronouns and proper nouns. In this process, researchers use the context to deeply explore the deep meaning of words in order to achieve accurate correspondence with specific things, thereby improving the understanding of the text content. In recent years, with the rapid development of deep learning theory, the construction of coreference resolution models has gradually shifted from traditional machine learning methods to neural network models based on deep learning. This shift has significantly promoted the development of coreference resolution technology, making it one of the hot spots in the field of natural language processing research. These technologies have been widely used in many fields such as information extraction, automatic summarization, question-answering systems, machine translation, sentiment analysis, and text semantic analysis.

[0003] The coreference resolution task aims to establish an effective connection between the referential statement and all possible candidate antecedents. Traditional methods usually use external tools to identify all statements in the text, and then use clustering algorithms to classify statements referring to the same entity into the same equivalence class. However, both stages are highly dependent on complex and fine-grained features. At present, the most classic model in the field of coreference resolution is the end-to-end coreference resolution model proposed by Lee et al. The uniqueness of this model is that it does not rely on any grammatical parser, manually designed statement detector or heuristic function, but only uses a simple log-likelihood function, and its training effect is significantly better than previous models. At present, most end-to-end coreference resolution models mainly focus on the analysis of features such as characters and parts of speech in the text. However, considering the complexity of entity relationship information in the text, features such as character parts of speech, contextual semantics and logical relations will have an important impact on the semantics of different words. In addition, the existing model only considers the product of two vectors in the score function of the candidate antecedent, and fails to fully utilize the linear characteristics of the vector space. This limitation may affect the model's comprehensive capture of semantic information. More importantly, current models mainly focus on coreference relationships within a sentence, often ignoring possible coreference relationships between sentences, which results in many expressions being unable to find corresponding antecedents. To this end, future research should focus on improving feature representation and expanding the contextual scope of coreference resolution to improve the overall performance and accuracy of the model. This may involve a deeper understanding of entity relationships and systematic modeling of cross-sentence coreference relationships, thereby achieving more efficient and accurate coreference resolution.

[0004] Therefore, the present invention starts from the perspective of lexical features, integrates the features such as part of speech, contextual semantics and logical relations of characters, and adopts word embedding method to vectorize text to obtain vector representation of words. Subsequently, these word vectors are input into the feature extraction layer composed of bidirectional gated recurrent unit (BiGRU) and convolutional neural network (CNN), and context-dependent features and local-dependent features are extracted respectively. In order to further enhance the recognition ability of the model to key information, the attention mechanism is used to strengthen the important information in these two features respectively. Then, the score function of the candidate antecedent is modified to obtain more semantic information of the expression. Finally, this information will be input into the output layer composed of the fully connected layer to generate the final coreference chain. Through this method, it is expected to significantly improve the accuracy and efficiency of coreference resolution.

[0005] Most of the existing end-to-end models for solving the coreference resolution task are similar to the first end-to-end learning coreference resolution model proposed by Lee in 2017. This method decomposes coreference resolution into two subtasks: entity mention recognition and antecedent prediction, and processes them through joint learning. Its main goal is to identify the mentions in the text and pair each pair of mentions to determine whether they have a coreference relationship, thereby treating all mention pairs as a binary classification task.

[0006] Specifically, the model first performs high-dimensional embedding encoding of text symbols through a bidirectional long short-term memory network (Bi-LSTM) to calculate the representation of each potential mention span. In this process, span representations with low scores are pruned to improve computational efficiency. Subsequently, based on the span representation, the model calculates the probability score of the coreference relationship between the two words, which takes into account the score of the span word as a mention word and the probability of one word being the antecedent of another word. Finally, the clustering result, i.e., the mention word, will be constructed using a greedy strategy based on the model's prediction results for the antecedent word. This method effectively integrates mention identification and antecedent word prediction, which helps to improve the accuracy and efficiency of coreference resolution.

[0007] Although such models have superior performance, they dynamically construct span representations of mentioned words and span pair representations of mentioned pairs, resulting in frequent span representation construction that consumes a lot of memory and video memory, thus limiting the length of text that the model can handle and the number of instances per training batch. Current methods usually use various pruning techniques, however, the length of text they can handle generally does not exceed 512 words. If the article exceeds this length, it usually needs to be truncated or processed using a sliding window method, which makes existing methods less applicable to long text processing.

[0008] In addition, the mention pair model has some inherent flaws in its architectural design. When judging the relationship between one mention word and another mention word, it is essentially calculating conditional probability, and the co-reference relationship between any two mention words is independent and based on local judgments. To solve this problem, some recent optimization works have attempted to iteratively update the representation of the mention word and integrate the information of the possible corresponding antecedent words in the process. However, despite the improvement in performance, the final model result is still based on local features and fails to make full use of global information. This shows that improving the model's applicability to long texts and global understanding capabilities remains an important direction for future research. Summary of the invention

[0009] In view of the defects in the prior art, the object of the present invention is to provide an improved method and system for a multi-feature fusion entity co-reference resolution model.

[0010] According to the present invention, a method for deploying an improved multi-feature fusion entity co-reference resolution model includes:

[0011] Step S1: construct a multi-feature fusion entity co-reference resolution model; the multi-feature fusion entity co-reference resolution model includes a word embedding layer, a feature extraction layer, a to-be-resolved item pruning layer and an output layer;

[0012] Step S2: Use different layers of the multi-feature fusion entity coreference resolution model to process the input text and deploy the multi-feature fusion entity coreference resolution model.

[0013] Preferably, step S2 includes the following sub-steps:

[0014] Step S2.1: In the word embedding layer, the input text is segmented and the text is embedded into a vector representation;

[0015] Step S2.2: In the feature extraction layer, a bidirectional gated recurrent unit and a convolutional neural network are used to extract the context distance dependency features and the distance local features between adjacent words in the text, and the key information in these two features is strengthened through the attention mechanism.

[0016] Step S2.3: In the to-be-resolved item pruning layer, the multi-feature fusion entity coreference resolution model obtains more semantic information of the expression by modifying the score function of the candidate antecedent;

[0017] Step S2.4: Input the semantic information into the output layer consisting of a fully connected layer to generate the final co-reference chain.

[0018] Preferably, step S2.1 includes performing word segmentation processing on the sentences of the input text, encoding the word array after word segmentation into digital form, and inputting the RoBERTa model to generate a corresponding word embedding vector.

[0019] Preferably, step S2.3 includes calculating the association score for any two words in the text, the higher the score, the stronger the association, and serving as a rule for determining demonstrative pronouns and corresponding entity information; using a feedforward neural network to map the phrase feature vector into a scalar to obtain a corresponding numerical form.

[0020] Preferably, the output layer includes a softmax unit for predicting whether there is a co-referential relationship between two expressions, and selecting the candidate word with the highest probability as the top priority word of the phrase.

[0021] According to the present invention, a deployment system of an improved multi-feature fusion entity co-reference resolution model includes:

[0022] Module M1: construct a multi-feature fusion entity co-reference resolution model; the multi-feature fusion entity co-reference resolution model includes a word embedding layer, a feature extraction layer, a to-be-resolved item pruning layer and an output layer;

[0023] Module M2: Use different layers of the multi-feature fusion entity coreference resolution model to process the input text and deploy the multi-feature fusion entity coreference resolution model.

[0024] Preferably, the module M2 includes the following submodules:

[0025] Module M2.1: In the word embedding layer, the input text is segmented and the text is embedded into a vector representation;

[0026] Module M2.2: In the feature extraction layer, bidirectional gated recurrent units and convolutional neural networks are used to extract contextual distance dependency features and local distance features between adjacent words in the text, and the key information in these two features is strengthened through the attention mechanism.

[0027] Module M2.3: In the to-be-resolved item pruning layer, the multi-feature fusion entity coreference resolution model obtains more semantic information of the expression by modifying the score function of the candidate antecedent;

[0028] Module M2.4: Input semantic information into the output layer consisting of fully connected layers to generate the final co-reference chain.

[0029] Preferably, the module M2.1 includes performing word segmentation processing on sentences of the input text, encoding the word array after word segmentation into digital form, and inputting the RoBERTa model to generate corresponding word embedding vectors.

[0030] Preferably, the module M2.3 includes calculating the correlation score for any two words in the text, the higher the score, the stronger the correlation, and serving as a rule for determining demonstrative pronouns and corresponding entity information; using a feedforward neural network to map the phrase feature vector into a scalar to obtain a corresponding numerical form.

[0031] Preferably, the output layer includes a softmax unit for predicting whether there is a co-referential relationship between two expressions, and selecting the candidate word with the highest probability as the top priority word of the phrase.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. The present invention proposes an end-to-end multi-feature fusion entity coreference resolution model, which effectively captures the semantic information of words and their contextual context, deeply mines the potential coreference relationship in the document, and helps the machine understand and analyze complex texts more efficiently and accurately.

[0034] 2. Traditional models often show insufficient adaptability when processing longer texts. The present invention designs a more robust structure by integrating bidirectional gated recurrent units and convolutional neural networks, thereby improving the model's ability to capture co-reference relationships in long texts, enabling it to maintain efficient and accurate decomposition effects when facing complex texts.

[0035] 3. When obtaining text vocabulary features, existing methods are often unable to fully capture the multi-level semantic information of words. The present invention uses the RoBERTa model to achieve multi-feature extraction of parts of speech, combines the attention mechanism, and comprehensively considers the semantics, syntax, and contextual information of vocabulary, thereby greatly improving the integrity and accuracy of feature extraction.

[0036] 4. The present invention introduces a vector subtraction method and an improved scoring mechanism to more accurately score candidate antecedents, thereby avoiding misjudgment or omissions caused by a relatively simple scoring function and significantly improving the accuracy of antecedent selection.

[0037] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0039] Figure 1 It is a flow chart of the present invention.

[0040] Figure 2 It is the overall structure diagram of the model of the present invention. DETAILED DESCRIPTION

[0041] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0042] Reference Figure 1 and Figure 2 As shown, the present invention proposes an end-to-end multi-feature fusion entity co-reference resolution model, which includes four components: a word embedding layer, a feature extraction layer, a to-be-resolved item pruning layer and an output layer.

[0043] First, the text is segmented and embedded into a vector representation using the robustly optimized BERT approach (RoBERTa). Next, the bidirectional gated recurrent unit (BiGRU) and convolutional neural network (CNN) are used to extract the long-distance dependency features of the context and the short-distance local features between adjacent words in the text. Then, the key information in these two features is strengthened through the attention mechanism to improve the feature extraction ability of the model. In the pruning layer of the items to be resolved, the model obtains more semantic information of the expression by modifying the score function of the candidate antecedent. Finally, this information is input into the output layer composed of the fully connected layer to generate the final coreference chain.

[0044] In the four layers of the end-to-end multi-feature fusion entity coreference resolution model:

[0045] (1) Word embedding layer: First, the input sentence is segmented, the word array after segmentation is encoded into digital form, and then input into the RoBERTa model to generate the corresponding word embedding vector.

[0046] (2) Feature extraction layer. The feature extraction layer is mainly composed of three modules: local feature extraction, context feature extraction and feature fusion module.

[0047] a. Local feature extraction

[0048] CNN is mainly used to mine local features of data during feature extraction. The network consists of two core parts: convolution layer and pooling layer. Through multi-layer convolution operations, local dependencies in sentences can be effectively extracted, which is particularly suitable for capturing short-distance entity co-reference relationships. After the convolution operation, the pooling layer further reduces the feature dimension, reduces computational complexity, and retains important information. Through the pooling operation, the model can reduce computational overhead while avoiding overfitting, ensuring that the extracted local features have good robustness.

[0049] In order to further improve the expressiveness of local features, the present invention combines the attention mechanism and introduces it into the feature extraction process. This mechanism can weight local features and focus on enhancing those features that are critical to coreference resolution, so that the model can more accurately identify and focus on key information in the text.

[0050] b. Context feature extraction

[0051] Recurrent Neural Network (RNN) is widely used in extracting long-distance features due to its unique memory characteristics, and can effectively capture contextual information in text. However, when processing long sequences, traditional RNN is prone to gradient vanishing or exploding problems, which limits its performance in capturing long-distance dependent features.

[0052] To solve this problem, the present invention adopts a gated recurrent unit (GRU), which is an improved RNN architecture. By introducing update gate and reset gate mechanisms, GRU can more effectively capture longer dependencies than traditional RNNs, and improve the feature extraction performance of the model while reducing computational complexity. Furthermore, the present invention uses a bidirectional GRU (BiGRU) to capture contextual features in the data.

[0053] c. Feature fusion

[0054] In the last step of the feature extraction layer, the local feature vector needs to be concatenated with the context feature vector to construct a feature representation vector f representing the entire text. Through this concatenation operation, the local features in the text can be integrated with the overall context information, so that the model can retain the local fine information while also being able to globally understand the overall semantics of the text.

[0055] (3) Item Pruning Layer: The main principle of coreference resolution is to calculate the correlation score of any two words in the text. The higher the score, the stronger the correlation, which can be used as the rule for determining demonstrative pronouns and corresponding entity information. Feed Forward Neural Networks (FFNN) is used to map the phrase feature vector into a scalar to obtain the corresponding numerical form.

[0056] (4) Output layer: The output layer consists of softmax units, which are mainly used to predict whether there is a co-reference relationship between two expressions and select the candidate word with the highest probability as the top priority word of the phrase.

[0057] The above is a basic embodiment of the present invention. The technical solution of the present invention is further described below through a preferred embodiment.

[0058] Example 1

[0059] Reference Figure 1 and Figure 2 As shown, the present invention proposes an end-to-end multi-feature fusion entity co-reference resolution model, including:

[0060] 1. Word embedding layer: First, the input sentence is segmented and the word array after segmentation is encoded into digital form. According to the position where the word first appears in the text, it is encoded starting from 1. The encoded word is represented by e, and the encoded text is represented by T = {e1, e2, ..., e n}, then T is input into the RoBERTa model for encoding to obtain a vector representation containing word position information and context information. RoBERTa is a deep bidirectional encoder language model based on BERT. Compared with BERT, it improves static masks to dynamic masks and uses a mixed representation of character level and word level to achieve text encoding. This improvement enables RoBERTa to fully learn the sentence-level, character-level, and word-level features in the text. Based on the position information and context of the word, RoBERTa generates the corresponding word embedding vector. After word embedding processing, the data is represented as a word vector:

[0061] E={X1,X2,…,X n};

[0062] Where X i ∈R d , represents the encoded word e i The d-dimensional word vectors obtained after embedding with RoBERTa.

[0063] 2. Feature extraction layer: The feature extraction layer takes the output E of the word embedding layer as input and finally generates the combined feature F as output.

[0064] a. Local feature extraction

[0065] CNN is mainly used to mine local features of data during feature extraction. The network consists of two core parts: convolution layer and pooling layer. The convolution layer uses three groups of convolution kernels of different sizes, each group contains two convolution kernels of the same size, aiming to capture local patterns and important information in the text through multi-scale receptive fields. Through multi-layer convolution operations, local dependencies in sentences can be effectively extracted, which is particularly suitable for capturing short-distance entity co-reference relationships.

[0066] After the convolution operation, the pooling layer further reduces the feature dimension and reduces the computational complexity while retaining important information. Through the pooling operation, the model can reduce the computational overhead while avoiding overfitting, ensuring that the extracted local features have good robustness. The extracted local features are shown in equations (1) and (2):

[0067] L i =Max(conv(E)) (1)

[0068] L=Concat(L1,L2,L3) (2)

[0069] Among them, Max represents the maximum pooling operation, Conv represents the convolution operation, and the final output feature moment L is formed by splicing the convolution results of three groups of convolution kernels. This splicing strategy can make full use of the features extracted by convolution kernels of different scales, improve the model's perception of complex text structures, and thus improve the accuracy of coreference resolution.

[0070] In order to further improve the expressive power of local features, the present invention combines the attention mechanism and introduces it into the feature extraction process. This mechanism can weight local features and focus on enhancing those features that are critical to coreference resolution, so that the model can more accurately identify and focus on key information in the text. The final extracted local dependency feature representation is shown in Equation (3) and Equation (4).

[0071]

[0072] in, are trainable parameter matrices corresponding to the linear transformations of the three features: query, key, and value. These matrices are adaptively learned and updated through training so that the model can more effectively extract and match key information in local features. ω represents the dimension of the local feature vector. L It is the final extracted local dependency feature representation.

[0073] b. Context feature extraction

[0074] The present invention uses a bidirectional GRU (BiGRU) to capture contextual features in the data. BiGRU consists of two GRUs, which perform calculations in the forward and reverse directions respectively, ensuring that information can be extracted from both ends of the sequence. Through this bidirectional structure, the model can not only obtain forward contextual dependency features, but also capture reverse dependency information at the same time, further enhancing the breadth and depth of text understanding.

[0075] In this embodiment, the vector matrix E generated by the word embedding layer is used as the input of BiGRU, and the two GRUs process the text sequence in their respective calculation directions. Finally, the feature vectors captured in these two directions are concatenated to form a complete context feature representation. Similar to the convolutional neural network, the attention mechanism is used to highlight the key context dependencies and obtain the final context feature vector f C , as shown in formula (5).

[0076]

[0077] in, is a trainable parameter matrix.

[0078] c. Feature fusion

[0079] In the last step of the feature extraction layer, the local feature vector needs to be concatenated with the context feature vector to construct a feature representation vector f representing the entire text. Through this concatenation operation, the local features in the text can be integrated with the overall context information, so that the model can understand the overall semantics of the text while retaining the local fine information. This combination improves the performance of the model in processing complex language structures and makes it more accurate in entity coreference resolution tasks. The final feature vector is shown in formula (6):

[0080] f = Concat(f L ,f C ) (6)

[0081] 3. Item Pruning Layer,The main principle of coreference resolution is to calculate the correlation score of any two words in the text. The higher the score, the stronger the correlation, which can be used as the determination rule of demonstrative pronouns and corresponding entity information. Feed Forward Neural Networks (FFNN) is used to map the phrase feature vector into a scalar to obtain the corresponding numerical form. The calculation formula is shown in formula (7).

[0082] score(i)=FFNN(g i ) (7)

[0083] Generally, the probability score of phrase i being the antecedent of phrase j is calculated as shown in formula (8):

[0084]

[0085] The traditional candidate antecedent scoring function mainly considers the representation of phrase i and phrase j, as well as the multiplication operation of their corresponding vectors. However, in natural language processing, this method ignores the linear characteristics of the vector space. The vector space belongs to a linear space, and the addition and subtraction operations of adding vectors may help the model capture more semantic information. Through experiments, it is found that although the use of vector addition can combine more semantics, it will lose a lot of fine-grained information between words, making it difficult for the model to accurately and efficiently capture the correlation between phrases. The use of vector subtraction operations can reflect the difference between the two phrases in the semantic space, which is equivalent to the difference in the projection of the vectors represented in the coordinate system. This difference is measured by cosine similarity, which can more accurately express the difference between vectors, thereby revealing their linear relationship in the semantic space.

[0086] Specifically, the vector subtraction operation can not only better capture the subtle differences between phrases, but also establish the correlation between phrases through analogical reasoning. Therefore, vector subtraction is introduced in the candidate antecedent score function, and the cosine similarity is combined to measure this difference. This method not only produces different information from the traditional vector multiplication operation, but also effectively captures richer semantic relationships between candidate phrases, thereby improving the performance of the model in the coreference resolution task. Finally, the candidate antecedent score function is shown in formula (9):

[0087]

[0088] in, Represents the distance between phrases i and j.

[0089] Finally, the association score between phrases i and j is calculated by formula (10), where the phrase with the largest score is regarded as the phrase with a referential relationship:

[0090] score(i,j)=score(i)+score(j)+score a (i,h) (10)

[0091] 4. Output layer, which consists of softmax units, is mainly used to predict whether there is a co-referential relationship between two expressions. Specifically, the softmax function is used to normalize the scores of phrase i and each phrase j in its candidate antecedent set, and the probability of a co-referential relationship between phrase i and phrase j in the candidate antecedent set is calculated. After obtaining the probability of each candidate antecedent j, the model sorts these probabilities and selects the candidate with the highest probability as the top priority word of phrase i.

[0092] In order to further construct the coreference relationship chain in the text, the model puts all expressions with coreference relationship into the same equivalence class according to the transitivity principle of coreference resolution to form a set of coreference chains. This process ensures that in the entire document, all coreference expressions can be effectively aggregated together to achieve global coreference resolution. The specific parameters of the model are set as follows:

[0093] Table 1 Model parameter settings

[0094]

[0095] In order to verify the effectiveness of the model, the present invention uses the English coreference resolution corpus of the CONLL-2012 shared task for evaluation and model training, namely the Ontonotes 5.0 dataset. This corpus is designed to achieve unlimited coreference resolution and provide annotated training sets, validation sets, and test sets for coreference resolution researchers. The Ontonotes 5.0 dataset is the most standard and classic dataset for studying the coreference resolution task to date.

[0096] In order to verify the effectiveness of the model, the present invention uses the English coreference resolution corpus in the CONLL-2012 shared task, namely the OntoNotes 5.0 dataset, for model training and evaluation. This corpus is designed to support coreference resolution in a wide range of application scenarios, and is specifically designed to provide well-annotated training sets, validation sets, and test sets for researchers in the field of coreference resolution. As the most standard and authoritative coreference resolution dataset, OntoNotes 5.0 covers a variety of text types, such as news, conversations, etc., and contains rich annotation information, including coreference relationships, semantic role annotations, and syntactic tree structures.

[0097] Table 2 Comparison of experimental results of different models (%)

[0098]

[0099] Compared with the classic traditional entity-level model, this model has achieved significant performance improvements, with accuracy increased by 6.2% and 0.9%, recall increased by 13.9% and 3.8%, and F1 value increased by 10.2% and 2.4%. The model combines the contextual features extracted by BiGRU and the local dependency features extracted by CNN, comprehensively capturing data features from different dimensions, and thus has more significant advantages in accuracy and F1 value than other methods; the RoBERTa word embedding technology is used, so that the generated word vector not only contains richer semantic information, but also incorporates position information, effectively improving the performance of the model in dealing with polysemous words in sentences; in addition, the model optimizes the scoring mechanism of candidate antecedents, and by introducing the vector subtraction method, the improved scoring mechanism scores the candidate antecedents more accurately, significantly improving the accuracy of antecedent selection. Therefore, the model performs better in capturing the complex coreference relationship of the text.

[0100] Through various technical improvements, the present invention significantly improves the accuracy, robustness and adaptability of the entity coreference resolution model, and has broad application prospects.

[0101] The present invention also provides an improved deployment system for a multi-feature fusion entity co-reference resolution model. The improved deployment system for a multi-feature fusion entity co-reference resolution model can be implemented by executing the process steps of the improved deployment method for a multi-feature fusion entity co-reference resolution model, that is, a person skilled in the art can understand the improved deployment method for a multi-feature fusion entity co-reference resolution model as a preferred implementation of the improved deployment system for a multi-feature fusion entity co-reference resolution model.

[0102] Specifically, a deployment system of an improved multi-feature fusion entity coreference resolution model includes:

[0103] Module M1: construct a multi-feature fusion entity co-reference resolution model; the multi-feature fusion entity co-reference resolution model includes a word embedding layer, a feature extraction layer, a to-be-resolved item pruning layer and an output layer;

[0104] Module M2: Use different layers of the multi-feature fusion entity coreference resolution model to process the input text and deploy the multi-feature fusion entity coreference resolution model.

[0105] The module M2 includes the following submodules:

[0106] Module M2.1: In the word embedding layer, the input text is segmented and the text is embedded into a vector representation;

[0107] Module M2.2: In the feature extraction layer, bidirectional gated recurrent units and convolutional neural networks are used to extract contextual distance dependency features and local distance features between adjacent words in the text, and the key information in these two features is strengthened through the attention mechanism.

[0108] Module M2.3: In the to-be-resolved item pruning layer, the multi-feature fusion entity coreference resolution model obtains more semantic information of the expression by modifying the score function of the candidate antecedent;

[0109] Module M2.4: Input semantic information into the output layer consisting of fully connected layers to generate the final co-reference chain.

[0110] The module M2.1 includes word segmentation processing on the sentences of the input text, encoding the word array after word segmentation into digital form, and inputting the RoBERTa model to generate the corresponding word embedding vector.

[0111] The module M2.3 includes calculating the association score of any two words in the text. The higher the score, the stronger the association, and serving as a rule for determining demonstrative pronouns and corresponding entity information; a feedforward neural network is used to map the phrase feature vector into a scalar to obtain a corresponding numerical form.

[0112] The output layer includes a softmax unit, which is used to predict whether there is a co-referential relationship between two expressions and select the candidate word with the highest probability as the top priority word of the phrase.

[0113] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0114] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. An improved deployment method of a multi-feature fusion entity co-reference resolution model, characterized in that: include: Step S1: construct a multi-feature fusion entity co-reference resolution model; the multi-feature fusion entity co-reference resolution model includes a word embedding layer, a feature extraction layer, a to-be-resolved item pruning layer and an output layer; Step S2: Use different layers of the multi-feature fusion entity coreference resolution model to process the input text and deploy the multi-feature fusion entity coreference resolution model.

2. The method for deploying the improved multi-feature fusion entity co-reference resolution model according to claim 1 is characterized in that: The step S2 comprises the following sub-steps: Step S2.1: In the word embedding layer, the input text is segmented and the text is embedded into a vector representation; Step S2.2: In the feature extraction layer, a bidirectional gated recurrent unit and a convolutional neural network are used to extract the context distance dependency features and the distance local features between adjacent words in the text, and the key information in these two features is strengthened through the attention mechanism. Step S2.3: In the to-be-resolved item pruning layer, the multi-feature fusion entity coreference resolution model obtains more semantic information of the expression by modifying the score function of the candidate antecedent; Step S2.4: Input the semantic information into the output layer consisting of a fully connected layer to generate the final co-reference chain.

3. The method for deploying the improved multi-feature fusion entity co-reference resolution model according to claim 2 is characterized in that: The step S2.1 includes performing word segmentation processing on the sentences of the input text, encoding the word array after word segmentation into digital form, and inputting it into the RoBERTa model to generate the corresponding word embedding vector.

4. The method for deploying the improved multi-feature fusion entity co-reference resolution model according to claim 2 is characterized in that: The step S2.3 includes calculating the association score for any two words in the text. The higher the score, the stronger the association, and it is used as a rule for determining demonstrative pronouns and corresponding entity information; a feedforward neural network is used to map the phrase feature vector into a scalar to obtain a corresponding numerical form.

5. The method for deploying the improved multi-feature fusion entity co-reference resolution model according to claim 2 is characterized in that: The output layer includes a softmax unit, which is used to predict whether there is a co-referential relationship between two expressions and select the candidate word with the highest probability as the top priority word of the phrase.

6. An improved deployment system for multi-feature fusion entity co-reference resolution model, characterized in that: include: Module M1: construct a multi-feature fusion entity co-reference resolution model; the multi-feature fusion entity co-reference resolution model includes a word embedding layer, a feature extraction layer, a to-be-resolved item pruning layer and an output layer; Module M2: Use different layers of the multi-feature fusion entity coreference resolution model to process the input text and deploy the multi-feature fusion entity coreference resolution model.

7. The improved multi-feature fusion entity coreference resolution model deployment system according to claim 6 is characterized in that: The module M2 includes the following submodules: Module M2.1: In the word embedding layer, the input text is segmented and the text is embedded into a vector representation; Module M2.2: In the feature extraction layer, bidirectional gated recurrent units and convolutional neural networks are used to extract contextual distance dependency features and local distance features between adjacent words in the text, and the key information in these two features is strengthened through the attention mechanism. Module M2.3: In the to-be-resolved item pruning layer, the multi-feature fusion entity coreference resolution model obtains more semantic information of the expression by modifying the score function of the candidate antecedent; Module M2.4: Input semantic information into the output layer consisting of fully connected layers to generate the final co-reference chain.

8. The improved multi-feature fusion entity co-reference resolution model deployment system according to claim 7 is characterized in that: The module M2.1 includes word segmentation processing on the sentences of the input text, encoding the word array after word segmentation into digital form, and inputting the RoBERTa model to generate the corresponding word embedding vector.

9. The improved multi-feature fusion entity coreference resolution model deployment system according to claim 7 is characterized in that: The module M2.3 includes calculating the association score of any two words in the text. The higher the score, the stronger the association, and serving as a rule for determining demonstrative pronouns and corresponding entity information; a feedforward neural network is used to map the phrase feature vector into a scalar to obtain a corresponding numerical form.

10. The improved multi-feature fusion entity co-reference resolution model deployment system according to claim 7, characterized in that: The output layer includes a softmax unit, which is used to predict whether there is a co-referential relationship between two expressions and select the candidate word with the highest probability as the top priority word of the phrase.

Images