A word processing method and device based on multi-task model

Abstract

The present application provides a word processing method and device based on a multi-task model, the method comprising: obtaining a candidate entity segment, and generating a candidate entity segment encoding vector based on the candidate entity segment; performing entity recognition processing and classification pruning processing on the candidate entity segment encoding vector through an entity recognition model to obtain a recognition pruned encoding vector; inputting the recognition pruned encoding vector into a relationship extraction model for scoring twice to generate a final encoding vector of a first relationship pair, and processing the final encoding vector of the first relationship pair through the relationship extraction model, so as to accurately determine the relationship between words in the candidate entity segment; inputting the recognition pruned encoding vector into a coreference resolution model for scoring twice to generate a final encoding vector of a second relationship pair, and processing the final encoding vector of the second relationship pair through the coreference resolution model, so as to accurately determine the coreference relationship between words in the candidate entity segment.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a word processing method and device based on a multi-task model, a training method and device for a multi-task model, a computing device, and a computer-readable storage medium. Background Art

[0002] Entity recognition refers to identifying and extracting entities with specific meaning or strong referentiality in unstructured text, such as names of people, places, organizational structures, dates and times, proper nouns, etc.

[0003] A relationship is a connection between two or more entities. Relationship extraction is to detect and identify certain semantic relationships between entities from text. For example, in the sentence "Beijing is the capital, political center and cultural center of China", the relationship expressed can be (China, capital, Beijing), (China, political center, Beijing) or (China, cultural center, Beijing).

[0004] Coreference resolution is a special type of relationship extraction. One of the entities in coreference resolution is usually a different way of saying another entity in the current context. The relationship between the two entities can be expressed as (entity 1, coreference, entity 2).

[0005] At present, the entity recognition task, relationship extraction task, and coreference resolution task of a sentence are all carried out separately. Information cannot be shared between multiple tasks, and they cannot constrain each other, which leads to unsatisfactory results of entity recognition, relationship extraction, and coreference resolution. Summary of the invention

[0006] In view of this, the embodiments of the present application provide a word processing method and device based on a multi-task model, a training method and device for a multi-task model, a computing device and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0007] The present application embodiment provides a word processing method based on a multi-task model, including:

[0008] Obtaining a candidate entity segment, and generating a candidate entity segment encoding vector based on the candidate entity segment;

[0009] Performing entity recognition processing and classification pruning processing on the candidate entity fragment encoding vector through an entity recognition model to obtain a recognition pruning encoding vector;

[0010] Inputting the identified pruned coding vector into a relation extraction model for scoring twice to generate a final coding vector for a first relation pair, processing the final coding vector for the first relation pair through the relation extraction model to determine the relationship between words in the candidate entity segment;

[0011] The identified pruned coding vector is input into a coreference resolution model for scoring twice to generate a final coding vector for a second relationship pair, and the final coding vector for the second relationship pair is processed through the coreference resolution model to determine the coreference relationship between the words in the candidate entity fragment.

[0012] Optionally, inputting the identified pruned coding vector into a relation extraction model for scoring twice to generate a final coding vector for a first relation pair, processing the final coding vector for the first relation pair through the relation extraction model, and determining the relationship between words in the candidate entity fragments, includes:

[0013] Scoring the identification pruned coding vector for the first time through a relation extraction model, and pruning the identification pruned coding vector based on the score to obtain a relation extraction coding vector;

[0014] A second scoring is performed based on the relationship extraction coding vector to generate a final coding vector for the first relationship pair, relationship extraction processing is performed on the final coding vector of the first relationship pair through the relationship extraction model to obtain a relationship extraction result, and the relationship between the words in the candidate entity fragment is determined based on the relationship extraction result.

[0015] Optionally, the identified pruned coding vector is input into a coreference resolution model for scoring twice to generate a final coding vector for a second relationship pair, and the final coding vector for the second relationship pair is processed by the coreference resolution model to determine the coreference relationship between words in the candidate entity segment, including:

[0016] Scoring the recognition pruned coding vector for the first time through a coreference resolution model, and pruning the recognition pruned coding vector based on the score to obtain a coreference resolution coding vector;

[0017] A second scoring is performed based on the coreference resolution coding vector to generate a final coding vector for a second relationship pair, coreference resolution processing is performed on the final coding vector of the second relationship pair through the coreference resolution model to obtain a coreference resolution result, and the coreference relationship between the words in the candidate entity fragment is determined based on the coreference resolution result.

[0018] Optionally, the entity recognition model, the relationship extraction model, and the coreference resolution model share a feedforward neural network for scoring;

[0019] The step of performing entity recognition processing and classification pruning processing on the candidate entity segment encoding vector through the entity recognition model to obtain the recognition pruning encoding vector includes:

[0020] Inputting the candidate entity segment encoding vector into an entity recognition model, and scoring the candidate entity segment encoding vector through the feedforward neural network;

[0021] The candidate entity encoding vector is classified based on the score of the candidate entity fragment encoding vector to obtain a classification label of the candidate entity encoding vector, and the candidate entity encoding vector is pruned to obtain a recognition pruned encoding vector.

[0022] Optionally, the relationship extraction model and the entity recognition model share a feedforward neural network for scoring;

[0023] The identification pruned coding vector is scored for the first time by using a relation extraction model, and the identification pruned coding vector is pruned based on the score, including:

[0024] Inputting the recognition pruned coding vector into the relation extraction model, and scoring the recognition pruned coding vector for the first time through the feedforward neural network to obtain a score of the recognition pruned coding vector;

[0025] The recognition pruned coding vectors with scores greater than or equal to a preset threshold are used as relation extraction coding vectors.

[0026] Optionally, the coreference resolution model and the entity recognition model share a feedforward neural network for scoring;

[0027] Scoring the identification pruned coding vector for the first time through a coreference resolution model, and pruning the identification pruned coding vector based on the score to obtain a coreference resolution coding vector, including:

[0028] Inputting the recognition pruned code vector into the coreference resolution model, and scoring the recognition pruned code vector for the first time through the feedforward neural network to obtain the score of the recognition pruned code vector;

[0029] The recognition pruning coding vector with a score greater than or equal to a preset threshold is used as the coreference resolution coding vector.

[0030] Optionally, the extracting the encoding vector based on the relationship and performing a second scoring to generate a final encoding vector for the first relationship pair includes:

[0031] Obtaining a first initial relation pair encoding vector based on the relation extraction encoding vector and the classification label of the relation extraction encoding vector;

[0032] Performing classification prediction processing on the first initial relationship pair encoding vector, and pruning the first initial relationship pair encoding vector according to a preset ratio based on a result of the classification prediction to obtain a first relationship pair encoding vector;

[0033] The first relationship pair encoding vector is scored for a second time through the feedforward neural network, and a final encoding vector of the first relationship pair is obtained based on the score of the second scoring and the first relationship pair encoding vector.

[0034] Optionally, obtaining a final encoding vector of the first relationship pair based on the second scoring score and the encoding vector of the first relationship pair includes:

[0035] Generate a first weight score based on the score of the second scoring of the encoding vector by the first relationship;

[0036] Based on the first weight score, sequentially performing attention processing on the relation extraction encoding vectors corresponding to the first relation pair encoding vector, obtaining a first intermediate vector;

[0037] Performing gating processing on the relation extraction encoding vector corresponding to the encoding vector based on the first intermediate vector and the first relation to obtain a first gating vector;

[0038] Re-encode the relation extraction encoding vector corresponding to the encoding vector according to the first gating vector, the first intermediate vector and the first relation to generate a re-encoded first updated encoding vector;

[0039] A first relationship pair final encoding vector is generated based on the first updated encoding vector and the classification label of the first updated encoding vector.

[0040] Optionally, the performing classification prediction processing on the first initial relationship pair encoding vector, and pruning the first initial relationship pair encoding vector according to a preset ratio based on a result of the classification prediction to obtain the first relationship pair encoding vector includes:

[0041] Scoring the first initial relationship pair encoding vector by a feedforward neural network to obtain a score of the first initial relationship pair encoding vector;

[0042] Performing classification prediction processing on the first initial relationship pair encoding vector to obtain a category of the first initial relationship pair encoding vector;

[0043] The first initial relationship pair encoding vector is pruned based on the category and score of the first initial relationship pair encoding vector to obtain a first relationship pair encoding vector.

[0044] Optionally, obtaining a first initial relationship pair encoding vector based on the relationship extraction encoding vector and a classification label of the relationship extraction encoding vector includes:

[0045] Encoding the classification label of the relation extraction encoding vector to generate a first label vector;

[0046] Based on any two of the relation extraction encoding vectors and their corresponding first label vectors, a first initial relation pair encoding vector of any two of the relation extraction encoding vectors is obtained.

[0047] Optionally, obtaining a first initial relationship pair encoding vector based on the relationship extraction encoding vector and a classification label of the relationship extraction encoding vector includes:

[0048] Encoding the classification label of the relation extraction encoding vector to generate a first label vector;

[0049] Determine a semantic vector between any two relation extraction encoding vectors based on positions of the any two relation extraction encoding vectors in the candidate entity fragment;

[0050] Based on any two of the relationship extraction encoding vectors, the semantic vector between the any two relationship extraction encoding vectors and the first label vector corresponding to each of the relationship extraction encoding vectors, a first initial relationship pair encoding vector of any two of the relationship extraction encoding vectors is obtained.

[0051] Optionally, determining the semantic vector between the arbitrary two relation extraction encoding vectors includes:

[0052] Determine multiple word vectors between the arbitrary two relation extraction encoding vectors;

[0053] Pooling or attention processing is performed on multiple word vectors between any two relationship extraction encoding vectors to obtain corresponding semantic vectors.

[0054] Optionally, obtaining the first initial relationship pair encoding vector of any two of the relationship extraction encoding vectors includes: according to the semantic relationship between any two of the relationship extraction encoding vectors in the candidate entity fragment, respectively obtaining the first initial relationship pair encoding vector of the forward semantic relationship and the first initial relationship pair encoding vector of the reverse semantic relationship of any two of the relationship extraction encoding vectors.

[0055] Optionally, scoring the first initial relationship pair encoding vector by a feedforward neural network to obtain a score of the first initial relationship pair encoding vector includes:

[0056] The first initial relationship pair encoding vector of the positive semantic relationship and the first initial relationship pair encoding vector of the reverse semantic relationship of any two of the relationship extraction encoding vectors are scored through a feedforward neural network, and the score of the first initial relationship pair encoding vector of the reverse semantic relationship is made less than a preset threshold.

[0057] Optionally, the generating a second relationship pair final encoding vector by performing a second scoring based on the coreference resolution encoding vector comprises:

[0058] Obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector;

[0059] Performing classification prediction processing on the second initial relationship pair encoding vector, and pruning the second initial relationship pair encoding vector according to a preset ratio based on a result of the classification prediction to obtain a second relationship pair encoding vector;

[0060] The second relationship pair encoding vector is scored for a second time through the feedforward neural network, and a final encoding vector of the second relationship pair is obtained based on the score of the second scoring and the second relationship pair encoding vector.

[0061] Optionally, obtaining a final encoding vector of the second relationship pair based on the second scoring score and the encoding vector of the second relationship pair includes:

[0062] generating a second weight score based on a second scoring of the encoding vector of the second relationship pair;

[0063] Attention processing is performed on the coreference resolution encoding vectors corresponding to the second relationship pair encoding vector in sequence based on the second weight score to obtain a second intermediate vector;

[0064] Performing gating processing on a coreference elimination coding vector corresponding to the coding vector based on the second intermediate vector and the second relationship to obtain a second gating vector;

[0065] Re-encode the coreference elimination code vector corresponding to the code vector according to the second gating vector, the second intermediate vector and the second relationship to generate a re-encoded second updated code vector;

[0066] A second relationship pair final encoding vector is generated based on the second updated encoding vector and the classification label of the second updated encoding vector.

[0067] Optionally, obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and a classification label of the coreference resolution encoding vector includes:

[0068] Encoding the classification labels of the coreference elimination coding vector to generate a second label vector;

[0069] Based on any two of the coreference resolution coding vectors and their corresponding second label vectors, a second initial relation pair coding vector of any two of the coreference resolution coding vectors is obtained.

[0070] Optionally, obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and a classification label of the coreference resolution encoding vector includes:

[0071] Encoding the classification labels of the coreference elimination coding vector to generate a second label vector;

[0072] Determine a semantic vector between any two coreference resolution coding vectors based on positions of the any two coreference resolution coding vectors in the candidate entity segment;

[0073] Based on any two of the coreference resolution coding vectors, the semantic vector between the any two coreference resolution coding vectors, and the second label vector corresponding to each of the coreference resolution coding vectors, a second initial relationship pair coding vector of any two of the coreference resolution coding vectors is obtained.

[0074] Optionally, determining the semantic vector between the arbitrary two coreference resolution coding vectors includes:

[0075] Determine a plurality of word vectors between the arbitrary two coreference resolution encoding vectors;

[0076] Pooling or attention processing is performed on multiple word vectors between any two coreference resolution encoding vectors to obtain corresponding semantic vectors.

[0077] Optionally, performing classification prediction processing on the second initial relationship pair encoding vector, and pruning the second initial relationship pair encoding vector according to a preset ratio based on a result of the classification prediction to obtain a second relationship pair encoding vector, includes:

[0078] Scoring the second initial relationship pair encoding vector by a feedforward neural network to obtain a score of the second initial relationship pair encoding vector;

[0079] Performing classification prediction processing on the second initial relationship pair encoding vector to obtain a category of the second initial relationship pair encoding vector;

[0080] The second initial relationship pair encoding vector is pruned based on the category and score of the second initial relationship pair encoding vector to obtain a second relationship pair encoding vector.

[0081] Optionally, the final encoding vector of the first relationship pair is subjected to relationship extraction processing through the relationship extraction model to obtain a relationship extraction result, including: scoring the final encoding vector of the first relationship pair through the relationship extraction model, and performing classification prediction processing on the final encoding vector of the first relationship pair based on the scoring result to obtain a relationship extraction result.

[0082] Optionally, the coreference resolution model is used to perform coreference resolution processing on the final coding vector of the second relationship pair to obtain a coreference resolution result, including: scoring the final coding vector of the second relationship pair through the coreference resolution model, and classifying and predicting the final coding vector of the second relationship pair based on the scoring result to obtain a coreference resolution result.

[0083] Optionally, the method further includes: inputting the final encoding vector of the first relationship pair and the final encoding vector of the second relationship pair into the entity recognition model for entity recognition processing.

[0084] The present application embodiment provides a multi-task model training method, including:

[0085] Obtain at least two sample candidate entity pairs and a classification label of each of the sample candidate entity pairs, and generate a sample candidate entity encoding vector based on the sample candidate entity of each sample candidate entity pair;

[0086] Performing entity recognition processing and classification pruning processing on the sample candidate entity encoding vector through an entity recognition model to obtain a sample recognition pruning encoding vector;

[0087] Inputting the sample identification pruning coding vector into the relation extraction model and the coreference resolution model for two scoring processes respectively, obtaining the final coding vector of the first sample relation pair and the final coding vector of the second sample relation pair;

[0088] Based on the final encoding vector of the first sample relationship pair and the final encoding vector of the second sample relationship pair, the loss values ​​of the entity recognition model, the relationship extraction model and the coreference resolution model are respectively determined, and the entity recognition model, the relationship extraction model and the coreference resolution model are trained.

[0089] Optionally, the sample identification pruning coding vector is respectively input into a relation extraction model and a coreference resolution model for two scoring processes to obtain a final coding vector of a first sample relation pair and a final coding vector of a second sample relation pair, including:

[0090] Inputting the sample identification pruned coding vector into the relationship extraction model, scoring the sample identification pruned coding vector for the first time through the relationship extraction model, and pruning the sample identification pruned coding vector based on the score to obtain a sample relationship extraction coding vector, and performing a second scoring based on the sample relationship extraction coding vector to generate a final coding vector for the first sample relationship pair;

[0091] The sample identification pruned coding vector is input into the coreference resolution model, the sample identification pruned coding vector is scored for the first time by the coreference resolution model, and the sample identification pruned coding vector is pruned based on the score to obtain a sample coreference resolution coding vector, and a second scoring is performed based on the sample coreference resolution coding vector to generate a final coding vector for a second sample relationship pair.

[0092] Optionally, extracting the encoding vector based on the sample relationship and performing a second scoring to generate a final encoding vector for the first sample relationship pair includes:

[0093] Obtaining a first initial sample relationship pair encoding vector based on the sample relationship extraction encoding vector and the classification label of the sample relationship extraction encoding vector;

[0094] Performing classification prediction processing on the first initial sample relationship pair encoding vector, and pruning the first initial sample relationship pair encoding vector according to a preset ratio based on the classification prediction result to obtain a first sample relationship pair encoding vector;

[0095] The first sample relationship pair encoding vector is scored for a second time through the feedforward neural network, and a final encoding vector of the first sample relationship pair is obtained based on the score of the second scoring and the first sample relationship pair encoding vector.

[0096] Optionally, obtaining a final encoding vector of the first sample relationship pair based on the second scoring score and the encoding vector of the first sample relationship pair includes:

[0097] Generate a first sample weight score based on the score of the second scoring of the encoding vector by the first sample relationship;

[0098] Based on the first sample weight score, the sample relationship extraction encoding vectors corresponding to the first sample relationship pair encoding vectors are sequentially subjected to attention processing to obtain a first sample intermediate vector;

[0099] Based on the first sample intermediate vector and the first sample relationship, extract the encoding vector corresponding to the encoding vector and perform gating processing to obtain a first sample gating vector;

[0100] Re-encode the sample relationship extraction encoding vector corresponding to the encoding vector according to the first sample gating vector, the first sample intermediate vector and the first sample relationship to generate a re-encoded first sample update encoding vector;

[0101] Based on the first sample updated encoding vector and the classification label of the first sample updated encoding vector, a first sample relationship pair final encoding vector is generated.

[0102] Optionally, performing a second scoring based on the sample coreference resolution coding vector to generate a second sample relationship pair final coding vector includes:

[0103] Obtaining a second initial sample relationship pair encoding vector based on the sample coreference elimination encoding vector and the classification label of the sample coreference elimination encoding vector;

[0104] Performing classification prediction processing on the second initial sample relationship pair encoding vector, and pruning the second initial sample relationship pair encoding vector according to a preset ratio based on the classification prediction result to obtain a second sample relationship pair encoding vector;

[0105] The second sample relationship pair encoding vector is scored for a second time through the feedforward neural network, and a final encoding vector of the second sample relationship pair is obtained based on the score of the second scoring and the second sample relationship pair encoding vector.

[0106] Optionally, obtaining a final encoding vector of the second sample relationship pair based on the second scoring score and the encoding vector of the second sample relationship pair includes:

[0107] Generate a second sample weight score based on the score of the second scoring of the encoding vector by the second sample relationship;

[0108] Based on the second sample weight score, sequentially performing attention processing on the sample coreference elimination encoding vectors corresponding to the second sample relationship pair encoding vector, obtaining a second sample intermediate vector;

[0109] Performing gating processing on the sample coreference elimination coding vector corresponding to the coding vector based on the second sample intermediate vector and the second sample relationship to obtain a second sample gating vector;

[0110] Generate a re-encoded second sample update code vector according to the second sample gating vector, the second sample intermediate vector, and the sample coreference elimination code vector corresponding to the second sample relationship pair code vector;

[0111] Based on the second sample updated encoding vector and the classification label of the second sample updated encoding vector, a second sample relationship pair final encoding vector is generated.

[0112] Optionally, determining the loss values ​​of the entity recognition model, the relationship extraction model, and the coreference resolution model based on the final encoding vector of the first sample relationship pair and the final encoding vector of the second sample relationship pair, respectively, includes:

[0113] Calculate the loss value of the entity recognition model using a cross entropy loss function based on the score and classification label of the final encoding vector of the first sample relationship and the score and classification label of the final encoding vector of the second sample relationship;

[0114] Calculate the loss value of the relationship extraction model using a cross entropy loss function based on the score of the first sample relationship to the final encoded vector and the classification label of the first sample relationship to the final encoded vector;

[0115] Based on the score of the second sample relationship to the final encoded vector and the classification label of the second sample relationship to the final encoded vector, the loss value of the coreference resolution model is calculated using a cross entropy loss function.

[0116] Optionally, the first sample relationship to the final encoding vector comprises two sample relationship extraction encoding vectors;

[0117] The loss value of the relationship extraction model is calculated based on the score of the first sample relationship to the final encoded vector and the classification label of the first sample relationship to the final encoded vector using a cross entropy loss function, including:

[0118] In the case where the classification label is a symmetric relationship label, two sample relationship extraction encoding vectors of the first sample relationship pair final encoding vector are reversely arranged to generate a reverse first sample relationship pair final encoding vector;

[0119] The score of the final encoding vector of the first sample relationship and the corresponding classification label, the final encoding vector of the reverse first sample relationship and the corresponding classification label are respectively used to calculate the loss value of the relationship extraction model using the cross entropy loss function.

[0120] The embodiment of the present application provides a word processing device based on a multi-task model, including:

[0121] An entity segment acquisition module is configured to acquire candidate entity segments and generate candidate entity segment encoding vectors based on the candidate entity segments;

[0122] An entity recognition pruning module is configured to perform entity recognition processing and classification pruning processing on the candidate entity segment encoding vector through an entity recognition model to obtain a recognition pruned encoding vector;

[0123] a relation extraction processing module, configured to input the identified pruned coding vector into a relation extraction model for secondary scoring, generate a final coding vector for a first relation pair, process the final coding vector for the first relation pair through the relation extraction model, and determine the relationship between words in the candidate entity segment;

[0124] The coreference resolution processing module is configured to input the identified pruned coding vector into the coreference resolution model for secondary scoring, generate a final coding vector for a second relationship pair, process the final coding vector of the second relationship pair through the coreference resolution model, and determine the coreference relationship between the words in the candidate entity fragment.

[0125] The present application embodiment provides a multi-task model training device, including:

[0126] A sample acquisition module is configured to acquire at least two sample candidate entity pairs and a classification label of each of the sample candidate entity pairs, and generate a sample candidate entity encoding vector based on the sample candidate entity of each sample candidate entity pair;

[0127] A sample recognition module is configured to perform entity recognition processing and classification pruning processing on the sample candidate entity fragment encoding vector through an entity recognition model to obtain a sample recognition pruning encoding vector;

[0128] A sample processing module is configured to input the sample identification pruning coding vector into a relation extraction model and a coreference resolution model for two scoring processes, respectively, to obtain a final coding vector of a first sample relation pair and a final coding vector of a second sample relation pair;

[0129] The model training module is configured to determine the loss values ​​of the entity recognition model, the relationship extraction model and the coreference resolution model based on the final encoding vector of the first sample relationship pair and the final encoding vector of the second sample relationship pair, and train the entity recognition model, the relationship extraction model and the coreference resolution model.

[0130] An embodiment of the present application provides a computing device, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein when the processor executes the instructions, the steps of the multi-task model-based word processing method or the multi-task model training method as described above are implemented.

[0131] An embodiment of the present application provides a computer-readable storage medium storing computer instructions, characterized in that when the instructions are executed by a processor, the steps of the multi-task model-based word processing method or the multi-task model training method as described above are implemented.

[0132] The word processing method and device based on the multi-task model provided in the present application first perform entity recognition processing and classification pruning processing on the candidate entity fragment coding vector through the entity recognition model to obtain the recognition pruned coding vector, so as to reduce the negative examples in the candidate entity fragment coding vector, and then input the recognition pruned coding vector into the relationship extraction model and the coreference resolution model for scoring twice, respectively, to obtain the final coding vector of the first relationship pair and the final coding vector of the second relationship pair, and then process the final coding vector of the first relationship pair through the relationship extraction model to determine the relationship between the words in the candidate entity fragment, and process the final coding vector of the second relationship pair through the coreference resolution model to determine the coreference relationship between the words in the candidate entity fragment, thereby adding constraints to the relationship extraction task and the coreference resolution task respectively through the two scorings, improving the evaluation indicators of the relationship extraction model and the coreference resolution model, and can more accurately obtain the relationship between the words in the candidate entity fragment and the coreference relationship respectively.

[0133] Secondly, in this application, the recognition pruned coding vector is first scored for the first time through the feedforward neural network of the relationship extraction model to obtain the relationship extraction coding vector, and then classification prediction and pruning are performed in sequence based on the relationship extraction coding vector to obtain the first relationship pair coding vector, and the first relationship pair coding vector is scored for the second time through the feedforward neural network of the relationship extraction model. Finally, attention processing and gating processing are performed based on the score of the second scoring and the relationship extraction coding vector corresponding to the first relationship pair coding vector, and finally the final coding vector of the first relationship pair is obtained, thereby realizing the information update of the relationship extraction coding vector and integrating the information of other relationship extraction coding vectors, which significantly improves the relationship extraction task.

[0134] Again, in this application, the identified pruned coding vector is first scored for the first time through the feedforward neural network of the coreference resolution model to obtain the coreference resolution coding vector, and then classification prediction and pruning are performed in sequence based on the coreference resolution coding vector to obtain the second relationship pair coding vector, and the second relationship pair coding vector is scored for the second time through the feedforward neural network of the coreference resolution model, and finally, attention processing and gating processing are performed based on the score of the second scoring and the coreference resolution coding vector corresponding to the second relationship pair coding vector, and finally the final coding vector of the second relationship pair is obtained, thereby realizing the information update of the coreference resolution coding vector, incorporating the information of other coreference resolution coding vectors, and significantly improving the coreference resolution task.

[0135] In addition, the final encoding vector of the first relationship pair and the final encoding vector of the second relationship pair are input into the entity recognition model for entity recognition processing, so that the evaluation index of the entity recognition task is significantly improved.

[0136] The training method and device of the multi-task model provided in the present application first organically combine the relationship extraction task, the coreference resolution task, and the entity recognition task during the training process, and then train the relationship extraction model, the coreference resolution model, and the entity recognition model separately, which can effectively improve the effect of model training and improve the performance of the relationship extraction model, the coreference resolution model, and the entity recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0137] Figure 1 is a structural diagram of a multi-task model according to an embodiment of the present application;

[0138] Figure 2 is a schematic diagram of the steps of a word processing method based on a multi-task model according to an embodiment of the present application;

[0139] Figure 3 is a schematic diagram of the steps of a word processing method based on a multi-task model according to an embodiment of the present application;

[0140] Figure 4 It is a flowchart of the steps of the training method of the multi-task model described in one embodiment of the present application;

[0141] Figure 5 is a structural diagram of a word processing device based on a multi-task model according to an embodiment of the present application;

[0142] Figure 6 It is a structural schematic diagram of a training device for a multi-task model according to an embodiment of the present application;

[0143] Figure 7 It is a schematic diagram of the structure of a computing device described in an embodiment of the present application. DETAILED DESCRIPTION

[0144] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0145] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0146] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0147] First, the terms involved in one or more embodiments of the present invention are explained.

[0148] Entity recognition: refers to identifying and extracting entities with specific meaning or strong referentiality in unstructured text, such as names of people, places, organizational structure names, dates and times, proper nouns, etc.

[0149] Entity Recognition Model: A model used to perform entity recognition tasks.

[0150] Relation extraction: Detect and identify certain semantic relationships between entities from the text, such as the sentence "Beijing is the capital, political center and cultural center of China", where the relationship expressed can be (China, capital, Beijing), (China, political center, Beijing) or (China, cultural center, Beijing).

[0151] Relation extraction model: A model used to perform relation extraction tasks.

[0152] Co-reference resolution: a special relationship extraction. One of the entities in co-reference resolution is usually a different way of saying another entity in the current context. The relationship between the two entities can be expressed as (entity 1, co-reference, entity 2).

[0153] Coreference Resolution Model: A model for performing coreference resolution tasks.

[0154] Candidate entity fragment (span): a fragment consisting of one or more words in a sentence.

[0155] Candidate entity span encoding vector (span embedding): The vector generated by the candidate entity span after the encoder's encoding process.

[0156] Identify pruned encoding vector: the encoding vector remaining after the candidate entity segment encoding vector is pruned.

[0157] Pruning: Filter according to preset rules.

[0158] Relation extraction encoding vector: the encoding vector remaining after pruning the recognition pruned encoding vector based on the relation extraction processing result.

[0159] First label vector: an encoding vector obtained by encoding the classification label of the relation extraction encoding vector.

[0160] The first initial relationship pair encoding vector is an encoding vector formed by combining the two relationship extraction encoding vectors and their first label vectors and the distance feature vectors.

[0161] First relation pair encoding vector: the encoding vector remaining after pruning the first initial relation pair encoding vector based on the score of the first initial relation pair encoding vector.

[0162] Co-reference resolution code vector: the code vector remaining after pruning the recognition pruned code vector based on the result of the coreference resolution process.

[0163] Second label vector: an encoding vector obtained by encoding the classification label of the coreference resolution encoding vector.

[0164] Second relationship pair encoding vector: an encoding vector formed by combining two coreference resolution encoding vectors and their second label vectors and a distance feature vector.

[0165] Feedforward Neural Network (FFNN): A simplest neural network, where neurons are arranged in layers, each neuron is only connected to the neurons in the previous layer, receives the output of the previous layer, and outputs it to the next layer. There is no feedback between the layers. It is one of the most widely used and fastest growing artificial neural networks. In this application, the entity recognition model, the relationship extraction model, and the coreference resolution model share a feedforward neural network for scoring.

[0166] Convolutional Neural Networks (CNN): is a type of feedforward neural network that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning. In this application, the character-level features corresponding to each word are obtained through CNN network encoding.

[0167] Classification label: A label used to identify the type of encoded vector.

[0168] Accuracy: refers to the ratio of the number of correctly identified entities to the number of identified entities. The value is between 0 and 1. The larger the value, the higher the accuracy.

[0169] Recall rate: refers to the ratio of the number of correctly identified entities to the number of entities in the sample. The value is between 0 and 1. The larger the value, the higher the recall rate.

[0170] Weighted harmonic mean: also known as F1 value, F1 value = (2*accuracy*recall) / (accuracy+recall).

[0171] In the present application, a word processing method and apparatus based on a multi-task model, a training method and apparatus for a multi-task model, a computing device and a computer-readable storage medium are provided, which are described in detail one by one in the following embodiments.

[0172] like Figure 1 As shown, this embodiment provides a multi-task model, which is used in the multi-task model-based word processing method described in this application, including an encoder, an entity recognition model, a relationship extraction model and a coreference resolution model, wherein the entity recognition model, the relationship extraction model and the coreference resolution model share a feedforward neural network.

[0173] The entity recognition model is a model for identifying entities in unstructured input text. In this embodiment, the entity recognition model performs entity recognition processing and classification pruning processing on the candidate entity fragment encoding vector to obtain a recognition pruned encoding vector.

[0174] The relationship extraction model is a model used to detect, identify, and extract semantic relationships between entities. In this embodiment, the relationship extraction model performs a first score on the recognition pruned coding vector, and prunes the recognition pruned coding vector based on the score to obtain a relationship extraction coding vector; a second score is performed based on the relationship extraction coding vector to generate a final coding vector for the first relationship pair, and the final coding vector of the first relationship pair is subjected to relationship extraction processing by the relationship extraction model to obtain a relationship extraction result, and the relationship between words in the candidate entity fragment is determined based on the relationship extraction result.

[0175] The coreference resolution model is a model for detecting, identifying, and extracting entities with coreference relationships. In this embodiment, the coreference resolution model scores the recognition pruned coding vector for the first time, and prunes the recognition pruned coding vector based on the score to obtain a coreference resolution coding vector; generates a final coding vector for a second relationship pair based on the coreference resolution coding vector, and performs coreference resolution processing on the final coding vector of the second relationship pair through the coreference resolution model to obtain a coreference resolution result, and determines the coreference relationship between words in the candidate entity fragment based on the coreference resolution result.

[0176] Furthermore, the final encoding vector of the first relationship pair and the final encoding vector of the second relationship pair are input into the entity recognition model for entity recognition processing to improve the entity recognition effect of the entity recognition model.

[0177] The multi-task model provided in this embodiment organically combines the relationship extraction model, entity recognition model, and coreference resolution model. The three share a feedforward neural network for scoring, which can realize information sharing among the relationship extraction model, entity recognition model, and coreference resolution model, thereby improving the accuracy and recall rate of the above-mentioned relationship extraction model, entity recognition model, and coreference resolution model.

[0178] like Figure 2 As shown, Figure 2 It is a flowchart showing the steps of a word processing method based on a multi-task model according to an embodiment of the present application, including steps S210 to S240, wherein step S230 and step S240 are parallel steps and there is no necessary order of execution.

[0179] S210: Obtain a candidate entity segment, and generate a candidate entity segment encoding vector based on the candidate entity segment.

[0180] The candidate entity segment is a word set composed of one or more words in the target sentence, target paragraph, or target article, and each word represents an entity. Specifically, the candidate entity segment can be obtained by performing word segmentation on the target sentence, target paragraph, or target article, and then extracting one or more target words from the result of the word segmentation to form a word set.

[0181] For example, suppose that after word segmentation processing is performed on the target sentence, 10 words including word segmentation processing results A1-A10 are obtained, and a word set consisting of A1-A6 is extracted from the above word segmentation processing results, and the above word set is used as a candidate entity segment.

[0182] In practical applications, the candidate entity fragments may be input into an encoder for encoding processing to generate a candidate entity fragment encoding vector.

[0183] In this embodiment, the encoder includes a bidirectional LSTM, a pre-trained BERT model, a CNN network, and any combination thereof.

[0184] Preferably, a sentence containing several candidate entity fragments is first encoded by a pre-trained BERT model to obtain a word-level feature vector of the sentence, and then encoded by a CNN network to obtain a character-level feature vector of the sentence. The word-level feature vector and the character-level feature vector are concatenated to obtain a concatenated vector, and then the concatenated vector is encoded by a bidirectional LSTM network to obtain a feature vector with context features. Finally, the encoding vector of each candidate entity fragment is calculated based on the extracted candidate entity fragments using an attention mechanism. The encoding vector of the candidate entity fragment can be expressed by the following formula:

[0185]

[0186] Among them, g i The encoding vector representing the candidate entity fragment, x START(i) * 、x END(i) * The vector representing the start and end positions of the candidate entity fragment, Φ (i) represents additional features, It represents the result of calculating the words in each candidate entity segment based on the attention mechanism. The specific calculation process is as follows:

[0187]

[0188]

[0189]

[0190]

[0191] Specifically, t represents the candidate entity segment, i represents the word in the candidate entity segment, and formula (2) represents the encoding vector x corresponding to each word in the candidate entity segment t * The vector (h) output by the forward propagation of the bidirectional LSTM t，1 ) and the vector of back-propagation output (h t，-1 ), formula (3) indicates that the parameter α of the candidate entity segment t is obtained by multiplying its parameter w with the score of the candidate entity segment given by the feedforward neural network, and formula (4) indicates the weight a of each word in the candidate entity segment i，t Based on the parameter α of the candidate entity segment in which it is located and the total parameter of the word in the candidate entity segment, formula (5) represents the encoding vector corresponding to each word in the candidate entity segment: The weight parameter a of the word in the candidate entity segment i，t and the candidate entity fragment encoding vector x t get.

[0192] This embodiment obtains candidate entity segments and performs encoding processing on the candidate entity segments to prepare for the execution of other subsequent tasks, thereby improving the efficiency of the execution of subsequent tasks.

[0193] S220. Perform entity recognition processing and classification pruning processing on the candidate entity fragment encoding vector through an entity recognition model to obtain a recognition pruning encoding vector.

[0194] It should be noted that, in this embodiment, the entity recognition model, the relationship extraction model, and the coreference resolution model share a feedforward neural network for scoring.

[0195] Specifically, the step S220 may also include steps S221 and S222.

[0196] S221. Input the candidate entity segment encoding vector into an entity recognition model, and score the candidate entity segment encoding vector through the feedforward neural network.

[0197] The score of each candidate entity segment coding vector is composed of a basic score (Mention score) and a classification score (classifier score), and the above basic score and classification score are obtained by scoring through a feedforward neural network. The score of the candidate entity segment coding vector can be the sum of the basic score and the classification score, the average value, the weighted average value, etc., and this application does not limit this.

[0198] The feedforward neural network uses the principle of deep learning to score the candidate entity fragment coding vector. Specifically, the candidate entity fragment coding vector is calculated or encoded again by using the feedforward neural network, and the corresponding score is mapped to obtain the score of the candidate entity fragment coding vector. It should be noted that the mapping of the score by the feedforward neural network can be continuously adjusted through the execution of subsequent tasks, the calculation of loss values, the return of gradients, etc. The score of the candidate entity fragment coding vector can be a ten-point score, a hundred-point score, a thousand-point score, etc., and this application does not limit this.

[0199] This embodiment can improve the accuracy of entity recognition processing and the effect of the entity recognition model by scoring the candidate entity segment encoding vectors and then performing entity recognition processing.

[0200] S222. Classify the candidate entity coding vector based on the score of the candidate entity fragment coding vector to obtain a classification label of the candidate entity coding vector, and prune the candidate entity coding vector to obtain a recognized pruned coding vector.

[0201] In practical applications, the candidate entity fragment coding vectors are classified based on their scores to obtain a classification label for each candidate entity fragment coding vector, and then the category to which each candidate entity fragment coding vector belongs can be obtained. A certain proportion of the candidate entity fragment coding vectors in one or several categories can be pruned according to the classification results, and the remaining candidate entity coding vectors can be used as recognition pruned coding vectors. Alternatively, candidate entity coding vectors with scores less than a preset threshold can be pruned, and the remaining candidate entity coding vectors can be used as recognition pruned coding vectors. This application does not impose any restrictions on this.

[0202] For example, suppose the candidate entity fragment encoding vectors exist in n categories, namely, the first category, the second category, ... the nth category, among which the nth category is a negative example, that is, all candidate entity fragment encoding vectors that do not belong to the previous category are classified as the nth category. After classification, m1 of the m candidate entity fragment encoding vectors belong to the first category, m2 belong to the second category, ... m n belongs to the nth class (m1+m2+…m n =m), then the candidate entity segment encoding vectors can be pruned in the following three ways: (1) p1% of the candidate entity segment encoding vectors are pruned in the first category, p2% of the candidate entity segment encoding vectors are pruned in the second category, and so on. n % of the candidate entity fragment encoding vectors, and the remaining candidate entity encoding vectors are used as recognition pruning encoding vectors, where p1, p2, ... n The values ​​of can be the same or different; (2) The encoding vectors of candidate entity fragments of the first, second, ..., n-1th categories are not pruned, and the encoding vectors of candidate entity fragments of the nth category are pruned. n % of the candidate entity fragment coding vectors, and the remaining candidate entity coding vectors are used as recognition pruned coding vectors; (3) the candidate entity fragment coding vectors whose scores of one or more categories are less than a preset threshold are pruned, and the remaining candidate entity coding vectors are used as recognition pruned coding vectors, wherein the preset threshold of the score can be determined according to the specific situation, and the present application does not impose any restrictions on this.

[0203] In this embodiment, classification and pruning of candidate entity fragment encoding vectors helps to improve the quality of input encoding vectors for subsequent relationship extraction models and coreference resolution models, laying a foundation for the execution of relationship extraction tasks and coreference resolution tasks.

[0204] S230: input the identified pruned coding vector into a relation extraction model for scoring twice to generate a final coding vector for a first relation pair, process the final coding vector for the first relation pair through the relation extraction model, and determine the relationship between the words in the candidate entity fragment.

[0205] Specifically, the step S230 may also include steps S231 to S232.

[0206] S231. Score the recognition pruned coding vector for the first time through a relation extraction model, and prune the recognition pruned coding vector based on the score to obtain a relation extraction coding vector.

[0207] Specifically, the relation extraction model and the entity recognition model share a feed-forward neural network for scoring;

[0208] Step S231 includes: inputting the recognition pruned coding vector into the relationship extraction model, scoring the recognition pruned coding vector for the first time through a feedforward neural network to obtain the score of the recognition pruned coding vector, and then using the recognition pruned coding vector with a score greater than or equal to a preset threshold as the relationship extraction coding vector.

[0209] Among them, the score of each recognition pruned code vector is composed of a basic score and a classification score, and the above basic score and classification score are both obtained by scoring through a feedforward neural network.

[0210] This embodiment scores the recognition pruned coding vector and further processes the recognition pruned coding vector based on the entity recognition task, which helps to pave the way for the execution of the subsequent relationship extraction task.

[0211] S232. Perform a second scoring based on the relationship extraction coding vector to generate a final coding vector for the first relationship pair, perform relationship extraction processing on the final coding vector of the first relationship pair through the relationship extraction model to obtain a relationship extraction result, and determine the relationship between the words in the candidate entity fragment based on the relationship extraction result.

[0212] Specifically, in step S232, the second scoring is performed based on the relationship extraction coding vector to generate the final coding vector of the first relationship pair, including the following steps S2321 to S2323:

[0213] S2321. Obtain a first initial relationship pair encoding vector based on the relationship extraction encoding vector and the classification label of the relationship extraction encoding vector.

[0214] Specifically, the classification label of the relationship extraction encoding vector may be encoded to generate a first label vector; and a first initial relationship pair encoding vector may be obtained based on the relationship extraction encoding vector and the first label vector.

[0215] In practical applications, each first initial relationship pair encoding vector is composed of two relationship extraction encoding vectors and classification label encoding vectors corresponding to the two relationship extraction encoding vectors. In other words, the relationship extraction encoding vectors of the two words in the candidate entity segment and the first label vectors of the two words can be concatenated to obtain the first initial relationship pair encoding vector, as shown below:

[0216] span_pair_embeddings=torch.cat([span1_embeddings, span2_embeddings, span1_embeddings*span2_embeddings, span1_label_embedding, span2_label_embedding], -1).

[0217] Among them, torch.cat is a function used to splice two or more vectors together, span_pair_embeddings represents the first initial relationship pair encoding vector, span1_embeddings represents the relationship extraction encoding vector 1, span2_embeddings represents the relationship extraction encoding vector 2, span1_label_embedding represents the label vector of the relationship extraction encoding vector 1, and span2_label_embedding represents the label vector of the relationship extraction encoding vector 2.

[0218] It should be noted that the classification label is a type of feature information of the candidate entity fragment. In addition, other types of feature information, such as distance, etc., can be combined when generating the first initial relationship pair encoding vector, depending on the specific situation. This application does not impose any restrictions on this.

[0219] For example, the relationship extraction encoding vectors of two words in the candidate entity segment, the first label vectors of the two words, and the distance feature vector between the two words can be concatenated to obtain the first initial relationship pair encoding vector, as shown below:

[0220] span_pair_embeddings=torch.cat([span1_embeddings, span2_embeddings, span1_embeddings*span2_embeddings, antecedent_distance_embeddings, span1_label_embedding, span2_label_embedding], -1).

[0221] Among them, torch.cat is a function used to splice two or more vectors together, span_pair_embeddings represents the first initial relationship pair encoding vector, span1_embeddings represents the relationship extraction encoding vector 1, span2_embeddings represents the relationship extraction encoding vector 2, antecedent_distance_embeddings represents the distance feature vector between the relationship extraction encoding vector 1 and the relationship extraction encoding vector 2, span1_label_embedding represents the label vector of the relationship extraction encoding vector 1, and span2_label_embedding represents the label vector of the relationship extraction encoding vector 2.

[0222] For another example, a specific method for generating the first initial relationship pair encoding vector includes:

[0223] Encoding the classification label of the relation extraction encoding vector to generate a first label vector;

[0224] Determine a semantic vector between any two relation extraction encoding vectors based on positions of the any two relation extraction encoding vectors in the candidate entity fragment;

[0225] Based on any two of the relationship extraction encoding vectors, the semantic vector between the any two relationship extraction encoding vectors and the first label vector corresponding to each of the relationship extraction encoding vectors, a first initial relationship pair encoding vector of any two of the relationship extraction encoding vectors is obtained.

[0226] Specifically, determining the semantic vector between any two relationship extraction encoding vectors includes: determining multiple word vectors between the any two relationship extraction encoding vectors; performing pooling or attention processing on the multiple word vectors between any two relationship extraction encoding vectors to obtain a corresponding semantic vector.

[0227] In a specific application, for the candidate entity fragment [w1, w2, …, wx-1, wx], [w1, w2] is the relationship extraction coding vector span1, [wx-1, wx] is the relationship extraction coding vector span2, then [w3, …, wx-2] is the word vector between any two relationship extraction coding vectors span1 and span2, and the word vector [w3, …, wx-2] between the two relationship extraction coding vectors span1 and span2 is pooled or paid attention to obtain the corresponding semantic vector, thereby increasing the semantic information of the first initial relationship pair coding vector and enhancing the expressiveness of the first initial relationship pair coding vector.

[0228] Regarding the pooling process, it should be noted that for the word vector [w3, ..., wx-2], if all this information is retained, there will be information redundancy, which increases the difficulty of calculation. At this time, the pooling process is equivalent to a dimensionality reduction operation to compress the vector and retain the main features. The pooling process is to take the maximum value or average value of a small matrix area to replace the area, so as to obtain the final semantic vector. The size of the small matrix can be set according to actual needs.

[0229] For attention processing, the calculation formula is as shown in the above formulas (3) to (5), which will not be repeated here.

[0230] Specifically, the first initial relationship is calculated for the encoding vector as follows:

[0231] span_pair_embeddings=torch.cat([span1_embeddings, span2_embeddings, span1_embeddings*span2_embeddings, segment_info, span1_label_embedding, span2_label_embedding], -1).

[0232] Among them, torch.cat is a function used to splice two or more vectors together, span_pair_embeddings represents the first initial relation pair encoding vector, span1_embeddings represents relation extraction encoding vector 1, span2_embeddings represents relation extraction encoding vector 2, segment_info represents the word vector between relation extraction encoding vector 1 and relation extraction encoding vector 2, span1_label_embedding represents the label vector of relation extraction encoding vector 1, and span2_label_embedding represents the label vector of relation extraction encoding vector 2.

[0233] It should be noted that in the process of calculating span_pair_embeddings, not all words can have a relationship. For example, in a paragraph of text, words that are far apart basically have no relationship. Therefore, a distance threshold is set in this embodiment. If the distance between span1 and span2 exceeds the threshold, the span_pair_embeddings of span1 and span2 are directly pruned. The value of the distance threshold can be set according to actual needs, for example, the distance threshold is set to 60 word units.

[0234] Optionally, obtaining the first initial relationship pair encoding vector of any two of the relationship extraction encoding vectors includes: according to the semantic relationship between any two of the relationship extraction encoding vectors in the candidate entity fragment, respectively obtaining the first initial relationship pair encoding vector of the forward semantic relationship and the first initial relationship pair encoding vector of the reverse semantic relationship of any two of the relationship extraction encoding vectors.

[0235] That is, when span_pair_embeddings are generated, both span_pair_embedding12 and span_pair_embedding21 are generated.

[0236] It should be noted that the positive and negative semantic relationships are not necessarily consistent with the specific arrangement order. Here, it is assumed that the relationship is an asymmetric relationship, and the positive semantic relationship is considered to be correct and consistent with the annotation, while the negative semantic relationship is incorrect and inconsistent with the data annotation. For example, if method is used for task, then [span1(method)span2(task)] is considered to be the correct relationship of the positive semantic relationship, and the reverse order is the wrong negative relationship. At this time, span1 and span2 are arranged in the order of first and last.

[0237] But if we change the notation, if it is part of, the possible expressions in the sentence are 1) span1 is part of span2; 2) span 1 includes span2.

[0238] In both sentences, the span pairs are marked as a part of relationship, but the positive semantic relationship in 1) is [span1 span2], while the positive semantic relationship in 2) is [span2 span1].

[0239] This embodiment obtains a first initial relationship pair encoding vector based on the relationship extraction encoding vector and the classification label of the relationship extraction encoding vector, which helps to improve the execution efficiency and effect of the relationship extraction task.

[0240] S2322. Perform classification prediction processing on the first initial relationship pair coding vector, and prune the first initial relationship pair coding vector according to a preset ratio based on the classification prediction result to obtain a first relationship pair coding vector.

[0241] Specifically, the first initial relationship pair encoding vector can be scored through a feedforward neural network to obtain the score of the first initial relationship pair encoding vector; the first initial relationship pair encoding vector can be classified and predicted to obtain the category of the first initial relationship pair encoding vector; the first initial relationship pair encoding vector can be pruned based on the category and score of the first initial relationship pair encoding vector to obtain the first relationship pair encoding vector.

[0242] In practical applications, the category of the first initial relationship pair encoding vector can be determined by the softmax function. The formula of the softmax function is shown in the following formula (6):

[0243]

[0244] Among them, S i represents the softmax value corresponding to the i-th first initial relationship pair encoding vector; i represents the i-th first initial relationship pair encoding vector; j represents the total number of first initial relationship pair encoding vectors.

[0245] Specifically, the category of the first initial relationship pair encoding vector is the category of the relationship between the two words therein. In practical applications, the first initial relationship pair encoding vector can also be pruned based on the category and score of the first initial relationship pair encoding vector. A certain proportion of one or more categories of first initial relationship pair encoding vectors can be pruned, or first initial relationship pair encoding vectors with scores less than a preset threshold can be pruned. This can be determined based on the specific situation, and the present application does not impose any restrictions on this.

[0246] Optionally, in the case where the classification label is a directional classification label, it is a label attribute with obvious directionality, such as used for. Usually the labeled data will be [method (head entity label), use for (relationship label), task (tail entity label)], and not the other way around. Therefore, according to this classification score, pruning is performed according to the preset proportions according to the categories. When obtaining the pair span-embeddings that make up the high scores, it is necessary to especially compare the scores of pair span-embedding12 and pair span-embedding21 in the current category. If pair span-embedding12 is relatively high in the current category, then pair span-embedding21 should be a negative example, so it must be forced to be given a very low score. In the actual application stage, the score can be set to 0 or to other scores.

[0247] Specifically, the first initial relationship pair encoding vector is scored through a feedforward neural network to obtain the score of the first initial relationship pair encoding vector, including: scoring the first initial relationship pair encoding vector of the forward semantic relationship and the first initial relationship pair encoding vector of the reverse semantic relationship of any two of the relationship extraction encoding vectors through a feedforward neural network, and making the score of the first initial relationship pair encoding vector of the reverse semantic relationship less than a preset threshold.

[0248] In this embodiment, the preset threshold can be set to multiple values, such as 0.

[0249] This embodiment obtains the first relationship pair encoding vector by scoring, classifying and pruning the first initial relationship pair encoding vector, which helps to further improve the execution efficiency and effect of the relationship extraction task.

[0250] S2323. Score the first relationship pair encoding vector for a second time through the feedforward neural network, and obtain a final encoding vector for the first relationship pair based on the score of the second scoring and the first relationship pair encoding vector.

[0251] In this embodiment, obtaining the final encoding vector of the first relationship pair based on the second scoring score and the encoding vector of the first relationship pair in step S2323 includes the following steps S21 to S24:

[0252] S21. Generate a first weight score based on the score of the second scoring of the encoded vector based on the first relationship.

[0253] The calculation of the first weight score is as follows:

[0254] s(i, j) = s m (i)+s m (j)+s a (i,j) (7)

[0255] Among them, i, j represent the relation extraction encoding vector;

[0256] s m (i) represents the score of the relation extraction encoding vector i in the encoding vector of the first relation pair as an entity;

[0257] s m (j) represents the score of the relation extraction encoding vector j in the encoding vector of the first relation pair as an entity;

[0258] s a (i, j) represents the fraction of relation extraction encoding vector j that may be the antecedent of relation extraction encoding vector i;

[0259] s(i,j) represents the first weight score.

[0260] S22. Based on the first weight score, the relationship extraction encoding vectors corresponding to the first relationship pair encoding vector are sequentially subjected to attention processing to obtain a first intermediate vector.

[0261] The calculation of the first intermediate vector refers to the following formulas (8) to (9):

[0262]

[0263]

[0264] Wherein, g represents the relation extraction encoding vector corresponding to the encoding vector of the first relation pair;

[0265] P represents the weight coefficient of attention;

[0266] a represents the first intermediate vector;

[0267] s represents the first weight score;

[0268] i, j both represent a parameter in the [0,n] set.

[0269] S23. Perform gating processing on the relationship extraction coding vector corresponding to the coding vector based on the first intermediate vector and the first relationship to obtain a first gating vector.

[0270] The calculation of the first gating vector is shown in the following formula (10):

[0271]

[0272] Among them, σ represents the logical sigmoid function,

[0273] W f represents the weight parameter,

[0274] represents the i-th first intermediate vector;

[0275] Relation extraction encoding vector representing the encoding vector corresponding to the i-th first relation pair;

[0276] i represents a parameter in the [0,n] set;

[0277] Represents the i-th first gating vector.

[0278] S24. Re-encode the relation extraction encoding vector corresponding to the encoding vector according to the first gating vector, the first intermediate vector and the first relation to generate a re-encoded first updated encoding vector.

[0279] Specifically, the calculation formula of the first updated encoding vector is shown in the following formula (11):

[0280]

[0281] in, represents the i-th first gating vector;

[0282] represents the i-th first intermediate vector;

[0283] Relation extraction encoding vector representing the encoding vector corresponding to the i-th first relation pair;

[0284] Represents the i-th first updated encoding vector.

[0285] According to this formula, the calculation of the first updated code vector is to use the first gating vector to re-encode the relation extraction code vector corresponding to the first relation pair code vector. This value is between (0, 1). It can be understood as how much (f) of the newly obtained first updated code vector comes from the original relation extraction code vector. How much (1-f) comes from the first intermediate vector

[0286] S25. Generate a final encoding vector for a first relationship pair based on the first updated encoding vector and the classification label of the first updated encoding vector.

[0287] Specifically, the calculation of the final encoding vector by the first relation is as follows:

[0288] fspan_pair_embeddings=torch.cat([f1_embeddings, f2_embeddings, f1_embeddings*f2_embeddings, segment_info, f1_label_embedding, f2_label_embedding], -1).

[0289] Among them, torch.cat is a function used to concatenate two or more vectors together, fspan_pair_embeddings represents the final encoding vector of the first relationship pair, f1_embeddings represents the first updated encoding vector 1, f2_embeddings represents the first updated encoding vector 2, segment_info represents the word vector between the first updated encoding vector 1 and the first updated encoding vector 2, f1_label_embedding represents the label vector of the first updated encoding vector 1, and f2_label_embedding represents the label vector of the first updated encoding vector 2.

[0290] Through steps S21 to S24, the recognition pruned coding vector is first scored for the first time through the feedforward neural network of the relationship extraction model to obtain the relationship extraction coding vector, and then classification prediction and pruning are performed in sequence based on the relationship extraction coding vector to obtain the first relationship pair coding vector, and the first relationship pair coding vector is scored for the second time through the feedforward neural network of the relationship extraction model, and finally attention processing and gating processing are performed based on the score of the second scoring and the relationship extraction coding vector corresponding to the first relationship pair coding vector, and then re-encoding is performed to finally obtain the final coding vector of the first relationship pair, thereby realizing the information update of the relationship extraction coding vector and integrating the information of other relationship extraction coding vectors, so that the relationship extraction task has been significantly improved.

[0291] Finally, after obtaining the final encoding vector of the first relationship pair, the final encoding vector of the first relationship pair is scored through the relationship extraction model, and the final encoding vector of the first relationship pair is classified and predicted based on the scoring result to obtain the relationship extraction result.

[0292] S240, input the identified pruned coding vector into a coreference resolution model for scoring twice to generate a final coding vector for a second relationship pair, process the final coding vector of the second relationship pair through the coreference resolution model, and determine the coreference relationship between the words in the candidate entity segment.

[0293] Specifically, the step S240 may further include steps S241 to S242:

[0294] S241. Perform a first score on the identification pruned coding vector using a coreference resolution model, and prune the identification pruned coding vector based on the score to obtain a coreference resolution coding vector.

[0295] Specifically, the coreference resolution model and the entity recognition model share a feed-forward neural network for scoring;

[0296] Step S241 includes: inputting the recognition pruned coding vector into the coreference resolution model, scoring the recognition pruned coding vector for the first time through a feedforward neural network to obtain the score of the recognition pruned coding vector, and then using the recognition pruned coding vector with a score greater than or equal to a preset threshold as the coreference resolution coding vector.

[0297] Among them, the score of each recognition pruned code vector is composed of a basic score and a classification score, and the above basic score and classification score are both obtained by scoring through a feedforward neural network.

[0298] This embodiment scores the recognition pruned coding vectors and further processes the recognition pruned coding vectors based on the entity recognition task, which helps to pave the way for the subsequent coreference resolution task.

[0299] S242. Perform a second scoring based on the coreference resolution coding vector to generate a final coding vector for a second relationship pair, perform coreference resolution processing on the final coding vector of the second relationship pair through the coreference resolution model to obtain a coreference resolution result, and determine the coreference relationship between the words in the candidate entity fragment based on the coreference resolution result.

[0300] Specifically, in step S242, performing a second scoring based on the coreference resolution coding vector to generate a final coding vector for the second relationship pair includes the following steps S2421 to S2423:

[0301] S2421. Obtain a second initial relationship pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector.

[0302] Specifically, the classification labels of the coreference resolution coding vectors may be encoded to generate a second label vector; based on any two of the coreference resolution coding vectors and their corresponding second label vectors, a second initial relationship pair coding vector of any two of the coreference resolution coding vectors may be obtained.

[0303] In practical applications, each second relationship pair encoding vector is composed of two coreference resolution encoding vectors and classification label encoding vectors corresponding to two coreference resolution encoding vectors. In other words, the coreference resolution encoding vectors of the two words in the candidate entity segment and the second label vectors of the two words can be concatenated to obtain the second relationship pair encoding vector, as shown below:

[0304] span_pair_embeddings=torch.cat([span1_embeddings, span2_embeddings, span1_embeddings*span2_embeddings, span1_label_embedding, span2_label_embedding], -1).

[0305] Among them, torch.cat is a function used to concatenate two or more vectors together, span_pair_embeddings represents the second relationship pair encoding vector, span1_embeddings represents the coreference resolution encoding vector 1, span2_embeddings represents the coreference resolution encoding vector 2, span1_label_embedding represents the label vector of the coreference resolution encoding vector 1, and span2_label_embedding represents the label vector of the coreference resolution encoding vector 2.

[0306] It should be noted that the classification label is a type of feature information of the candidate entity fragment. In addition, other types of feature information, such as distance, etc., can be combined when generating the second relationship pair encoding vector, depending on the specific situation. This application does not impose any restrictions on this.

[0307] For example, the coreference resolution encoding vectors of the two words in the candidate entity segment, the second label vectors of the two words, and the distance feature vector between the two words can be concatenated to obtain the second relationship pair encoding vector, as shown below:

[0308] span_pair_embeddings=torch.cat([span1_embeddings, span2_embeddings, span1_embeddings*span2_embeddings, antecedent_distance_embeddings, span1_label_embedding, span2_label_embedding], -1).

[0309] Among them, torch.cat is a function used to concatenate two or more vectors together, span_pair_embeddings represents the second relationship pair encoding vector, span1_embeddings represents the coreference resolution encoding vector 1, span2_embeddings represents the coreference resolution encoding vector 2, antecedent_distance_embeddings represents the distance feature vector between the coreference resolution encoding vector 1 and the coreference resolution encoding vector 2, span1_label_embedding represents the label vector of the coreference resolution encoding vector 1, and span2_label_embedding represents the label vector of the coreference resolution encoding vector 2.

[0310] This embodiment obtains a second relationship pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector, which helps to improve the execution efficiency and effect of the coreference resolution task.

[0311] In another specific implementation scheme, step S2421 includes: encoding the classification labels of the coreference resolution coding vectors to generate a second label vector; determining the semantic vector between any two coreference resolution coding vectors based on the positions of the any two coreference resolution coding vectors in the candidate entity fragment; obtaining the second initial relationship pair coding vector of any two coreference resolution coding vectors based on any two coreference resolution coding vectors, the semantic vector between the any two coreference resolution coding vectors and the second label vector corresponding to each coreference resolution coding vector.

[0312] Specifically, determining the semantic vector between any two coreference resolution coding vectors includes: determining multiple word vectors between any two coreference resolution coding vectors; performing pooling or attention processing on the multiple word vectors between any two coreference resolution coding vectors to obtain a corresponding semantic vector.

[0313] In a specific application, for the candidate entity fragment [z1, z2, …, zx-1, zx], [z1, z2] is the coreference resolution coding vector span3, [zx-1, zx] is the coreference resolution coding vector span4, then [z3, …, zx-2] is the word vector between any two coreference resolution coding vectors span3 and span4, and the word vector [z3, …, zx-2] between the two coreference resolution coding vectors span3 and span4 is pooled or attended to obtain the corresponding semantic vector, thereby increasing the semantic information of the second initial relationship pair coding vector and enhancing the expressiveness of the second initial relationship pair coding vector.

[0314] Regarding the pooling process, it should be noted that for the word vector [z3, ..., zx-2], if all this information is retained, there will be information redundancy, which increases the difficulty of calculation. At this time, the pooling process is equivalent to a dimensionality reduction operation to compress the vector and retain the main features. The pooling process is to take the maximum value or average value of a small matrix area to replace the area, so as to obtain the final semantic vector. The size of the small matrix can be set according to actual needs.

[0315] For attention processing, the calculation formula is as shown in the above formulas (3) to (5), which will not be repeated here.

[0316] Specifically, the second initial relation is calculated as follows:

[0317] span_pair_embeddings=torch.cat([span3_embeddings, span4_embeddings, span3_embeddings*span4_embeddings, segment_info, span3_label_embedding, span4_label_embedding], -1).

[0318] Among them, torch.cat is a function used to concatenate two or more vectors together, span_pair_embeddings represents the second initial relationship pair encoding vector, span3_embeddings represents the coreference resolution encoding vector 1, span4_embeddings represents the coreference resolution encoding vector 2, segment_info represents the word vector between the coreference resolution encoding vector 1 and the coreference resolution encoding vector 2, span3_label_embedding represents the label vector of the coreference resolution encoding vector 1, and span4_label_embedding represents the label vector of the coreference resolution encoding vector 2.

[0319] It should be noted that in the process of calculating span_pair_embeddings, not all words can have a relationship. For example, in a paragraph of text, words that are far apart basically have no relationship. Therefore, a distance threshold is set in this embodiment. If the distance between span3 and span4 exceeds the threshold, the span_pair_embeddings of span3 and span4 are directly pruned. The value of the distance threshold can be set according to actual needs, for example, the distance threshold is set to 60 word units.

[0320] S2422. Perform classification prediction processing on the second initial relationship pair coding vector, and prune the second initial relationship pair coding vector according to a preset ratio based on the classification prediction result to obtain a second relationship pair coding vector.

[0321] Specifically, step S2422 includes:

[0322] Scoring the second initial relationship pair encoding vector by a feedforward neural network to obtain a score of the second initial relationship pair encoding vector;

[0323] Performing classification prediction processing on the second initial relationship pair encoding vector to obtain a category of the second initial relationship pair encoding vector;

[0324] The second initial relationship pair encoding vector is pruned based on the category and score of the second initial relationship pair encoding vector to obtain a second relationship pair encoding vector.

[0325] In practical applications, the category of the second initial relationship pair encoding vector can be determined by a softmax function. The formula of the softmax function is shown in formula (6), which will not be repeated here.

[0326] Specifically, the process of coreference resolution includes scoring and classification prediction processing. In other words, the second relationship pair encoding vector is scored through the coreference resolution model, and the second relationship pair encoding vector is classified and predicted based on the scoring result, that is, the coreference resolution processing is completed, the coreference resolution result is obtained, and the coreference relationship between words is determined.

[0327] This embodiment determines the coreference relationship between words by performing coreference resolution processing on the second relationship pair encoding vector, which can improve the accuracy of the coreference resolution task and further effectively improve the accuracy of determining the coreference relationship between words.

[0328] S2423. Score the second relationship pair encoding vector for a second time through the feedforward neural network, and obtain a final encoding vector for the second relationship pair based on the score of the second scoring and the second relationship pair encoding vector.

[0329] In this embodiment, obtaining the final encoding vector of the second relationship pair based on the second scoring score and the encoding vector of the second relationship pair in step S2423 includes the following steps S31 to S35:

[0330] S31. Generate a second weight score based on a second scoring of the second relationship pair encoding vector.

[0331] The calculation of the second weight score is as follows:

[0332] s(i, j) = sm (i)+s m (i)+s a (i,j) (12)

[0333] Where i, j represent the coreference resolution encoding vector;

[0334] s m (i) represents the score of the coreference resolution encoding vector i in the encoding vector of the second relation pair that can be regarded as an entity;

[0335] s m (j) represents the score of the coreference resolution encoding vector j in the encoding vector of the second relation pair that can be regarded as an entity;

[0336] s a (i,j) represents the fraction of coreference resolution code vector j that is likely to be the antecedent of coreference resolution code vector i;

[0337] s(i,j) represents the second weight score.

[0338] S32. Perform attention processing on the coreference resolution coding vectors corresponding to the second relationship pair coding vector in sequence based on the second weight score to obtain a second intermediate vector.

[0339] The calculation of the second intermediate vector is shown in the following formulas (13) and (14):

[0340]

[0341]

[0342] Wherein, g represents the coreference elimination encoding vector corresponding to the encoding vector of the second relation pair;

[0343] P represents the weight coefficient of attention;

[0344] a represents the second intermediate vector;

[0345] s represents the first weight score;

[0346] i, j both represent a parameter in the [0,n] set.

[0347] S33. Perform gating processing on the coreference elimination coding vector corresponding to the coding vector based on the second intermediate vector and the second relationship to obtain a second gating vector.

[0348] The calculation of the second gating vector is shown in the following formula (15):

[0349]

[0350] Among them, σ represents the logical sigmoid function,

[0351] W f represents the weight parameter,

[0352] represents the i-th second intermediate vector;

[0353] represents the coreference resolution encoding vector corresponding to the i-th second relation pair encoding vector;

[0354] i represents a parameter in the [0,n] set;

[0355] Represents the i-th second gating vector.

[0356] S34. Re-encode the coreference resolution coding vector corresponding to the coding vector according to the second gating vector, the second intermediate vector and the second relationship to generate a re-encoded second updated coding vector.

[0357] Specifically, the calculation formula of the first updated coding vector is shown in the following formula (16):

[0358]

[0359] in, represents the i-th second gating vector;

[0360] represents the i-th second intermediate vector;

[0361] represents the coreference resolution encoding vector corresponding to the i-th second relation pair encoding vector;

[0362] represents the i-th second updated encoding vector.

[0363] According to this formula, the calculation of the second updated code vector is to use the second gating vector to re-encode the coreference elimination code vector corresponding to the second relationship pair code vector. This value is between (0, 1). It can be understood as how much (f) of the newly obtained second updated code vector comes from the original coreference elimination code vector. How much (1-f) comes from the second intermediate vector

[0364] S35. Generate a second relationship pair final encoding vector based on the second updated encoding vector and the classification label of the second updated encoding vector.

[0365] Specifically, the calculation of the final encoding vector by the second relationship is as follows:

[0366] fspan_pair_embeddings=torch.cat([f3_embeddings, f4_embeddings, f3_embeddings*f4_embeddings, segment_info, f3_label_embedding, f4_label_embedding], -1).

[0367] Among them, torch.cat is a function used to concatenate two or more vectors together, fspan_pair_embeddings represents the final encoding vector of the second relation pair, f3_embeddings represents the second updated encoding vector 1, f4_embeddings represents the second updated encoding vector 2, segment_info represents the word vector between the second updated encoding vector 1 and the second updated encoding vector 2, f3_label_embedding represents the label vector of the second updated encoding vector 1, and f4_label_embedding represents the label vector of the second updated encoding vector 2.

[0368] Through steps S31 to S35, this embodiment first scores the identified pruned coding vector for the first time through the feedforward neural network of the coreference resolution model to obtain a coreference resolution coding vector, then performs classification prediction and pruning in sequence based on the coreference resolution coding vector to obtain a second relationship pair coding vector, scores the second relationship pair coding vector for a second time through the feedforward neural network of the coreference resolution model, and finally performs attention processing and gating processing based on the scores of the two scores and the coreference resolution coding vector corresponding to the second relationship pair coding vector, and then re-encodes to obtain a second updated coding vector, and finally obtains the second relationship pair final coding vector based on the second updated coding vector and the classification label of the second updated coding vector, thereby achieving an information update of the coreference resolution coding vector, incorporating the information of other coreference resolution coding vectors, and significantly improving the coreference resolution task.

[0369] Finally, after obtaining the final coding vector of the second relationship pair, the final coding vector of the second relationship pair is scored using the coreference resolution model, and classification prediction processing is performed on the final coding vector of the second relationship pair based on the scoring result to obtain the coreference resolution result.

[0370] In addition, after obtaining the final coding vector of the first relationship pair and the final coding vector of the second relationship pair, this embodiment inputs the final coding vector of the first relationship pair and the final coding vector of the second relationship pair into the entity recognition model for entity recognition processing, so that the evaluation index (average F1) of the entity recognition task is significantly improved.

[0371] The word processing method based on the multi-task model provided in this embodiment first performs entity recognition processing and classification pruning processing on the candidate entity fragment coding vector through the entity recognition model to obtain a recognition pruned coding vector to reduce negative examples in the candidate entity fragment coding vector, and then inputs the recognition pruned coding vector into the relationship extraction model and the coreference resolution model for scoring twice to obtain the final coding vector of the first relationship pair and the final coding vector of the second relationship pair respectively, and then processes the final coding vector of the first relationship pair through the relationship extraction model to determine the relationship between the words in the candidate entity fragment, and processes the final coding vector of the second relationship pair through the coreference resolution model to determine the coreference relationship between the words in the candidate entity fragment, thereby adding constraints to the relationship extraction task and the coreference resolution task respectively through the two scorings, improving the evaluation indicators of the relationship extraction model and the coreference resolution model, and can more accurately obtain the relationship between the words in the candidate entity fragment and the coreference relationship respectively.

[0372] In addition, in this embodiment, based on the semantic relationship between any two relationship extraction coding vectors in the candidate entity fragment, the first initial relationship pair coding vector of the forward semantic relationship and the first initial relationship pair coding vector of the reverse semantic relationship of any two relationship extraction coding vectors are obtained respectively, thereby introducing the consideration of semantic relationship in the process of obtaining the relationship extraction result, so that the relationship extraction result can be more accurate.

[0373] See also Figure 3 This embodiment provides a word processing method based on a multi-task model, including:

[0374] S310: Obtain a candidate entity segment, and generate a candidate entity segment encoding vector based on the candidate entity segment.

[0375] Among them, step S310 is the same as step S210 in the aforementioned embodiment. For a specific explanation of step S310, please refer to the detailed description of the aforementioned embodiment, which will not be repeated here.

[0376] S311, inputting the candidate entity segment encoding vector into an entity recognition model, and scoring the candidate entity segment encoding vector through the feedforward neural network.

[0377] S312. Classify the candidate entity coding vector based on the score of the candidate entity fragment coding vector to obtain a classification label of the candidate entity coding vector, and prune the candidate entity coding vector to obtain a recognized pruned coding vector.

[0378] S313. Score the recognition pruned coding vector for the first time through the relationship extraction model, and prune the recognition pruned coding vector based on the score to obtain a relationship extraction coding vector.

[0379] Specifically, step S313 includes: inputting the recognition pruned coding vector into the relationship extraction model, scoring the recognition pruned coding vector for the first time through the feedforward neural network to obtain the score of the recognition pruned coding vector; and using the recognition pruned coding vector with a score greater than or equal to a preset threshold as the relationship extraction coding vector.

[0380] S314: Extract the encoding vector based on the relationship and perform a second scoring to generate a final encoding vector for the first relationship pair.

[0381] Specifically, step S314 includes the following steps S3141 to S3143:

[0382] S3141. Obtain a first initial relationship pair encoding vector based on the relationship extraction encoding vector and the classification label of the relationship extraction encoding vector.

[0383] S3142. Perform classification prediction processing on the first initial relationship pair coding vector, and prune the first initial relationship pair coding vector according to a preset ratio based on the classification prediction result to obtain a first relationship pair coding vector.

[0384] S3143. Score the first relationship pair encoding vector for a second time through the feedforward neural network, and obtain the final encoding vector of the first relationship pair based on the score of the second scoring and the first relationship pair encoding vector.

[0385] Specifically, for the detailed explanation of steps S3141 to S3143, please refer to the detailed contents of steps S2321 to S2323 of the aforementioned embodiment, which will not be repeated here.

[0386] S315. Perform relation extraction processing on the final encoding vector of the first relation pair through the relation extraction model to obtain a relation extraction result, and determine the relationship between the words in the candidate entity fragment based on the relation extraction result.

[0387] Specifically, step S315 includes: scoring the final encoding vector of the first relationship pair through the relationship extraction model, and performing classification prediction processing on the final encoding vector of the first relationship pair based on the scoring result to obtain a relationship extraction result.

[0388] S316. Perform a first score on the identification pruned code vector using a coreference resolution model, and prune the identification pruned code vector based on the score to obtain a coreference resolution code vector.

[0389] Step S316 includes: inputting the recognition pruned coding vector into the coreference resolution model, scoring the recognition pruned coding vector for the first time through the feedforward neural network to obtain the score of the recognition pruned coding vector; and taking the recognition pruned coding vector whose score is greater than or equal to a preset threshold as the coreference resolution coding vector.

[0390] S317: Perform a second scoring based on the coreference resolution coding vector to generate a final coding vector for a second relationship pair.

[0391] Specifically, step S317 includes steps S3171 to S3173:

[0392] S3171. Obtain a second initial relationship pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector.

[0393] S3172. Perform classification prediction processing on the second initial relationship pair coding vector, and prune the second initial relationship pair coding vector according to a preset ratio based on the result of the classification prediction to obtain a second relationship pair coding vector.

[0394] S3173. Score the second relationship pair encoding vector for a second time through the feedforward neural network, and obtain the final encoding vector of the second relationship pair based on the score of the second scoring and the second relationship pair encoding vector.

[0395] Specifically, for the detailed explanation of steps S3171 to S3173, please refer to the detailed contents of steps S2421 to S2423 of the aforementioned embodiment, which will not be repeated here.

[0396] S318. Perform coreference resolution processing on the final encoding vector of the second relationship pair through the coreference resolution model to obtain a coreference resolution result, and determine the coreference relationship between the words in the candidate entity segment based on the coreference resolution result.

[0397] Specifically, step S318 includes: scoring the final encoding vector of the second relationship pair through the coreference resolution model, and performing classification prediction processing on the final encoding vector of the second relationship pair based on the scoring result to obtain a coreference resolution result.

[0398] S319: Input the final encoding vector of the first relationship pair and the final encoding vector of the second relationship pair into the entity recognition model for entity recognition processing.

[0399] In this embodiment, entity recognition processing and classification pruning processing are performed on the candidate entity fragment coding vector through the entity recognition model to obtain a recognition pruned coding vector to reduce negative examples in the candidate entity fragment coding vector, and then the recognition pruned coding vector is respectively input into the relationship extraction model and the coreference resolution model for scoring twice to obtain the final coding vector of the first relationship pair and the final coding vector of the second relationship pair respectively, and then the final coding vector of the first relationship pair is processed by the relationship extraction model to determine the relationship between the words in the candidate entity fragment, and the final coding vector of the second relationship pair is processed by the coreference resolution model to determine the coreference relationship between the words in the candidate entity fragment, thereby adding constraints to the relationship extraction task and the coreference resolution task respectively through two scorings, improving the evaluation indicators of the relationship extraction model and the coreference resolution model, and can more accurately obtain the relationship between the words in the candidate entity fragment and the coreference relationship respectively.

[0400] In addition, the final encoding vector of the first relationship pair and the final encoding vector of the second relationship pair are input into the entity recognition model for entity recognition processing, so that the evaluation index of the entity recognition task is significantly improved.

[0401] like Figure 4 As shown, this embodiment provides a multi-task model training method, including steps S410 to S440.

[0402] S410: Obtain at least two sample candidate entity pairs and a classification label of each of the sample candidate entity pairs, and generate a sample candidate entity encoding vector based on the sample candidate entity of each sample candidate entity pair.

[0403] For the specific content, please refer to the above embodiments, which will not be described again here.

[0404] S420, performing entity recognition processing and classification pruning processing on the sample candidate entity fragment encoding vector through an entity recognition model to obtain a sample recognition pruning encoding vector.

[0405] Specifically, assuming that there are n categories of sample candidate entity fragments (n≥1, and n is an integer), then the sample candidate entity fragments that do not belong to the above n categories are all negative examples. The sample candidate entity fragment coding vectors are classified based on the scores of the sample candidate entity fragment coding vectors to obtain the classification label of each sample candidate entity fragment coding vector, and then the category to which each sample candidate entity fragment coding vector belongs can be obtained. After pruning some sample candidate entity fragment coding vectors in the negative examples according to a preset ratio, the remaining other sample candidate entity fragment coding vectors are the identification pruned coding vectors. Among them, the proportion of sample candidate entity fragment coding vectors pruned in the negative examples can be determined according to the specific situation, such as one-sixth, one-fifth, etc., and this application does not limit this.

[0406] This embodiment classifies the sample candidate entity encoding vectors and prunes some encoding vectors in the negative examples, so that the model can learn positive examples and negative examples in appropriate proportions at the same time, that is, learn from both correct and incorrect aspects at the same time, which helps to improve the model training effect.

[0407] S430, respectively inputting the sample identification pruning coding vector into a relation extraction model and a coreference resolution model for two scoring processes to obtain a final coding vector of a first sample relation pair and a final coding vector of a second sample relation pair.

[0408] Specifically, step S430 includes:

[0409] S431. Input the sample identification pruned coding vector into the relationship extraction model, score the sample identification pruned coding vector for the first time through the relationship extraction model, prune the sample identification pruned coding vector based on the score to obtain a sample relationship extraction coding vector, and score the sample relationship extraction coding vector for the second time to generate a final coding vector for the first sample relationship pair.

[0410] Specifically, step S431 includes the following steps S4311 to S4313:

[0411] S4311. Obtain a first initial sample relationship pair encoding vector (span_pair_embedding) based on the sample relationship extraction encoding vector and the classification label of the sample relationship extraction encoding vector.

[0412] Among them, for the generation process of the first initial sample relationship pair encoding vector, reference may be made to the generation process of the first initial relationship pair encoding vector in the aforementioned embodiment, which will not be repeated here.

[0413] S4312: Perform classification prediction processing on the first initial sample relationship pair coding vector, and prune the first initial sample relationship pair coding vector according to a preset ratio based on the classification prediction result to obtain a first sample relationship pair coding vector.

[0414] S4313. Score the first sample relationship pair encoding vector for a second time through the feedforward neural network, and obtain a final encoding vector of the first sample relationship pair based on the score of the second scoring and the first sample relationship pair encoding vector.

[0415] Specifically, obtaining the final encoding vector of the first sample relationship pair based on the scores of the two scorings and the encoding vector of the first sample relationship pair in step S4313 includes:

[0416] S41. Generate a first sample weight score based on the score of the second scoring of the encoding vector based on the first sample relationship.

[0417] S42. Based on the first sample weight score, the sample relationship extraction encoding vectors corresponding to the first sample relationship pair encoding vectors are sequentially subjected to attention processing to obtain a first sample intermediate vector.

[0418] S43: Perform gate processing on the sample relationship extraction coding vector corresponding to the coding vector based on the first sample intermediate vector and the first sample relationship to obtain a first sample gating vector.

[0419] S44. Re-encode the sample relationship extraction encoding vector corresponding to the encoding vector according to the first sample gating vector, the first sample intermediate vector and the first sample relationship to generate a re-encoded first sample update encoding vector.

[0420] S45. Generate a final encoding vector for a first sample relationship based on the first sample updated encoding vector and the classification label of the first sample updated encoding vector.

[0421] Among them, for the process of generating the final encoding vector of the first sample relationship, reference may be made to the process of generating the final encoding vector of the first relationship in the aforementioned embodiment, namely steps S21 to S25, which will not be repeated here.

[0422] S432. Input the sample identification pruned coding vector into the coreference resolution model, score the sample identification pruned coding vector for the first time through the coreference resolution model, prune the sample identification pruned coding vector based on the score, obtain a sample coreference resolution coding vector, and score the sample coreference resolution coding vector for the second time to generate a final coding vector for a second sample relationship.

[0423] Specifically, in step S432, performing a second scoring based on the sample coreference elimination coding vector to generate a second sample relationship pair final coding vector includes:

[0424] S4321. Obtain a second initial sample relationship pair encoding vector (span_pair_embedding) based on the sample coreference elimination encoding vector and the classification label of the sample coreference elimination encoding vector.

[0425] S4322. Perform classification prediction processing on the second initial sample relationship pair coding vector, and prune the second initial sample relationship pair coding vector according to a preset ratio based on the classification prediction result to obtain a second sample relationship pair coding vector.

[0426] S4323. Score the second sample relationship pair encoding vector for a second time through the feedforward neural network, and obtain a final encoding vector of the second sample relationship pair based on the scores of the two scores and the second sample relationship pair encoding vector.

[0427] Specifically, in step S4323, obtaining the final encoding vector of the second sample relationship pair based on the scores of the two scorings and the encoding vector of the second sample relationship pair includes the following steps S51 to S54:

[0428] S51. Generate a second sample weight score based on the second sample relationship for the second scoring of the encoding vector.

[0429] S52. Perform attention processing on the sample coreference elimination coding vectors corresponding to the second sample relationship pair coding vector in sequence based on the second sample weight score to obtain a second sample intermediate vector.

[0430] S53: Perform gating processing on the sample coreference elimination coding vector corresponding to the coding vector based on the second sample intermediate vector and the second sample relationship to obtain a second sample gating vector.

[0431] S54. Re-encode the sample coreference elimination coding vector corresponding to the coding vector according to the second sample gating vector, the second sample intermediate vector and the second sample relationship to generate a re-encoded second sample updated coding vector.

[0432] S55: Generate a final encoding vector for a second sample relationship based on the second sample updated encoding vector and the classification label of the second sample updated encoding vector.

[0433] Among them, for the generation process of the final encoding vector of the second sample relationship, reference may be made to the generation process of the final encoding vector of the second relationship in the aforementioned embodiment, namely steps S31 to S35, which will not be repeated here.

[0434] S440. Based on the final encoding vector of the first sample relationship pair and the final encoding vector of the second sample relationship pair, respectively determine the loss values ​​of the entity recognition model, the relationship extraction model and the coreference resolution model, and train the entity recognition model, the relationship extraction model and the coreference resolution model.

[0435] Specifically, step S440 includes the following steps S441 to S443:

[0436] S441. Calculate the loss value of the entity recognition model using a cross entropy loss function based on the score and classification label of the final encoding vector of the first sample relationship and the score and classification label of the final encoding vector of the second sample relationship.

[0437] S442. Calculate the loss value of the relationship extraction model based on the score of the first sample relationship to the final encoded vector and the classification label of the first sample relationship to the final encoded vector using a cross entropy loss function.

[0438] Specifically, the first sample relationship pair final encoding vector includes two sample relationship extraction encoding vectors, and step S442 includes the following steps S4421-S4422:

[0439] S4421. When the classification label is a symmetric relationship label, the two sample relationship extraction coding vectors of the first sample relationship pair final coding vector are reversely arranged to generate a reverse first sample relationship pair final coding vector.

[0440] In a specific usage scenario, when the classification label is a symmetric relationship label, such as conjunction, it is usually (span1, span2, conjunction) when annotating, so data enhancement can be done to add the reverse first sample relationship pair encoding vector (span2, span1) to obtain (span2, span1, conjunction) and then calculate the loss value.

[0441] S4422. Calculate the loss value of the relationship extraction model using the cross entropy loss function for the score of the final encoding vector of the first sample relationship pair and the corresponding classification label, and the final encoding vector of the reverse first sample relationship pair and the corresponding classification label.

[0442] Specifically, based on the loss values ​​of the entity recognition model, the relationship extraction model, and the coreference resolution, the weight values ​​of each layer of neuron nodes in the model are reversely adjusted from the output layer to the input layer to train the model.

[0443] In this embodiment, the semantic relationship of the coding vectors in the candidate entity fragment is extracted according to any two sample relationships, and the final coding vector of the first sample relationship pair of the positive semantic relationship and the final coding vector of the reverse first sample relationship pair of the reverse semantic relationship of any two sample relationship extracted coding vectors are obtained respectively, thereby introducing the consideration of semantic relationship in the training process, so that the relationship extraction result can be more accurate.

[0444] S443. Calculate the loss value of the coreference resolution model based on the score of the second sample relationship to the final encoded vector and the classification label of the second sample relationship to the final encoded vector using a cross entropy loss function.

[0445] For example, during the entity recognition model training process, a set of loss values ​​obtained by cross entropy calculation may be [-0.0000, -6.8651, -9.8858, -9.3611, -9.4160, -8.8986, -10.0036], where the 7 numbers correspond to the numbers 0 to 6, and each number represents a classification label.

[0446] Through F.softmax, it becomes the classification probability [9.9856e-01, 1.0421e-03, 5.0818e-05, 8.5878e-05, 8.1292e-05, 1.3638e-04, 4.5174e-05], and the maximum value is the final loss value.

[0447] The formula for the cross entropy loss function is as follows:

[0448]

[0449] Cross entropy is the difference between two probability distributions p and q, where p represents the encoding vector of the classification label corresponding to the true distribution, i.e., the sample identification pruned encoding vector, the first sample relationship pair encoding vector, and the second sample relationship pair encoding vector, q represents the non-true distribution, i.e., the sample identification pruned encoding vector, the first sample relationship pair encoding vector, and the second sample relationship pair encoding vector, and H(p, q) represents the loss value.

[0450] The training method of the multi-task model provided in the present application first organically combines the relationship extraction task, the coreference resolution task, and the entity recognition task during the training process, and then trains the relationship extraction model, the coreference resolution model, and the entity recognition model separately, which can effectively improve the effect of model training and improve the performance of the relationship extraction model, the coreference resolution model, and the entity recognition model.

[0451] like Figure 5 As shown, this embodiment discloses a word processing device based on a multi-task model, including:

[0452] The entity segment acquisition module 510 is configured to acquire candidate entity segments and generate candidate entity segment encoding vectors based on the candidate entity segments;

[0453] An entity recognition pruning module 520 is configured to perform entity recognition processing and classification pruning processing on the candidate entity segment encoding vector through an entity recognition model to obtain a recognition pruned encoding vector;

[0454] The relation extraction processing module 530 is configured to input the identified pruned coding vector into the relation extraction model for scoring twice, generate a final coding vector for a first relation pair, process the final coding vector for the first relation pair through the relation extraction model, and determine the relationship between the words in the candidate entity segment;

[0455] The coreference resolution processing module 540 is configured to input the identified pruned coding vector into the coreference resolution model for scoring twice, generate a final coding vector for a second relationship pair, process the final coding vector of the second relationship pair through the coreference resolution model, and determine the coreference relationship between the words in the candidate entity fragment.

[0456] Optionally, the relationship extraction processing module 530 is further configured to:

[0457] Scoring the identification pruned coding vector for the first time through a relation extraction model, and pruning the identification pruned coding vector based on the score to obtain a relation extraction coding vector;

[0458] A second scoring is performed based on the relationship extraction coding vector to generate a final coding vector for the first relationship pair, relationship extraction processing is performed on the final coding vector of the first relationship pair through the relationship extraction model to obtain a relationship extraction result, and the relationship between the words in the candidate entity fragment is determined based on the relationship extraction result.

[0459] Optionally, the coreference resolution processing module 540 is further configured to:

[0460] Scoring the recognition pruned coding vector for the first time through a coreference resolution model, and pruning the recognition pruned coding vector based on the score to obtain a coreference resolution coding vector;

[0461] A second scoring is performed based on the coreference resolution coding vector to generate a final coding vector for a second relationship pair, coreference resolution processing is performed on the final coding vector of the second relationship pair through the coreference resolution model to obtain a coreference resolution result, and the coreference relationship between the words in the candidate entity fragment is determined based on the coreference resolution result.

[0462] Optionally, the entity recognition model, the relationship extraction model, and the coreference resolution model share a feedforward neural network for scoring;

[0463] The entity recognition pruning module 520 is further configured to:

[0464] Inputting the candidate entity segment encoding vector into an entity recognition model, and scoring the candidate entity segment encoding vector through the feedforward neural network;

[0465] The candidate entity encoding vector is classified based on the score of the candidate entity fragment encoding vector to obtain a classification label of the candidate entity encoding vector, and the candidate entity encoding vector is pruned to obtain a recognition pruned encoding vector.

[0466] Optionally, the relationship extraction model and the entity recognition model share a feedforward neural network for scoring;

[0467] The relationship extraction processing module 530 is further configured to:

[0468] Inputting the recognition pruned coding vector into the relation extraction model, and scoring the recognition pruned coding vector for the first time through the feedforward neural network to obtain a score of the recognition pruned coding vector;

[0469] The recognition pruned coding vectors with scores greater than or equal to a preset threshold are used as relation extraction coding vectors.

[0470] Optionally, the coreference resolution model and the entity recognition model share a feedforward neural network for scoring;

[0471] The coreference resolution processing module 540 is further configured to:

[0472] Inputting the recognition pruned code vector into the coreference resolution model, and scoring the recognition pruned code vector for the first time through the feedforward neural network to obtain the score of the recognition pruned code vector;

[0473] The recognition pruning coding vector with a score greater than or equal to a preset threshold is used as the coreference resolution coding vector.

[0474] Optionally, the relationship extraction processing module 530 is further configured to:

[0475] Obtaining a first initial relation pair encoding vector based on the relation extraction encoding vector and the classification label of the relation extraction encoding vector;

[0476] Performing classification prediction processing on the first initial relationship pair encoding vector, and pruning the first initial relationship pair encoding vector according to a preset ratio based on a result of the classification prediction to obtain a first relationship pair encoding vector;

[0477] The first relationship pair encoding vector is scored for a second time through the feedforward neural network, and a final encoding vector of the first relationship pair is obtained based on the score of the second scoring and the first relationship pair encoding vector.

[0478] Optionally, the relationship extraction processing module 530 is further configured to:

[0479] Generate a first weight score based on the score of the second scoring of the encoding vector by the first relationship;

[0480] Based on the first weight score, sequentially performing attention processing on the relation extraction encoding vectors corresponding to the first relation pair encoding vector, obtaining a first intermediate vector;

[0481] Performing gating processing on the relation extraction encoding vector corresponding to the encoding vector based on the first intermediate vector and the first relation to obtain a first gating vector;

[0482] Re-encode the relation extraction encoding vector corresponding to the encoding vector according to the first gating vector, the first intermediate vector and the first relation to generate a re-encoded first updated encoding vector;

[0483] A first relationship pair final encoding vector is generated based on the first updated encoding vector and the classification label of the first updated encoding vector.

[0484] Optionally, the relationship extraction processing module 530 is further configured to:

[0485] Scoring the first initial relationship pair encoding vector by a feedforward neural network to obtain a score of the first initial relationship pair encoding vector;

[0486] Performing classification prediction processing on the first initial relationship pair encoding vector to obtain a category of the first initial relationship pair encoding vector;

[0487] The first initial relationship pair encoding vector is pruned based on the category and score of the first initial relationship pair encoding vector to obtain a first relationship pair encoding vector.

[0488] Optionally, the relationship extraction processing module 530 is further configured to:

[0489] Encoding the classification label of the relation extraction encoding vector to generate a first label vector;

[0490] Based on any two of the relation extraction encoding vectors and their corresponding first label vectors, a first initial relation pair encoding vector of any two of the relation extraction encoding vectors is obtained.

[0491] Optionally, the relationship extraction processing module 530 is further configured to:

[0492] Encoding the classification label of the relation extraction encoding vector to generate a first label vector;

[0493] Determine a semantic vector between any two relation extraction encoding vectors based on positions of the any two relation extraction encoding vectors in the candidate entity fragment;

[0494] Based on any two of the relationship extraction encoding vectors, the semantic vector between the any two relationship extraction encoding vectors and the first label vector corresponding to each of the relationship extraction encoding vectors, a first initial relationship pair encoding vector of any two of the relationship extraction encoding vectors is obtained.

[0495] Optionally, the relationship extraction processing module 530 is further configured to:

[0496] Determine multiple word vectors between the arbitrary two relation extraction encoding vectors;

[0497] Pooling or attention processing is performed on multiple word vectors between any two relationship extraction encoding vectors to obtain corresponding semantic vectors.

[0498] Optionally, the relationship extraction processing module 530 is further configured to:

[0499] According to the semantic relationship between any two of the relationship extraction coding vectors in the candidate entity fragment, the first initial relationship pair coding vector of the forward semantic relationship and the first initial relationship pair coding vector of the reverse semantic relationship of any two of the relationship extraction coding vectors are obtained respectively.

[0500] Optionally, the relationship extraction processing module 530 is further configured to score the first initial relationship pair encoding vector of the forward semantic relationship and the first initial relationship pair encoding vector of the reverse semantic relationship of any two of the relationship extraction encoding vectors through a feedforward neural network, and make the score of the first initial relationship pair encoding vector of the reverse semantic relationship less than a preset threshold.

[0501] Optionally, the coreference resolution processing module 540 is further configured to:

[0502] Obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector;

[0503] Performing classification prediction processing on the second initial relationship pair encoding vector, and pruning the second initial relationship pair encoding vector according to a preset ratio based on a result of the classification prediction to obtain a second relationship pair encoding vector;

[0504] The second relationship pair encoding vector is scored for a second time through the feedforward neural network, and a final encoding vector of the second relationship pair is obtained based on the score of the second scoring and the second relationship pair encoding vector.

[0505] Optionally, the coreference resolution processing module 540 is further configured to:

[0506] generating a second weight score based on a second scoring of the encoding vector of the second relationship pair;

[0507] Attention processing is performed on the coreference resolution encoding vectors corresponding to the second relationship pair encoding vector in sequence based on the second weight score to obtain a second intermediate vector;

[0508] Performing gating processing on a coreference elimination coding vector corresponding to the coding vector based on the second intermediate vector and the second relationship to obtain a second gating vector;

[0509] Re-encode the coreference elimination code vector corresponding to the code vector according to the second gating vector, the second intermediate vector and the second relationship to generate a re-encoded second updated code vector;

[0510] A second relationship pair final encoding vector is generated based on the second updated encoding vector and the classification label of the second updated encoding vector.

[0511] Optionally, the coreference resolution processing module 540 is further configured to:

[0512] Encoding the classification labels of the coreference elimination coding vector to generate a second label vector;

[0513] Based on any two of the coreference resolution coding vectors and their corresponding second label vectors, a second initial relation pair coding vector of any two of the coreference resolution coding vectors is obtained.

[0514] Optionally, the coreference resolution processing module 540 is further configured to:

[0515] Encoding the classification labels of the coreference elimination coding vector to generate a second label vector;

[0516] Determine a semantic vector between any two coreference resolution coding vectors based on positions of the any two coreference resolution coding vectors in the candidate entity segment;

[0517] Based on any two of the coreference resolution coding vectors, the semantic vector between the any two coreference resolution coding vectors, and the second label vector corresponding to each of the coreference resolution coding vectors, a second initial relationship pair coding vector of any two of the coreference resolution coding vectors is obtained.

[0518] Optionally, the coreference resolution processing module 540 is further configured to:

[0519] Determine a plurality of word vectors between the arbitrary two coreference resolution encoding vectors;

[0520] Pooling or attention processing is performed on multiple word vectors between any two coreference resolution encoding vectors to obtain corresponding semantic vectors.

[0521] Optionally, the coreference resolution processing module 540 is further configured to:

[0522] Scoring the second initial relationship pair encoding vector by a feedforward neural network to obtain a score of the second initial relationship pair encoding vector;

[0523] Performing classification prediction processing on the second initial relationship pair encoding vector to obtain a category of the second initial relationship pair encoding vector;

[0524] The second initial relationship pair encoding vector is pruned based on the category and score of the second initial relationship pair encoding vector to obtain a second relationship pair encoding vector.

[0525] Optionally, the relationship extraction processing module 530 is further configured to: score the final encoding vector of the first relationship pair through the relationship extraction model, and perform classification prediction processing on the final encoding vector of the first relationship pair based on the scoring result to obtain a relationship extraction result.

[0526] Optionally, the coreference resolution processing module 540 is further configured to: score the final encoding vector of the second relationship pair through the coreference resolution model, and perform classification prediction processing on the final encoding vector of the second relationship pair based on the scoring result to obtain a coreference resolution result.

[0527] Optionally, the device further includes: an entity recognition module configured to: input the final encoding vector of the first relationship pair and the final encoding vector of the second relationship pair into the entity recognition model for entity recognition processing.

[0528] The word processing device based on the multi-task model provided in the present application first performs entity recognition processing and classification pruning processing on the candidate entity fragment coding vector through the entity recognition model to obtain the recognition pruned coding vector, so as to reduce the negative examples in the candidate entity fragment coding vector, and then inputs the recognition pruned coding vector into the relationship extraction model and the coreference resolution model for scoring twice, respectively, to obtain the final coding vector of the first relationship pair and the final coding vector of the second relationship pair, and then processes the final coding vector of the first relationship pair through the relationship extraction model to determine the relationship between the words in the candidate entity fragment, and processes the final coding vector of the second relationship pair through the coreference resolution model to determine the coreference relationship between the words in the candidate entity fragment, thereby adding constraints to the relationship extraction task and the coreference resolution task respectively through the two scorings, improving the evaluation indicators of the relationship extraction model and the coreference resolution model, and can more accurately obtain the relationship between the words in the candidate entity fragment and the coreference relationship respectively.

[0529] The word processing device based on the multi-task model provided in this embodiment realizes the organic combination of the relationship extraction model, the coreference resolution model, and the entity recognition model, realizes the organic combination of the relationship extraction task, the coreference resolution task, and the entity recognition task, and realizes the sharing of information in the above three tasks, which can effectively improve the accuracy and recall rate of relationship extraction, coreference resolution, and entity recognition, and effectively improve the accuracy of word processing based on the multi-task model for determining word relationships.

[0530] The above is a schematic scheme of a word processing device based on a multi-task model of this embodiment. It should be noted that the technical scheme of the device and the technical scheme of the word processing method based on the multi-task model belong to the same concept, and the details not described in detail in the technical scheme of the device can be found in the description of the technical scheme of the word processing method based on the multi-task model.

[0531] like Figure 6 As shown, this embodiment discloses a training device for a multi-task model, including:

[0532] The sample segment acquisition module 610 is configured to acquire a sample candidate entity segment and generate a sample candidate entity segment encoding vector based on the sample candidate entity segment;

[0533] The sample recognition and pruning module 620 is configured to perform entity recognition processing and classification pruning processing on the sample candidate entity segment encoding vector through an entity recognition model to obtain a sample recognition and pruning encoding vector;

[0534] The sample processing module 630 is configured to input the sample identification pruning coding vector into the relationship extraction model and the coreference resolution model for two scoring processes, respectively, to obtain the final coding vector of the first sample relationship pair and the final coding vector of the second sample relationship pair;

[0535] The model training module 640 is configured to determine the loss values ​​of the entity recognition model, the relationship extraction model and the coreference resolution model based on the final encoding vector of the first sample relationship pair and the final encoding vector of the second sample relationship pair, respectively, and train the entity recognition model, the relationship extraction model and the coreference resolution model.

[0536] Optionally, the sample processing module 630 is further configured to:

[0537] Inputting the sample identification pruned coding vector into the relationship extraction model, scoring the sample identification pruned coding vector for the first time through the relationship extraction model, and pruning the sample identification pruned coding vector based on the score to obtain a sample relationship extraction coding vector, and performing a second scoring based on the sample relationship extraction coding vector to generate a final coding vector for the first sample relationship pair;

[0538] The sample identification pruned coding vector is input into the coreference resolution model, the sample identification pruned coding vector is scored for the first time by the coreference resolution model, and the sample identification pruned coding vector is pruned based on the score to obtain a sample coreference resolution coding vector, and a second scoring is performed based on the sample coreference resolution coding vector to generate a final coding vector for a second sample relationship pair.

[0539] Optionally, the sample processing module 630 is further configured to:

[0540] Obtaining a first initial sample relationship pair encoding vector based on the sample relationship extraction encoding vector and the classification label of the sample relationship extraction encoding vector;

[0541] Performing classification prediction processing on the first initial sample relationship pair encoding vector, and pruning the first initial sample relationship pair encoding vector according to a preset ratio based on the classification prediction result to obtain a first sample relationship pair encoding vector;

[0542] The first sample relationship pair encoding vector is scored a second time through the feedforward neural network, and a final encoding vector of the first sample relationship pair is obtained based on the scores of the two scores and the first sample relationship pair encoding vector.

[0543] Optionally, the sample processing module 630 is further configured to:

[0544] Generate a first sample weight score based on the score of the second scoring of the encoding vector by the first sample relationship;

[0545] Based on the first sample weight score, the sample relationship extraction encoding vectors corresponding to the first sample relationship pair encoding vectors are sequentially subjected to attention processing to obtain a first sample intermediate vector;

[0546] Based on the first sample intermediate vector and the first sample relationship, extract the encoding vector corresponding to the encoding vector and perform gating processing to obtain a first sample gating vector;

[0547] Re-encode the sample relationship extraction encoding vector corresponding to the encoding vector according to the first sample gating vector, the first sample intermediate vector and the first sample relationship to generate a re-encoded first sample update encoding vector;

[0548] Based on the first sample updated encoding vector and the classification label of the first sample updated encoding vector, a first sample relationship pair final encoding vector is generated.

[0549] Optionally, the sample processing module 630 is further configured to:

[0550] Obtaining a second initial sample relationship pair encoding vector based on the sample coreference elimination encoding vector and the classification label of the sample coreference elimination encoding vector;

[0551] Performing classification prediction processing on the second initial sample relationship pair encoding vector, and pruning the second initial sample relationship pair encoding vector according to a preset ratio based on the classification prediction result to obtain a second sample relationship pair encoding vector;

[0552] The second sample relationship pair encoding vector is scored a second time through the feedforward neural network, and a final encoding vector of the second sample relationship pair is obtained based on the scores of the two scores and the second sample relationship pair encoding vector.

[0553] Optionally, the sample processing module 630 is further configured to:

[0554] Generate a second sample weight score based on the score of the second scoring of the encoding vector by the second sample relationship;

[0555] Based on the second sample weight score, sequentially performing attention processing on the sample coreference elimination encoding vectors corresponding to the second sample relationship pair encoding vector, obtaining a second sample intermediate vector;

[0556] Performing gating processing on the sample coreference elimination coding vector corresponding to the coding vector based on the second sample intermediate vector and the second sample relationship to obtain a second sample gating vector;

[0557] Generate a re-encoded second sample update code vector according to the second sample gating vector, the second sample intermediate vector, and the sample coreference elimination code vector corresponding to the second sample relationship pair code vector;

[0558] Based on the second sample updated encoding vector and the classification label of the second sample updated encoding vector, a second sample relationship pair final encoding vector is generated.

[0559] Optionally, the model training module 640 is further configured to:

[0560] Calculate the loss value of the entity recognition model using a cross entropy loss function based on the score and classification label of the final encoding vector of the first sample relationship and the score and classification label of the final encoding vector of the second sample relationship;

[0561] Calculate the loss value of the relationship extraction model using a cross entropy loss function based on the score of the first sample relationship to the final encoded vector and the classification label of the first sample relationship to the final encoded vector;

[0562] Based on the score of the second sample relationship to the final encoded vector and the classification label of the second sample relationship to the final encoded vector, the loss value of the coreference resolution model is calculated using a cross entropy loss function.

[0563] Optionally, the first sample relationship pair final encoding vector includes two sample relationship extraction encoding vectors, and the model training module 640 is further configured as follows:

[0564] In the case where the classification label is a symmetric relationship label, two sample relationship extraction encoding vectors of the first sample relationship pair final encoding vector are reversely arranged to generate a reverse first sample relationship pair final encoding vector;

[0565] The score of the final encoding vector of the first sample relationship and the corresponding classification label, the final encoding vector of the reverse first sample relationship and the corresponding classification label are respectively used to calculate the loss value of the relationship extraction model using the cross entropy loss function.

[0566] The training device of the multi-task model provided in the present application first organically combines the relationship extraction task, the coreference resolution task, and the entity recognition task during the training process, and then trains the relationship extraction model, the coreference resolution model, and the entity recognition model separately, which can effectively improve the effect of model training and improve the performance of the relationship extraction model, the coreference resolution model, and the entity recognition model.

[0567] like Figure 7 As shown, Figure 7 7 is a block diagram showing a computing device 700 according to an embodiment of the present specification. The components of the computing device 700 include but are not limited to a memory 770 and a processor 720. The processor 720 is connected to the memory 770 via a bus 730, and the database 750 is used to store data.

[0568] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.77 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0569] In one embodiment of the present specification, the above components of the computing device 700 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0570] The computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a PC. The computing device 700 may also be a mobile or stationary server. The computing device may be the method described in any of the above embodiments.

[0571] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the multi-task model-based word processing method or the multi-task model training method as described above.

[0572] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the multi-task model-based word processing method or the multi-task model training method are of the same concept, and the details not described in detail in the technical scheme of the storage medium can be found in the description of the technical scheme of the multi-task model-based word processing method or the multi-task model training method.

[0573] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0574] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0575] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0576] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is limited only by the claims and their full scope and equivalents.

Claims

1. A word processing method based on a multi-task model, characterized in that: include: Obtaining a candidate entity segment, and generating a candidate entity segment encoding vector based on the candidate entity segment; Performing entity recognition processing and classification pruning processing on the candidate entity fragment encoding vector through an entity recognition model to obtain a recognition pruning encoding vector; Inputting the identified pruned coding vector into a relation extraction model for scoring twice to generate a final coding vector for a first relation pair, processing the final coding vector for the first relation pair through the relation extraction model to determine the relationship between words in the candidate entity segment; The identified pruned coding vector is input into a coreference resolution model for scoring twice to generate a final coding vector for a second relationship pair, and the final coding vector for the second relationship pair is processed by the coreference resolution model to determine the coreference relationship between words in the candidate entity fragment, wherein the relationship extraction model and the coreference resolution model share a feedforward neural network for scoring twice, the score of the first scoring is used to prune the identified pruned coding vector, and the final coding vector of the first relationship pair and the final coding vector of the second relationship pair are generated based on classification prediction of the pruning results and the first relationship pair coding vector and the second relationship pair coding vector obtained by pruning.

2. The word processing method based on the multi-task model according to claim 1 is characterized in that: Inputting the identified pruned coding vector into a relation extraction model for scoring twice to generate a final coding vector for a first relation pair, processing the final coding vector for the first relation pair through the relation extraction model, and determining the relationship between words in the candidate entity segment, including: Scoring the identification pruned coding vector for the first time through a relation extraction model, and pruning the identification pruned coding vector based on the score to obtain a relation extraction coding vector; A second scoring is performed based on the relationship extraction coding vector to generate a final coding vector for the first relationship pair, relationship extraction processing is performed on the final coding vector of the first relationship pair through the relationship extraction model to obtain a relationship extraction result, and the relationship between the words in the candidate entity fragment is determined based on the relationship extraction result.

3. The word processing method based on the multi-task model according to claim 1, characterized in that: Inputting the identified pruned coding vector into a coreference resolution model for scoring twice to generate a final coding vector for a second relationship pair, processing the final coding vector for the second relationship pair through the coreference resolution model, and determining the coreference relationship between words in the candidate entity segment, including: Scoring the recognition pruned coding vector for the first time through a coreference resolution model, and pruning the recognition pruned coding vector based on the score to obtain a coreference resolution coding vector; A second scoring is performed based on the coreference resolution coding vector to generate a final coding vector for a second relationship pair, coreference resolution processing is performed on the final coding vector of the second relationship pair through the coreference resolution model to obtain a coreference resolution result, and the coreference relationship between the words in the candidate entity fragment is determined based on the coreference resolution result.

4. The word processing method based on the multi-task model according to claim 1 is characterized in that: The entity recognition model, the relationship extraction model, and the coreference resolution model share a feedforward neural network for scoring; The step of performing entity recognition processing and classification pruning processing on the candidate entity segment encoding vector through the entity recognition model to obtain the recognition pruning encoding vector includes: Inputting the candidate entity segment encoding vector into an entity recognition model, and scoring the candidate entity segment encoding vector through the feedforward neural network; The candidate entity encoding vector is classified based on the score of the candidate entity fragment encoding vector to obtain a classification label of the candidate entity encoding vector, and the candidate entity encoding vector is pruned to obtain a recognition pruned encoding vector.

5. The word processing method based on the multi-task model according to claim 2 is characterized in that: The relationship extraction model and the entity recognition model share a feedforward neural network for scoring; The identification pruned coding vector is scored for the first time by using a relation extraction model, and the identification pruned coding vector is pruned based on the score, including: Inputting the recognition pruned coding vector into the relation extraction model, and scoring the recognition pruned coding vector for the first time through the feedforward neural network to obtain a score of the recognition pruned coding vector; The recognition pruned coding vectors with scores greater than or equal to a preset threshold are used as relation extraction coding vectors.

6. The word processing method based on the multi-task model according to claim 3 is characterized in that: The coreference resolution model and the entity recognition model share a feedforward neural network for scoring; Scoring the identification pruned coding vector for the first time through a coreference resolution model, and pruning the identification pruned coding vector based on the score to obtain a coreference resolution coding vector, including: Inputting the recognition pruned code vector into the coreference resolution model, and scoring the recognition pruned code vector for the first time through the feedforward neural network to obtain the score of the recognition pruned code vector; The recognition pruning coding vector with a score greater than or equal to a preset threshold is used as the coreference resolution coding vector.

7. The word processing method based on the multi-task model according to claim 5, characterized in that: The extracting the encoding vector based on the relationship and performing a second scoring to generate a final encoding vector for the first relationship pair includes: Obtaining a first initial relation pair encoding vector based on the relation extraction encoding vector and the classification label of the relation extraction encoding vector; Performing classification prediction processing on the first initial relationship pair encoding vector, and pruning the first initial relationship pair encoding vector according to a preset ratio based on a result of the classification prediction to obtain a first relationship pair encoding vector; The first relationship pair encoding vector is scored for a second time through the feedforward neural network, and a final encoding vector of the first relationship pair is obtained based on the score of the second scoring and the first relationship pair encoding vector.

8. The word processing method based on the multi-task model according to claim 7 is characterized in that: Obtaining a final encoding vector of the first relationship pair based on the second scoring score and the encoding vector of the first relationship pair includes: Generate a first weight score based on the score of the second scoring of the encoding vector by the first relationship; Based on the first weight score, sequentially performing attention processing on the relation extraction encoding vectors corresponding to the first relation pair encoding vector, obtaining a first intermediate vector; Performing gating processing on the relation extraction encoding vector corresponding to the encoding vector based on the first intermediate vector and the first relation to obtain a first gating vector; Re-encode the relation extraction encoding vector corresponding to the encoding vector according to the first gating vector, the first intermediate vector and the first relation to generate a re-encoded first updated encoding vector; A first relationship pair final encoding vector is generated based on the first updated encoding vector and the classification label of the first updated encoding vector.

9. The word processing method based on the multi-task model according to claim 7, characterized in that: The performing classification prediction processing on the first initial relationship pair encoding vector, and pruning the first initial relationship pair encoding vector according to a preset ratio based on the classification prediction result to obtain the first relationship pair encoding vector, includes: Scoring the first initial relationship pair encoding vector by a feedforward neural network to obtain a score of the first initial relationship pair encoding vector; Performing classification prediction processing on the first initial relationship pair encoding vector to obtain a category of the first initial relationship pair encoding vector; The first initial relationship pair encoding vector is pruned based on the category and score of the first initial relationship pair encoding vector to obtain a first relationship pair encoding vector.

10. The word processing method based on the multi-task model according to claim 9, characterized in that: The step of obtaining a first initial relationship pair encoding vector based on the relationship extraction encoding vector and the classification label of the relationship extraction encoding vector includes: Encoding the classification label of the relation extraction encoding vector to generate a first label vector; Based on any two of the relation extraction encoding vectors and their corresponding first label vectors, a first initial relation pair encoding vector of any two of the relation extraction encoding vectors is obtained.

11. The word processing method based on the multi-task model according to claim 9, characterized in that: The step of obtaining a first initial relationship pair encoding vector based on the relationship extraction encoding vector and the classification label of the relationship extraction encoding vector includes: Encoding the classification label of the relation extraction encoding vector to generate a first label vector; Determine a semantic vector between any two relation extraction encoding vectors based on positions of the any two relation extraction encoding vectors in the candidate entity fragment; Based on any two of the relationship extraction encoding vectors, the semantic vector between the any two relationship extraction encoding vectors and the first label vector corresponding to each of the relationship extraction encoding vectors, a first initial relationship pair encoding vector of any two of the relationship extraction encoding vectors is obtained.

12. The word processing method based on the multi-task model according to claim 11, characterized in that: Determining the semantic vector between the arbitrary two relation extraction encoding vectors includes: Determine multiple word vectors between the arbitrary two relation extraction encoding vectors; Pooling or attention processing is performed on multiple word vectors between any two relationship extraction encoding vectors to obtain corresponding semantic vectors.

13. The word processing method based on the multi-task model according to claim 10 or 11, characterized in that: Obtaining a first initial relation pair encoding vector of any two of the relation extraction encoding vectors, comprising: According to the semantic relationship between any two of the relationship extraction coding vectors in the candidate entity fragment, the first initial relationship pair coding vector of the forward semantic relationship and the first initial relationship pair coding vector of the reverse semantic relationship of any two of the relationship extraction coding vectors are obtained respectively.

14. The word processing method based on the multi-task model according to claim 13, characterized in that: Scoring the first initial relationship pair encoding vector by a feedforward neural network to obtain a score of the first initial relationship pair encoding vector includes: The first initial relationship pair encoding vector of the positive semantic relationship and the first initial relationship pair encoding vector of the reverse semantic relationship of any two of the relationship extraction encoding vectors are scored through a feedforward neural network, and the score of the first initial relationship pair encoding vector of the reverse semantic relationship is made less than a preset threshold.

15. The word processing method based on the multi-task model according to claim 6, characterized in that: The generating a second relationship pair final coding vector by performing a second scoring based on the coreference resolution coding vector comprises: Obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector; Performing classification prediction processing on the second initial relationship pair encoding vector, and pruning the second initial relationship pair encoding vector according to a preset ratio based on a result of the classification prediction to obtain a second relationship pair encoding vector; The second relationship pair encoding vector is scored for a second time through the feedforward neural network, and a final encoding vector of the second relationship pair is obtained based on the score of the second scoring and the second relationship pair encoding vector.

16. The word processing method based on the multi-task model according to claim 15, characterized in that: Obtaining a final encoding vector of the second relationship pair based on the second scoring score and the encoding vector of the second relationship pair includes: generating a second weight score based on a second scoring of the encoding vector of the second relationship pair; Attention processing is performed on the coreference resolution encoding vectors corresponding to the second relationship pair encoding vector in sequence based on the second weight score to obtain a second intermediate vector; Performing gating processing on a coreference elimination coding vector corresponding to the coding vector based on the second intermediate vector and the second relationship to obtain a second gating vector; Re-encode the coreference elimination code vector corresponding to the code vector according to the second gating vector, the second intermediate vector and the second relationship to generate a re-encoded second updated code vector; A second relationship pair final encoding vector is generated based on the second updated encoding vector and the classification label of the second updated encoding vector.

17. The word processing method based on the multi-task model according to claim 15, characterized in that: Obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector includes: Encoding the classification labels of the coreference elimination coding vector to generate a second label vector; Based on any two of the coreference resolution coding vectors and their corresponding second label vectors, a second initial relation pair coding vector of any two of the coreference resolution coding vectors is obtained.

18. The word processing method based on the multi-task model according to claim 15, characterized in that: Obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector includes: Encoding the classification labels of the coreference elimination coding vector to generate a second label vector; Determine a semantic vector between any two coreference resolution coding vectors based on positions of the any two coreference resolution coding vectors in the candidate entity segment; Based on any two of the coreference resolution coding vectors, the semantic vector between the any two coreference resolution coding vectors, and the second label vector corresponding to each of the coreference resolution coding vectors, a second initial relationship pair coding vector of any two of the coreference resolution coding vectors is obtained.

19. The word processing method based on the multi-task model according to claim 18, characterized in that: Determining the semantic vector between the arbitrary two coreference resolution encoding vectors includes: Determine a plurality of word vectors between the arbitrary two coreference resolution encoding vectors; Pooling or attention processing is performed on multiple word vectors between any two coreference resolution encoding vectors to obtain corresponding semantic vectors.

20. The word processing method based on the multi-task model according to claim 17 or 18, characterized in that: The method of performing classification prediction processing on the second initial relationship pair encoding vector and pruning the second initial relationship pair encoding vector according to a preset ratio based on a result of the classification prediction to obtain a second relationship pair encoding vector includes: Scoring the second initial relationship pair encoding vector by a feedforward neural network to obtain a score of the second initial relationship pair encoding vector; Performing classification prediction processing on the second initial relationship pair encoding vector to obtain a category of the second initial relationship pair encoding vector; The second initial relationship pair encoding vector is pruned based on the category and score of the second initial relationship pair encoding vector to obtain a second relationship pair encoding vector.

21. The word processing method based on the multi-task model according to claim 2, characterized in that: Performing relation extraction processing on the final encoding vector of the first relation pair through the relation extraction model to obtain a relation extraction result, including: The final encoding vector of the first relationship pair is scored using the relationship extraction model, and classification prediction processing is performed on the final encoding vector of the first relationship pair based on the scoring result to obtain a relationship extraction result.

22. The word processing method based on the multi-task model according to claim 3, characterized in that: Performing coreference resolution processing on the final encoding vector of the second relationship pair by using the coreference resolution model to obtain a coreference resolution result includes: The final encoding vector of the second relationship pair is scored using the coreference resolution model, and classification prediction processing is performed on the final encoding vector of the second relationship pair based on the scoring result to obtain a coreference resolution result.

23. The word processing method based on the multi-task model according to claim 1, 8 or 16, characterized in that: The method further comprises: The final encoding vector of the first relationship pair and the final encoding vector of the second relationship pair are input into the entity recognition model for entity recognition processing.

24. A multi-task model training method, characterized in that: include: Obtain at least two sample candidate entity pairs and a classification label of each of the sample candidate entity pairs, and generate a sample candidate entity encoding vector based on the sample candidate entity of each sample candidate entity pair; Performing entity recognition processing and classification pruning processing on the sample candidate entity encoding vector through an entity recognition model to obtain a sample recognition pruning encoding vector; The sample identification pruned coding vector is respectively input into a relation extraction model and a coreference resolution model for two scoring processes to obtain a final coding vector of a first sample relation pair and a final coding vector of a second sample relation pair, wherein the relation extraction model and the coreference resolution model share a feedforward neural network for two scoring processes, the score of the first scoring process is used to prune the sample identification pruned coding vector, and the final coding vector of the first sample relation pair and the final coding vector of the second sample relation pair are generated by a second scoring process based on the classification prediction of the pruning results and the first sample relation pair coding vector and the second sample relation pair coding vector obtained by pruning; Based on the final encoding vector of the first sample relationship pair and the final encoding vector of the second sample relationship pair, the loss values ​​of the entity recognition model, the relationship extraction model and the coreference resolution model are respectively determined, and the entity recognition model, the relationship extraction model and the coreference resolution model are trained.

25. The multi-task model training method according to claim 24, characterized in that: The sample identification pruning coding vector is respectively input into the relation extraction model and the coreference resolution model for two scoring processes to obtain the final coding vector of the first sample relation pair and the final coding vector of the second sample relation pair, including: Inputting the sample identification pruned coding vector into the relationship extraction model, scoring the sample identification pruned coding vector for the first time through the relationship extraction model, and pruning the sample identification pruned coding vector based on the score to obtain a sample relationship extraction coding vector, and performing a second scoring based on the sample relationship extraction coding vector to generate a final coding vector for the first sample relationship pair; The sample identification pruned coding vector is input into the coreference resolution model, the sample identification pruned coding vector is scored for the first time by the coreference resolution model, and the sample identification pruned coding vector is pruned based on the score to obtain a sample coreference resolution coding vector, and a second scoring is performed based on the sample coreference resolution coding vector to generate a final coding vector for a second sample relationship pair.

26. The multi-task model training method according to claim 25, characterized in that: Extracting the encoding vector based on the sample relationship and performing a second scoring to generate a final encoding vector for the first sample relationship pair includes: Obtaining a first initial sample relationship pair encoding vector based on the sample relationship extraction encoding vector and the classification label of the sample relationship extraction encoding vector; Performing classification prediction processing on the first initial sample relationship pair encoding vector, and pruning the first initial sample relationship pair encoding vector according to a preset ratio based on the classification prediction result to obtain a first sample relationship pair encoding vector; The first sample relationship pair encoding vector is scored for a second time through the feedforward neural network, and a final encoding vector of the first sample relationship pair is obtained based on the score of the second scoring and the first sample relationship pair encoding vector.

27. The multi-task model training method according to claim 26, characterized in that: Obtaining a final encoding vector of the first sample relationship pair based on the second scoring score and the encoding vector of the first sample relationship pair includes: Generate a first sample weight score based on the score of the second scoring of the encoding vector by the first sample relationship; Based on the first sample weight score, the sample relationship extraction encoding vectors corresponding to the first sample relationship pair encoding vectors are sequentially subjected to attention processing to obtain a first sample intermediate vector; Based on the first sample intermediate vector and the first sample relationship, extract the encoding vector corresponding to the encoding vector and perform gating processing to obtain a first sample gating vector; Re-encode the sample relationship extraction encoding vector corresponding to the encoding vector according to the first sample gating vector, the first sample intermediate vector and the first sample relationship to generate a re-encoded first sample update encoding vector; Based on the first sample updated encoding vector and the classification label of the first sample updated encoding vector, a first sample relationship pair final encoding vector is generated.

28. The multi-task model training method according to claim 25, characterized in that: Performing a second scoring based on the sample coreference resolution coding vector to generate a second sample relationship pair final coding vector includes: Obtaining a second initial sample relationship pair encoding vector based on the sample coreference elimination encoding vector and the classification label of the sample coreference elimination encoding vector; Performing classification prediction processing on the second initial sample relationship pair encoding vector, and pruning the second initial sample relationship pair encoding vector according to a preset ratio based on the classification prediction result to obtain a second sample relationship pair encoding vector; The second sample relationship pair encoding vector is scored for a second time through the feedforward neural network, and a final encoding vector of the second sample relationship pair is obtained based on the score of the second scoring and the second sample relationship pair encoding vector.

29. The multi-task model training method according to claim 28, characterized in that: Obtaining a final encoding vector of the second sample relationship pair based on the second scoring score and the encoding vector of the second sample relationship pair includes: Generate a second sample weight score based on the score of the second scoring of the encoding vector by the second sample relationship; Based on the second sample weight score, sequentially performing attention processing on the sample coreference elimination encoding vectors corresponding to the second sample relationship pair encoding vector, obtaining a second sample intermediate vector; Performing gating processing on the sample coreference elimination coding vector corresponding to the coding vector based on the second sample intermediate vector and the second sample relationship to obtain a second sample gating vector; Generate a re-encoded second sample update code vector according to the second sample gating vector, the second sample intermediate vector, and the sample coreference elimination code vector corresponding to the second sample relationship pair code vector; Based on the second sample updated encoding vector and the classification label of the second sample updated encoding vector, a second sample relationship pair final encoding vector is generated.

30. The multi-task model training method according to claim 25, characterized in that: Determining loss values ​​of the entity recognition model, the relationship extraction model, and the coreference resolution model based on the final encoding vector of the first sample relationship pair and the final encoding vector of the second sample relationship pair, respectively, includes: Calculate the loss value of the entity recognition model using a cross entropy loss function based on the score and classification label of the final encoding vector of the first sample relationship and the score and classification label of the final encoding vector of the second sample relationship; Calculate the loss value of the relationship extraction model using a cross entropy loss function based on the score of the first sample relationship to the final encoded vector and the classification label of the first sample relationship to the final encoded vector; The loss value of the coreference resolution model is calculated based on the score of the second sample relationship to the final encoded vector and the classification label of the second sample relationship to the final encoded vector using a cross entropy loss function.

31. The multi-task model training method according to claim 30, characterized in that: The final encoding vector of the first sample relationship pair includes two sample relationship extraction encoding vectors; The loss value of the relationship extraction model is calculated based on the score of the first sample relationship to the final encoded vector and the classification label of the first sample relationship to the final encoded vector using a cross entropy loss function, including: In the case where the classification label is a symmetric relationship label, two sample relationship extraction encoding vectors of the first sample relationship pair final encoding vector are reversely arranged to generate a reverse first sample relationship pair final encoding vector; The score of the final encoding vector of the first sample relationship and the corresponding classification label, the final encoding vector of the reverse first sample relationship and the corresponding classification label are respectively used to calculate the loss value of the relationship extraction model using the cross entropy loss function.

32. A word processing device based on a multi-task model, characterized in that: include: An entity segment acquisition module is configured to acquire candidate entity segments and generate candidate entity segment encoding vectors based on the candidate entity segments; An entity recognition pruning module is configured to perform entity recognition processing and classification pruning processing on the candidate entity segment encoding vector through an entity recognition model to obtain a recognition pruning encoding vector; a relation extraction processing module, configured to input the identified pruned coding vector into a relation extraction model for secondary scoring, generate a final coding vector for a first relation pair, process the final coding vector for the first relation pair through the relation extraction model, and determine the relationship between words in the candidate entity segment; The coreference resolution processing module is configured to input the recognition pruned coding vector into the coreference resolution model for secondary scoring, generate a final coding vector for a second relationship pair, process the final coding vector of the second relationship pair through the coreference resolution model, and determine the coreference relationship between the words in the candidate entity fragment, wherein the relationship extraction model and the coreference resolution model share a feedforward neural network for twice scoring, the score of the first scoring is used to prune the recognition pruned coding vector, and the final coding vector of the first relationship pair and the final coding vector of the second relationship pair are generated based on the classification prediction of the pruning results and the first relationship pair coding vector and the second relationship pair coding vector obtained by pruning.

33. A multi-task model training device, characterized in that: include: A sample acquisition module is configured to acquire at least two sample candidate entity pairs and a classification label of each of the sample candidate entity pairs, and generate a sample candidate entity encoding vector based on the sample candidate entity of each sample candidate entity pair; A sample recognition module is configured to perform entity recognition processing and classification pruning processing on the sample candidate entity fragment encoding vector through an entity recognition model to obtain a sample recognition pruning encoding vector; A sample processing module is configured to input the sample identification pruned coding vector into a relation extraction model and a coreference resolution model for two scoring processes respectively, to obtain a final coding vector of a first sample relation pair and a final coding vector of a second sample relation pair, wherein the relation extraction model and the coreference resolution model share a feedforward neural network for two scoring processes, the score of the first scoring process is used to prune the sample identification pruned coding vector, and the final coding vector of the first sample relation pair and the final coding vector of the second sample relation pair are generated by performing a second scoring process based on the classification prediction of the pruning results and the first sample relation pair coding vector and the second sample relation pair coding vector obtained by pruning; The model training module is configured to determine the loss values ​​of the entity recognition model, the relationship extraction model and the coreference resolution model based on the final encoding vector of the first sample relationship pair and the final encoding vector of the second sample relationship pair, and train the entity recognition model, the relationship extraction model and the coreference resolution model.

34. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the instructions, the steps of the method described in any one of claims 1-23 or 24-31 are implemented.

35. A computer-readable storage medium storing computer instructions, characterized in that: When the instruction is executed by the processor, the steps of the method described in any one of claims 1-23 or 24-31 are implemented.

Images