A word processing method and device based on dual-task model

Through the combination of the dual-task model, entity recognition and co-referential digestion model sharing information has been solved, the problem of insufficient task information sharing in the existing technology has been significantly improved, and the effect of entity recognition and co-referential digestion tasks has been improved, and the accuracy of word processing has been improved. .

CN113761920BActive Publication Date: 2024-11-22BEIJING KINGSOFT DIGITAL ENTERTAINMENT CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010507257.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-06-05
Publication Date
2024-11-22
Estimated Expiration
2040-06-05

AI Technical Summary

Technical Problem

In the prior art, entity recognition tasks and co-referential digestion tasks cannot effectively share information, resulting in unsatisfactory results in entity recognition and co-referential digestion tasks.

Method used

Using the word processing method based on the dual-task model, the candidate entity fragment encoding vector is subject to entity recognition processing and classification pruning through the entity recognition model, and the recognition pruning encoding vector is generated, and the total reference digestion model is input to determine the candidate entity fragment. The common reference relationship between words in China.

Benefits of technology

The organic combination of entity recognition and co-referential digestion tasks is realized, the sharing of information in the task is improved, the accuracy and recall rate of entity recognition and co-referential digestion tasks is significantly improved, and the accuracy of word processing is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113761920B_ABST
    Figure CN113761920B_ABST
Patent Text Reader

Abstract

The present application provides a word processing method and device based on a dual-task model, wherein the method comprises: obtaining a candidate entity segment, and generating a candidate entity segment encoding vector based on the candidate entity segment; performing entity recognition processing and classification pruning processing on the candidate entity segment encoding vector through an entity recognition model to obtain a recognition pruning encoding vector; inputting the recognition pruning encoding vector into a coreference resolution model for processing to determine the coreference relationship between words in the candidate entity segment. The method and device provided by the present application can improve the accuracy and recall rate of coreference resolution and entity recognition, and improve the accuracy of word processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a word processing method and device based on a dual-task model, a dual-task model training method and device, a computing device, and a computer-readable storage medium. Background Art

[0002] Entity recognition refers to identifying and extracting entities with specific meaning or strong referentiality in unstructured text, such as names of people, places, organizational structure names, dates and times, proper nouns, etc.

[0003] A relationship is a connection between two or more entities. Relationship extraction is to detect and identify certain semantic relationships between entities from text. For example, in the sentence "Beijing is the capital, political center and cultural center of China", the relationship expressed can be (China, capital, Beijing), (China, political center, Beijing) or (China, cultural center, Beijing).

[0004] Coreference resolution is a special type of relationship extraction. One of the entities in coreference resolution is usually a different way of saying another entity in the current context. The relationship between the two entities can be expressed as (entity 1, coreference, entity 2).

[0005] At present, the entity recognition task and coreference resolution task of a sentence are carried out separately. Information cannot be shared between the tasks, and they cannot constrain each other, which leads to unsatisfactory results of both the entity recognition task and the coreference resolution task. Summary of the invention

[0006] In view of this, the embodiments of the present application provide a word processing method and device based on a dual-task model, a training method and device for a dual-task model, a computing device and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0007] The present application embodiment provides a word processing method based on a dual-task model, including:

[0008] Obtaining a candidate entity segment, and generating a candidate entity segment encoding vector based on the candidate entity segment;

[0009] Performing entity recognition processing and classification pruning processing on the candidate entity fragment encoding vector through an entity recognition model to obtain a recognition pruning encoding vector;

[0010] The identified pruned coding vector is input into a coreference resolution model for processing to determine the coreference relationship between the words in the candidate entity segment.

[0011] Optionally, inputting the identified pruned encoding vector into a coreference resolution model for processing to determine the coreference relationship between words in the candidate entity segment includes:

[0012] Scoring the identification pruned coding vector by a coreference resolution model, and pruning the identification pruned coding vector based on the score to obtain a coreference resolution coding vector;

[0013] A second relationship pair encoding vector is generated based on the coreference resolution encoding vector, coreference resolution processing is performed on the second relationship pair encoding vector through the coreference resolution model to obtain a coreference resolution result, and the coreference relationship between the words in the candidate entity fragment is determined based on the coreference resolution result.

[0014] Optionally, the entity recognition model and the coreference resolution model share a feedforward neural network for scoring;

[0015] The step of performing entity recognition processing and classification pruning processing on the candidate entity segment encoding vector through the entity recognition model to obtain the recognition pruning encoding vector includes:

[0016] Inputting the candidate entity segment encoding vector into an entity recognition model, and scoring the candidate entity segment encoding vector through the feedforward neural network;

[0017] The candidate entity fragment coding vectors are classified based on the scores of the candidate entity fragment coding vectors to obtain classification labels of the candidate entity fragment coding vectors, and the candidate entity fragment coding vectors are pruned to obtain recognized pruned coding vectors.

[0018] Optionally, the coreference resolution model and the entity recognition model share a feedforward neural network for scoring;

[0019] Scoring the recognition pruned coding vector by a coreference resolution model, and pruning the recognition pruned coding vector based on the score, including:

[0020] Inputting the recognition pruned coding vector into the coreference resolution model, scoring the recognition pruned coding vector through the feedforward neural network, and obtaining a score of the recognition pruned coding vector;

[0021] The recognition pruning coding vector with a score greater than or equal to a preset threshold is used as the coreference resolution coding vector.

[0022] Optionally, generating a second relation pair encoding vector based on the coreference resolution encoding vector includes:

[0023] Obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector;

[0024] The second initial relationship pair coding vector is subjected to classification prediction processing, and the second initial relationship pair coding vector is pruned according to a preset ratio based on a result of the classification prediction to obtain a second relationship pair coding vector.

[0025] Optionally, obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and a classification label of the coreference resolution encoding vector includes:

[0026] Encoding the classification labels of the coreference elimination coding vector to generate a second label vector;

[0027] Based on any two of the coreference resolution coding vectors and their corresponding second label vectors, a second initial relation pair coding vector of any two of the coreference resolution coding vectors is obtained.

[0028] Optionally, obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and a classification label of the coreference resolution encoding vector includes:

[0029] Encoding the classification labels of the coreference elimination coding vector to generate a second label vector;

[0030] Determine a semantic vector between any two coreference resolution coding vectors based on positions of the any two coreference resolution coding vectors in the candidate entity segment;

[0031] Based on any two of the coreference resolution coding vectors, the semantic vector between the any two coreference resolution coding vectors, and the second label vector corresponding to each of the coreference resolution coding vectors, a second initial relationship pair coding vector of any two of the coreference resolution coding vectors is obtained.

[0032] Optionally, determining the semantic vector between the arbitrary two coreference resolution coding vectors includes:

[0033] Determine a plurality of word vectors between the arbitrary two coreference resolution encoding vectors;

[0034] Pooling or attention processing is performed on multiple word vectors between any two coreference resolution encoding vectors to obtain corresponding semantic vectors.

[0035] Optionally, performing classification prediction processing on the second initial relationship pair encoding vector, and pruning the second initial relationship pair encoding vector according to a preset ratio based on a result of the classification prediction to obtain a second relationship pair encoding vector, includes:

[0036] Scoring the second initial relationship pair encoding vector by a feedforward neural network to obtain a score of the second initial relationship pair encoding vector;

[0037] Performing classification prediction processing on the second initial relationship pair encoding vector to obtain a category of the second initial relationship pair encoding vector;

[0038] The second initial relationship pair encoding vector is pruned based on the category and score of the second initial relationship pair encoding vector to obtain a second relationship pair encoding vector.

[0039] Optionally, performing coreference resolution processing on the second relationship pair encoding vector by using the coreference resolution model to obtain a coreference resolution result includes:

[0040] The second relationship pair encoding vector is scored using the coreference resolution model, and classification prediction processing is performed on the second relationship pair encoding vector based on the scoring result to obtain a coreference resolution result.

[0041] The present application embodiment provides a dual-task model training method, including:

[0042] Obtain at least two sample candidate entity pairs and a classification label of each of the sample candidate entity pairs, and generate a sample candidate entity fragment encoding vector based on the sample candidate entities of each sample candidate entity pair;

[0043] Performing entity recognition processing and classification pruning processing on the sample candidate entity fragment encoding vector through an entity recognition model to obtain a sample recognition pruning encoding vector;

[0044] Inputting the sample identification pruning coding vector into a coreference resolution model for processing to obtain a second sample relationship pair coding vector;

[0045] Based on the sample recognition pruning coding vector and the second sample relationship pair coding vector, the loss values ​​of the entity recognition model and the coreference resolution model are respectively determined, and the entity recognition model and the coreference resolution model are trained.

[0046] Optionally, the sample identification pruning coding vectors are respectively input into a coreference resolution model for processing to obtain a second sample relationship pair coding vector, including:

[0047] The sample identification pruned coding vector is input into the coreference resolution model, the sample identification pruned coding vector is scored by the coreference resolution model, and the sample identification pruned coding vector is pruned based on the score to obtain a sample coreference resolution coding vector, and a second sample relationship pair coding vector is generated based on the sample coreference resolution coding vector.

[0048] Optionally, determining the loss values ​​of the entity recognition model and the coreference resolution model based on the sample recognition pruning encoding vector and the second sample relationship pair encoding vector, respectively, includes:

[0049] The loss value of the entity recognition model is calculated using a cross entropy loss function based on the score of the sample recognition pruned coding vector and the classification label of the sample recognition pruned coding vector;

[0050] Based on the score of the second sample relationship pair encoding vector and the classification label of the second sample relationship pair encoding vector, a cross entropy loss function is used to calculate the loss value of the coreference resolution model.

[0051] The embodiment of the present application provides a word processing device based on a dual-task model, comprising:

[0052] An entity segment acquisition module is configured to acquire candidate entity segments and generate candidate entity segment encoding vectors based on the candidate entity segments;

[0053] An entity recognition pruning module is configured to perform entity recognition processing and classification pruning processing on the candidate entity segment encoding vector through an entity recognition model to obtain a recognition pruning encoding vector;

[0054] The coreference resolution processing module is configured to input the identified pruned coding vector into a coreference resolution model for processing, and determine the coreference relationship between the words in the candidate entity segment.

[0055] The present application embodiment provides a dual-task model training device, including:

[0056] A sample acquisition module is configured to acquire at least two sample candidate entity pairs and a classification label of each of the sample candidate entity pairs, and generate a sample candidate entity fragment encoding vector based on the sample candidate entity of each sample candidate entity pair;

[0057] A sample recognition module is configured to perform entity recognition processing and classification pruning processing on the sample candidate entity fragment encoding vector through an entity recognition model to obtain a sample recognition pruning encoding vector;

[0058] A sample processing module is configured to input the sample identification pruning coding vector into a coreference resolution model for processing to obtain a second sample relationship pair coding vector;

[0059] The model training module is configured to determine the loss values ​​of the entity recognition model and the coreference resolution model respectively based on the sample recognition pruning coding vector and the second sample relationship coding vector, and train the entity recognition model and the coreference resolution model.

[0060] An embodiment of the present application provides a computing device, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein when the processor executes the instructions, the steps of the word processing method based on the dual-task model or the training method of the dual-task model as described above are implemented.

[0061] An embodiment of the present application provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the word processing method based on the dual-task model or the training method of the dual-task model as described above.

[0062] The word processing method and device based on the dual-task model provided in the present application first perform entity recognition processing and classification pruning processing on the candidate entity fragment coding vector through the entity recognition model to obtain the recognition pruned coding vector, so as to reduce the negative examples in the candidate entity fragment coding vector, and then process the recognition pruned coding vector through the coreference resolution model, which can enhance the coreference resolution model's understanding of the candidate entity fragment, provide a basis for the execution of the coreference resolution task, and effectively improve the accuracy of word processing.

[0063] The word processing method and device based on the dual-task model provided in the present application realize the organic combination of the coreference resolution model and the entity recognition model, realize the organic combination of the coreference resolution task and the entity recognition task, and realize the sharing of information in the above tasks. It can effectively improve the accuracy and recall rate of the coreference resolution task and the entity recognition task, and effectively improve the accuracy of word processing based on the dual-task model for determining word relationships.

[0064] The training method and device of the dual-task model provided in the present application first organically combine the coreference resolution task and the entity recognition task during the training process, and then train the coreference resolution model and the entity recognition model separately, which can effectively improve the effect of model training and improve the performance of the coreference resolution model and the entity recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a schematic diagram of the structure of a dual-task model according to an embodiment of the present application;

[0066] Figure 2 It is a schematic diagram of the steps of a word processing method based on a dual-task model according to an embodiment of the present application;

[0067] Figure 3 is a schematic flow chart of steps of a word processing method based on a dual-task model according to another embodiment of the present application;

[0068] Figure 4 It is a schematic diagram of the steps of the dual-task model training method described in one embodiment of the present application;

[0069] Figure 5 is a structural diagram of a word processing device based on a dual-task model according to an embodiment of the present application;

[0070] Figure 6 It is a structural schematic diagram of a training device for a dual-task model according to an embodiment of the present application;

[0071] Figure 7 It is a schematic diagram of the structure of a computing device described in an embodiment of the present application. DETAILED DESCRIPTION

[0072] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0073] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0074] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0075] First, the terms involved in one or more embodiments of the present invention are explained.

[0076] Entity recognition: refers to identifying and extracting entities with specific meaning or strong referentiality in unstructured text, such as names of people, places, organizational structure names, dates and times, proper nouns, etc.

[0077] Entity Recognition Model: A model used to perform entity recognition tasks.

[0078] Relation extraction: Detect and identify certain semantic relationships between entities from the text, such as the sentence "Beijing is the capital, political center and cultural center of China", where the relationship expressed can be (China, capital, Beijing), (China, political center, Beijing) or (China, cultural center, Beijing).

[0079] Co-reference resolution: a special relationship extraction. One of the entities in co-reference resolution is usually a different way of saying another entity in the current context. The relationship between the two entities can be expressed as (entity 1, co-reference, entity 2).

[0080] Coreference Resolution Model: A model for performing coreference resolution tasks.

[0081] Candidate entity fragment (span): a fragment consisting of one or more words in a sentence.

[0082] Candidate entity span encoding vector (span embedding): The vector generated by the candidate entity span after the encoder's encoding process.

[0083] Identify pruned encoding vectors: the encoding vectors remaining after the candidate entity segment encoding vectors are pruned.

[0084] Pruning: Filter according to preset rules.

[0085] Co-reference resolution code vector: the code vector remaining after pruning the recognition pruned code vector based on the result of the coreference resolution process.

[0086] Second label vector: an encoding vector obtained by encoding the classification label of the coreference resolution encoding vector.

[0087] Second relationship pair encoding vector: an encoding vector formed by combining two coreference resolution encoding vectors and their second label vectors and a distance feature vector.

[0088] Feedforward Neural Network (FFNN): A simplest neural network, where neurons are arranged in layers, each neuron is only connected to the neurons in the previous layer, receives the output of the previous layer, and outputs it to the next layer. There is no feedback between the layers. It is one of the most widely used and fastest growing artificial neural networks. In this application, the entity recognition model and the coreference resolution model share a feedforward neural network for scoring.

[0089] Convolutional Neural Networks (CNN): is a type of feedforward neural network that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning. In this application, the character-level features corresponding to each word are obtained through CNN network encoding.

[0090] Classification label: A label used to identify the type of encoded vector.

[0091] Accuracy: refers to the ratio of the number of correctly identified entities to the number of identified entities. The value is between 0 and 1. The larger the value, the higher the accuracy.

[0092] Recall rate: refers to the ratio of the number of correctly identified entities to the number of entities in the sample. The value is between 0 and 1. The larger the value, the higher the recall rate.

[0093] Weighted harmonic mean: also known as F1 value, F1 value = (2*accuracy*recall) / (accuracy+recall).

[0094] In the present application, a word processing method and apparatus based on a dual-task model, a training method and apparatus for a dual-task model, a computing device, and a computer-readable storage medium are provided, which are described in detail one by one in the following embodiments.

[0095] like Figure 1 As shown, this embodiment provides a dual-task model, which is used in the dual-task model-based word processing method described in this application, including an encoder, an entity recognition model and a coreference resolution model, wherein the entity recognition model and the coreference resolution model share a feedforward neural network.

[0096] The entity recognition model is a model for identifying entities in unstructured input text. In this embodiment, the entity recognition model performs entity recognition processing and classification pruning processing on the candidate entity fragment encoding vector to obtain a recognition pruned encoding vector.

[0097] The coreference resolution model is a model for detecting, identifying, and extracting entities with coreference relationships. In this embodiment, the coreference resolution model scores the recognition pruned coding vector and prunes the recognition pruned coding vector based on the score to obtain a coreference resolution coding vector; a second relationship pair coding vector is generated based on the coreference resolution coding vector, and the second relationship pair coding vector is processed by the coreference resolution model to obtain a coreference resolution result, and the relationship between words in the candidate entity fragment is determined based on the coreference resolution result.

[0098] The dual-task model provided in this embodiment organically combines the entity recognition model and the coreference resolution model. The two models share a feedforward neural network for scoring, thereby realizing information sharing between the entity recognition model and the coreference resolution model, thereby improving the accuracy and recall rate of the above-mentioned entity recognition model and the coreference resolution model.

[0099] like Figure 2 As shown, Figure 2 is a flowchart showing the steps of a word processing method based on a dual-task model according to an embodiment of the present application, including steps S210 to S230.

[0100] S210: Obtain a candidate entity segment, and generate a candidate entity segment encoding vector based on the candidate entity segment.

[0101] The candidate entity segment is a word set composed of one or more words in the target sentence, target paragraph, or target article, and each word represents an entity. Specifically, the candidate entity segment can be obtained by performing word segmentation on the target sentence, target paragraph, or target article, and then extracting one or more target words from the result of the word segmentation to form a word set.

[0102] For example, suppose that after word segmentation processing is performed on the target sentence, 10 words including word segmentation processing results A1-A10 are obtained, and a word set consisting of A1-A6 is extracted from the above word segmentation processing results, and the above word set is used as a candidate entity segment.

[0103] In practical applications, the candidate entity fragments may be input into an encoder for encoding processing to generate a candidate entity fragment encoding vector.

[0104] In this embodiment, the encoder includes a bidirectional LSTM, a pre-trained BERT model, a CNN network, and any combination thereof.

[0105] Preferably, a sentence containing several candidate entity fragments is first encoded by a pre-trained BERT model to obtain a word-level feature vector of the sentence, and then encoded by a CNN network to obtain a character-level feature vector of the sentence. The word-level feature vector and the character-level feature vector are concatenated to obtain a concatenated vector, and then the concatenated vector is encoded by a bidirectional LSTM network to obtain a feature vector with context features. Finally, the encoding vector of each candidate entity fragment is calculated based on the extracted candidate entity fragments using an attention mechanism. The encoding vector of the candidate entity fragment can be expressed by the following formula:

[0106]

[0107] Among them, g i The encoding vector representing the candidate entity fragment, x START(i) * 、x END(i) * The vector representing the start and end positions of the candidate entity fragment, Φ (i) represents additional features, It represents the result of calculating the words in each candidate entity segment based on the attention mechanism. The specific calculation process is as follows:

[0108]

[0109]

[0110]

[0111]

[0112] Specifically, t represents the candidate entity segment, i represents the word in the candidate entity segment, and formula (2) represents the encoding vector x corresponding to each word in the candidate entity segment t * The vector (h) output by the forward propagation of the bidirectional LSTM t,1 ) and the vector of back-propagation output (h t,-1 ), formula (3) indicates that the parameter α of the candidate entity segment t is obtained by multiplying its parameter w with the score of the candidate entity segment given by the feedforward neural network, and formula (4) indicates the weight a of each word in the candidate entity segment i,t Based on the parameter α of the candidate entity segment in which it is located and the total parameter of the word in the candidate entity segment, formula (5) represents the encoding vector corresponding to each word in the candidate entity segment: The weight parameter a of the word in the candidate entity segment i,t and the candidate entity fragment encoding vector x t get.

[0113] This embodiment obtains candidate entity segments and performs encoding processing on the candidate entity segments to prepare for the execution of other subsequent tasks, thereby improving the efficiency of the execution of subsequent tasks.

[0114] S220. Perform entity recognition processing and classification pruning processing on the candidate entity fragment encoding vector through an entity recognition model to obtain a recognition pruning encoding vector.

[0115] It should be noted that, in this embodiment, the entity recognition model and

[0116] The coreference resolution models share a feed-forward neural network for scoring.

[0117] Specifically, the step S220 may also include steps S221 and S222.

[0118] S221. Input the candidate entity segment encoding vector into an entity recognition model, and score the candidate entity segment encoding vector through the feedforward neural network.

[0119] The score of each candidate entity segment coding vector is composed of a basic score (Mention score) and a classification score (classifier score), and the above basic score and classification score are obtained by scoring through a feedforward neural network. The score of the candidate entity segment coding vector can be the sum of the basic score and the classification score, the average value, the weighted average value, etc., and this application does not limit this.

[0120] The feedforward neural network uses the principle of deep learning to score the candidate entity fragment coding vector. Specifically, the candidate entity fragment coding vector is calculated or encoded again by using the feedforward neural network, and the corresponding score is mapped to obtain the score of the candidate entity fragment coding vector. It should be noted that the mapping of the score by the feedforward neural network can be continuously adjusted through the execution of subsequent tasks, the calculation of loss values, the return of gradients, etc. The score of the candidate entity fragment coding vector can be a ten-point score, a hundred-point score, a thousand-point score, etc., and this application does not limit this.

[0121] This embodiment scores the candidate entity segment encoding vectors and then performs entity recognition processing, which can improve the accuracy of entity recognition processing and improve the effect of the entity recognition model.

[0122] S222. Classify the candidate entity fragment coding vector based on the score of the candidate entity fragment coding vector to obtain a classification label of the candidate entity fragment coding vector, and prune the candidate entity fragment coding vector to obtain a recognized pruned coding vector.

[0123] In practical applications, the candidate entity fragment coding vectors are classified based on their scores to obtain a classification label for each candidate entity fragment coding vector, and then the category to which each candidate entity fragment coding vector belongs can be obtained. A certain proportion of the candidate entity fragment coding vectors in one or several categories can be pruned according to the classification results, and the remaining candidate entity fragment coding vectors can be used as identification pruned coding vectors. Alternatively, candidate entity fragment coding vectors with scores less than a preset threshold can be pruned, and the remaining candidate entity fragment coding vectors can be used as identification pruned coding vectors. The present application does not impose any restrictions on this.

[0124] For example, suppose that the candidate entity fragment encoding vectors exist in the first category, the second category, ... the nth category, etc., where the nth category is a negative example, that is, all candidate entity fragment encoding vectors that do not belong to the previous category are classified as the nth category. After classification, m of the m candidate entity fragment encoding vectors 1 belongs to the first category, m 2 belongs to the second category...m n belongs to the nth class (m1 +m 2 +……m n =m), then the candidate entity segment encoding vectors can be pruned in the following three ways: (1) The first type prunes p 1 % of the candidate entity fragment encoding vectors, the second category is pruned away 2 % of the candidate entity fragment encoding vectors... the nth category is pruned away n % of the candidate entity fragment encoding vectors, and the remaining candidate entity fragment encoding vectors are used as recognition pruning encoding vectors, where p 1 、p 2 …… n The values ​​of can be the same or different; (2) The encoding vectors of candidate entity fragments of the first, second, ..., n-1th categories are not pruned, and the encoding vectors of candidate entity fragments of the nth category are pruned. n % of the candidate entity fragment coding vectors, and the remaining candidate entity fragment coding vectors are used as recognition pruned coding vectors; (3) the candidate entity fragment coding vectors whose scores of one or more categories are less than a preset threshold are pruned, and the remaining candidate entity fragment coding vectors are used as recognition pruned coding vectors, wherein the preset threshold of the score can be determined according to the specific situation, and the present application does not impose any restrictions on this.

[0125] In this embodiment, classification and pruning are performed on the candidate entity segment encoding vectors, which helps to improve the quality of the subsequent coreference resolution model input encoding vectors and lays a foundation for the execution of the coreference resolution task.

[0126] S230: Input the identified pruned coding vector into a coreference resolution model for processing to determine the coreference relationship between the words in the candidate entity segment.

[0127] Specifically, the step S230 may also include steps S231 to S232.

[0128] S231. Score the recognition pruned code vector using a coreference resolution model, and prune the recognition pruned code vector based on the score to obtain a coreference resolution code vector.

[0129] Specifically, the recognition pruned coding vector can be input into the coreference resolution model, and the recognition pruned coding vector can be scored by the feedforward neural network to obtain the score of the recognition pruned coding vector, and then the recognition pruned coding vector with a score greater than or equal to a preset threshold is used as the coreference resolution coding vector.

[0130] Among them, the score of each recognition pruned code vector is composed of a basic score and a classification score, and the above basic score and classification score are both obtained by scoring through a feedforward neural network.

[0131] This embodiment scores the recognition pruned coding vectors and further processes the recognition pruned coding vectors based on the entity recognition task, which helps to pave the way for the subsequent coreference resolution task.

[0132] S232. Generate a second relationship pair encoding vector based on the coreference resolution encoding vector, perform coreference resolution processing on the second relationship pair encoding vector through the coreference resolution model to obtain a coreference resolution result, and determine the coreference relationship between the words in the candidate entity fragment based on the coreference resolution result.

[0133] Specifically, in the step S232, generating a second relationship pair encoding vector based on the coreference resolution encoding vector includes S2321-S2322:

[0134] S2321. Obtain a second initial relationship pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector.

[0135] In a specific implementation scheme, step S2421 includes: encoding the classification label of the coreference elimination coding vector to generate a second label vector; based on any two of the coreference elimination coding vectors and their corresponding second label vectors, obtaining a second initial relationship pair coding vector of any two of the coreference elimination coding vectors.

[0136] In practical applications, each second relationship pair encoding vector is composed of two coreference resolution encoding vectors and classification label encoding vectors corresponding to two coreference resolution encoding vectors. In other words, the coreference resolution encoding vectors of the two words in the candidate entity segment and the second label vectors of the two words can be concatenated to obtain the second relationship pair encoding vector, as shown below:

[0137] span_pair_embeddings=torch.cat([span1_embeddings, span2_embeddings, span1_embeddings*span2_embeddings, span1_label_embedding, span2_label_embedding], -1).

[0138] Among them, torch.cat is a function used to concatenate two or more vectors together, span_pair_embeddings represents the second relationship pair encoding vector, span1_embeddings represents the coreference resolution encoding vector 1, span2_embeddings represents the coreference resolution encoding vector 2, span1_label_embedding represents the label vector of the coreference resolution encoding vector 1, and span2_label_embedding represents the label vector of the coreference resolution encoding vector 2.

[0139] It should be noted that the classification label is a type of feature information of the candidate entity fragment. In addition, other types of feature information, such as distance, etc., can be combined when generating the second relationship pair encoding vector, depending on the specific situation. This application does not impose any restrictions on this.

[0140] For example, the coreference resolution encoding vectors of the two words in the candidate entity segment, the second label vectors of the two words, and the distance feature vector between the two words can be concatenated to obtain the second relationship pair encoding vector, as shown below:

[0141] span_pair_embeddings=torch.cat([span1_embeddings, span2_embeddings, span1_embeddings*span2_embeddings, antecedent_distance_embeddings, span1_label_embedding, span2_label_embedding], -1).

[0142] Among them, torch.cat is a function used to concatenate two or more vectors together, span_pair_embeddings represents the second relationship pair encoding vector, span1_embeddings represents the coreference resolution encoding vector 1, span2_embeddings represents the coreference resolution encoding vector 2, antecedent_distance_embeddings represents the distance feature vector between the coreference resolution encoding vector 1 and the coreference resolution encoding vector 2, span1_label_embedding represents the label vector of the coreference resolution encoding vector 1, and span2_label_embedding represents the label vector of the coreference resolution encoding vector 2.

[0143] This embodiment obtains a second relationship pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector, which helps to improve the execution efficiency and effect of the coreference resolution task.

[0144] In another specific implementation scheme, step S2421 includes: encoding the classification labels of the coreference resolution coding vectors to generate a second label vector; determining the semantic vector between any two coreference resolution coding vectors based on the positions of the any two coreference resolution coding vectors in the candidate entity fragment; obtaining the second initial relationship pair coding vector of any two coreference resolution coding vectors based on any two coreference resolution coding vectors, the semantic vector between the any two coreference resolution coding vectors and the second label vector corresponding to each coreference resolution coding vector.

[0145] Specifically, determining the semantic vector between any two coreference resolution coding vectors includes: determining multiple word vectors between any two coreference resolution coding vectors; performing pooling or attention processing on the multiple word vectors between any two coreference resolution coding vectors to obtain a corresponding semantic vector.

[0146] In a specific application, for the candidate entity fragment [z1, z2, …, zx-1, zx], [z1, z2] is the coreference resolution coding vector span3, [zx-1, zx] is the coreference resolution coding vector span4, then [z3, …, zx-2] is the word vector between any two coreference resolution coding vectors span3 and span4, and the word vector [z3, …, zx-2] between the two coreference resolution coding vectors span3 and span4 is pooled or attended to obtain the corresponding semantic vector, thereby increasing the semantic information of the second initial relationship pair coding vector and enhancing the expressiveness of the second initial relationship pair coding vector.

[0147] Specifically, the calculation formula of the second initial relationship encoding vector is as follows:

[0148] span_pair_embeddings=torch.cat([span3_embeddings, span4_embeddings, span3_embeddings*span4_embeddings, segment_info, span3_label_embedding, span4_label_embedding], -1).

[0149] Among them, torch.cat is a function used to concatenate two or more vectors together, span_pair_embeddings represents the second initial relationship pair encoding vector, span3_embeddings represents the coreference resolution encoding vector 1, span4_embeddings represents the coreference resolution encoding vector 2, segment_info represents the word vector between the coreference resolution encoding vector 1 and the coreference resolution encoding vector 2, span3_label_embedding represents the label vector of the coreference resolution encoding vector 1, and span4_label_embedding represents the label vector of the coreference resolution encoding vector 2.

[0150] It should be noted that in the process of calculating span_pair_embeddings, not all words can have a relationship. For example, in a paragraph of text, words that are far apart basically have no relationship. Therefore, a distance threshold is set in this embodiment. If the distance between span3 and span4 exceeds the threshold, the span_pair_embeddings of span3 and span4 are directly pruned. The value of the distance threshold can be set according to actual needs, for example, the threshold is set to 60 word units.

[0151] S2322. Perform classification prediction processing on the second initial relationship pair coding vector, and prune the second initial relationship pair coding vector according to a preset ratio based on the classification prediction result to obtain a second relationship pair coding vector.

[0152] Specifically, step S2322 includes:

[0153] Scoring the second initial relationship pair encoding vector by a feedforward neural network to obtain a score of the second initial relationship pair encoding vector;

[0154] Performing classification prediction processing on the second initial relationship pair encoding vector to obtain a category of the second initial relationship pair encoding vector;

[0155] The second initial relationship pair encoding vector is pruned based on the category and score of the second initial relationship pair encoding vector to obtain a second relationship pair encoding vector.

[0156] Specifically, the process of coreference resolution includes scoring and classification prediction processing. In other words, the second relationship pair encoding vector is scored through the coreference resolution model, and the second relationship pair encoding vector is classified and predicted based on the scoring result, that is, the coreference resolution processing is completed, the coreference resolution result is obtained, and the coreference relationship between words is determined.

[0157] This embodiment determines the coreference relationship between words by performing coreference resolution processing on the second relationship pair encoding vector, which can improve the accuracy of the coreference resolution task and further effectively improve the accuracy of determining the coreference relationship between words.

[0158] The word processing method based on the dual-task model provided in this embodiment first performs entity recognition processing and classification pruning processing on the candidate entity fragment coding vector through the entity recognition model to obtain a recognition pruned coding vector to reduce negative examples in the candidate entity fragment coding vector, and then performs coreference resolution processing through the coreference resolution model, and prunes the recognition pruned coding vector again based on the result to obtain a coreference resolution coding vector, thereby achieving further screening of the recognition pruned coding vector based on different task requirements, wherein generating a second relationship pair coding vector based on the coreference resolution coding vector can further and more deeply enhance the coreference resolution model's understanding of the candidate entity fragment, and provide a basis for the execution of the coreference resolution task, and finally performing coreference resolution processing separately, and determining the relationship between words based on the above processing results, thereby effectively improving the accuracy of word processing.

[0159] The word processing method based on the dual-task model provided in this embodiment realizes the organic combination of the coreference resolution model and the entity recognition model, realizes the organic combination of the coreference resolution task and the entity recognition task, and realizes the sharing of information in the above two tasks. It can effectively improve the accuracy and recall rate of the coreference resolution task and the entity recognition task, and effectively improve the accuracy of word processing based on the dual-task model for determining word relationships.

[0160] See also Figure 3 , this embodiment provides a word processing method based on a dual-task model, including:

[0161] S310: Obtain a candidate entity segment, and generate a candidate entity segment encoding vector based on the candidate entity segment.

[0162] Among them, step S310 is the same as step S210 in the aforementioned embodiment. For a specific explanation of step S310, please refer to the detailed description of the aforementioned embodiment, which will not be repeated here.

[0163] S311, inputting the candidate entity segment encoding vector into an entity recognition model, and scoring the candidate entity segment encoding vector through the feedforward neural network.

[0164] S312. Classify the candidate entity fragment coding vector based on the score of the candidate entity fragment coding vector to obtain a classification label of the candidate entity fragment coding vector, and prune the candidate entity fragment coding vector to obtain a recognized pruned coding vector.

[0165] Among them, step S311 and step S312 are the same as steps S221 to S222 in the aforementioned embodiment. For the specific explanation of step S311 and step S312, please refer to the detailed description of the aforementioned embodiment, which will not be repeated here.

[0166] S313. Score the recognition pruned code vector using a coreference resolution model, and prune the recognition pruned code vector based on the score to obtain a coreference resolution code vector.

[0167] S314. Obtain a second initial relationship pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector.

[0168] S315. Perform classification prediction processing on the second initial relationship pair coding vector, and prune the second initial relationship pair coding vector according to a preset ratio based on the classification prediction result to obtain a second relationship pair coding vector.

[0169] For the detailed explanation of steps S313 to S315, please refer to the detailed content of step S230 in the aforementioned embodiment, which will not be described again here.

[0170] like Figure 4 As shown, this embodiment provides a dual-task model training method, including steps S410 to S440.

[0171] S410: Obtain at least two sample candidate entity pairs and a classification label of each of the sample candidate entity pairs, and generate a sample candidate entity segment encoding vector based on the sample candidate entities of each sample candidate entity pair.

[0172] For the specific content, please refer to the above embodiments, which will not be described again here.

[0173] S420, performing entity recognition processing and classification pruning processing on the sample candidate entity fragment encoding vector through an entity recognition model to obtain a sample recognition pruning encoding vector.

[0174] Specifically, assuming that there are n categories of sample candidate entity fragments (n≥1, and n is an integer), then the sample candidate entity fragments that do not belong to the above n categories are all negative examples. The sample candidate entity fragment encoding vectors are classified based on the scores of the sample candidate entity fragment encoding vectors to obtain the classification label of each sample candidate entity fragment encoding vector, and then the category to which each sample candidate entity fragment encoding vector belongs can be obtained. After pruning some sample candidate entity fragment encoding vectors in the negative examples according to a preset ratio, the remaining other sample candidate entity fragment encoding vectors are the identification pruned encoding vectors. Among them, the proportion of sample candidate entity fragment encoding vectors pruned in the negative examples can be determined according to the specific situation, such as one-sixth, one-fifth, etc., and this application does not limit this.

[0175] This embodiment classifies the encoding vectors of sample candidate entity fragments and prunes some encoding vectors in negative examples, so that the model can learn positive examples and negative examples in appropriate proportions at the same time, that is, learn from both correct and incorrect aspects at the same time, which helps to improve the model training effect.

[0176] S430: Input the sample identification pruning coding vector into a coreference resolution model for processing to obtain a second sample relationship pair coding vector.

[0177] Specifically, the step S430 includes: inputting the sample identification pruned coding vector into the coreference resolution model, scoring the sample identification pruned coding vector through the coreference resolution model, and pruning the sample identification pruned coding vector based on the score to obtain a sample coreference resolution coding vector, and generating a second sample relationship pair coding vector based on the sample coreference resolution coding vector.

[0178] For the specific content, please refer to the above embodiments, which will not be described again here.

[0179] S440, determining the loss values ​​of the entity recognition model and the coreference resolution model respectively based on the sample recognition pruning coding vector and the second sample relationship coding vector, and training the entity recognition model and the coreference resolution model.

[0180] Specifically, step S440 includes:

[0181] The loss value of the entity recognition model is calculated using a cross entropy loss function based on the score of the sample recognition pruned coding vector and the classification label of the sample recognition pruned coding vector;

[0182] Based on the score of the second sample relationship pair encoding vector and the classification label of the second sample relationship pair encoding vector, a cross entropy loss function is used to calculate the loss value of the coreference resolution model.

[0183] For example, during the entity recognition model training process, a set of loss values ​​obtained by cross entropy calculation may be [-0.0000, -6.8651, -9.8858, -9.3611, -9.4160, -8.8986, -10.0036], where the 7 numbers correspond to the numbers 0 to 6, and each number represents a classification label.

[0184] Through F.softmax, it becomes the classification probability [9.9856e-01, 1.0421e-03, 5.0818e-05, 8.5878e-05, 8.1292e-05, 1.3638e-04, 4.5174e-05], and the maximum value is the final loss value.

[0185] The formula for the cross entropy loss function is as follows:

[0186]

[0187] Cross entropy is the difference between two probability distributions p and q, where p represents the encoding vector of the classification label corresponding to the true distribution, i.e., the sample identification pruned encoding vector and the second sample relationship pair encoding vector, q represents the non-true distribution, i.e., the sample identification pruned encoding vector and the second sample relationship pair encoding vector, and H(p, q) represents the loss value.

[0188] Specifically, based on the loss values ​​of the entity recognition model and the coreference resolution model, respectively, the weight values ​​of each layer of neuron nodes in the model are reversely adjusted from the output layer to the input layer to train the model.

[0189] The training method of the dual-task model of this embodiment first organically combines the coreference resolution task and the entity recognition task during the training process, and then trains the coreference resolution model and the entity recognition model separately, which can effectively improve the effect of model training and improve the performance of the coreference resolution model and the entity recognition model.

[0190] like Figure 5 As shown, this embodiment discloses a word processing device based on a dual-task model, including:

[0191] The entity segment acquisition module 510 is configured to acquire a candidate entity segment and generate a candidate entity segment encoding vector based on the candidate entity segment;

[0192] An entity recognition pruning module 520 is configured to perform entity recognition processing and classification pruning processing on the candidate entity segment encoding vector through an entity recognition model to obtain a recognition pruned encoding vector;

[0193] The coreference resolution processing module 530 is configured to input the identified pruned coding vector into a coreference resolution model for processing, and determine the coreference relationship between the words in the candidate entity segment.

[0194] Optionally, the coreference resolution processing module 530 is further configured to:

[0195] Scoring the identification pruned coding vector by a coreference resolution model, and pruning the identification pruned coding vector based on the score to obtain a coreference resolution coding vector;

[0196] A second relationship pair encoding vector is generated based on the coreference resolution encoding vector, coreference resolution processing is performed on the second relationship pair encoding vector through the coreference resolution model to obtain a coreference resolution result, and the relationship between the words in the candidate entity fragment is determined based on the coreference resolution result.

[0197] Optionally, the entity recognition model and the coreference resolution model share a feed-forward neural network for scoring;

[0198] The entity recognition pruning module 520 is further configured to:

[0199] Inputting the candidate entity segment encoding vector into an entity recognition model, and scoring the candidate entity segment encoding vector through the feedforward neural network;

[0200] The candidate entity fragment coding vectors are classified based on the scores of the candidate entity fragment coding vectors, and the candidate entity fragment coding vectors are pruned based on the results of the classification process to obtain a recognized pruned coding vector.

[0201] Optionally, the coreference resolution model and the entity recognition model share a feed-forward neural network for scoring;

[0202] The coreference resolution processing module 530 is further configured to:

[0203] Inputting the recognition pruned coding vector into the coreference resolution model, scoring the recognition pruned coding vector through the feedforward neural network, and obtaining a score of the recognition pruned coding vector;

[0204] The recognition pruning coding vector with a score greater than or equal to a preset threshold is used as the coreference resolution coding vector.

[0205] Optionally, the coreference resolution processing module 530 is further configured to:

[0206] Obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector;

[0207] The second initial relationship pair coding vector is subjected to classification prediction processing, and the second initial relationship pair coding vector is pruned according to a preset ratio based on the result of the classification prediction to obtain a second relationship pair coding vector.

[0208] Optionally, the coreference resolution processing module 530 is further configured to:

[0209] Encoding the classification labels of the coreference elimination coding vector to generate a second label vector;

[0210] Based on any two of the coreference resolution coding vectors and their corresponding second label vectors, a second initial relation pair coding vector of any two of the coreference resolution coding vectors is obtained.

[0211] Optionally, the coreference resolution processing module 530 is further configured to:

[0212] Encoding the classification labels of the coreference elimination coding vector to generate a second label vector;

[0213] Determine a semantic vector between any two coreference resolution coding vectors based on positions of the any two coreference resolution coding vectors in the candidate entity segment;

[0214] Based on any two of the coreference resolution coding vectors, the semantic vector between the any two coreference resolution coding vectors, and the second label vector corresponding to each of the coreference resolution coding vectors, a second initial relationship pair coding vector of any two of the coreference resolution coding vectors is obtained.

[0215] Optionally, the coreference resolution processing module 530 is further configured to:

[0216] Determine a plurality of word vectors between the arbitrary two coreference resolution encoding vectors;

[0217] Pooling or attention processing is performed on multiple word vectors between any two coreference resolution encoding vectors to obtain corresponding semantic vectors.

[0218] Optionally, the coreference resolution processing module 530 is further configured to:

[0219] Scoring the second initial relationship pair encoding vector by a feedforward neural network to obtain a score of the second initial relationship pair encoding vector;

[0220] Performing classification prediction processing on the second initial relationship pair encoding vector to obtain a category of the second initial relationship pair encoding vector;

[0221] The second initial relationship pair encoding vector is pruned based on the category and score of the second initial relationship pair encoding vector to obtain a second relationship pair encoding vector.

[0222] Optionally, the coreference resolution processing module 540 is further configured to: score the second relationship pair encoding vector using the coreference resolution model, and perform classification prediction processing on the second relationship pair encoding vector based on the scoring result to obtain a coreference resolution result.

[0223] The word processing device based on the dual-task model of this embodiment realizes the organic combination of the coreference resolution model and the entity recognition model, realizes the organic combination of the coreference resolution task and the entity recognition task, and realizes the sharing of information in the above two tasks. It can effectively improve the accuracy and recall rate of the coreference resolution task and the entity recognition task, and effectively improve the accuracy of word processing based on the dual-task model for determining word relationships.

[0224] like Figure 6 As shown, this embodiment discloses a dual-task model training device, including:

[0225] The sample acquisition module 610 is configured to acquire at least two sample candidate entity segments and a classification label of each sample candidate entity pair, and generate a sample candidate entity segment encoding vector based on the sample candidate entity segments;

[0226] The sample identification module 620 is configured to perform entity identification processing and classification pruning processing on the sample candidate entity fragment encoding vector through an entity identification model to obtain a sample identification pruning encoding vector;

[0227] The sample processing module 630 is configured to input the sample identification pruning coding vector into the coreference resolution model for processing to obtain a second sample relationship pair coding vector;

[0228] The model training module 640 is configured to determine the loss values ​​of the entity recognition model and the coreference resolution model based on the sample recognition pruning coding vector and the second sample relationship coding vector, respectively, and train the entity recognition model and the coreference resolution model.

[0229] Optionally, the sample processing module 630 is further configured to: input the sample identification pruned coding vector into the coreference resolution model, score the sample identification pruned coding vector through the coreference resolution model, and prune the sample identification pruned coding vector based on the score to obtain a sample coreference resolution coding vector, and generate a second sample relationship pair coding vector based on the sample coreference resolution coding vector.

[0230] Optionally, the model training module 640 is further configured to:

[0231] The loss value of the entity recognition model is calculated using a cross entropy loss function based on the score of the sample recognition pruned coding vector and the classification label of the sample recognition pruned coding vector;

[0232] Based on the score of the second sample relationship pair encoding vector and the classification label of the second sample relationship pair encoding vector, a cross entropy loss function is used to calculate the loss value of the coreference resolution model.

[0233] The training device of the dual-task model provided in the present application first organically combines the coreference resolution task and the entity recognition task during the training process, and then trains the coreference resolution model and the entity recognition model separately, which can effectively improve the effect of model training and improve the performance of the coreference resolution model and the entity recognition model.

[0234] It should be noted that the components in the device claim should be understood as the functional modules that must be established to implement the steps of the program flow or the steps of the method, and the functional modules are not actually divided or separated. The device claim defined by such a group of functional modules should be understood as the functional module architecture of the solution mainly implemented by the computer program recorded in the specification, and should not be understood as the physical device that mainly implements the solution by hardware.

[0235] like Figure 7 As shown, Figure 7 7 is a block diagram showing a computing device 700 according to an embodiment of the present specification. The components of the computing device 700 include but are not limited to a memory 770 and a processor 720. The processor 720 is connected to the memory 770 via a bus 730, and the database 750 is used to store data.

[0236] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.77 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0237] In one embodiment of the present specification, the above components of the computing device 700 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0238] The computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a PC. The computing device 700 may also be a mobile or stationary server. The computing device may execute the method described in any of the above embodiments.

[0239] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the word processing method based on the dual-task model or the training method of the dual-task model as described above.

[0240] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the word processing method based on the dual-task model or the training method of the dual-task model are of the same concept, and the details not described in detail in the technical scheme of the storage medium can be found in the description of the technical scheme of the word processing method based on the dual-task model or the training method of the dual-task model.

[0241] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0242] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0243] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0244] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is limited only by the claims and their full scope and equivalents.

Claims

1. A word processing method based on a dual-task model, characterized in that: include: Obtaining a candidate entity segment, and generating a candidate entity segment encoding vector based on the candidate entity segment; Inputting the candidate entity fragment coding vector into an entity recognition model, scoring the candidate entity fragment coding vector through a feedforward neural network in the entity recognition model, and performing entity recognition processing and classification pruning processing on the candidate entity fragment coding vector based on the score of the candidate entity fragment coding vector to obtain a recognition pruned coding vector, wherein the score of the entity fragment coding vector includes a basic score and a classification score; The recognition pruned coding vector is input into a coreference resolution model, and the recognition pruned coding vector is scored by a feedforward neural network in the coreference resolution model; the recognition pruned coding vector is pruned based on the score of the recognition pruned coding vector to obtain a coreference resolution coding vector; a second relationship pair coding vector is generated based on the coreference resolution coding vector, and the second relationship pair coding vector is processed by the coreference resolution model to obtain a coreference resolution result, and the coreference relationship between the words in the candidate entity fragment is determined based on the coreference resolution result, wherein the entity recognition model and the coreference resolution model share a feedforward neural network for scoring, the second relationship pair coding vector is the coding vector remaining after the second initial relationship pair coding vector is pruned based on the score of the second initial relationship pair coding vector, and the second initial relationship pair coding vector is a coding vector composed of two coreference resolution coding vectors and their second label vectors and a distance feature vector.

2. The word processing method based on the dual-task model according to claim 1, characterized in that: Performing entity recognition processing and classification pruning processing on the candidate entity segment encoding vector based on the score of the candidate entity segment encoding vector, including: The candidate entity fragment coding vectors are classified based on the scores of the candidate entity fragment coding vectors to obtain classification labels of the candidate entity fragment coding vectors, and the candidate entity fragment coding vectors are pruned to obtain recognized pruned coding vectors.

3. The word processing method based on the dual-task model according to claim 1, characterized in that: Pruning the identified pruned code vector based on the score of the identified pruned code vector includes: The recognition pruning coding vector with a score greater than or equal to a preset threshold is used as the coreference resolution coding vector.

4. The word processing method based on the dual-task model according to claim 3 is characterized in that: The generating a second relation pair encoding vector based on the coreference resolution encoding vector comprises: Obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector; The second initial relationship pair coding vector is subjected to classification prediction processing, and the second initial relationship pair coding vector is pruned according to a preset ratio based on the result of the classification prediction to obtain a second relationship pair coding vector.

5. The word processing method based on the dual-task model according to claim 4 is characterized in that: Obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector includes: Encoding the classification labels of the coreference elimination coding vector to generate a second label vector; Based on any two of the coreference resolution coding vectors and their corresponding second label vectors, a second initial relation pair coding vector of any two of the coreference resolution coding vectors is obtained.

6. The word processing method based on the dual-task model according to claim 4 is characterized in that: Obtaining a second initial relation pair encoding vector based on the coreference resolution encoding vector and the classification label of the coreference resolution encoding vector includes: Encoding the classification labels of the coreference elimination coding vector to generate a second label vector; Determine a semantic vector between any two coreference resolution coding vectors based on positions of the any two coreference resolution coding vectors in the candidate entity segment; Based on any two of the coreference resolution coding vectors, the semantic vector between the any two coreference resolution coding vectors, and the second label vector corresponding to each of the coreference resolution coding vectors, a second initial relationship pair coding vector of any two of the coreference resolution coding vectors is obtained.

7. The word processing method based on the dual-task model according to claim 6 is characterized in that: Determining the semantic vector between the arbitrary two coreference resolution encoding vectors includes: Determine a plurality of word vectors between the arbitrary two coreference resolution encoding vectors; Pooling or attention processing is performed on multiple word vectors between any two coreference resolution encoding vectors to obtain corresponding semantic vectors.

8. The word processing method based on the dual-task model according to claim 5 or 6, characterized in that: The method of performing classification prediction processing on the second initial relationship pair encoding vector and pruning the second initial relationship pair encoding vector according to a preset ratio based on a result of the classification prediction to obtain a second relationship pair encoding vector includes: Scoring the second initial relation pair encoding vector by a feedforward neural network in the coreference resolution model to obtain a score of the second initial relation pair encoding vector; Performing classification prediction processing on the second initial relationship pair encoding vector to obtain a category of the second initial relationship pair encoding vector; The second initial relationship pair encoding vector is pruned based on the category and score of the second initial relationship pair encoding vector to obtain a second relationship pair encoding vector.

9. The word processing method based on the dual-task model according to claim 1, characterized in that: Performing coreference resolution processing on the second relationship pair encoding vector by using the coreference resolution model to obtain a coreference resolution result includes: The second relationship pair encoding vector is scored using the coreference resolution model, and classification prediction processing is performed on the second relationship pair encoding vector based on the scoring result to obtain a coreference resolution result.

10. A dual-task model training method, characterized in that: include: Obtain at least two sample candidate entity pairs and a classification label of each of the sample candidate entity pairs, and generate a sample candidate entity fragment encoding vector based on the sample candidate entities of each sample candidate entity pair; Inputting the candidate entity fragment coding vector into an entity recognition model, scoring the candidate entity fragment coding vector through a feedforward neural network in the entity recognition model, and performing entity recognition processing and classification pruning processing based on the score of the candidate entity fragment coding vector to obtain a sample recognition pruned coding vector, wherein the score of the entity fragment coding vector includes a basic score and a classification score; Inputting the identified pruned coding vector into a coreference resolution model, scoring the identified pruned coding vector through a feedforward neural network in the coreference resolution model, and obtaining a second sample relationship pair coding vector based on the score of the identified pruned coding vector; Based on the sample recognition pruning coding vector and the second sample relationship pair coding vector, the loss values ​​of the entity recognition model and the coreference resolution model are respectively determined, and the entity recognition model and the coreference resolution model are trained, wherein the entity recognition model and the coreference resolution model share a feedforward neural network for scoring.

11. The dual-task model training method according to claim 10, characterized in that: Obtaining a second sample relationship pair encoding vector based on the score of the identified pruned encoding vector includes: The sample identification pruning code vector is pruned based on the score of the identification pruning code vector to obtain a sample coreference elimination code vector, and a second sample relationship pair code vector is generated based on the sample coreference elimination code vector.

12. The dual-task model training method according to claim 11, characterized in that: Determining the loss values ​​of the entity recognition model and the coreference resolution model based on the sample recognition pruning encoding vector and the second sample relationship pair encoding vector respectively includes: The loss value of the entity recognition model is calculated using a cross entropy loss function based on the score of the sample recognition pruned coding vector and the classification label of the sample recognition pruned coding vector; Based on the score of the second sample relationship pair encoding vector and the classification label of the second sample relationship pair encoding vector, a cross entropy loss function is used to calculate the loss value of the coreference resolution model.

13. A word processing device based on a dual-task model, characterized in that: include: An entity segment acquisition module is configured to acquire candidate entity segments and generate candidate entity segment encoding vectors based on the candidate entity segments; an entity recognition pruning module, configured to input the candidate entity segment encoding vector into an entity recognition model, score the candidate entity segment encoding vector through a feedforward neural network in the entity recognition model, and perform entity recognition processing and classification pruning processing on the candidate entity segment encoding vector based on the score of the candidate entity segment encoding vector to obtain a recognition pruned encoding vector, wherein the score of the entity segment encoding vector includes a basic score and a classification score; The coreference resolution processing module is configured to input the recognition pruned coding vector into the coreference resolution model, score the recognition pruned coding vector through the feedforward neural network in the coreference resolution model; prune the recognition pruned coding vector based on the score of the recognition pruned coding vector to obtain a coreference resolution coding vector; generate a second relationship pair coding vector based on the coreference resolution coding vector, perform coreference resolution processing on the second relationship pair coding vector through the coreference resolution model to obtain a coreference resolution result, and determine the coreference relationship between the words in the candidate entity fragment based on the coreference resolution result, wherein the entity recognition model and the coreference resolution model share a feedforward neural network for scoring, the second relationship pair coding vector is the coding vector remaining after pruning the second initial relationship pair coding vector based on the score of the second initial relationship pair coding vector, and the second initial relationship pair coding vector is a coding vector composed of two coreference resolution coding vectors and their second label vectors and distance feature vectors.

14. A dual-task model training device, characterized in that: include: A sample acquisition module is configured to acquire at least two sample candidate entity pairs and a classification label of each of the sample candidate entity pairs, and generate a sample candidate entity fragment encoding vector based on the sample candidate entity of each sample candidate entity pair; A sample recognition module is configured to input the candidate entity segment encoding vector into an entity recognition model, score the candidate entity segment encoding vector through a feedforward neural network in the entity recognition model, and perform entity recognition processing and classification pruning processing on the sample candidate entity segment encoding vector based on the score of the candidate entity segment encoding vector to obtain a sample recognition pruned encoding vector, wherein the score of the entity segment encoding vector includes a basic score and a classification score; A sample processing module is configured to input the recognition pruned code vector into a coreference resolution model, score the recognition pruned code vector through a feedforward neural network in the coreference resolution model, and obtain a second sample relationship pair code vector based on the score of the recognition pruned code vector; The model training module is configured to determine the loss values ​​of the entity recognition model and the coreference resolution model respectively based on the sample recognition pruning coding vector and the second sample relationship coding vector, and train the entity recognition model and the coreference resolution model, wherein the entity recognition model and the coreference resolution model share a feedforward neural network for scoring.

15. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the instructions, the steps of the method according to any one of claims 1 to 12 are implemented.

16. A computer-readable storage medium storing computer instructions, characterized in that: When the instruction is executed by a processor, the steps of the method described in any one of claims 1 to 12 are implemented.