Method and system for identifying named entities based on multi-target learning

Abstract

The invention discloses a named entity recognition method and system based on multi-target learning, and relates to the technical field of natural language processing. The method comprises the steps of obtaining a source text sequence and a target text sequence; according to the source text sequence, the prediction language model and double affine transformation, constructing a span representation matrix to obtain a representation coding layer; according to the span representation matrix, the inter-span relative position information and the matrix region information, span feature representation is constructed by using cavity convolution and a nonlinear activation function, and a span dependency layer is obtained; according to the span feature representation and a multi-layer perception mechanism, calculating a span category probability to obtain a span classification task; and according to the span feature representation, the automatic regression bidirectional transducer and the multi-layer perception mechanism, calculating a probability maximum value, obtaining a boundary generation task, finally obtaining a multi-target learning named entity recognition model, and performing named entity recognition on a target text sequence. According to the method, the problem that span boundary feature representation is fuzzy due to context sharing is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and system for recognizing named entities based on multi-objective learning. Background Art

[0002] Named entity recognition (NER) is a core natural language processing technology that aims to identify and extract named entities with specific semantic roles, such as names of people, places, and organizations, from large amounts of unstructured text data. As a key foundational task in information extraction, NER plays a crucial role in promoting the development of various natural language processing applications, such as information retrieval, question-answering systems, knowledge graphs, text understanding, and machine translation. For this reason, the NER task has long been a hot topic and focus of research in the field of natural language processing, attracting widespread attention and research.

[0003] Natural language is iterative and recursive in expression, so there are a large number of nested named entities in the text. Current methods for nested NER include: multi-layer stacking-based sequence tagging methods, hypergraph-based methods, sequence-to-sequence methods, and span-based methods. Among them, the multi-layer stacking-based sequence tagging method identifies nested entities by stacking multiple layers of flat NER recognition layers, but this method will have problems such as error propagation between layers; the hypergraph-based method uses hypergraphs to represent nested entity structures, but the hypergraph structure is complex, and there are problems such as pseudo-structure and structural ambiguity during the reasoning process; the sequence-to-sequence method uses a generative model to directly generate entities in the text, which has problems such as low decoding efficiency and exposure bias; and the span-based method enumerates all possible spans and classifies them to achieve the recognition of nested entities at different levels. Because different entities belong to different subsequences in the text, the span-based method can effectively capture these subsequences and thus understand the complex multi-level structure and semantics of the text. Therefore, the span-based method avoids the problems of multi-layer stacking-based sequence labeling methods, hypergraph-based methods, and sequence-to-sequence-based methods, and shows advantages in nested entity recognition and understanding the deep structure of text.

[0004] Span-based named entity recognition methods typically identify entities based on local context. However, nested entities share the same context, which results in high semantic similarity between the generated nested entity feature representations. This blurs the feature representation of span boundaries, making it difficult for the model to effectively capture the dependencies between these spans. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for recognizing named entities based on multi-objective learning, which not only realizes multi-objective learning named entity recognition with joint boundary generation, but also solves the problem of fuzzy representation of span boundary features caused by shared context in traditional methods.

[0006] To achieve the above objectives, the present invention provides a method for identifying named entities based on multi-objective learning, the specific steps of which are as follows:

[0007] Step S1: Obtain a source text sequence and a target text sequence; the source text sequence is a recognized text sequence; the target text sequence is a text sequence to be recognized;

[0008] Step S2: constructing a span representation matrix according to the source text sequence, the predicted language model and the biaffine transformation to obtain the representation coding layer;

[0009] Step S3: constructing a span feature representation based on the span representation matrix, the relative position information between spans, and the matrix region information using a dilated convolution and a nonlinear activation function to obtain the span dependency layer;

[0010] Step S4: Calculate the span category probability based on the span feature representation and the multi-layer perception mechanism to obtain a span classification task;

[0011] Step S5: Calculate the maximum probability based on the span feature representation, the auto-regressive bidirectional transformer, and the multi-layer perception mechanism to obtain a boundary generation task;

[0012] Step S6: Based on the span classification task and the boundary generation task, a prediction layer is obtained to obtain a multi-objective learning named entity recognition model;

[0013] Step S7: input the target text sequence into the multi-objective learning named entity recognition model to perform named entity recognition.

[0014] Preferably, in step S2, constructing a span representation matrix according to the source text sequence, the predicted language model and the biaffine transformation to obtain the representation coding layer includes:

[0015] Step S21: constructing a vector set of the source text according to the predicted language model and the source text sequence; the vector set includes the vector of each word in the target text sequence;

[0016] Step S22: capturing the context-dependent features of the vector set according to the bidirectional long short-term memory network to obtain a context representation of the vector set;

[0017] Step S23: Mapping the context representation to the entity head vector and the entity tail vector using a multi-layer perception mechanism to obtain the head vector and the tail vector of the vector set;

[0018] Step S24: Utilize double affine transformation to fuse the head vector and the tail vector, construct a span representation matrix, and obtain the representation coding layer.

[0019] Preferably, the span representation matrix is:

[0020] E=BERT(X)={e1,e2,…,e N};

[0021] X={x1,x2,…,x N};

[0022]

[0023]

[0024] Where X is the source text sequence; x1 is the first word of the source text sequence; x2 is the second word of the source text sequence; x N is the Nth word in the source text sequence; E is the vector set of the source text; e1 is the first vector value of the source text; e2 is the second vector value of the source text; e N is the Nth vector value of the source text; BERT(X) is the vector transformation of the source text sequence by the pre-trained language model; LSTM(·) is the long short-term memory network function; e i is the i-th vector value of the source text; is the hidden state of the forward LSTM position i; is the hidden state of the backward LSTM position i; [;] is the concatenation operation of and; h i is the context representation of the vector set; h i head is the head vector; h i tail is the tail vector; MLP head (·) is the head mapping function of the multi-layer perception mechanism; MLP tail (·) is the tail mapping function of the multi-layer perception mechanism; M ij is the span representation matrix; Biaffine(h i head ,h i tail ) is the double affine transformation fusion h i head Head vector and h i tail Tail vector.

[0025] Preferably, in step S3, based on the span representation matrix, the relative position information between spans, and the matrix area information, a span feature representation is constructed using a dilated convolution and a nonlinear activation function to obtain the span dependency layer, including:

[0026] Step S31: using a multi-layer perception mechanism, fusing the relative position information between spans, the matrix region information, and the span representation matrix to obtain a position region perception representation;

[0027] Step S32: construct a span feature representation based on the position region perception representation, the dilated convolution and the nonlinear activation function to obtain the span dependency layer.

[0028] Preferably, the span feature is expressed as:

[0029]

[0030] Among them, V is the position region perception representation; MLP(·) is the multi-layer perception mechanism fusion function; M ij is the span representation matrix; E d is the relative position information between spans; E t is the matrix region information; R is the real number set; N is the dimension; d v is the vector dimension of span feature representation; Q l is the output of the dilated convolution with a dilation rate of l; σ(·) is the nonlinear activation function; DConv l (·) is the dilated convolution calculation function; ld Q is the vector dimension of the output of the dilated convolution with a dilation rate of l; Q is the span feature representation; Q 1 is the output of the dilated convolution with a dilation rate of 1; Q 2 is the output of the dilated convolution with a dilation rate of 2.

[0031] Preferably, in step S4, the span category probability is calculated based on the span feature representation and the multi-layer perception mechanism to obtain the span classification task, including:

[0032] Step S41: Calculate a span classification score based on the span feature representation and the multi-layer perception mechanism;

[0033] Step S42: Calculate the span category probability based on the span classification score and the span representation matrix. Preferably, the span category probability is:

[0034] P ij =MLP(Q);

[0035] Y=Softmax(P ij +M ij );

[0036] Among them, P ij is the span classification score; Y is the span category probability; Softmax(·) is the score maximum function.

[0037] Preferably, in step S5, the maximum probability is calculated based on the span feature representation, the auto-regressive bidirectional transformer and the multi-layer perception mechanism to obtain the boundary generation task, including:

[0038] Step S51: Decode the span feature representation using an auto-regressive bidirectional transformer to obtain a decoded output feature;

[0039] Step S52: using the multi-layer perception mechanism to perform word prediction on the decoded output features to obtain a prediction score;

[0040] Step S53: Use the Softmax function to normalize the prediction scores and calculate the probability of each word being selected;

[0041] Step S54: Calculate the maximum probability based on the probability of each word being selected to obtain a boundary generation task.

[0042] Preferably, the maximum probability is:

[0043] Decode ij,τ =Decoder(Q,input);

[0044] Word ij,τ =MLP(Decode ij,τ );

[0045] W=Softmax(Word ij,τ );

[0046] Among them, Decode ij,τ is the decoding output feature; Decoder(·) is the decoding operation; input is the input data; Word ij,τ is the prediction score; W is the maximum probability.

[0047] The present invention also provides a named entity recognition system based on multi-objective learning, comprising:

[0048] A sequence acquisition module is used to acquire a source text sequence and a target text sequence; the source text sequence is a recognized text sequence; the target text sequence is a text sequence to be recognized;

[0049] A span representation matrix construction module is used to construct a span representation matrix according to the source text sequence, the prediction language model and the biaffine transformation to obtain the representation coding layer;

[0050] A span feature representation construction module, which constructs a span feature representation based on the span representation matrix, the relative position information between spans, and the matrix area information, using a dilated convolution and a nonlinear activation function to obtain the span dependency layer;

[0051] A span classification task module is used to calculate the span category probability based on the span feature representation and the multi-layer perception mechanism to obtain a span classification task;

[0052] A boundary generation task module is used to calculate the maximum probability based on the span feature representation, the auto-regressive bidirectional transformer and the multi-layer perception mechanism to obtain a boundary generation task;

[0053] A model building module is used to obtain a prediction layer based on the span classification task and the boundary generation task, thereby obtaining a multi-objective learning named entity recognition model;

[0054] The recognition module is used to input the target text sequence into the multi-objective learning named entity recognition model to perform named entity recognition.

[0055] Therefore, the present invention proposes a method and system for identifying named entities based on multi-objective learning, which has the following beneficial effects:

[0056] (1) The source text sequence is converted into a vector representation in a high-dimensional vector space through a predictive language model, which better captures the contextual semantic information of the sentence;

[0057] (2) The double affine transformation mechanism is used to capture the deep semantic relationship between words in the text, thereby expanding the sentence into a two-dimensional semantic plane. This can more effectively integrate the information between words and provide a richer and more accurate span representation for entity recognition tasks;

[0058] (3) The resulting boundary generation task can extract key boundary semantic information, which helps the model locate entity boundaries more accurately, thereby enhancing the model's attention to entity boundary information and solving the problem of fuzzy representation of span boundary features caused by shared context in traditional methods.

[0059] (4) The boundary information is decoded through the auto-regressive bidirectional transformer, guiding the model network to focus more on the boundary information of the span, thereby strengthening the semantic expression of the boundary features and improving the performance and accuracy of the overall model. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 Schematic diagram of a process for identifying named entities based on multi-objective learning in Example 1 of the present invention;

[0062] Figure 2 This is a technical roadmap for the multi-objective learning named entity recognition method provided in the first embodiment of the present invention; wherein, Figure 2 (1) is the input, Figure 2 (2) in is the representation coding layer, Figure 2 (3) in is the span dependency layer, Figure 2 (4) in is the prediction layer;

[0063] Figure 3 This is a schematic diagram of the boundary generation task provided in the first embodiment of the present invention. DETAILED DESCRIPTION

[0064] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0065] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0066] Example 1

[0067] like Figure 1 As shown, the present invention provides a method for identifying named entities based on multi-objective learning, which is implemented based on a multi-objective learning named entity recognition model; wherein the multi-objective learning named entity recognition model includes: a representation encoding layer, a span dependency layer and a prediction layer; the prediction layer includes a span classification task and a boundary generation task. Figure 2 As shown in (1), take "the Secretary of Homeland Security" as the target text input. The specific steps are as follows:

[0068] Step S1: Obtain a source text sequence and a target text sequence; wherein the source text sequence is a recognized text sequence, and the target text sequence is a text sequence to be recognized.

[0069] Step S2: Based on the source text sequence, the predicted language model and the bi-affine transformation, a span representation matrix is ​​constructed to obtain the representation coding layer. The technical roadmap of the representation coding layer can be found in Figure 2 (2) in the above specifically includes:

[0070] Step S21, construct a vector set of the source text based on the predicted language model and the source text sequence. The vector set includes the vector of each word in the target text sequence and is in a high-dimensional vector space. The pre-trained language model (Bidirectional Encoder Representations from Transformers, BERT) is initially trained using a large amount of unlabeled text data to learn and capture the deep representation of language, and then apply it to specific natural language processing downstream tasks, which can significantly improve the performance of various natural language processing tasks. The pre-trained language model can better obtain the contextual semantic information of the sentence by introducing powerful context modeling and bidirectional encoding mechanisms.

[0071] Step S22: Capture the context-dependent features of the vector set using a bidirectional long short-term memory network to obtain a contextual representation of the vector set. To further enhance the ability of context modeling, a bidirectional long short-term memory network (BiLSTM) is used. By combining the forward and backward information transfer mechanisms of the long short-term memory network, BiLSTM can simultaneously capture the context-dependent features of each word in the input sequence, thereby generating a richer and more comprehensive word context representation.

[0072] Step S23: Using the multi-layer perception mechanism, the context representation is mapped to the entity head vector and the entity tail vector to obtain the head vector and tail vector of the vector set. Figure 2 The span dependency layer shown in (3) is initially a span representation matrix Where R is a set of real numbers, N is the dimension, N×N is the span representing the dimension of the matrix, and M ij is the span representation matrix; d MEach span represents a vector dimension. Each row and column of the matrix corresponds to the vector representation of a different word in the input sentence. The intersection of a row and a column represents the semantic representation of a span consisting of two words. Considering that the head and tail of an entity span have different features, the contextual representation of each word in the text is mapped to the entity head vector and entity tail vector respectively through a multi-layer perception mechanism (MLP).

[0073] Step S24: Using double affine transformation, fuse the head vector and the tail vector to construct a span representation matrix to obtain a representation coding layer.

[0074] The expression used to construct the span representation matrix is ​​as follows:

[0075] E=BERT(X)={e1,e2,…,e N};

[0076] X={x1,x2,…,x N};

[0077]

[0078] Among them, V is the position region perception representation; MLP(·) is the multi-layer perception mechanism fusion function; M ij is the span representation matrix; E d is the relative position information between spans; E t is the matrix region information; R is the real number set; N is the dimension; d v is the vector dimension of span feature representation; Q l is the output of the dilated convolution with a dilation rate of l; σ(·) is the nonlinear activation function; DConv l (·) is the dilated convolution calculation function; ld Q is the vector dimension of the output of the dilated convolution with a dilation rate of l; Q is the span feature representation; Q 1 is the output of the dilated convolution with a dilation rate of 1; Q 2 is the output of the dilated convolution with a dilation rate of 2.

[0079] Step S3: Based on the span representation matrix, the relative position information between spans, and the matrix area information, a span feature representation is constructed using a dilated convolution and a nonlinear activation function to obtain a span dependency layer, specifically including:

[0080] Step S31: Utilizing a multi-layer perception mechanism, the relative position information between spans, the matrix region information, and the span representation matrix are fused to obtain a position-region-aware representation. To enhance the multi-objective learning named entity recognition model's ability to capture complex inter-span dependencies, the present invention incorporates two types of position information into the existing span representation matrix: information representing the relative position between spans and matrix region information used to distinguish upper and lower triangular regions. This enables the model to understand the spatial layout of the span matrix, providing the multi-objective learning named entity recognition model with a more comprehensive perspective for determining contextual relationships between spans.

[0081] Step S32, based on the position region perception representation, hole convolution and nonlinear activation function, construct the span feature representation and obtain the span dependency layer. In the hole convolution, the field of view is expanded by inserting holes in the standard convolution kernel size, so that the farther span semantic relationship can be captured without increasing the convolution kernel size. The greater the dilation rate, the greater the capture ability. The model can capture information at a farther distance and obtain the global semantic information of the span matrix. The nonlinear activation function is the GELU activation function. Using the output of the dilated convolution with a dilation rate of [1,2,3,4], the feature map with a dilation rate of [1,2,3,4] is spliced ​​to obtain the span matrix representation.

[0082] The expression of the span feature is:

[0083]

[0084] Among them, Q 3 is the output of the dilated convolution with a dilation rate of 3; Q 4 is the output of the dilated convolution with a dilation rate of 4; 4d Q is the vector dimension of the output of the dilated convolution with a dilation rate of 4.

[0085] Step S4: Calculate the span category probability based on the span feature representation and the multi-layer perception mechanism to obtain the span classification task, which specifically includes:

[0086] Step S41: Calculate the span classification score based on the span feature representation and the multi-layer perception mechanism. The expression of the span classification score is:

[0087] P ij =MLP(Q);

[0088] Among them, P ij Score for span classification.

[0089] Step S42: Calculate the span category probability based on the span classification score and the span representation matrix. The span category probability is obtained by combining the double affine transformation and the maximum score of the MLP classifier.

[0090] The calculation expression of span category probability is:

[0091] Y=Softmax(P ij +M ij );

[0092] Where Y is the span category probability; Softmax(·) is the maximum score function.

[0093] In span classification tasks, a collaborative strategy of multiple classifiers is often employed to improve the generalization and accuracy of the model. This strategy leverages the strengths of different classifiers to more comprehensively capture and understand complex patterns in the data. As a feedforward neural network, the MLP is known for its powerful nonlinear mapping capabilities and excels at capturing local features of the data. However, the MLP alone cannot capture the global dependencies between spans, while the biaffine transform is more adept at capturing global dependencies. By using these two classifiers simultaneously, the advantages of both can be combined.

[0094] Step S5: Calculate the maximum probability based on the span feature representation, the auto-regressive bidirectional transformer, and the multi-layer perception mechanism to obtain the boundary generation task, specifically including:

[0095] Step S51: Decode the span feature representation using an auto-regressive bidirectional transformer to obtain a decoded output feature. Specifically, the span encoding is decoded using a start identifier and an end identifier to obtain a decoded output feature. The auto-regressive bidirectional transformer (BART) is a transformer-based sequence generation model, and its decoder consists of 12 layers of transformer decoding blocks. The span boundary generation framework structure is as follows: Figure 3 As shown, in the span boundary generation task, the framework directly uses the decoder of the BART model to decode the encoded span semantics. The span encoding represents the key value and the value. The position information input = { <h> , <t>} represents the query guidance information, reflecting the meaning representation of the current input code being paid attention to. Among them, input is the input data, <h>is the start identifier, <t>As the end identifier, the start identifier and the end identifier are introduced to decode the span encoding representation.

[0096] The expression of the decoding output feature is:

[0097] Decode ij,τ =Decoder(Q,input);

[0098] Among them, Decode ij,τ is the decoding output feature; Decoder(·) is the decoding operation; input is the input data.

[0099] Step S52: Use the multi-layer perception mechanism to predict the word of the decoded output features and obtain the prediction score. The expression of the prediction score is:

[0100] Word ij,τ =MLP(Decode ij,τ );

[0101] Among them, Word ij,τ Score for prediction.

[0102] Step S53: Use the Softmax function to normalize the predicted scores and determine the probability of each word being selected. In order to select the final result from the predicted score vector, the scores are normalized using the Softmax function to obtain the probability of each word being selected.

[0103] Step S54: Determine the maximum probability based on the probability of each word being selected, and obtain the boundary generation task. The expression of the maximum probability is:

[0104] W=Softmax(Word ij,τ );

[0105] Among them, W is the maximum probability.

[0106] The cross entropy function is used as the loss function of the maximum probability, and the formula is as follows:

[0107]

[0108] Among them, L boundary is the loss function of the maximum probability, D is the vocabulary set, and S is the number of samples;

[0109] Step S6: Figure 2 As shown in (4), based on the span classification task and the boundary generation task, a multi-objective learning named entity recognition model is obtained; the span classification task and the boundary generation task are jointly trained by utilizing the shared information in the model network, so that the model can learn better feature representation and more accurate task prediction; Among them, the loss function of multi-task learning is defined as follows:

[0110] L=L span +λL boundary ;

[0111] Among them, L span and L boundary are the cross entropy losses for span classification and boundary generation tasks respectively; L boundary The boundary generation task is an auxiliary task; λ is an adjustable weight parameter of the boundary generation task.

[0112] In order to minimize the difference between the true label and the predicted label, the present invention calculates the predicted label for each sentence X = {x1, x2, ..., x N }, the training objective of the model is to minimize the negative log-likelihood loss with respect to the corresponding true label, which can be formalized as:

[0113]

[0114] Among them, L span is the negative log-likelihood loss; N is the number of words;

[0115] Step S7: input the target text sequence into the multi-objective learning named entity recognition model to perform named entity recognition.

[0116] The present invention obtains a multi-objective learning named entity recognition model by constructing a representation coding layer, a span dependency layer and a prediction layer. Named entity recognition is performed on the target text sequence through the multi-objective learning named entity recognition model, which solves the problem of fuzzy span boundary feature representation caused by shared context in traditional methods.

[0117] The present invention provides an evaluation of a multi-objective learning named entity recognition model and other recognition models on three commonly used nested named entity recognition datasets, namely ACE2004, ACE2005 and Genia. The ACE2004 and ACE2005 datasets contain seven entity types, namely: person name (PER), organization name (ORG), geopolitical entity name (GPE), place name (LOC), infrastructure (FAC), weapon name (WEA) and vehicle (VEH). According to statistical analysis, 30% of the sentences in the ACE2004 and ACE2005 datasets contain nested entities, while 17% of the sentences in the Genia dataset contain nested entities. The evaluation results are shown in Table 1. The average F1 of the multi-objective learning named entity recognition model on the ACE04, ACE05 and Genia datasets reaches 87.83%, 86.90% and 81.65% respectively.

[0118] Table 1 Evaluation results

[0119]

[0120] Example 2

[0121] The present invention also provides a named entity recognition system based on multi-objective learning, which specifically includes:

[0122] The sequence acquisition module is used to obtain the source text sequence and the target text sequence. The source text sequence is the recognized text sequence, and the target text sequence is the text sequence to be recognized;

[0123] The span representation matrix construction module is used to construct the span representation matrix according to the source text sequence, the predictive language model and the bi-affine transformation to obtain the representation encoding layer.

[0124] The span feature representation construction module constructs the span feature representation based on the span representation matrix, the relative position information between spans, and the matrix area information, using void convolution and nonlinear activation function to obtain the span dependency layer;

[0125] The span classification task module is used to calculate the span category probability based on the span feature representation and multi-layer perception mechanism to obtain the span classification task.

[0126] The boundary generation task module is used to calculate the maximum probability based on the span feature representation, the auto-regressive bidirectional transformer and the multi-layer perception mechanism to obtain the boundary generation task.

[0127] The model building module is used to obtain the prediction layer based on the span classification task and the boundary generation task, and obtain the multi-objective learning named entity recognition model.

[0128] The recognition module is used to input the target text sequence into the multi-objective learning named entity recognition model for named entity recognition.

[0129] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

[0131] Therefore, the present invention provides a method and system for recognizing named entities based on multi-objective learning, which not only realizes multi-objective learning named entity recognition with joint boundary generation, but also solves the problem of fuzzy representation of span boundary features caused by shared context in traditional methods.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.< / t> < / h> < / t> < / h>

Claims

1. A method for identifying named entities based on multi-objective learning, characterized in that: The following steps are involved: Step S1: obtaining a source text sequence and a target text sequence; the source text sequence is a recognized text sequence; The target text sequence is the text sequence to be recognized; Step S2: constructing a span representation matrix according to the source text sequence, the predicted language model and the biaffine transformation to obtain the representation coding layer; Step S3: constructing a span feature representation based on the span representation matrix, the relative position information between spans, and the matrix region information using a dilated convolution and a nonlinear activation function to obtain the span dependency layer; Step S4: Calculate the span category probability based on the span feature representation and the multi-layer perception mechanism to obtain a span classification task; Step S5: Calculate the maximum probability based on the span feature representation, the auto-regressive bidirectional transformer, and the multi-layer perception mechanism to obtain a boundary generation task; Step S6: Based on the span classification task and the boundary generation task, a prediction layer is obtained to obtain a multi-objective learning named entity recognition model; Step S7: input the target text sequence into the multi-objective learning named entity recognition model to perform named entity recognition.

2. The method for identifying named entities based on multi-objective learning according to claim 1, characterized in that: In step S2, a span representation matrix is ​​constructed based on the source text sequence, the predicted language model, and the biaffine transformation to obtain the representation coding layer, including: Step S21: constructing a vector set of the source text according to the predicted language model and the source text sequence; the vector set includes the vector of each word in the target text sequence; Step S22: capturing the context-dependent features of the vector set according to the bidirectional long short-term memory network to obtain a context representation of the vector set; Step S23: Mapping the context representation to the entity head vector and the entity tail vector using a multi-layer perception mechanism to obtain the head vector and the tail vector of the vector set; Step S24: Utilize double affine transformation to fuse the head vector and the tail vector, construct a span representation matrix, and obtain the representation coding layer.

3. The method for identifying named entities based on multi-objective learning according to claim 2, characterized in that: The span representation matrix is: E=BERT(X)={e1,e2,…,e N }; X={x1,x2,…,x N }; Where X is the source text sequence; x1 is the first word of the source text sequence; x2 is the second word of the source text sequence; x N is the Nth word in the source text sequence; E is the vector set of the source text; e1 is the first vector value of the source text; e2 is the second vector value of the source text; e N is the Nth vector value of the source text; BERT(X) is the vector transformation of the source text sequence by the pre-trained language model; LSTM(·) is the long short-term memory network function; e i is the i-th vector value of the source text; is the hidden state of the forward LSTM position i; is the hidden state of the backward LSTM position i; [;] is the concatenation operation of and; h i is the context representation of the vector set; h i head is the head vector; h i tail is the tail vector; MLP head (·) is the head mapping function of the multi-layer perception mechanism; MLP tail (·) is the tail mapping function of the multi-layer perception mechanism; M ij is the span representation matrix; Biaffine(h i head ,h i tail ) is the double affine transformation fusion h i head Head vector and h i tail Tail vector.

4. The method for identifying named entities based on multi-objective learning according to claim 1, characterized in that: In step S3, based on the span representation matrix, the relative position information between spans, and the matrix area information, a span feature representation is constructed using a dilated convolution and a nonlinear activation function to obtain the span dependency layer, including: Step S31: using a multi-layer perception mechanism, fusing the relative position information between spans, the matrix region information, and the span representation matrix to obtain a position region perception representation; Step S32: construct a span feature representation based on the position region perception representation, the dilated convolution and the nonlinear activation function to obtain the span dependency layer.

5. The method for identifying named entities based on multi-objective learning according to claim 4, characterized in that: The span feature is expressed as: Among them, V is the position region perception representation; MLP(·) is the multi-layer perception mechanism fusion function; M ij is the span representation matrix; E d is the relative position information between spans; E t is the matrix region information; R is the real number set; N is the dimension; d v is the vector dimension of span feature representation; Q l is the output of the dilated convolution with a dilation rate of l; σ(·) is the nonlinear activation function; DConv l (·) is the dilated convolution calculation function; ld Q is the vector dimension of the output of the dilated convolution with a dilation rate of l; Q is the span feature representation; Q 1 is the output of the dilated convolution with a dilation rate of 1; Q 2 is the output of the dilated convolution with a dilation rate of 2.

6. The method for identifying named entities based on multi-objective learning according to claim 1, characterized in that: In step S4, the span category probability is calculated based on the span feature representation and the multi-layer perception mechanism to obtain the span classification task, including: Step S41: Calculate a span classification score based on the span feature representation and the multi-layer perception mechanism; Step S42: Calculate the span category probability according to the span classification score and the span representation matrix.

7. The method for identifying named entities based on multi-objective learning according to claim 6, characterized in that: The span class probability is: P ij =MLP(Q); Y=Soft max(P ij +M ij ); Among them, P ij is the span classification score; Y is the span category probability; Soft max(·) is the score maximum function.

8. The method for identifying named entities based on multi-objective learning according to claim 1, characterized in that: In step S5, the maximum probability is calculated based on the span feature representation, the auto-regressive bidirectional transformer and the multi-layer perception mechanism to obtain the boundary generation task, including: Step S51: Decode the span feature representation using an auto-regressive bidirectional transformer to obtain a decoded output feature; Step S52: using the multi-layer perception mechanism to perform word prediction on the decoded output features to obtain a prediction score; Step S53: Use the Softmax function to normalize the prediction scores and calculate the probability of each word being selected; Step S54: Calculate the maximum probability based on the probability of each word being selected to obtain a boundary generation task.

9. The method for identifying named entities based on multi-objective learning according to claim 8, characterized in that: The maximum probability is: Decode ij,τ =Decoder(Q,input); Word ij,τ =MLP(Decode ij,τ ); W=Soft max(Word ij,τ ); Among them, Decode ij,τ is the decoding output feature; Decoder(·) is the decoding operation; input is the input data; Word ij,τ is the prediction score; W is the maximum probability.

10. A named entity recognition system based on multi-objective learning, characterized in that: include: A sequence acquisition module is used to acquire a source text sequence and a target text sequence; the source text sequence is a recognized text sequence; the target text sequence is a text sequence to be recognized; A span representation matrix construction module is used to construct a span representation matrix according to the source text sequence, the prediction language model and the biaffine transformation to obtain the representation coding layer; A span feature representation construction module, which constructs a span feature representation based on the span representation matrix, the relative position information between spans, and the matrix area information, using a dilated convolution and a nonlinear activation function to obtain the span dependency layer; A span classification task module is used to calculate the span category probability based on the span feature representation and the multi-layer perception mechanism to obtain a span classification task; A boundary generation task module is used to calculate the maximum probability based on the span feature representation, the auto-regressive bidirectional transformer and the multi-layer perception mechanism to obtain a boundary generation task; A model building module is used to obtain a prediction layer based on the span classification task and the boundary generation task, thereby obtaining a multi-objective learning named entity recognition model; The recognition module is used to input the target text sequence into the multi-objective learning named entity recognition model to perform named entity recognition.

Images