A nested named entity extraction method for parallel tasks of ancient book text sequence

Abstract

The application discloses a kind of for ancient book text sequence parallel task nested named entity extraction method, belong to named entity recognition field, this method includes: obtaining ancient book text, and respectively carries out named entity information marking to each character in ancient book text, obtains labeled dataset;Based on labeled dataset, for each character corresponding class matrix is constructed, and the labeled dataset containing span information is formed;The semantic vector of each character is extracted by encoding ancient book text through BERT model;The nested named entity extraction model of sequence parallel task is constructed, and the semantic vector of each character is used as input feature vector to train the nested named entity extraction model of sequence parallel task, and the trained nested named entity extraction model is obtained;Obtain text to be measured, according to text to be measured, determine nested entity and category information using the trained nested named entity extraction model.The application solves the problem of the limitation of boundary assembly in the traditional method.

Description

Technical Field

[0001] This invention belongs to the field of named entity recognition, and in particular relates to a method for extracting nested named entities for parallel tasks of ancient text sequences. Background Technology

[0002] As the preservation of ancient books shifts from physical regenerative protection to the transmission of content, Named Entity Recognition (NER) technology plays a crucial role in ancient book research. NER can automatically extract key information such as personal names, place names, and official titles from ancient books, facilitating digital organization and knowledge graph construction.

[0003] However, traditional Named Entity Recognition (NER) technology has significant limitations when dealing with the nested structures commonly found in ancient texts, struggling to effectively identify and resolve multi-level entity relationships. For example, expressions like "Marquis of Yingchuan" and "King of Yunnan" not only involve official titles but also nest the place names "Yingchuan" and "Yunnan." To address this, Nested Named Entity Recognition (NER) has emerged, capable of simultaneously identifying nested entities and their hierarchical relationships, thus processing complex texts more accurately. This technology provides a more refined means of information extraction for the preservation and research of ancient texts, helping to comprehensively uncover the deeper cultural connotations within them.

[0004] In traditional nested named entity recognition tasks, given a text sequence, the goal is to extract (h, t, c) triples (where h represents the left boundary of the entity, t represents the right boundary, and c represents the entity's class). However, this task suffers from several problems: First, the identification of the left and right boundaries is performed independently, lacking effective boundary information fusion, leading to incomplete information. Second, error propagation is prone to occur during boundary assembly, increasing the complexity of the assembly process. Furthermore, the separate tasks of class classification and entity recognition prevent end-to-end joint modeling. Summary of the Invention

[0005] To address the aforementioned shortcomings in existing technologies, this invention provides a nested named entity extraction method for parallel tasks involving ancient text sequences, which solves the limitation of boundary assembly in traditional methods.

[0006] To achieve the aforementioned objectives, the technical solution adopted by this invention is: a method for extracting nested named entities for parallel tasks involving ancient text sequences, comprising:

[0007] Obtain the ancient text and annotate the named entity information of each character in the ancient text to obtain the annotated dataset A1;

[0008] Based on the labeled dataset A1, a corresponding category matrix is ​​constructed for each character according to the different span distributions of entities, forming a labeled dataset A2 containing span information;

[0009] The text input sequence in the labeled dataset A2 is encoded using the BERT model, and the semantic vectors of each character are extracted.

[0010] A nested named entity extraction model for a sequence parallel task is constructed, and the semantic vector of each character is used as the input feature vector to train the nested named entity extraction model for the sequence parallel task, so as to obtain the trained nested named entity extraction model.

[0011] Obtain the text to be tested. Based on the text, use the trained nested named entity extraction model to determine the nested entities and their category information.

[0012] Furthermore, based on the labeled dataset A1, a corresponding category matrix is ​​constructed for each character according to the different span distributions of the entities, forming a labeled dataset A2 containing span information, specifically as follows:

[0013] Based on the labeled dataset A1, obtain the text input sequence and entity category label set, and extract the feature triples of each entity in the labeled dataset A1 based on the text input sequence and entity category label set:

[0014] e j ={s j ,c j ,l j}

[0015] l j =t j -s j +1

[0016] Among them, e j For the j-th entity; s j For entity e j The starting position; c j For entity e j Category labels; j For entity e j The span; t j This is the end position of the entity;

[0017] Based on the span of each entity l j Initialize the corresponding all-zero matrix M for each different span value. l :

[0018] M l =0 n ×C

[0019] Where, matrix M l The dimension is [n, C]; 0 n The vector is composed of all zeros; C is the number of category labels; n is the length of the text input sequence.

[0020] For each triple {s j ,c j ,l j}, respectively based on the span l of the entity j The values ​​will fill the zero matrix M with the entity's starting position and category label, corresponding to the span value. l In this process, the category matrix is ​​obtained.

[0021]

[0022] The category matrices of all entities are integrated to form a labeled dataset A2; the labeled dataset A2 includes the text input sequence and the category matrix of each entity.

[0023] Furthermore, the text input sequence in the labeled dataset A2 is encoded using the BERT model, and the semantic vector of each character is extracted, specifically as follows:

[0024] The text input sequence X in the labeled dataset A2 and the vocabulary V of the BERT model. BERT By comparing and filtering, character sets X not included in the BERT model were identified. UNK :

[0025]

[0026] Where, x i Input the i-th character in the text sequence;

[0027] Character set X UNK The vocabulary V added to the BERT model BERT In the process, an expanded thesaurus is generated.

[0028] Based on the extended vocabulary, the BERT model is used to encode the text input sequence X in the labeled dataset A2, and the semantic vector E of each character is extracted:

[0029] E = BERT(X).

[0030] Furthermore, the nested named entity extraction model includes a probabilistic-semantic fusion network, a parallel convolutional network, a prior knowledge module, a semantic vector fusion layer, a multilayer perceptron, and a classification layer;

[0031] The probabilistic-semantic fusion network is used to perform a linear transformation on the input feature vector E to generate a class probability distribution p for each input position; the class probability distribution p is then compared with the transpose p of p. Τ Perform matrix operations to obtain the category distribution similarity probability matrix P; transpose the input feature vector E to obtain Ei. Τ and EΤ Performing matrix operations with P yields semantic vectors rich in probabilistic dependencies. The expression for the category probability distribution p is:

[0032]

[0033] Where e is the natural constant; k is the feature index; d is the feature set; W pred (c,k) represents the weights of the c-th category and the k-th feature; E(i,k) represents the value of the k-th feature of the i-th character in the input feature vector E; b pred (c) represents the bias for the c-th category;

[0034] The parallel convolutional network is used to utilize several CNN convolution pairs with different kernel sizes. Parallel convolution operations are performed to obtain a set of semantic vectors ε containing a specific span l; the size of the convolution kernel is the span of the entity corresponding to the current CNN convolution.

[0035] The prior knowledge network is used to obtain the span information set S. l :

[0036] S l ={S (1) ,S (2) ,...,S (L)}

[0037] S = W S ·E+b S

[0038] Where S is the span matrix generated by the prior knowledge network; S (1) This refers to the span information for spans with a span value of 1 after splitting the span matrix S; S (2) This refers to the span information for a span value of 2 after splitting the span matrix S; S (L) This refers to the span information of the span matrix S after splitting it into span values ​​L; W S To generate the weight matrix for the span matrix S; b S The bias is used to generate the span matrix S; L is the maximum entity span value;

[0039] The semantic vector fusion layer is used to weight and fuse the semantic vector set ε and the input feature vector E based on the span value to obtain the fused feature matrix F; the semantic vector set ε and the fused feature matrix F are then integrated to obtain the final word vector set X.

[0040] X = {X (1) ,X (2) ,...,X (L)}

[0041] Among them, X (1) For the final word vector with a span value of 1; X (2) For the final word vector with a span value of 2; X (L) This is the final word vector for a span value of L;

[0042] The multilayer perceptron is used to process the span information set S l The final fused feature vector set X is obtained by concatenating the final word vector set X with the word vector set X based on the span value. input ; and based on the final fused feature vector set X input The feature representation set A is obtained through a multilayer perceptron:

[0043] A = {A (1) A (2) ,...,A (L)}

[0044]

[0045] Among them, A (1) A represents the final fused feature vector processed by a multilayer perceptron for a span value of 1; (2) For the final fused feature vector with a span of 2 after processing by a multilayer perceptron; A (L) This represents the final fused feature vector after processing by a multilayer perceptron for a span value of L; MLP stands for Multilayer Perceptron; X input This is the final set of fused feature vectors; For S (1) and X (1) The final fused feature vector after concatenation; For S (2) and X (2) The final fused feature vector after concatenation; For S (L) and X (L) The final fused feature vector after concatenation;

[0046] The classification layer is used to perform category prediction based on the feature representation set A, resulting in a category prediction matrix set P:

[0047] P = {P (1) ,P (2) ,...,P (L)}

[0048] P (L) =sigmoid(logits) (L) )

[0049] logits (L) =A (L) ·W out +bout

[0050] Among them, P (1) P is the category prediction matrix for entities with a span value of 1; (2) P is the category prediction matrix for entities with a span of 2; (L) Here is the class prediction matrix for entities with a span value of L; sigmoid is the activation function; logits (L) The classification score for entities with a span value of L; W out To output the weight matrix; b out This is a bias term.

[0051] Furthermore, the semantic vector rich in probabilistic dependencies The expression is:

[0052]

[0053] Among them, E Τ is the transpose of the input feature vector; P is the category distribution similarity probability matrix; T is the transpose.

[0054] Furthermore, the expression for the semantic vector set ε is:

[0055] ε={E (1) E (2) ,...,E (L)}

[0056] E (L) =Conv L (E)

[0057] Among them, E (1) E represents the semantic vector after convolution by a CNN with a kernel of 1. (2) E represents the semantic vector after convolution by a CNN with a kernel of 2. (L) The semantic vector is generated after convolution by a CNN with kernel L; Conv L This is a CNN convolution with kernel L.

[0058] Furthermore, the expression for the fused feature matrix F is:

[0059]

[0060] Where F(v,a) is the element in the v-th row and a-th column of the fused feature matrix F; k is the feature index; d is the feature set; E (l) (v,k) represents the k-th dimension feature at the v-th position in the semantic vector after convolution with a CNN kernel of size l; E T (k,a) represents the k-th dimension feature at the a-th position in the input feature vector E; l is the span value index.

[0061] Furthermore, the expression for the final word vector is:

[0062]

[0063] Among them, X (l) (v,o) is the final word vector X with a span value of l. (l) The element in the v-th row and o-th column of the fusion feature matrix F; n is the length of the text input sequence; g is the entity index; F(v,a) is the element in the v-th row and a-th column of the fusion feature matrix F; E (l) (a,o) is the semantic vector E after convolution by a CNN with kernel l. (l) The o-th dimension feature at position a; l is the span value index.

[0064] Furthermore, the loss function L of the nested named entity extraction model is:

[0065]

[0066] in, The loss for entities with a span value of 1; The loss for entities with a span of 2; The loss for an entity with a span value of L; for The weight of ; C is the number of category labels; c is the category label index; y c This represents the value of class c in the actual label vector; denoted as , where is the predicted probability of the model for class c; T is the number of positive samples; and N is the number of negative samples.

[0067] Furthermore, the step of obtaining the text to be tested, and extracting named entities based on the text using the trained nested named entity extraction model, specifically involves:

[0068] Obtain the test text. Based on the test text, use the trained nested named entity extraction model to obtain a set of category prediction matrices for the test text. Extract the predicted left boundaries of entities from the transposed category prediction matrices of the test text. Based on the span information of the category prediction matrices of the test text, obtain the right boundaries of entities. right The extracted entity's left boundary, entity's right boundary, and category C are then analyzed. j Combine these elements to determine the nested entities and their categories:

[0069] entity = {entity left entity right C j}

[0070]

[0071] entity right =entity left +l'-1

[0072] Where f is a conditional judgment function that extracts the left boundary of entities with a confidence level greater than 0.5; The category information C is represented in the category prediction matrix of the transposed text. j The row; l' represents the span information of the prediction matrix of the text to be tested; C j This represents the category information of the category prediction matrix for the text to be tested.

[0073] The beneficial effects of this invention are as follows: By designing a category matrix, building a probability-semantic fusion module, and combining span information and left boundary extraction to extract nested entities, this invention effectively solves the problem of the separation between traditional entity recognition and classification tasks, achieving end-to-end entity recognition and classification; it overcomes the error propagation problem in traditional boundary assembly, enhancing the accuracy of nested entity recognition; it simplifies the complex process of step-by-step left and right boundary recognition in traditional nested recognition, improving recognition efficiency; and it simplifies the task complexity in traditional BIO sequence labeling methods by improving the labeling method. This invention merges entity recognition and classification into a single task, introducing span information by designing a span matrix before training, thus requiring only the identification of the entity's starting position and category. Specifically, the model directly extracts the entity's left boundary from the input and combines it with the span matrix information to determine the entity's complete range, avoiding the error propagation problem caused by traditional multi-step processing. This method not only simplifies the boundary assembly process but also significantly improves the accuracy and efficiency of recognition. Attached Figure Description

[0074] Figure 1 This is a flowchart of the method of the present invention.

[0075] Figure 2 The span matrix generated for the prior knowledge module.

[0076] Figure 3 A modeling diagram for nested named entity extraction methods for parallel tasks. Detailed Implementation

[0077] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0078] like Figure 1As shown, in one embodiment of the present invention, a method for nested named entity extraction for parallel tasks of ancient text sequences includes:

[0079] Obtain the ancient text and annotate the named entity information of each character in the ancient text to obtain the annotated dataset A1;

[0080] Based on the labeled dataset A1, a corresponding category matrix is ​​constructed for each character according to the different span distributions of entities, forming a labeled dataset A2 containing span information;

[0081] The text input sequence in the labeled dataset A2 is encoded using the BERT model, and the semantic vectors of each character are extracted.

[0082] A nested named entity extraction model for a sequence parallel task is constructed, and the semantic vector of each character is used as the input feature vector to train the nested named entity extraction model for the sequence parallel task, so as to obtain the trained nested named entity extraction model.

[0083] Obtain the text to be tested. Based on the text, use the trained nested named entity extraction model to determine the nested entities and their category information.

[0084] In this embodiment, the ancient text corpus used in this invention is the "Collection and Research of Ancient Texts from Yunnan, Guizhou, and Guangxi" (Project No.: 21XTQ003). Its annotation rules include categories such as full name, alias, location, time, book, official title, and appellation. Annotators classify and label each character in the ancient text according to these rules to determine its category. The final result is the Ancient Text Corpus Dataset (APEC) A1, stored in JSON format. Table 1 shows the category label information.

[0085] Table 1

[0086]

[0087] Based on the labeled dataset A1, a category matrix is ​​constructed for each character according to the different span distributions of entities, forming a labeled dataset A2 containing span information, specifically:

[0088] Based on the labeled dataset A1, obtain the text input sequence and entity category label set, and extract the feature triples of each entity in the labeled dataset A1 based on the text input sequence and entity category label set:

[0089] e j ={s j ,c j ,l j}

[0090] l j =t j -sj +1

[0091] Among them, e j For the j-th entity; s j For entity e j The starting position; c j For entity e j Category labels; j For entity e j The span; t j This is the end position of the entity;

[0092] Based on the span of each entity l j Initialize the corresponding all-zero matrix M for each different span value. l :

[0093] M l =0 n ×C

[0094] Where, matrix M l The dimension is [n, C]; 0 n The vector is composed of all zeros; C is the number of category labels; n is the length of the text input sequence.

[0095] For each triple {s j ,c j ,l j}, respectively based on the span l of the entity j The values ​​will fill the zero matrix M with the entity's starting position and category label, corresponding to the span value. l In this process, the category matrix is ​​obtained.

[0096]

[0097] The category matrices of all entities are integrated to form a labeled dataset A2; the labeled dataset A2 includes the text input sequence and the category matrix of each entity.

[0098] In this embodiment, the maximum length of the input sequence is set to L, and the number of entity category labels is set to R. Multiple span matrices M are defined. l Each matrix is ​​L×R in size, and its core is to design corresponding matrices for different entity spans. When entity information is filled in, it is placed into the matrix with the corresponding span, and the right boundary of the entity is determined by the position of the left boundary and the span value of the matrix.

[0099] The text input sequence in the labeled dataset A2 is encoded using the BERT model, and the semantic vector of each character is extracted. Specifically:

[0100] The text input sequence X in the labeled dataset A2 and the vocabulary V of the BERT model.BERT By comparing and filtering, character sets X not included in the BERT model were identified. UNK :

[0101]

[0102] Where, x i Input the i-th character in the text sequence;

[0103] Character set X UNK The vocabulary V added to the BERT model BERT In the process, an expanded thesaurus is generated.

[0104] Based on the extended vocabulary, the BERT model is used to encode the text input sequence X in the labeled dataset A2, and the semantic vector E of each character is extracted:

[0105] E = BERT(X).

[0106] In this embodiment, since ancient texts contain a large number of traditional Chinese characters and special characters, and the BERT model's pre-trained vocabulary has limited recognition of these characters, it is necessary to expand the BERT vocabulary before semantic encoding to better handle these characters. The text data is input into the BERT model, and the BERT encoding layer performs contextual implicit encoding on each character, generating a semantic tag corresponding to each character. This step captures the core semantic association between characters and their context through BERT's multi-layered Transformer encoding layer.

[0107] like Figure 2 and Figure 3 As shown, the nested named entity extraction model includes a probabilistic-semantic fusion network, a parallel convolutional network, a prior knowledge module, a semantic vector fusion layer, a multilayer perceptron, and a classification layer;

[0108] The probabilistic-semantic fusion network is used to perform a linear transformation on the input feature vector E to generate a class probability distribution p for each input position; the class probability distribution p is then compared with the transpose p of p. Τ Perform matrix operations to obtain the category distribution similarity probability matrix P; transpose the input feature vector E to obtain Ei. Τ and E Τ Performing matrix operations with P yields semantic vectors rich in probabilistic dependencies. The expression for the category probability distribution p is:

[0109]

[0110] Where e is the natural constant; k is the feature index; d is the feature set; W pred(c,k) represents the weights of the c-th category and the k-th feature; E(i,k) represents the value of the k-th feature of the i-th character in the input feature vector E; b pred (c) represents the bias for the c-th category;

[0111] The parallel convolutional network is used to utilize several CNN convolution pairs with different kernel sizes. Parallel convolution operations are performed to obtain a set of semantic vectors ε containing a specific span l; the size of the convolution kernel is the span of the entity corresponding to the current CNN convolution.

[0112] The prior knowledge network is used to obtain the span information set S. l :

[0113] S l ={S (1) ,S (2) ,...,S (L)}

[0114] S = W S ·E+b S

[0115] Where S is the span matrix generated by the prior knowledge network; S (1) This refers to the span information for spans with a span value of 1 after splitting the span matrix S; S (2) This refers to the span information for a span value of 2 after splitting the span matrix S; S (L) This refers to the span information of the span matrix S after splitting it into span values ​​L; W S To generate the weight matrix for the span matrix S; b S The bias is used to generate the span matrix S; L is the maximum entity span value;

[0116] The semantic vector fusion layer is used to weight and fuse the semantic vector set ε and the input feature vector E based on the span value to obtain the fused feature matrix F; the semantic vector set ε and the fused feature matrix F are then integrated to obtain the final word vector set X.

[0117] X = {X (1) ,X (2) ,...,X (L)}

[0118] Among them, X (1) For the final word vector with a span value of 1; X (2) For the final word vector with a span value of 2; X (L) This is the final word vector for a span value of L;

[0119] The multilayer perceptron is used to process the span information set S lThe final fused feature vector set X is obtained by concatenating the final word vector set X with the word vector set X based on the span value. input ; and based on the final fused feature vector set X input The feature representation set A is obtained through a multilayer perceptron:

[0120] A = {A (1) A (2) ,...,A (L)}

[0121]

[0122] Among them, A (1) A represents the final fused feature vector processed by a multilayer perceptron for a span value of 1; (2) For the final fused feature vector with a span of 2 after processing by a multilayer perceptron; A (L) This represents the final fused feature vector after processing by a multilayer perceptron for a span value of L; MLP stands for Multilayer Perceptron; X input This is the final set of fused feature vectors; For S (1) and X (1) The final fused feature vector after concatenation; For S (2) and X (2) The final fused feature vector after concatenation; For S (L) and X (L) The final fused feature vector after concatenation;

[0123] The classification layer is used to perform category prediction based on the feature representation set A, resulting in a category prediction matrix set P:

[0124] P = {P (1) ,P (2) ,...,P (L)}

[0125] P (L) =sigmoid(logits) (L) )

[0126] logits (L) =A (L) ·W out +b out

[0127] Among them, P (1) P is the category prediction matrix for entities with a span value of 1; (2) P is the category prediction matrix for entities with a span of 2; (L)Here is the class prediction matrix for entities with a span value of L; sigmoid is the activation function; logits (L) The classification score for entities with a span value of L; W out To output the weight matrix; b out This is a bias term.

[0128] In this embodiment, the category distribution similarity probability matrix P describes the similarity of the category probability distributions of the j-th position and the i-th position in the text input sequence.

[0129] In this embodiment, to extract entity features across different spans more comprehensively, multiple convolutional neural networks (CNNs) are designed. Each CNN uses a different kernel size to extract features for different entity spans. These CNNs will run in parallel, thereby capturing multiple features simultaneously within local contexts of varying lengths. Different CNNs correspond to different kernel sizes to extract feature information for their respective spans.

[0130] Parallel convolution operations: Each convolutional network pairs Parallel operations are performed, with the convolutional kernel sliding across sequence E and convolving character by character with a stride of 1. Each convolutional network extracts local features across different spans in parallel.

[0131] The semantic vector rich in probabilistic dependencies The expression is:

[0132]

[0133] Among them, E Τ is the transpose of the input feature vector; P is the category distribution similarity probability matrix; T is the transpose.

[0134] The expression for the semantic vector set ε is:

[0135] ε={E (1) E (2) ,...,E (L)}

[0136] E (L) =Conv L (E)

[0137] Among them, E (1) E represents the semantic vector after convolution by a CNN with a kernel of 1. (2) E represents the semantic vector after convolution by a CNN with a kernel of 2. (L) The semantic vector is generated after convolution by a CNN with kernel L; Conv L This is a CNN convolution with kernel L.

[0138] The expression for the fused feature matrix F is:

[0139]

[0140] Where F(v,a) is the element in the v-th row and a-th column of the fused feature matrix F; k is the feature index; d is the feature set; E (l) (v,k) represents the k-th dimension feature at the v-th position in the semantic vector after convolution with a CNN kernel of size l; E T (k,a) represents the k-th dimension feature at the a-th position in the input feature vector E; l is the span value index.

[0141] The expression for the final word vector is:

[0142]

[0143] Among them, X (l) (v,o) is the final word vector X with a span value of l. (l) The element in the v-th row and o-th column of the fusion feature matrix F; n is the length of the text input sequence; g is the entity index; F(v,a) is the element in the v-th row and a-th column of the fusion feature matrix F; E (l) (a,o) is the semantic vector E after convolution by a CNN with kernel l. (l) The o-th dimension feature at position a; l is the span value index.

[0144] The loss function L of the nested named entity extraction model is:

[0145]

[0146] in, The loss for entities with a span value of 1; The loss for entities with a span of 2; The loss for an entity with a span value of L; for The weight of ; C is the number of category labels; c is the category label index; y c This represents the value of class c in the actual label vector; denoted as , where is the predicted probability of the model for class c; T is the number of positive samples; and N is the number of negative samples.

[0147] In this embodiment, during the model training process, a weighted binary cross-entropy loss function with multiple weight coefficients is designed for the task of feature extraction and recognition of nested named entities with different spans. By setting different weight coefficients for each span, this loss function can dynamically adjust the weight of each span entity in the loss calculation according to the relative importance of the entity span in the sequence, thereby improving the model's recognition ability.

[0148] The core idea of ​​this design is to assign corresponding weight coefficients to each span, reflecting the relative importance of each entity span in the overall sequence, ensuring that the model pays more attention to complex, long-span entities. Through weight allocation in parallel tasks, entities with larger spans receive higher weights, ensuring that the model gives greater attention to these complex entities during the learning process.

[0149] The process of obtaining the text to be tested involves extracting named entities from the trained nested named entity extraction model, specifically as follows:

[0150] Obtain the test text. Based on the test text, use the trained nested named entity extraction model to obtain a set of category prediction matrices for the test text. Extract the predicted left boundaries of entities from the transposed category prediction matrices of the test text. Based on the span information of the category prediction matrices of the test text, obtain the right boundaries of entities. right The extracted entity left boundary, entity right boundary, and category C j Combine these elements to determine the nested entities and their categories:

[0151] entity = {entity left entity right C j}

[0152]

[0153] entity right =entity left +l'-1

[0154] Where f is a conditional judgment function that extracts the left boundary of entities with a confidence level greater than 0.5; The category information C is represented in the category prediction matrix of the transposed text. j The row; l' represents the span information of the prediction matrix of the text to be tested; C j This represents the category information of the category prediction matrix for the text to be tested.

[0155] In this embodiment, the process of extracting nested entities is achieved by transposing the category prediction matrix of the text to be tested, extracting the left boundary of the entity, and locating the right boundary based on the span matrix.

[0156] First, the category prediction matrix of the given text to be tested has dimensions (n', C), where: n' represents the predicted value of each character in the sequence; C represents the number of entity categories.

[0157] First, the category prediction matrix of the text to be tested is transposed. Before transposition, the dimension of the category prediction matrix of the text to be tested is (n', C), and after transposition, it becomes (C, n').

[0158] The transposed class prediction matrix of the test text is such that each row corresponds to a prediction sequence for a class, and each column corresponds to a character in the sequence.

[0159] Next, nested entities are extracted from the category prediction matrix of the transposed text to be tested. Specifically, the head position (i.e., the left boundary of the entity) of the corresponding category is extracted from each row of the matrix, with an extraction threshold of 0.5. Each row corresponds to a sequence of IDs for a category, representing the prediction score of each character in that category.

Claims

1. A method for nested named entity extraction in parallel tasks of ancient text sequences, characterized in that, include: Obtain the ancient text and annotate the named entity information of each character in the ancient text to obtain the annotated dataset A1; Based on the labeled dataset A1, a corresponding category matrix is ​​constructed for each character according to the different span distributions of entities, forming a labeled dataset A2 containing span information; The text input sequence in the labeled dataset A2 is encoded using the BERT model, and the semantic vectors of each character are extracted. A nested named entity extraction model for a sequence parallel task is constructed, and the semantic vector of each character is used as the input feature vector to train the nested named entity extraction model for the sequence parallel task, so as to obtain the trained nested named entity extraction model. Obtain the text to be tested. Based on the text, use the trained nested named entity extraction model to determine the nested entities and their category information.

2. The nested named entity extraction method for parallel tasks of ancient text sequences according to claim 1, characterized in that, Based on the labeled dataset A1, a category matrix is ​​constructed for each character according to the different span distributions of entities, forming a labeled dataset A2 containing span information, specifically: Based on the labeled dataset A1, obtain the text input sequence and entity category label set, and extract the feature triples of each entity in the labeled dataset A1 based on the text input sequence and entity category label set: e j ={s j ,c j ,l j } l j =t j -s j +1 Among them, e j For the j-th entity; s j For entity e j The starting position; c j For entity e j Category labels; j For entity e j The span; t j This is the end position of the entity; Based on the span of each entity l j Initialize the corresponding all-zero matrix M for each different span value. l : M l =0 n ×C Where, matrix M l The dimension is [n, C]; 0 n The vector is composed of all zeros; C is the number of category labels; n is the length of the text input sequence. For each triple {s j ,c j ,l j }, respectively based on the span l of the entity j The values ​​will fill the zero matrix M with the entity's starting position and category label, corresponding to the span value. l In this process, the category matrix is ​​obtained. The category matrices of all entities are integrated to form a labeled dataset A2; the labeled dataset A2 includes the text input sequence and the category matrix of each entity.

3. The nested named entity extraction method for parallel tasks of ancient text sequences according to claim 1, characterized in that, The text input sequence in the labeled dataset A2 is encoded using the BERT model, and the semantic vector of each character is extracted. Specifically: The text input sequence X in the labeled dataset A2 and the vocabulary V of the BERT model. BERT By comparing and filtering, character sets X not included in the BERT model were identified. UNK : Where, x i Input the i-th character in the text sequence; Character set X UNK The vocabulary V added to the BERT model BERT In the process, an expanded thesaurus is generated. Based on the extended vocabulary, the BERT model is used to encode the text input sequence X in the labeled dataset A2, and the semantic vector E of each character is extracted: E = BERT(X).

4. The nested named entity extraction method for parallel tasks of ancient text sequences according to claim 1, characterized in that, The nested named entity extraction model includes a probabilistic-semantic fusion network, a parallel convolutional network, a prior knowledge module, a semantic vector fusion layer, a multilayer perceptron, and a classification layer. The probabilistic-semantic fusion network is used to perform a linear transformation on the input feature vector E to generate a class probability distribution p for each input position; the class probability distribution p is then compared with the transpose p of p. Τ Perform matrix operations to obtain the category distribution similarity probability matrix P; transpose the input feature vector E to obtain Ei. Τ and E Τ Performing matrix operations with P yields semantic vectors rich in probabilistic dependencies. The expression for the category probability distribution p is: Where e is the natural constant; k is the feature index; d is the feature set; W pred (c,k) represents the weights of the c-th category and the k-th feature; E(i,k) represents the value of the k-th feature of the i-th character in the input feature vector E; b pred (c) represents the bias for the c-th category; The parallel convolutional network is used to utilize several CNN convolution pairs with different kernel sizes. Parallel convolution operations are performed to obtain a set of semantic vectors ε containing a specific span l; the size of the convolution kernel is the span of the entity corresponding to the current CNN convolution. The prior knowledge network is used to obtain the span information set S. l : S l ={S (1) ,S (2) ,...,S (L) } S=W S ·E+b S Where S is the span matrix generated by the prior knowledge network; S (1) This refers to the span information for spans with a span value of 1 after splitting the span matrix S; S (2) This refers to the span information for a span value of 2 after splitting the span matrix S; S (L) This refers to the span information of the span matrix S after splitting it into span values ​​L; W S To generate the weight matrix for the span matrix S; b S The bias is used to generate the span matrix S; L is the maximum entity span value; The semantic vector fusion layer is used to weight and fuse the semantic vector set ε and the input feature vector E based on the span value to obtain the fused feature matrix F; the semantic vector set ε and the fused feature matrix F are then integrated to obtain the final word vector set X. X={X (1) ,X (2) ,...,X (L) } Among them, X (1) For the final word vector with a span value of 1; X (2) For the final word vector with a span value of 2; X (L) This is the final word vector for a span value of L; The multilayer perceptron is used to process the span information set S l The final fused feature vector set X is obtained by concatenating the final word vector set X with the word vector set X based on the span value. input ; and based on the final fused feature vector set X input The feature representation set A is obtained through a multilayer perceptron: A={A (1) ,A (2) ,...,A (L) } Among them, A (1) A represents the final fused feature vector processed by a multilayer perceptron for a span value of 1; (2) For the final fused feature vector with a span of 2 after processing by a multilayer perceptron; A (L) This represents the final fused feature vector after processing by a multilayer perceptron for a span value of L; MLP stands for Multilayer Perceptron; X input This is the final set of fused feature vectors; For S (1) and X (1) The final fused feature vector after concatenation; For S (2) and X (2) The final fused feature vector after concatenation; For S (L) and X (L) The final fused feature vector after concatenation; The classification layer is used to perform category prediction based on the feature representation set A, resulting in a category prediction matrix set P: P = {P (1) ,P (2) ,...,P (L) } P (L) =sigmoid(logits (L) ) logits (L) =A (L) ·W out +b out Among them, P (1) P is the category prediction matrix for entities with a span value of 1; (2) P is the category prediction matrix for entities with a span of 2; (L) Here is the class prediction matrix for entities with a span value of L; sigmoid is the activation function; logits (L) The classification score for entities with a span value of L; W out To output the weight matrix; b out This is a bias term.

5. The nested named entity extraction method for parallel tasks of ancient text sequences according to claim 4, characterized in that, The semantic vector rich in probability dependencies The expression is: Among them, E Τ is the transpose of the input feature vector; P is the category distribution similarity probability matrix; T is the transpose.

6. The nested named entity extraction method for parallel tasks of ancient text sequences according to claim 4, characterized in that, The expression for the semantic vector set ε is: ε={E (1) ,AND (2) ,...,AND (L) } AND (L) =Conv L (AND) Among them, E (1) E represents the semantic vector after convolution by a CNN with a kernel of 1. (2) E represents the semantic vector after convolution by a CNN with a kernel of 2. (L) The semantic vector is generated after convolution by a CNN with kernel L; Conv L This is a CNN convolution with kernel L.

7. The nested named entity extraction method for parallel tasks of ancient text sequences according to claim 4, characterized in that, The expression for the fused feature matrix F is: Where F(v,a) is the element in the v-th row and a-th column of the fused feature matrix F; k is the feature index; d is the feature set; E (l) (v,k) represents the k-th dimension feature at the v-th position in the semantic vector after convolution with a CNN kernel of size l; E T (k,a) represents the k-th dimension feature at the a-th position in the input feature vector E; l is the span value index.

8. The nested named entity extraction method for parallel tasks of ancient text sequences according to claim 4, characterized in that, The expression for the final word vector is: Among them, X (l) (v,o) is the final word vector X with a span value of l. (l) The element in the v-th row and o-th column of the fusion feature matrix F; n is the length of the text input sequence; g is the entity index; F(v,a) is the element in the v-th row and a-th column of the fusion feature matrix F; E (l) (a,o) is the semantic vector E after convolution by a CNN with kernel l. (l) The o-th dimension feature at position a; l is the span value index.

9. The nested named entity extraction method for parallel tasks of ancient text sequences according to claim 1, characterized in that, The loss function L of the nested named entity extraction model is: in, The loss for entities with a span value of 1; The loss for an entity with a span of 2; The loss for an entity with a span value of L; for The weight of ; C is the number of category labels; c is the category label index; y c This represents the value of class c in the actual label vector; denoted as , where is the predicted probability of the model for class c; T is the number of positive samples; and N is the number of negative samples.

10. The nested named entity extraction method for parallel tasks of ancient text sequences according to claim 1, characterized in that, The process of obtaining the text to be tested involves extracting named entities from the trained nested named entity extraction model, specifically as follows: Obtain the test text. Based on the test text, use the trained nested named entity extraction model to obtain a set of category prediction matrices for the test text. Extract the predicted left boundaries of entities from the transposed category prediction matrices of the test text. Based on the span information of the category prediction matrices of the test text, obtain the right boundaries of entities. right The extracted entity's left boundary, entity's right boundary, and category C are then analyzed. j Combine these elements to determine the nested entities and their categories: entity={entity left ,entity right ,C j } entity right =entity left +l'-1 Where f is a conditional judgment function that extracts the left boundary of entities with a confidence level greater than 0.5; The category information C is represented in the category prediction matrix of the transposed text. j The row; l' represents the span information of the prediction matrix for the text to be tested; C j This represents the category information of the category prediction matrix for the text to be tested.

Images