Nested teaching named entity recognition method based on span

Abstract

The invention discloses a nested teaching named entity recognition method based on span, and relates to the crossing field of artificial intelligence and intelligent education. Firstly, a method for fusing BERT hidden representation and part-of-speech tagging of sentences is provided, so that coding of a BERT language model and part-of-speech features of the sentences can be considered at the same time in nested teaching named entity recognition. And secondly, a Bi-LSTM network is used on a BERT language model to refine semantics so as to obtain finer context features, and left and right adjacent semantics are fused through a CNNs convolutional neural network in segment endpoint classification so as to improve the precision of segment endpoint classification. And finally, three sub-tasks of training section endpoint classification, adjacent section endpoint position relation classification and candidate teaching entity span teaching entity classification are combined to improve the generalization ability of the model and solve the nested teaching named entity identification problem in a more effective manner.

Description

Technical Field

[0001] This invention relates to the intersection of artificial intelligence and smart education, specifically a span-based nested instructional named entity recognition method that can be applied to intelligent teaching systems across various disciplines. Background Technology

[0002] Intelligent teaching systems require the processing of large amounts of text. For example, the organization of teaching content, automatic test questions, and automatic grading all rely heavily on natural language processing (NLP), and named entity recognition (NER) is a primary step in NLP. However, instructional named entities differ fundamentally from general named entities. General named entities refer to names of people, organizations, places, and all other entities identified by names. In contrast, instructional named entities primarily refer to named entities related to the teaching process, including concept names, attribute names, combinations of concept and attribute names, relational terms, and attribute values ​​and attribute text content. Therefore, instructional named entity recognition methods differ from general named entity recognition in processing methods and model structures. This invention proposes a span-based nested instructional named entity recognition method to address the problem of instructional named entity recognition more effectively. Summary of the Invention

[0003] This invention discloses a nested instructional named entity recognition method based on span, characterized by the following steps:

[0004] S1. A sentence to be identified containing nested teaching named entities is fed into the BERT language model for processing to obtain the BERT hidden representation of the sentence to be identified. At the same time, the sentence to be identified is fed into the language processing software Stanford CoreNLP for processing to obtain the part-of-speech tag sequence of the sentence to be identified, and it is converted into a part-of-speech embedding vector sequence. Then, the hidden states in the BERT hidden representation are concatenated with the corresponding part-of-speech embedding vectors and fed into a scaled dot product multi-head self-attention mechanism for semantic fusion to obtain the part-of-speech-aware sentence hidden representation.

[0005] S2. The part-of-speech-aware sentence hidden representation is fed into a Bi-LSTM bidirectional long short-term memory network for semantic refinement, resulting in a semantically refined sentence hidden representation;

[0006] S3. Feed the semantically refined sentence hidden representation into a CNN convolutional neural network to extract the semantics of the left and right adjacent elements, and obtain the sentence hidden representation with enhanced adjacent semantics;

[0007] S4. The hidden representations of adjacent semantically enhanced sentences are fed into the segment endpoint classification layer for classification to obtain the segment endpoint annotation sequence of the sentence to be identified;

[0008] S5. For all adjacent segment endpoints, connect the hidden states of the adjacent segment endpoints in the semantically refined sentence hidden representation, and send them to the positional relationship classification layer for classification to obtain the predicted positional relationship between all adjacent segment endpoints;

[0009] S6. Based on the predicted positional relationship of the endpoints of adjacent segments, analyze whether any two endpoints of adjacent segments exist in the same outermost segment, and automatically generate the span of all nested candidate teaching entities in the sentence to be identified based on the analysis results;

[0010] S7. Perform average pooling on the hidden states of all candidate teaching entity spans, and then connect them with the hidden states of segment endpoints through a gating mechanism to form the hidden representation of all candidate teaching entity spans.

[0011] S8. The hidden representations of the span of all candidate teaching entities are fed into the teaching entity classification layer for classification to obtain all nested teaching named entities in the sentence to be identified;

[0012] The BERT language model refers to the Bidirectional Encoder Representations from Transformers language model proposed by Google AI Language, and the Stanford CoreNLP language processing software refers to the natural language processing toolkit developed by Manning et al. at Stanford University.

[0013] Furthermore, step S1 specifically includes:

[0014] S1.1 feeds the sentence S to be recognized, which contains nested instructional named entities, into the BERT language model to obtain the BERT hidden representation H of the sentence to be recognized. b The calculation process is as follows:

[0015]

[0016] Where BERT(·) represents the BERT language model used. Let n be the hidden state sequence output by the BERT language model, and n be the sub-words {s1, s2, ..., sn} segmented from the sentence S to be recognized in the BERT language model. n The number of}, d b R represents the number of hidden units in the BERT language model, and R is the set of real numbers.

[0017] S1.2 The sentence S to be identified is fed into the language processing software Stanford CoreNLP for processing to obtain the part-of-speech tag sequence U of S. The calculation process is as follows:

[0018] U = CoreNLP(S) = {u1, u2, ..., u} m} (2)

[0019] Where CoreNLP(·) represents the language processing software used, Stanford CoreNLP, {u1,u2,…,u m} represents the part-of-speech tag of S in Stanford CoreNLP, and m represents the number of words containing S in Stanford CoreNLP;

[0020] Then, the words corresponding to the part-of-speech tag sequence U are aligned according to the subwords of S in the BERT language model, and the part-of-speech tag sequence U is expanded from the number m to the number n U′. The calculation process is as follows:

[0021] U′=Transform(Align(token(U),token(H) b )))={u′1,u′2,…,u′ n} (3)

[0022] Where U′ is the expanded part-of-speech tag sequence, Align(token(U), token(H) b )) indicates that token(U) and token(H) are... b Alignment is performed for token(U) and token(H). b The table represents the word sequence corresponding to U and H. b The corresponding sub-word sequence, {u′1,u′2,…,u′ n} represents the expanded part-of-speech tags. Transform(·) means expanding the part-of-speech tag sequence U from a quantity m to a quantity n U′ according to the following rules:

[0023] (1) For any u i ∈U, i=1,2,…,n, if u i The corresponding word is in token(H) b If it is not decomposed in U′, then the corresponding u′ in U′ is... i =u i ;

[0024] (2) For any u i ∈U, i=1,2,…,n, if u i The corresponding word is in token(H) b It is decomposed into n in ) l Each sub-word, then the corresponding u′ in U′ i Expand to n l There are part-of-speech tags, and there is u′ i,1 =u i+"-B",u′ i,j =u i +"-I", j = 2, 3, ..., n l The "-B" indicates the beginning position of a part-of-speech tag, and "-I" indicates the middle position of a part-of-speech tag. i,j Represents u′ i The j-th word attribute tag in the text;

[0025] S1.3 converts U′ into a part-of-speech embedding vector sequence V, and the calculation process is as follows:

[0026]

[0027] Among them, E POS (U′) represents the conversion of the part-of-speech tag sequence U′ into the corresponding part-of-speech embedding vector sequence {v1,v,…,v} through a table lookup operation. n} is E POS (U′) The result of the table lookup operation, d u The dimension of the part-of-speech embedding vector;

[0028] S1.4 hides the BERT representation H b The hidden states in V are concatenated with the corresponding part-of-speech embedding vectors in V to obtain the combined hidden representation H. c The calculation process is as follows:

[0029]

[0030] Where [;] represents a join operation;

[0031] S1.5 will hide the combined representation H c The semantic fusion is performed by feeding the data into a scaled dot product multi-head self-attention mechanism to obtain the part-of-speech-aware sentence hidden representation H. u The calculation process is as follows:

[0032]

[0033] Among them, MHSA(H c ) indicates that the input X = H c The scaling dot product multi-head self-attention mechanism MHSA(X) is calculated as follows:

[0034] MHSA(X)=tanh([head1;head2;...;head h ]w R (7)

[0035] head i =Attention(X) i ,Xi ,X i =Attention(Xw) h ,Xw h ,Xw h (8)

[0036]

[0037] Where, X∈R n×d The input to the multi-head self-attention mechanism MHSA(·) is given by d, where d is the vector dimension of the sequence in the input X, and head is the number of heads. i Let represent the i-th head in multi-head self-attention, i = 1, 2, ..., h, and tanh(·) denote the hyperbolic tangent function. w R ∈R d×d It is a learnable parameter matrix, d g = d ÷ h, where h is the number of attention heads in multi-head self-attention, X i Let X represent the input in the i-th multi-head self-attention. i T The superscript T in the code represents the matrix transpose operation, and Softmax(·) represents the normalized exponential function in the neural network.

[0038] Furthermore, step S2 specifically includes:

[0039]

[0040]

[0041]

[0042] in, This represents a time sequence H processed from left to right. u LSTM (Long Short-Term Memory) network, This represents a right-to-left timing process H. u LSTM networks, express Network output, for Hidden states in express Network output, for The hidden state in H L Sentence hiding representation that indicates semantic refinement express and The connection result, d L It represents the number of hidden units in the LSTM network.

[0043] Furthermore, step S3 specifically includes:

[0044] S3.1 Sentence Hiding Representation H in Semantic Refinement L Insert a vector of all zeros at the start and end positions respectively. H L Expand to As shown below:

[0045]

[0046] S3.2 will The data is fed into a CNN (Convolutional Neural Network) with a kernel size of 3 to extract adjacent semantics, thus obtaining the sentence hidden representation H of adjacent semantics. a The calculation process is as follows:

[0047]

[0048] Where CNNs(·) represents the convolutional neural network used. d represents the output hidden state sequence of a CNN network. a This indicates the number of convolutional kernels in a CNN network;

[0049] S3.3 Hiding the semantically refined sentence representation H L Sentence hidden representation H with adjacent semantics a By connecting them, we obtain the semantically enhanced hidden representation H of adjacent sentences. e The calculation process is as follows:

[0050]

[0051] in, For H L With H a The connection result.

[0052] Furthermore, step S4 specifically includes:

[0053] S4.1 Place the i-th subword of S in H e The corresponding hidden state The data is fed into the segment endpoint classification layer to obtain the probability distribution of whether the i-th word is a segment endpoint. The calculation process is as follows:

[0054]

[0055]

[0056] in, Let i be the confidence vector for whether the i-th word is a segment endpoint. b is the weight matrix in the segment endpoint classification layer. d ∈R |D| Let D be the bias vector in the segment endpoint classification layer, where D = {"1", "0"}, and let |D| be the size of D.

[0057] S4.2 Probability distribution of segment endpoints The probability that the segment endpoint label of the i-th word is "1" The probability that the segment endpoint label is "0" is greater than its value. Then the i-th word in sentence S is a segment endpoint; otherwise, the i-th word in sentence S is not a segment endpoint.

[0058] S4.3 Repeat steps S4.1 and S4.2, with i taking values ​​of 1, 2, ..., n, until segment endpoints of all words in sentence S are labeled, resulting in the segment endpoint label sequence of S.

[0059] Furthermore, step S5 specifically includes:

[0060] S5.1 The sequence of segment endpoints from sentence S. In the sentence S, find all segment endpoints to form a segment endpoint sequence. n d The number of endpoints in the middle segment of sentence S;

[0061] S5.2 Let S d Any two adjacent segment endpoints The corresponding subwords in sentence S are respectively Then and Sentence Hidden Representation H in Semantic Refinement L Hidden state in Connect them, and send them to the positional relationship classification layer for classification to obtain the endpoints of adjacent segments. Probability distribution of the positional relationship between them and the endpoints of adjacent segments Predicted positional relationship between The calculation process is as follows:

[0062]

[0063]

[0064]

[0065] in, endpoints of adjacent segments The confidence vector of the positional relationship between them. b is the weight matrix in the positional relationship classification layer. r ∈R |Ω| Let Ω be the bias vector in the positional relationship classification layer, Ω = {"connection", "tangency", "separation", "connection_tangency_separation", "tangency_separation", "connection_separation", "connection_tangency"}, where |Ω| is the size of Ω. It is a positional relationship in Ω. express The positional relationship is The probability, Expressing the requirement to make The maximum value

[0066] S5.3 Repeat step S5.2, with i taking the values ​​1, 2, ..., n respectively. d -1, until the segment endpoint sequence S d The positional relationships between the endpoints of all adjacent segments have been predicted.

[0067] Furthermore, step S6 specifically includes:

[0068] S6.1 Find all the outermost segments in sentence S according to the following rules to form the outermost segment sequence S. w :

[0069] (1) If there is a "tangent-separate" and "separate" relationship between the endpoints of two adjacent segments, then the endpoints of these two adjacent segments do not exist in the same outermost segment;

[0070] (2) If there is a "connection", "tangency", "connection-tangency", "connection-separation" or "connection-tangency-separation" relationship between the endpoints of two adjacent segments, then the endpoints of these two adjacent segments exist in the same outermost segment;

[0071] The outermost segment refers to a span that contains at least one teaching named entity in a sentence and can exist independently without being embedded in other segments; the span refers to a sentence segment.

[0072] S6.2 For the outermost segment sequence S w Any outermost segment Find according to the following rules Span of all candidate teaching entities:

[0073] (1) Outermost segment The sub-words corresponding to all segment endpoints are candidate teaching entity spans;

[0074] (2) The span between any two segment endpoints is a candidate teaching entity span, including the two segment endpoints themselves.

[0075] S6.3 Repeat step S6.2 until the outermost segment sequence S is found. w The spans of all candidate teaching entities in all outermost segments are used to form the set of candidate teaching entity spans for sentence S. n k This represents the number of candidate teaching entity spans in sentence S.

[0076] Furthermore, step S7 specifically includes:

[0077] S7.1 Let S k span of any candidate teaching entity The two segment endpoints correspond to the following subwords in sentence S: Then and Sub-words between words in semantically refined sentence hidden representation H L The hidden states in the middle are averaged to obtain and Sub-word hiding representation The calculation process is as follows:

[0078]

[0079] in, For H L The i-th hidden state;

[0080] S7.2 will In H L Hidden state in as well as Connecting them together yields the span of candidate teaching entities. Hidden representation A j The calculation process is as follows:

[0081]

[0082] S7.3 will A j The data is fed into a gating mechanism to obtain the span of candidate teaching entities. gating hidden representation The calculation process is as follows:

[0083]

[0084] in, b is the weight of the gating mechanism. g Here, σ(·) represents the bias of the gating mechanism, σ(·) represents the sigmoid activation function in the neural network, and ⊙ represents pointwise multiplication;

[0085] S7.4 Repeat steps S7.1 to S7.3, with j taking values ​​of 1, 2, ..., n respectively. k until S is calculated k Gated hidden representation of the span of all candidate teaching entities.

[0086] Furthermore, step S8 specifically includes:

[0087] S8.1 will S k span of any candidate teaching entity gating hidden representation The data is fed into the teaching entity classification layer for classification, resulting in the candidate teaching entity span. Probability distribution of teaching entity classification The calculation process is as follows:

[0088]

[0089]

[0090]

[0091] in, For candidate teaching entity span The confidence vector for classifying teaching entities. b is the weight matrix in the teaching entity classification layer. k ∈R |Ψ| Let Ψ be the bias vector in the teaching entity classification layer, where Ψ = {"concept name", "attribute name", "relation word", "attribute value", "content object", "non-teaching entity"}, and is the set of labels for teaching entity categories, where |Ψ| is the size of Ψ. Indicates the span of candidate teaching entities For teaching entity categories The probability, function Indicate the search in Ψ such that The teaching entity category with the maximum value For candidate teaching entity span Predicted labels for the classification of teaching entities;

[0092] S8.2 Repeat step S8.1, with j taking values ​​of 1, 2, ..., n respectively. k until S is calculated k Predicted labels for the teaching entity classification of all candidate teaching entity spans;

[0093] S8.3 will S kThe candidate teaching entities whose predicted labels are not "non-teaching entities" are selected from all teaching entity categories to form the nested teaching named entity set in the sentence S to be solved.

[0094] Furthermore, the final loss calculation of the established neural network model during the training process includes the following steps:

[0095] (1) The segment endpoint loss generated by sentence S in the segment endpoint classification process is calculated using the following formula.

[0096]

[0097] Where θ represents all the parameters of the established neural network model. Let the true segment endpoint label be the i-th word in sentence S. The segment endpoint label representing the i-th word in sentence S is... The predicted probability, To predict the segment endpoint probability distribution of the i-th word in sentence S, Calculate using formula (17);

[0098] (2) The positional relationship loss of sentence S during the classification of the positional relationship between the endpoints of adjacent segments is calculated using the following formula.

[0099]

[0100] in, Indicates the endpoints of adjacent segments The true label of the positional relationship between them Indicates the endpoints of adjacent segments The positional relationship labels between them are The predicted probability, To predict the endpoints of adjacent segments in sentence S The probability distribution of the positional relationship between them. Calculate using formula (19);

[0101] (3) The teaching entity classification loss generated by sentence S in the teaching entity classification process of candidate teaching entity span is calculated using the following formula.

[0102]

[0103] in, For the j-th candidate teaching entity span Real teaching entity classification tags, The teaching entity classification label representing the span of the j-th candidate teaching entity is: The predicted probability, To predict the span of the j-th candidate teaching entity in sentence S The probability distribution of the teaching entity classification. Calculate using formula (25);

[0104] (4) Calculate the final loss of the established neural network model during the training process using the following joint training formula.

[0105]

[0106] Where λ and β are learnable hyperparameters, and 0 < λ < 1, 0 < β < 1.

[0107] The present invention has the following advantages:

[0108] (1) A method is proposed to integrate the BERT hidden representation of a sentence with part-of-speech tagging, so that the encoding of the BERT language model and the part-of-speech features of the sentence can be considered simultaneously in the nested instructional named entity recognition.

[0109] (2) The Bi-LSTM network was used on top of the BERT language model to refine the semantics, which can obtain more delicate contextual features;

[0110] (3) In segment endpoint classification, the semantics of left and right adjacency are incorporated through CNNs convolutional neural networks, which further improves the accuracy of segment endpoint classification;

[0111] (4) Based on the nesting characteristics of teaching named entities, it is proposed that the endpoints of adjacent segments with a "tangent" positional relationship be classified as existing in the same outermost segment, rather than not existing in the same outermost segment in general named entity recognition, thus better conforming to the actual situation of nested teaching named entity recognition.

[0112] (5) Generating candidate teaching entity spans only within the outermost segment can effectively reduce the number of candidate teaching entity spans and improve computational efficiency. Attached Figure Description

[0113] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0114] Figure 2 This is a nested teaching example diagram of named entity recognition. Detailed Implementation

[0115] The present invention will be further described below with reference to specific embodiments, but the scope of protection of the present invention is not limited to the following embodiments.

[0116] For a sentence S to be recognized that contains nested instructional named entities, according to Figure 1 The flowchart of the method of the present invention shown below identifies nested teaching naming instances in sentence S through the following steps:

[0117] S1. The sentence to be identified is fed into the BERT language model for processing to obtain the BERT hidden representation of the sentence to be identified. At the same time, the sentence to be identified is fed into the language processing software Stanford CoreNLP for processing to obtain the part-of-speech tag sequence of the sentence to be identified, and it is converted into a part-of-speech embedding vector sequence. Then, the hidden state in the BERT hidden representation is concatenated with the corresponding part-of-speech embedding vector, and fed into a scaling dot product multi-head self-attention mechanism for semantic fusion to obtain the part-of-speech-aware sentence hidden representation.

[0118] S2. The part-of-speech-aware sentence hidden representation is fed into a Bi-LSTM bidirectional long short-term memory network for semantic refinement, resulting in a semantically refined sentence hidden representation;

[0119] S3. Feed the semantically refined sentence hidden representation into a CNN convolutional neural network to extract the semantics of the left and right adjacent elements, and obtain the sentence hidden representation with enhanced adjacent semantics;

[0120] S4. The hidden representations of adjacent semantically enhanced sentences are fed into the segment endpoint classification layer for classification to obtain the segment endpoint annotation sequence of the sentence to be identified;

[0121] S5. For all adjacent segment endpoints, connect the hidden states of the adjacent segment endpoints in the semantically refined sentence hidden representation, and send them to the positional relationship classification layer for classification to obtain the predicted positional relationship between all adjacent segment endpoints;

[0122] S6. Based on the predicted positional relationship of the endpoints of adjacent segments, analyze whether any two endpoints of adjacent segments exist in the same outermost segment, and automatically generate the span of all nested candidate teaching entities in the sentence to be identified based on the analysis results;

[0123] S7. Perform average pooling on the hidden states of all candidate teaching entity spans, and then connect them with the hidden states of segment endpoints through a gating mechanism to form the hidden representation of all candidate teaching entity spans.

[0124] S8. The hidden representations of the span of all candidate teaching entities are fed into the teaching entity classification layer for classification to obtain all nested teaching named entities in the sentence to be identified;

[0125] The BERT language model refers to the Bidirectional Encoder Representations from Transformers language model proposed by Google AI Language, and the Stanford CoreNLP language processing software refers to the natural language processing toolkit developed by Manning et al. at Stanford University.

[0126] Furthermore, step S1 specifically includes:

[0127] S1.1 feeds the sentence S to be recognized, which contains nested instructional named entities, into the BERT language model to obtain the BERT hidden representation H of the sentence to be recognized. b The calculation process is as follows:

[0128]

[0129] Where BERT(·) represents the BERT language model used. Let n be the hidden state sequence output by the BERT language model, and n be the sub-words {s1, s2, ..., sn} segmented from the sentence S to be recognized in the BERT language model. n The number of}, d b R represents the number of hidden units in the BERT language model, and R is the set of real numbers.

[0130] S1.2 The sentence S to be identified is fed into the language processing software Stanford CoreNLP for processing to obtain the part-of-speech tag sequence U of S. The calculation process is as follows:

[0131] U = CoreNLP(S) = {u1, u2, ..., u} m} (2)

[0132] Where CoreNLP(·) represents the language processing software used, Stanford CoreNLP, {u1,u2,…,u m} represents the part-of-speech tag of S in Stanford CoreNLP, and m represents the number of words containing S in Stanford CoreNLP;

[0133] Then, the words corresponding to the part-of-speech tag sequence U are aligned according to the subwords of S in the BERT language model, and the part-of-speech tag sequence U is expanded from the number m to the number n U′. The calculation process is as follows:

[0134] U′=Transform(Align(token(U),token(H) b )))={u′1,u′2,…,u′ n} (3)

[0135] Where U′ is the expanded part-of-speech tag sequence, Align(token(U), token(H) b )) indicates that token(U) and token(H) are... b Alignment is performed for token(U) and token(H). b The table represents the word sequence corresponding to U and H. b The corresponding sub-word sequence, {u′1,u′2,…,u′ n} represents the expanded part-of-speech tags. Transform(·) means expanding the part-of-speech tag sequence U from a quantity m to a quantity n U′ according to the following rules:

[0136] (1) For any u i ∈U, i=1,2,…,n, if u i The corresponding word is in token(H) b If it is not decomposed in U′, then the corresponding u′ in U′ is... i =u i ;

[0137] (2) For any u i ∈U, i=1,2,…,n, if u i The corresponding word is in token(H) b It is decomposed into n in ) l Each sub-word, then the corresponding u′ in U′ i Expand to n l There are part-of-speech tags, and there is u′ i,1 =u i +"-B",u′ i,j =u i +"-I", j = 2, 3, ..., n l The "-B" indicates the beginning position of a part-of-speech tag, and "-I" indicates the middle position of a part-of-speech tag. i,j Represents u′ i The j-th word attribute tag in the text;

[0138] S1.3 converts U′ into a part-of-speech embedding vector sequence V, and the calculation process is as follows:

[0139]

[0140] Among them, E POS (U′) represents the conversion of the part-of-speech tag sequence U′ into the corresponding part-of-speech embedding vector sequence {v1,v,…,v} through a table lookup operation. n} is E POS (U′) The result of the table lookup operation, d uThe dimension of the part-of-speech embedding vector;

[0141] S1.4 hides the BERT representation H b The hidden states in V are concatenated with the corresponding part-of-speech embedding vectors in V to obtain the combined hidden representation H. c The calculation process is as follows:

[0142]

[0143] Where [;] represents a join operation;

[0144] S1.5 will hide the combined representation H c The semantic fusion is performed by feeding the data into a scaled dot product multi-head self-attention mechanism to obtain the part-of-speech-aware sentence hidden representation H. u The calculation process is as follows:

[0145]

[0146] Among them, MHSA(H c ) indicates that the input X = H c The scaling dot product multi-head self-attention mechanism MHSA(X) is calculated as follows:

[0147] MHSA(X)=tanh([head1;head2;...;head h ]w R (7)

[0148] head i =Attention(X) i ,X i ,X i =Attention(Xw) h ,Xw h ,Xw h (8)

[0149]

[0150] Where, X∈R n×d The input to the multi-head self-attention mechanism MHSA(·) is given by d, where d is the vector dimension of the sequence in the input X, and head is the number of heads. i Let represent the i-th head in multi-head self-attention, i = 1, 2, ..., h, and tanh(·) denote the hyperbolic tangent function. w R ∈R d×d It is a learnable parameter matrix, d g = d ÷ h, where h is the number of attention heads in multi-head self-attention, X i Let X represent the input in the i-th multi-head self-attention.i T The superscript T in the code represents the matrix transpose operation, and Softmax(·) represents the normalized exponential function in the neural network.

[0151] Furthermore, step S2 specifically includes:

[0152]

[0153]

[0154]

[0155] in, This represents a time sequence H processed from left to right. u LSTM (Long Short-Term Memory) network,

[0156] This represents a right-to-left timing process H. u LSTM networks, express Network output, for Hidden states in express Network output, for The hidden state in H L Sentence hiding representation that indicates semantic refinement express and The connection result, d L It represents the number of hidden units in the LSTM network.

[0157] Furthermore, step S3 specifically includes:

[0158] S3.1 Sentence Hiding Representation H in Semantic Refinement L Insert a vector of all zeros at the start and end positions respectively. H L Expand to As shown below:

[0159]

[0160] S3.2 will The data is fed into a CNN (Convolutional Neural Network) with a kernel size of 3 to extract adjacent semantics, thus obtaining the sentence hidden representation H of adjacent semantics. a The calculation process is as follows:

[0161]

[0162] Where CNNs(·) represents the convolutional neural network used. d represents the output hidden state sequence of a CNN network. a This indicates the number of convolutional kernels in a CNN network;

[0163] S3.3 Hiding the semantically refined sentence representation H L Sentence hidden representation H with adjacent semantics a By connecting them, we obtain the semantically enhanced hidden representation H of adjacent sentences. e The calculation process is as follows:

[0164]

[0165] in, For H L With H a The connection result.

[0166] Furthermore, step S4 specifically includes:

[0167] S4.1 Place the i-th subword of S in H e The corresponding hidden state The data is fed into the segment endpoint classification layer to obtain the probability distribution of whether the i-th word is a segment endpoint. The calculation process is as follows:

[0168]

[0169]

[0170] in, Let i be the confidence vector for whether the i-th word is a segment endpoint. b is the weight matrix in the segment endpoint classification layer. d ∈R |D| Let D be the bias vector in the segment endpoint classification layer, where D = {"1", "0"}, and let |D| be the size of D.

[0171] S4.2 Probability distribution of segment endpoints The probability that the segment endpoint label of the i-th word is "1" The probability that the segment endpoint label is "0" is greater than its value. Then the i-th word in sentence S is a segment endpoint; otherwise, the i-th word in sentence S is not a segment endpoint.

[0172] S4.3 Repeat steps S4.1 and S4.2, with i taking values ​​of 1, 2, ..., n, until segment endpoints of all words in sentence S are labeled, resulting in the segment endpoint label sequence of S.

[0173] Furthermore, step S5 specifically includes:

[0174] S5.1 The sequence of segment endpoints from sentence S. In the sentence S, find all segment endpoints to form a segment endpoint sequence. n d The number of endpoints in the middle segment of sentence S;

[0175] S5.2 Let S d Any two adjacent segment endpoints The corresponding subwords in sentence S are respectively Then and Sentence Hidden Representation H in Semantic Refinement L Hidden state in Connect them, and send them to the positional relationship classification layer for classification to obtain the endpoints of adjacent segments. Probability distribution of the positional relationship between them and the endpoints of adjacent segments Predicted positional relationship between The calculation process is as follows:

[0176]

[0177]

[0178]

[0179] in, endpoints of adjacent segments The confidence vector of the positional relationship between them. b is the weight matrix in the positional relationship classification layer. r ∈R |Ω| Let Ω be the bias vector in the positional relationship classification layer, Ω = {"connection", "tangency", "separation", "connection_tangency_separation", "tangency_separation", "connection_separation", "connection_tangency"}, where |Ω| is the size of Ω. It is a positional relationship in Ω. express The positional relationship is The probability, Expressing the requirement to make The maximum value

[0180] S5.3 Repeat step S5.2, with i taking the values ​​1, 2, ..., n respectively. d -1, until the segment endpoint sequence S dThe positional relationships between the endpoints of all adjacent segments have been predicted.

[0181] Furthermore, step S6 specifically includes:

[0182] S6.1 Find all the outermost segments in sentence S according to the following rules to form the outermost segment sequence S. w :

[0183] (1) If there is a "tangent-separate" and "separate" relationship between the endpoints of two adjacent segments, then the endpoints of these two adjacent segments do not exist in the same outermost segment;

[0184] (2) If there is a "connection", "tangency", "connection-tangency", "connection-separation" or "connection-tangency-separation" relationship between the endpoints of two adjacent segments, then the endpoints of these two adjacent segments exist in the same outermost segment;

[0185] The outermost segment refers to a span that contains at least one teaching named entity in a sentence and can exist independently without being embedded in other segments; the span refers to a sentence segment.

[0186] S6.2 For the outermost segment sequence S w Any outermost segment Find according to the following rules Span of all candidate teaching entities:

[0187] (1) Outermost segment The sub-words corresponding to all segment endpoints are candidate teaching entity spans;

[0188] (2) The span between any two segment endpoints is a candidate teaching entity span, including the two segment endpoints themselves.

[0189] S6.3 Repeat step S6.2 until the outermost segment sequence S is found. w The spans of all candidate teaching entities in all outermost segments are used to form the set of candidate teaching entity spans for sentence S. n k This represents the number of candidate teaching entity spans in sentence S.

[0190] Furthermore, step S7 specifically includes:

[0191] S7.1 Let S k span of any candidate teaching entity The two segment endpoints correspond to the following subwords in sentence S: Then and Sub-words between words in semantically refined sentence hidden representation H LThe hidden states in the middle are averaged to obtain and Sub-word hiding representation The calculation process is as follows:

[0192]

[0193] in, For H L The i-th hidden state;

[0194] S7.2 will In H L Hidden state in as well as Connecting them together yields the span of candidate teaching entities. Hidden representation A j The calculation process is as follows:

[0195]

[0196] S7.3 will A j The data is fed into a gating mechanism to obtain the span of candidate teaching entities. gating hidden representation The calculation process is as follows:

[0197]

[0198] in, b is the weight of the gating mechanism. g Here, σ(·) represents the bias of the gating mechanism, σ(·) represents the sigmoid activation function in the neural network, and ⊙ represents pointwise multiplication;

[0199] S7.4 Repeat steps S7.1 to S7.3, with j taking values ​​of 1, 2, ..., n respectively. k until S is calculated k Gated hidden representation of the span of all candidate teaching entities.

[0200] Furthermore, step S8 specifically includes:

[0201] S8.1 will S k span of any candidate teaching entity gating hidden representation The data is fed into the teaching entity classification layer for classification, resulting in the candidate teaching entity span. Probability distribution of teaching entity classification The calculation process is as follows:

[0202]

[0203]

[0204]

[0205] in, For candidate teaching entity span The confidence vector for classifying teaching entities. b is the weight matrix in the teaching entity classification layer. k ∈R |Ψ| Let Ψ be the bias vector in the teaching entity classification layer, where Ψ = {"concept name", "attribute name", "relation word", "attribute value", "content object", "non-teaching entity"}, and is the set of labels for teaching entity categories, where |Ψ| is the size of Ψ. Indicates the span of candidate teaching entities For teaching entity categories The probability, function Indicate the search in Ψ such that The teaching entity category with the maximum value For candidate teaching entity span Predicted labels for the classification of teaching entities;

[0206] S8.2 Repeat step S8.1, with j taking values ​​of 1, 2, ..., n respectively. k until S is calculated k Predicted labels for the teaching entity classification of all candidate teaching entity spans;

[0207] S8.3 will S k The candidate teaching entities whose predicted labels are not "non-teaching entities" are selected from all teaching entity categories to form the nested teaching named entity set in the sentence S to be solved.

[0208] Furthermore, the final loss calculation of the established neural network model during the training process includes the following steps:

[0209] (1) The segment endpoint loss generated by sentence S in the segment endpoint classification process is calculated using the following formula.

[0210]

[0211] Where θ represents all the parameters of the established neural network model. Let the true segment endpoint label be the i-th word in sentence S. The segment endpoint label representing the i-th word in sentence S is... The predicted probability, To predict the segment endpoint probability distribution of the i-th word in sentence S Calculate using formula (17);

[0212] (2) The positional relationship loss of sentence S during the classification of the positional relationship between the endpoints of adjacent segments is calculated using the following formula.

[0213]

[0214] in, Indicates the endpoints of adjacent segments The true label of the positional relationship between them Indicates the endpoints of adjacent segments The positional relationship labels between them are Predicted probability To predict the endpoints of adjacent segments in sentence S The probability distribution of the positional relationship between them. Calculate using formula (19);

[0215] (3) The teaching entity classification loss generated by sentence S in the teaching entity classification process of candidate teaching entity span is calculated using the following formula.

[0216]

[0217] in, For the j-th candidate teaching entity span Real teaching entity classification tags, The teaching entity classification label representing the span of the j-th candidate teaching entity is: The predicted probability, To predict the span of the j-th candidate teaching entity in sentence S The probability distribution of the teaching entity classification. Calculate using formula (25);

[0218] (4) Calculate the final loss of the established neural network model during the training process using the following joint training formula.

[0219]

[0220] Where λ and β are learnable hyperparameters, and 0 < λ < 1, 0 < β < 1.

[0221] Application Examples

[0222] The process of recognizing nested instructional named entities in the sentence "One of the characteristics of computer networks is resource sharing" in this embodiment is as follows:

[0223] (1) The sub-words segmented from the sentence in the BERT language model are:

[0224] S = {"Computer", "Calculation", "Machine", "Network", "of", "Special", "Point", "Of", "One", "Is", "Resource", "Share", "."}, where n = 16 in formula (1);

[0225] (2) The part-of-speech tagging of the sentence in the language processing software Stanford CoreNLP is as follows:

[0226] U = {('computer','NN'),('network','NN'),('of','DEG'),('feature','NN'),('one of','NN'),('is','VC'),('resource','NN'),('shared','VV'),('.','PU')}, where m = 9 in formula (1);

[0227] (3) Expand the part-of-speech tag sequence U from a quantity m=9 to a quantity n=16 U′:

[0228] U′={('compute','NN-B'),('calculation','NN-I'),('machine','NN-I'),('network','NN-B'),('network','NN-I'),('of','DEG'),('special','NN-B'),('point','NN-I'),('of','NN-B'),('one','NN-I'),('is','VC'),('resource','NN-B'),('source','NN-I'),('share','VV-B'),('share','VV-I'),('。','PU')}

[0229] (4) In step S4 of the inventive method, the actual labels of the sentence's segment endpoints are obtained, such as... Figure 2 As shown:

[0230] S p ={"1","0","1","1","1","0","1","1","0","0","1","1","1","0","1","0"}

[0231] Here, "1" indicates a segment endpoint, and "0" indicates that it is not a segment endpoint.

[0232] (5) In step S5 of the inventive method, the true labels of the positional relationships between adjacent endpoints of the sentence are obtained, such as... Figure 2 As shown, L represents a connection relationship, S represents a separation relationship, T represents a tangent relationship, LS represents a connection + separation relationship, and TS represents a tangent + separation relationship.

[0233] (6) According to the rules in step S6.1 of the invention method, four outermost segments are obtained, such as Figure 2 As shown.

[0234] (7) According to the rules in step S6.2 of the invention method, the following set of candidate teaching entity spans is obtained:

[0235] S k ={'computer','network','characteristics','is','resource','sharing','computer','computer network','computer network','characteristics','resource','resource sharing'}, and n k =17.

[0236] (8) In step S8 of the inventive method, the true labels of the teaching entity classification for the following candidate teaching entity span are obtained: {('Computer','N'),('Machine','N'),('Network','N'),('Network','N'),('Special','N'),('Point','N'),('Is','RW'),('Resource','N'),('Source','N'),('Share','N'),('Computer','CN'),('Computer Network','N'),('Computer Network','CN'),('Feature','PN'),('Resource','CN'),('Resource Sharing','N'),('Resource Sharing','CO')}

[0237] Where 'N' represents "non-teaching entity", 'CN' represents "concept name", 'PN' represents "attribute name", and 'RW' represents "relationship term".

Claims

1. A nested instructional named entity recognition method based on span, characterized in that... Includes the following steps: S1. A sentence to be identified containing nested teaching named entities is fed into the BERT language model for processing to obtain the BERT hidden representation of the sentence to be identified. At the same time, the sentence to be identified is fed into the language processing software Stanford CoreNLP for processing to obtain the part-of-speech tag sequence of the sentence to be identified, and it is converted into a part-of-speech embedding vector sequence. Then, the hidden states in the BERT hidden representation are concatenated with the corresponding part-of-speech embedding vectors and fed into a scaled dot product multi-head self-attention mechanism for semantic fusion to obtain the part-of-speech-aware sentence hidden representation. S2. The part-of-speech-aware sentence hidden representation is fed into a Bi-LSTM bidirectional long short-term memory network for semantic refinement to obtain a semantically refined sentence hidden representation; S3. Feed the semantically refined sentence hidden representation into a CNN convolutional neural network to extract the semantics of the left and right adjacent parts, and obtain the sentence hidden representation with enhanced adjacent semantics; S4. The hidden representations of adjacent semantically enhanced sentences are fed into the segment endpoint classification layer for classification to obtain the segment endpoint annotation sequence of the sentence to be identified; S5. For all adjacent segment endpoints, connect the hidden states of the adjacent segment endpoints in the semantically refined sentence hidden representation, and send them into the positional relationship classification layer for classification to obtain the predicted positional relationship between all adjacent segment endpoints; S6. Based on the predicted positional relationship of the endpoints of adjacent segments, analyze whether any two endpoints of adjacent segments exist in the same outermost segment, and automatically generate the span of all nested candidate teaching entities in the sentence to be identified based on the analysis results; S7. Perform average pooling on the hidden states of all candidate teaching entity spans, and then connect them with the hidden states of segment endpoints through a gating mechanism to form the hidden representation of all candidate teaching entity spans. S8. The hidden representations of the span of all candidate teaching entities are fed into the teaching entity classification layer for classification to obtain all nested teaching named entities in the sentence to be identified; The BERT language model refers to the Bidirectional Encoder Representations from Transformers language model proposed by Google AI Language, and the Stanford CoreNLP language processing software refers to the natural language processing toolkit developed by Manning et al. at Stanford University. Step S1 specifically includes: S1.1 feeds the sentence S to be recognized, which contains nested instructional named entities, into the BERT language model to obtain the BERT hidden representation H of the sentence to be recognized. b The calculation process is as follows: Where BERT(·) represents the BERT language model used. Let n be the hidden state sequence output by the BERT language model, and n be the sub-words {s1, s2, ..., sn} segmented from the sentence S to be recognized in the BERT language model. n The number of}, d b R represents the number of hidden units in the BERT language model, and R is the set of real numbers. S1.2 The sentence S to be identified is fed into the language processing software Stanford CoreNLP for processing to obtain the part-of-speech tag sequence U of S. The calculation process is as follows: U=CoreNLP(S)={u1,u2,...,u m } (2) Where CoreNLP(·) represents the language processing software used, Stanford CoreNLP, {u1, u2, ..., u m } represents the part-of-speech tag of S in Stanford CoreNLP, and m represents the number of words containing S in Stanford CoreNLP; Then, the words corresponding to the part-of-speech tag sequence U are aligned according to the subwords of S in the BERT language model, and the part-of-speech tag sequence U is expanded from the number m to the number n U′. The calculation process is as follows: U′=Transform(Align(token(U),token(H b )))={u′1,u′2,...,u′ n } (3) Where U′ is the expanded part-of-speech tag sequence, Align(token(U), token(H) b )) indicates that token(U) and token(H) are... b Alignment is performed for token(U) and token(H). b The table represents the word sequence corresponding to U and H. b The corresponding sub-word sequence, {u′1, u′2, ..., u′ n } represents the expanded part-of-speech tags. Transform(·) means expanding the part-of-speech tag sequence U from a quantity m to a quantity n U′ according to the following rules: (1) For any u i ∈U, i = 1, 2, ..., n, if u i The corresponding word is in token(H) b If it is not decomposed in U′, then the corresponding u′ in U′ is... i =u i ; (2) For any u i ∈U, i = 1, 2, ..., n, if u i The corresponding word is in token(H) b It is decomposed into n in ) l Each sub-word, then the corresponding u′ in U′ i Expand to n l There are part-of-speech tags, and there is u′ i，1 =u i +″-B″,u′ i，j =u i +"-I",j=2,3,...,n l The "-B" indicates the start position of a part-of-speech tag, and the "-I" indicates the middle position of a part-of-speech tag. i，j Represents u′ i The j-th part-of-speech tag in the text; S1.3 converts U′ into a part-of-speech embedding vector sequence V, and the calculation process is as follows: Among them, E POS (U′) represents the conversion of the part-of-speech tag sequence U′ into the corresponding part-of-speech embedding vector sequence {v1, v, ..., v} through a table lookup operation. n } is E POS (U′) The result of the table lookup operation, d u The dimension of the part-of-speech embedding vector; S1.4 hides the BERT representation H b The hidden states in V are concatenated with the corresponding part-of-speech embedding vectors in V to obtain the combined hidden representation H. c The calculation process is as follows: Where [;] represents a join operation; S1.5 will combine the hidden representation H c The semantic fusion is performed by feeding the data into a scaled dot product multi-head self-attention mechanism to obtain the part-of-speech-aware sentence hidden representation H. u The calculation process is as follows: Among them, MHSA(H c ) indicates that the input X = H c The scaling dot product multi-head self-attention mechanism MHSA(X) is calculated as follows: MHSA(X)=tanh([head1;head2;…;head h ]w R ) (7) head i =Attention(X i ,X i ,X i )=Attention(XW h ,Xw h ,Xw h ) (8) Where, X∈R n×d The input to the multi-head self-attention mechanism MHSA(·) is given by d, where d is the vector dimension of the sequence in the input X, and head is the number of heads. i Let tanh(·) represent the i-th head in a multi-head self-attention system, where i = 1, 2, ..., h, and tanh(·) represent the hyperbolic tangent function. w R ∈R d×d It is a learnable parameter matrix, d h = d ÷ h, where h is the number of attention heads in multi-head self-attention, X i Let X represent the input in the i-th multi-head self-attention. i T The superscript T in the matrix indicates the matrix transpose operation, and Softmax(·) represents the normalized exponential function in the neural network; Step S2 specifically includes: in, This represents a time sequence H processed from left to right. u LSTM (Long Short-Term Memory) network, This represents a right-to-left timing process H. u LSTM networks, express Network output, for Hidden states in express Network output, for The hidden state in H L Sentence hiding representation that indicates semantic refinement express and The connection result, d L It is the number of hidden units in the LSTM network; Step S3 specifically includes: S3.1 Sentence Hiding Representation H in Semantic Refinement L Insert a vector of all zeros at the start and end positions respectively. H L Expand to As shown below: S3.2 will The data is fed into a CNN (Convolutional Neural Network) with a kernel size of 3 to extract adjacent semantics, thus obtaining the sentence hidden representation H of adjacent semantics. a The calculation process is as follows: Where CNNs(·) represents the convolutional neural network used. d represents the output hidden state sequence of a CNN network. a This indicates the number of convolutional kernels in a CNN network; S3.3 Hiding the semantically refined sentence representation H L Sentence hidden representation H with adjacent semantics a By connecting them, we obtain the semantically enhanced hidden representation H of adjacent sentences. e The calculation process is as follows: in, For H L With H a The connection result; Step S4 specifically includes: S4.1 Place the i-th subword of S in H e The corresponding hidden state The data is fed into the segment endpoint classification layer to obtain the probability distribution of whether the i-th word is a segment endpoint. The calculation process is as follows: in, Let i be the confidence vector for whether the i-th word is a segment endpoint. b is the weight matrix in the segment endpoint classification layer. d ∈R |D| Let D be the bias vector in the segment endpoint classification layer, where D = {"1", "0"}, and let |D| be the size of D. S4.2 Probability distribution of segment endpoints The probability that the segment endpoint label of the i-th word is "1" The probability that the segment endpoint label is "0" is greater than its value. Then the i-th word in sentence S is a segment endpoint; otherwise, the i-th word in sentence S is not a segment endpoint. S4.3 Repeat steps S4.1 and S4.2, with i taking values ​​of 1, 2, ..., n, until segment endpoints of all words in sentence S are labeled, resulting in the segment endpoint label sequence of S. Step S5 specifically includes: S5.1 The sequence of segment endpoints from sentence S. In the sentence S, find all segment endpoints to form a segment endpoint sequence. n d The number of endpoints in the middle segment of sentence S; S5.2 Let S d Any two adjacent segment endpoints The corresponding subwords in sentence S are respectively Then and Sentence Hidden Representation H in Semantic Refinement L Hidden state in Connect them, and send them to the positional relationship classification layer for classification to obtain the endpoints of adjacent segments. Probability distribution of the positional relationship between them and the endpoints of adjacent segments Predicted positional relationship between The calculation process is as follows: in, endpoints of adjacent segments The confidence vector of the positional relationship between them. b is the weight matrix in the positional relationship classification layer. r ∈R |Ω| Let Ω be the bias vector in the positional relationship classification layer, Ω = {"connected", "tangent", "separated", "connected_tangent_separated", "tangent_separated", "connected_separated", "connected_tangent"}, where Ω is the set of positional relationship labels, and |Ω| is the magnitude of Ω. It is a positional relationship in Ω. express The positional relationship is The probability, Expressing the requirement to make The maximum value S5.3 Repeat step S5.2, with i taking the values ​​1, 2, ..., n respectively. d -1, until the segment endpoint sequence S d The positional relationships between the endpoints of all adjacent segments have been predicted. Step S6 specifically includes: S6.1 Find all the outermost segments in sentence S according to the following rules to form the outermost segment sequence S. w : (1) If there is a "tangent-separation" and "separation" relationship between the endpoints of two adjacent segments, then the endpoints of these two adjacent segments do not exist in the same outermost segment; (2) If there is a “connection”, “tangency”, “connection-tangency”, “connection-separation” and “connection-tangency-separation” relationship between the endpoints of two adjacent segments, then the endpoints of these two adjacent segments exist in the same outermost segment; The outermost segment refers to a span that contains at least one teaching named entity in a sentence and can exist independently without being embedded in other segments; the span refers to a sentence segment. S6.2 For the outermost segment sequence S w Any outermost segment Find according to the following rules Span of all candidate teaching entities: (1) Outermost segment The sub-words corresponding to all segment endpoints are candidate teaching entity spans; (2) The span between any two segment endpoints is a candidate teaching entity span, including the two segment endpoints themselves. S6.3 Repeat step S6.2 until the outermost segment sequence S is found. w The spans of all candidate teaching entities in all outermost segments are used to form the set of candidate teaching entity spans for sentence S. n k The number of candidate teaching entity spans in sentence S; Step S7 specifically includes: S7.1 Let S k span of any candidate teaching entity The two segment endpoints correspond to the following subwords in sentence S: Then and Sub-words between words in semantically refined sentence hidden representation H L The hidden states in the equation are averaged to obtain the following: and Sub-word hiding representation The calculation process is as follows: in, For H L The i-th hidden state; S7.2 will In H L Hidden state in as well as Connecting them together yields the span of candidate teaching entities. Hidden representation A j The calculation process is as follows: S7.3 will A j The data is fed into a gating mechanism to obtain the span of candidate teaching entities. gating hidden representation The calculation process is as follows: in, b is the weight of the gating mechanism. g Here, σ(·) represents the bias of the gating mechanism, σ(·) represents the sigmoid activation function in the neural network, and ⊙ represents pointwise multiplication; S7.4 Repeat steps S7.1 to S7.3, with j taking values ​​of 1, 2, ..., n respectively. k until S is calculated k Gated hidden representation of the span of all candidate teaching entities; Step S8 specifically includes: S8.1 will S k span of any candidate teaching entity gating hidden representation The data is fed into the teaching entity classification layer for classification, resulting in the span of candidate teaching entities. Probability distribution of teaching entity classification The calculation process is as follows: in, For candidate teaching entity span The confidence vector for classifying teaching entities. b is the weight matrix in the teaching entity classification layer. k ∈R |Ψ| Let Ψ be the bias vector in the teaching entity classification layer, where Ψ = {"concept name", "attribute name", "relation word", "attribute value", "content object", "non-teaching entity"}, and represent the set of labels for each teaching entity category. Let |Ψ| be the magnitude of Ψ. Indicates the span of candidate teaching entities For teaching entity categories The probability, function Indicate the search in Ψ such that The teaching entity category with the maximum value For candidate teaching entity span Predicted labels for the classification of teaching entities; A8.2 Repeat step S8.1, with j taking values ​​of 1, 2, ..., n respectively. k until S is calculated k Predicted labels for the teaching entity classification of all candidate teaching entity spans; S8.3 will S k The candidate teaching entities whose predicted labels are not "non-teaching entities" are selected from all teaching entity categories to form the nested teaching named entity set in the sentence S to be solved.

2. The nested instructional named entity recognition method based on span according to claim 1, characterized in that: The final loss calculation of the established neural network model during the training process includes the following steps: (1) The segment endpoint loss generated by sentence S in the segment endpoint classification process is calculated using the following formula. Where θ represents all the parameters of the established neural network model. Let the true segment endpoint label be the i-th word in sentence S. The segment endpoint label representing the i-th word in sentence S is... The predicted probability, To predict the segment endpoint probability distribution of the i-th word in sentence S, Calculate using formula (17); (2) The positional relationship loss of sentence S during the classification of the positional relationship between the endpoints of adjacent segments is calculated using the following formula. in, Indicates the endpoints of adjacent segments The true label of the positional relationship between them Indicates the endpoints of adjacent segments The positional relationship labels between them are The predicted probability, To predict the endpoints of adjacent segments in sentence S The probability distribution of the positional relationship between them. Calculate using formula (19); (3) The teaching entity classification loss generated by sentence S in the teaching entity classification process of candidate teaching entity span is calculated using the following formula. in, For the j-th candidate teaching entity span Real teaching entity classification tags, The teaching entity classification label representing the span of the j-th candidate teaching entity is: The predicted probability, To predict the span of the j-th candidate teaching entity in sentence S The probability distribution of the teaching entity classification. Calculate using formula (25); (4) Calculate the final loss of the established neural network model during the training process using the following joint training formula. Where λ and β are learnable hyperparameters, and 0 < λ < 1, 0 < β < 1.

Images