Chinese named entity recognition method based on sequence interaction and syntactic integration

Abstract

The invention discloses a Chinese named entity recognition method based on sequence interaction and syntactic integration. A retrieval algorithm is adopted, a related context sequence is retrieved based on the input sequence, and a Chinese natural language processing kit is used for obtaining three kinds of syntactic information, syntactic components, part-of-speech tags and dependency relationships of the input sequence; respectively inputting an input sequence, a context sequence and three types of syntactic information into a named entity recognition model to obtain sequence interaction information fusing the input sequence and the context sequence and syntactic integration information fusing syntactic components, part-of-speech tags and dependency relationships, and finally dynamically fusing and inputting the two types of information into a conditional random field so as to recognize the named entity. And obtaining a named entity tag sequence of the input sequence. Through fusion of sequence interaction information and syntactic integration information, entity information lost by a sentence-level input sequence is effectively supplemented, text structure information about sentences in a model is introduced, and importance of the two kinds of information is flexibly adjusted, so that higher recognition performance is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a Chinese named entity recognition method based on sequence interaction and syntactic integration. Background Art

[0002] As a basic task in natural language processing, named entity recognition plays an important role in natural language processing tasks such as machine translation, information retrieval, and intelligent question answering. The main content of this task is to locate named entities with specific meanings in unstructured texts, and then sequentially classify the entities into predefined labels, such as names of people, places, organizations, and time.

[0003] The solutions to named entity recognition can be roughly divided into three methods: rule-based, statistical learning-based, and deep learning-based. Given that the rule-based and statistical learning-based methods are time-consuming and labor-intensive and have insufficient generalization capabilities, in recent years, the mainstream method of named entity recognition has shifted to a sequence labeling method based on deep learning. Currently, deep learning models have achieved good results in Chinese named entity recognition tasks, using the rich word embedding information in the pre-trained language model to help the model learn the semantic information of sentences.

[0004] Although pre-trained language models have made great progress in the field of named entity recognition, they only obtain feature representations of each character containing contextual semantic information, ignoring the syntactic information in Chinese text, such as part-of-speech tags, dependencies, and syntactic components. These features are very important for the task of named entity recognition because they can effectively identify the inherent structure in the text, thereby guiding the model to accurately predict named entities. In addition, due to the presence of ambiguous words in the input sequence, the model is still difficult to recognize. Therefore, retrieving contextual information related to the input sequence through a retrieval algorithm can supplement the semantic information of the input sequence in the model and improve the model's ability to understand the input sequence. Summary of the invention

[0005] The present invention aims at solving the technical problems that there are many deficiencies in the prior art for named entity recognition in Chinese text, and provides a Chinese named entity recognition method based on sequence interaction and syntactic integration.

[0006] The Chinese named entity recognition method based on sequence interaction and syntactic integration includes the following steps: Step S1: construct a data set, which includes a number of preprocessed sentence texts; Step S2: construct a named entity recognition model, which includes a sequence interaction module, a syntax integration module, an information fusion module and a classifier; Import the sentence text in step S1 into the sequence interaction module, obtain the input sequence and related context sequence, and further obtain the sequence interaction information feature vector through the cross attention mechanism; Step S3: import the input sequence of step S1 into the syntax integration module to obtain the syntax integration feature vector; Step S4: The information fusion module fuses the sequence interaction information feature vector in step S2 and the syntax integration feature vector in step S3 to obtain a fused feature encoding feature vector; Step S5: Input the fused feature encoding feature vector of step S4 into the conditional random field of the classifier, output the entity label sequence of the sentence text, and use the negative log-likelihood loss function to optimize the parameters of the model.

[0007] Furthermore, step S1 is specifically as follows: Collect several sentence texts, divide them into [training set: validation set: test set] = [8:1:1] ratio and construct a dataset based on them.

[0008] Furthermore, in step S2, the specific steps are: Step S21: The sequence interaction module obtains the input sequence corresponding to the input sentence text through the pre-trained language model BERT; it is expressed as: ; ; ; in, Represents sentence text, Represents the first character in the sentence text, Represents the second character in the sentence text, represents the i-th character in the sentence text, Represents the input sequence corresponding to the sentence text, represents the first character feature vector in the input sequence, represents the second character feature vector in the input sequence, Represents the i-th character feature vector in the input sequence; Step S22: the sequence interaction module obtains the relevant context corresponding to each sentence text through the BM25 retrieval algorithm; Step S23, the sequence interaction module processes the relevant context through the pre-trained language model BERT to obtain the relevant context sequence; it is expressed as: ; ; ; in, Represents the relevant context corresponding to each sentence text, represents the first character in the relevant context, represents the second character in the relevant context, represents the jth character in the relevant context; represents a sequence of relevant contexts, represents the first character feature vector in the relevant context sequence, represents the second character feature vector in the relevant context sequence, Represents the jth character feature vector in the relevant context sequence; Step S24: input the input sequence of step S21 and the relevant context sequence of step S23, and interact with each other to obtain sequence interaction information; Sequence interaction information includes inter-sequence interaction information and intra-sequence interaction information; Inter-sequence interaction information is modeled through cross-attention. The input sequence and the relevant context sequence are used as query vector, key vector and value vector respectively to input the sequence interaction module to obtain the interaction information between the two sequences, which is expressed as: ; ; in, Represents the query vector with the relevant context sequence , with the input sequence as the key vector Sum value vector The obtained sequence interaction information is Indicates that the feature vector of the input sequence is used as the query vector , with the context feature vector as the key vector Sum value vector The obtained sequence interaction information is represents the cross attention mechanism; as well as , , is the number of interactions between two sequences; In the process of sequence interaction, the relative meaning of the position between characters in different sequences is introduced, so that the characters have the ability of distance perception and direction perception, which is expressed as; ; in, Represents the relative position information between different sequences of characters. is the index of the character in the sequence that is the query vector, is the index of the character in the sequence used as the key vector, The value range is , , is the feature dimension of the input sentence text, is the number of heads of multi-head attention; Integrate the relative distance information into the attention mechanism, expressed as: ; in, Represents the attention value that incorporates distance information, is the index in the query vector The character feature vector of is the index in the key vector The character feature vector of and All are learnable parameters; A top-k selection strategy is introduced in the calculation of cross attention, so as to convert the cross attention into sparse cross attention to retain the attention value with the highest similarity, which is expressed as: ; in, Indicates that the characters in the query vector and the characters in the key vector are included. Attention value, is a hyperparameter, a mask constant , , indicating that it contains and The attention value between express Top ranking The attention value; Get the final attention value , expressed as: ; Among them, the final attention value The prefix between the characters in the query vector and the characters in the key vector is preserved. Attention value; After modeling the interaction information between the input sequence and the related context sequence, the cross sparse attention is used to calculate the intra-sequence interaction information, which is expressed as: ; in, is the intra-sequence interaction information, Represents the feature vector obtained by the pre-trained language model of the input sequence as the query vector, , represents the feature vector of the sequence interaction information with the relevant context sequence feature vector as the query vector and the input sequence feature vector as the key vector and value vector; Finally, the inter-sequence interaction information is concatenated with the intra-sequence interaction information to obtain the sequence interaction information feature vector, which is expressed as: ; In the formula, is the sequence mutual information feature vector, Indicates a connection operation. and are learnable parameters.

[0009] Furthermore, in step S3, the specific steps are: Step S31: using a natural language processing toolkit to obtain syntactic information of the sentence text of step S21, the syntactic information including syntactic components, part-of-speech tags, and dependency relationships of each character constituting the sentence text; Step S31 is specifically as follows: Step S311: obtaining the syntactic component of each character in the syntactic tree of the input sentence text through the natural language toolkit, then obtaining the syntactic node closest to each character in the input sentence text, obtaining the text information corresponding to the syntactic node as the syntactic node context feature information, and combining the syntactic node with the syntactic node context feature information to obtain the syntactic component feature information; Step S312: obtaining the part-of-speech tag of each character in the input sentence text through a natural language toolkit, obtaining the part-of-speech tags of the characters adjacent to each character in the input sentence text, using the text information of the character and the adjacent characters as the part-of-speech tag context feature information, and combining the part-of-speech tag with the part-of-speech tag context feature information to obtain the part-of-speech tag feature information; Step S313, obtaining the dependency relationship of each character in the input sentence text through the natural language toolkit, obtaining the core word and the subordinate word corresponding to each character in the input sentence text, using the text information of the core word and the subordinate word as the dependency relationship context feature information, and combining the dependency relationship with the dependency relationship context feature information to obtain the feature information of the part-of-speech tag; Step S32, calculating the feature vectors of the context feature information and three types of syntactic information: syntactic components, part-of-speech tags, and dependency relationships; Step S32 specifically includes: mapping the context feature information into a key vector, and mapping the feature information obtained by combining the context feature information with the corresponding syntactic information into a value vector; The text information corresponding to each syntactic information obtained by the natural language toolkit is matched with the feature vector of the input sentence text obtained in the pre-trained language model BERT, and the matched feature vectors are concatenated to obtain the key vector representation. ,in, Represents the first context information of the i-th character, Represents the second context information of the i-th character, Represents the first character of the context information, , Represents the set of all syntactic information, It is one of the three types of syntactic information; The value vector is represented as ,in Represents the first context information and syntactic information of the i-th character The characteristic information of the composition, Represents the second context information and syntactic information of the i-th character The characteristic information of the composition, Represents the first character of the contextual information and syntactic information Characteristic information of the composition; Step S33: Input the key vector and value vector of the syntactic information into the fully connected layer to obtain the key vector embedding representation Sum value vector embedding representation ; Key vector embedding representation It is expressed as: ; in, and is a learnable parameter, and the key vector embedding is expressed as , The embedding representation of the key vector mapping representing the first context information of the i-th character in the input sequence, The embedding representation of the key vector mapping representing the second context information of the i-th character in the input sequence, Indicates the i-th character in the input sequence The embedding representation of the key vector mapping of context information; Value vector embedding representation It is expressed as: ; in, and is a learnable parameter, and the value vector embedding is expressed as , Represents the first context information and syntactic information of the i-th character in the input sequence The embedded representation of the feature information value vector mapping composed of Represents the second context information and syntactic information of the i-th character in the input sequence The embedded representation of the feature information value vector mapping composed of Indicates the i-th character in the input sequence contextual information and syntactic information Embedded representation of the feature information value vector mapping composed; Step S34: Calculate the feature vector of each syntactic information using the key vector embedding representation and the value vector embedding representation; Use the softmax function to calculate the syntactic information of the i-th character The key vector embedding representation and the feature vector of the i-th character get the weight assigned to each value vector , expressed as: ; Where n is the nth embedding represented by the key vector embedding, is the feature vector of the i-th character in the input sequence, Indicates the i-th character in the input sequence The embedding representation of the key vector mapping of context information; Afterwards, The value vector embedding representation applied to the corresponding combination of contextual information and syntactic information is expressed as: ; in, is the feature vector of the syntactic information of the i-th character in the sentence text, Indicates the i-th character in the input sequence contextual information and syntactic information Embedded representation of the feature information value vector mapping composed of Step S35: fusing the syntactic component feature vector, the part-of-speech tag feature vector and the dependency feature vector to obtain a syntactic integrated information feature vector of the input sequence; Connect the input sequence and syntactic information, and assign different weights to each syntactic information, expressed as: ; In the formula, and is a learnable parameter, is the sigmoid function; Use the softmax function to Calculate the corresponding attention for each syntactic information , expressed as: ; In the formula, Syntactic information about the i-th character in the input sequence The attention , Represents the set of all syntactic information, It is one of the three types of syntactic information; The obtained attention of each syntactic information is weighted and summed with the corresponding syntactic information, expressed as: ; In the formula, is the syntactic integration information of the i-th character in the input sequence. The syntactic information of each input sequence is expressed as , which is the syntactically integrated feature vector.

[0010] Further, step S4, the specific steps are: Using a dynamic gating mechanism, the final fusion feature encoding feature vector is obtained based on the sequence interaction information feature vector and the syntactic integration information feature vector of the input sequence; Step S41: Use a reset function to dynamically generate the weights of the sequence interaction information and the syntactic integration information based on the feature vectors of the two types of information, expressed as: ; In the formula, is the feature vector of the i-th character of the input sequence and syntactic integration information The weight obtained, is the sigmoid function, , and is a learnable parameter; Step S42: Use weights To control the proportion of the feature vector of the input sequence and the syntactic integrated information feature vector in the final fusion feature encoding feature vector, expressed as: ; In the formula, For splicing operation, is the feature vector of the i-th character in the final fused feature encoding feature vector of the input sequence. The final fused feature encoding feature vector is expressed as , It is an element-wise multiplication operation.

[0011] Further, step S5, the specific steps are: Step S51: Input the final fusion feature encoding feature vector into the conditional random field, and output the entity label sequence of the input sequence, expressed as: ; In the formula, is the conditional probability of the entity tag sequence, is the length of the input sequence, is the potential function, is the predicted label sequence No. tags, is the set of all possible label sequences; Step S52: During the training process, a negative log-likelihood loss function is used. To optimize the parameters of the model, it is expressed as:

[0012] v ; in, is the true label of the input sequence, is the conditional probability of the true label of the input sequence.

[0013] Compared with the existing technology, the present invention has the following beneficial effects:

[0014] (1) The present invention effectively reveals the intrinsic structure of the text and guides the model to find appropriate named entities by considering three types of text structure information in the input sequence: part-of-speech tags, dependencies, and syntactic components.

[0015] (2) The present invention retrieves contextual information related to the input sequence through a retrieval algorithm, models the interactive information between the input sequence and the context sequence and between different levels of the input sequence, realizes the interaction between the input sequence and its related context, and can supplement semantic information for the input sequence in the model, thereby improving the model's ability to understand the input sequence.

[0016] (3) The present invention uses a dynamic gating mechanism to dynamically generate weights based on the sequence interaction information feature vector and the syntactic integration information feature vector, and weigh the importance of the two types of information in the final fusion feature vector. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The present invention is a flowchart of the steps of the Chinese named entity recognition method based on sequence interaction and syntax integration. DETAILED DESCRIPTION

[0018] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0019] Reference Figure 1 , a Chinese named entity recognition method based on sequence interaction and syntactic integration, in one example, includes the following steps: Step S1: construct a data set, which includes a number of preprocessed sentence texts; Step S2: construct a named entity recognition model, which includes a sequence interaction module, a syntax integration module, an information fusion module and a classifier; Import the sentence text in step S1 into the sequence interaction module, obtain the input sequence and related context sequence, and further obtain the sequence interaction information feature vector through the cross attention mechanism; Step S3: import the input sequence of step S1 into the syntax integration module to obtain the syntax integration feature vector; Step S4: The information fusion module fuses the sequence interaction information feature vector in step S2 and the syntax integration feature vector in step S3 to obtain a fused feature encoding feature vector; Step S5: Input the fused feature encoding feature vector of step S4 into the conditional random field of the classifier, output the entity label sequence of the sentence text, and use the negative log-likelihood loss function to optimize the parameters of the model.

[0020] Furthermore, in one example, step S1 is specifically as follows: Collect several sentence texts, divide them into [training set: validation set: test set] = [8:1:1] ratio and construct a dataset based on them.

[0021] Furthermore, in one example, in step S2, the specific steps are: Step S21: The sequence interaction module obtains the input sequence corresponding to the input sentence text through the pre-trained language model BERT; it is expressed as: ; ; ; in, Represents sentence text, Represents the first character in the sentence text, Represents the second character in the sentence text, represents the i-th character in the sentence text, Represents the input sequence corresponding to the sentence text, represents the first character feature vector in the input sequence, represents the second character feature vector in the input sequence, Represents the i-th character feature vector in the input sequence; Step S22: the sequence interaction module obtains the relevant context corresponding to each sentence text through the BM25 retrieval algorithm; Step S23, the sequence interaction module processes the relevant context through the pre-trained language model BERT to obtain the relevant context sequence; it is expressed as: ; ; ; in, Represents the relevant context corresponding to each sentence text, represents the first character in the relevant context, represents the second character in the relevant context, represents the jth character in the relevant context; represents a sequence of relevant contexts, represents the first character feature vector in the relevant context sequence, represents the second character feature vector in the relevant context sequence, Represents the jth character feature vector in the relevant context sequence; Step S24: input the input sequence of step S21 and the relevant context sequence of step S23, and interact with each other to obtain sequence interaction information; Sequence interaction information includes inter-sequence interaction information and intra-sequence interaction information; Inter-sequence interaction information is modeled through cross-attention. The input sequence and the relevant context sequence are used as query vector, key vector and value vector respectively to input the sequence interaction module to obtain the interaction information between the two sequences, which is expressed as: ; ; in, Represents the query vector with the relevant context sequence , with the input sequence as the key vector Sum value vector The obtained sequence interaction information is Indicates that the feature vector of the input sequence is used as the query vector , with the context feature vector as the key vector Sum value vector The obtained sequence interaction information is represents the cross attention mechanism; as well as , , is the number of interactions between two sequences; In the process of sequence interaction, the relative meaning of the position between characters in different sequences is introduced, so that the characters have the ability of distance perception and direction perception, which is expressed as; ; in, Represents the relative position information between different sequences of characters. is the index of the character in the sequence that is the query vector, is the index of the character in the sequence used as the key vector, The value range is , , is the feature dimension of the input sentence text, is the number of heads of multi-head attention; Integrate the relative distance information into the attention mechanism, expressed as: ; in, Represents the attention value that incorporates distance information, is the index in the query vector The feature vector of the character, is the index in the key vector The feature vector of the character, and All are learnable parameters; A top-k selection strategy is introduced in the calculation of cross attention, so as to convert the cross attention into sparse cross attention to retain the attention value with the highest similarity, which is expressed as: ; in, Indicates that the characters in the query vector and the characters in the key vector are included. Attention value, is a hyperparameter, a mask constant , , indicating that it contains and The attention value between express Top ranking The attention value; Get the final attention value , expressed as: ; Among them, the final attention value The prefix between the characters in the query vector and the characters in the key vector is preserved. Attention value; After modeling the interaction information between the input sequence and the related context sequence, the cross sparse attention is used to calculate the intra-sequence interaction information, which is expressed as: ; in, is the intra-sequence interaction information, Represents the feature vector obtained by the pre-trained language model of the input sequence as the query vector, , represents the feature vector of the sequence interaction information with the relevant context sequence feature vector as the query vector and the input sequence feature vector as the key vector and value vector; Finally, the inter-sequence interaction information is concatenated with the intra-sequence interaction information to obtain the sequence interaction information feature vector, which is expressed as: ; In the formula, is the sequence mutual information feature vector, Indicates a connection operation. and are learnable parameters.

[0022] Furthermore, in one example, in step S3, the specific steps are: Step S31: using a natural language processing toolkit to obtain syntactic information of the sentence text of step S21, the syntactic information including syntactic components, part-of-speech tags, and dependency relationships of each character constituting the sentence text; Step S31 is specifically as follows: Step S311: obtaining the syntactic component of each character in the syntactic tree of the input sentence text through the natural language toolkit, then obtaining the syntactic node closest to each character in the input sentence text, obtaining the text information corresponding to the syntactic node as the syntactic node context feature information, and combining the syntactic node with the syntactic node context feature information to obtain the syntactic component feature information; Step S312: obtaining the part-of-speech tag of each character in the input sentence text through a natural language toolkit, obtaining the part-of-speech tags of the characters adjacent to each character in the input sentence text, using the text information of the character and the adjacent characters as the part-of-speech tag context feature information, and combining the part-of-speech tag with the part-of-speech tag context feature information to obtain the part-of-speech tag feature information; Step S313, obtaining the dependency relationship of each character in the input sentence text through the natural language toolkit, obtaining the core word and the subordinate word corresponding to each character in the input sentence text, using the text information of the core word and the subordinate word as the dependency relationship context feature information, and combining the dependency relationship with the dependency relationship context feature information to obtain the feature information of the part-of-speech tag; Step S32, calculating the feature vectors of the context feature information and three types of syntactic information: syntactic components, part-of-speech tags, and dependency relationships; Step S32 specifically includes: mapping the context feature information into a key vector, and mapping the feature information obtained by combining the context feature information with the corresponding syntactic information into a value vector; The text information corresponding to each syntactic information obtained by the natural language toolkit is matched with the feature vector of the input sentence text obtained in the pre-trained language model BERT, and the matched feature vectors are concatenated to obtain the key vector representation. ,in, Represents the first context information of the i-th character, Represents the second context information of the i-th character, Represents the first character of the context information, , Represents the set of all syntactic information, It is one of the three types of syntactic information; The value vector is represented as ,in Represents the first context information and syntactic information of the i-th character The characteristic information of the composition, Represents the second context information and syntactic information of the i-th character The characteristic information of the composition, Represents the first character of the contextual information and syntactic information Characteristic information of the composition; Step S33: Input the key vector and value vector of the syntactic information into the fully connected layer to obtain the key vector embedding representation Sum value vector embedding representation ; Key vector embedding representation It is expressed as: ; in, and is a learnable parameter, and the key vector embedding is expressed as , The embedding representation of the key vector mapping representing the first context information of the i-th character in the input sequence, The embedding representation of the key vector mapping representing the second context information of the i-th character in the input sequence, Indicates the i-th character in the input sequence The embedding representation of the key vector mapping of context information; Value vector embedding representation It is expressed as: ; in, and is a learnable parameter, and the value vector embedding is expressed as , Represents the first context information and syntactic information of the i-th character in the input sequence The embedded representation of the feature information value vector mapping composed of Represents the second context information and syntactic information of the i-th character in the input sequence The embedded representation of the feature information value vector mapping composed of Indicates the i-th character in the input sequence contextual information and syntactic information Embedded representation of the feature information value vector mapping composed of Step S34: Calculate the feature vector of each syntactic information using the key vector embedding representation and the value vector embedding representation; Use the softmax function to calculate the syntactic information of the i-th character The key vector embedding representation and the feature vector of the i-th character get the weight assigned to each value vector , expressed as: ; Where n is the nth embedding represented by the key vector embedding, is the feature vector of the i-th character in the input sequence, Indicates the i-th character in the input sequence The embedding representation of the key vector mapping of context information; Afterwards, The value vector embedding representation applied to the corresponding combination of contextual information and syntactic information is expressed as: ; in, is the feature vector of the syntactic information of the i-th character in the sentence text, Indicates the i-th character in the input sequence contextual information and syntactic information Embedded representation of the feature information value vector mapping composed of Step S35: fusing the syntactic component feature vector, the part-of-speech tag feature vector and the dependency feature vector to obtain a syntactic integrated information feature vector of the input sequence; Connect the input sequence and syntactic information, and assign different weights to each syntactic information, expressed as: ; In the formula, and is a learnable parameter, is the sigmoid function; Use the softmax function to Calculate the corresponding attention for each syntactic information , expressed as: ; In the formula, Syntactic information about the i-th character in the input sequence The attention , Represents the set of all syntactic information, It is one of the three types of syntactic information; The obtained attention of each syntactic information is weighted and summed with the corresponding syntactic information, expressed as: ; In the formula, is the syntactic integration information of the i-th character in the input sequence. The syntactic information of each input sequence is expressed as , which is the syntactically integrated feature vector.

[0023] Furthermore, in one example, step S4 includes the following specific steps: Using a dynamic gating mechanism, the final fusion feature encoding feature vector is obtained based on the sequence interaction information feature vector and the syntactic integration information feature vector of the input sequence; Step S41: Use a reset function to dynamically generate the weights of the sequence interaction information and the syntactic integration information based on the feature vectors of the two types of information, expressed as: ; In the formula, is the feature vector of the i-th character of the input sequence and syntactic integration information The weight obtained, is the sigmoid function, , and is a learnable parameter; Step S42: Use weights To control the proportion of the feature vector of the input sequence and the syntactic integrated information feature vector in the final fusion feature encoding feature vector, expressed as: ; In the formula, For splicing operation, is the feature vector of the i-th character in the final fused feature encoding feature vector of the input sequence. The final fused feature encoding feature vector is expressed as , It is an element-wise multiplication operation.

[0024] Furthermore, in one example, step S5 includes the following specific steps: Step S51: Input the final fusion feature encoding feature vector into the conditional random field, and output the entity label sequence of the input sequence, expressed as: ; In the formula, is the conditional probability of the entity tag sequence, is the length of the input sequence, is the potential function, is the predicted label sequence No. tags, is the set of all possible label sequences; Step S52: During the training process, a negative log-likelihood loss function is used. To optimize the parameters of the model, it is expressed as:

[0025] v ; in, is the true label of the input sequence, is the conditional probability of the true label of the input sequence.

[0026] The present invention is described in detail above in conjunction with the embodiments of the accompanying drawings. A person skilled in the art can make various variations of the present invention according to the above description. Therefore, some details in the embodiments should not be construed as limiting the present invention, and the present invention shall be protected by the scope defined by the attached claims.

Claims

1. A Chinese named entity recognition method based on sequence interaction and syntactic integration, characterized in that: The steps include: Step S1: construct a data set, which includes a number of preprocessed sentence texts; Step S2: construct a named entity recognition model, which includes a sequence interaction module, a syntax integration module, an information fusion module and a classifier; Import the sentence text in step S1 into the sequence interaction module, obtain the input sequence and related context sequence, and further obtain the sequence interaction information feature vector through the cross attention mechanism; Step S3: import the input sequence of step S1 into the syntax integration module to obtain the syntax integration feature vector; Step S4: The information fusion module fuses the sequence interaction information feature vector in step S2 and the syntax integration feature vector in step S3 to obtain a fused feature encoding feature vector; Step S5: Input the fused feature encoding feature vector of step S4 into the conditional random field of the classifier, output the entity label sequence of the sentence text, and use the negative log-likelihood loss function to optimize the parameters of the model.

2. The Chinese named entity recognition method based on sequence interaction and syntax integration according to claim 1 is characterized in that: Step S1 is specifically as follows: Collect several sentence texts, divide them into [training set: validation set: test set] = [8:1:1] ratio and construct a dataset based on them.

3. The Chinese named entity recognition method based on sequence interaction and syntax integration according to claim 2 is characterized in that: In step S2, the specific steps are: Step S21: The sequence interaction module obtains the input sequence corresponding to the input sentence text through the pre-trained language model BERT; it is expressed as: ； ； ； in, Represents sentence text, Represents the first character in the sentence text, Represents the second character in the sentence text, represents the i-th character in the sentence text, Represents the input sequence corresponding to the sentence text, represents the first character feature vector in the input sequence, represents the second character feature vector in the input sequence, Represents the i-th character feature vector in the input sequence; Step S22: the sequence interaction module obtains the relevant context corresponding to each sentence text through the BM25 retrieval algorithm; Step S23, the sequence interaction module processes the relevant context through the pre-trained language model BERT to obtain the relevant context sequence; it is expressed as: ； ； ； in, Represents the relevant context corresponding to each sentence text, represents the first character in the relevant context, represents the second character in the relevant context, represents the jth character in the relevant context; represents a sequence of relevant contexts, represents the first character feature vector in the relevant context sequence, represents the second character feature vector in the relevant context sequence, Represents the jth character feature vector in the relevant context sequence; Step S24: input the input sequence of step S21 and the relevant context sequence of step S23, and interact with each other to obtain sequence interaction information; Sequence interaction information includes inter-sequence interaction information and intra-sequence interaction information; Inter-sequence interaction information is modeled through cross-attention. The input sequence and the relevant context sequence are used as query vector, key vector and value vector respectively to input the sequence interaction module to obtain the interaction information between the two sequences, which is expressed as: ； ； in, Represents the query vector with the relevant context sequence , with the input sequence as the key vector Sum value vector The obtained sequence interaction information is Indicates that the feature vector of the input sequence is used as the query vector , with the context feature vector as the key vector Sum value vector The obtained sequence interaction information is represents the cross attention mechanism; as well as , , is the number of interactions between two sequences; In the process of sequence interaction, the relative meaning of the position between characters in different sequences is introduced, so that the characters have the ability of distance perception and direction perception, which is expressed as; ； in, Represents the relative position information between different sequences of characters. is the index of the character in the sequence that is the query vector, is the index of the character in the sequence used as the key vector, The value range is , , is the feature dimension of the input sentence text, is the number of heads of multi-head attention; Integrate the relative distance information into the attention mechanism, expressed as: ； in, Represents the attention value that incorporates distance information, is the index in the query vector The feature vector of the character, is the index in the key vector The feature vector of the character, and All are learnable parameters; A top-k selection strategy is introduced in the calculation of cross attention, so as to convert the cross attention into sparse cross attention to retain the attention value with the highest similarity, which is expressed as: ； in, Indicates that the characters in the query vector and the characters in the key vector are included. Attention value, is a hyperparameter, a mask constant , , indicating that it contains and The attention value between express Top ranking The attention value; Get the final attention value , expressed as: ； Among them, the final attention value The prefix between the characters in the query vector and the characters in the key vector is preserved. Attention value; After modeling the interaction information between the input sequence and the related context sequence, the cross sparse attention is used to calculate the intra-sequence interaction information, which is expressed as: ； in, is the intra-sequence interaction information, Represents the feature vector obtained by the pre-trained language model of the input sequence as the query vector, , represents the feature vector of the sequence interaction information with the relevant context sequence feature vector as the query vector and the input sequence feature vector as the key vector and value vector; Finally, the inter-sequence interaction information is concatenated with the intra-sequence interaction information to obtain the sequence interaction information feature vector, which is expressed as: ； In the formula, is the sequence mutual information feature vector, Indicates a connection operation. and are learnable parameters.

4. The Chinese named entity recognition method based on sequence interaction and syntax integration according to claim 3 is characterized by: In step S3, the specific steps are: Step S31: using a natural language processing toolkit to obtain syntactic information of the sentence text of step S21, the syntactic information including syntactic components, part-of-speech tags, and dependency relationships of each character constituting the sentence text; Step S31 is specifically as follows: Step S311: obtaining the syntactic component of each character in the syntactic tree of the input sentence text through the natural language toolkit, then obtaining the syntactic node closest to each character in the input sentence text, obtaining the text information corresponding to the syntactic node as the syntactic node context feature information, and combining the syntactic node with the syntactic node context feature information to obtain the syntactic component feature information; Step S312: obtaining the part-of-speech tag of each character in the input sentence text through a natural language toolkit, obtaining the part-of-speech tags of the characters adjacent to each character in the input sentence text, using the text information of the character and the adjacent characters as the part-of-speech tag context feature information, and combining the part-of-speech tag with the part-of-speech tag context feature information to obtain the part-of-speech tag feature information; Step S313, obtaining the dependency relationship of each character in the input sentence text through the natural language toolkit, obtaining the core word and the subordinate word corresponding to each character in the input sentence text, using the text information of the core word and the subordinate word as the dependency relationship context feature information, and combining the dependency relationship with the dependency relationship context feature information to obtain the feature information of the part-of-speech tag; Step S32, calculating the feature vectors of the context feature information and three types of syntactic information: syntactic components, part-of-speech tags, and dependency relationships; Step S32 specifically includes: mapping the context feature information into a key vector, and mapping the feature information obtained by combining the context feature information with the corresponding syntactic information into a value vector; The text information corresponding to each syntactic information obtained by the natural language toolkit is matched with the feature vector of the input sentence text obtained in the pre-trained language model BERT, and the matched feature vectors are concatenated to obtain the key vector representation. ,in, Represents the first context information of the i-th character, Represents the second context information of the i-th character, Represents the first character of the context information, , Represents the set of all syntactic information, It is one of the three types of syntactic information; The value vector is represented as ,in Represents the first context information and syntactic information of the i-th character The characteristic information of the composition, Represents the second context information and syntactic information of the i-th character The characteristic information of the composition, Represents the first character of the contextual information and syntactic information Characteristic information of the composition; Step S33: Input the key vector and value vector of the syntactic information into the fully connected layer to obtain the key vector embedding representation Sum value vector embedding representation ; Key vector embedding representation It is expressed as: ； in, and is a learnable parameter, and the key vector embedding is expressed as , The embedding representation of the key vector mapping representing the first context information of the i-th character in the input sequence, The embedding representation of the key vector mapping representing the second context information of the i-th character in the input sequence, Indicates the i-th character in the input sequence The embedding representation of the key vector mapping of context information; Value vector embedding representation It is expressed as: ； in, and is a learnable parameter, and the value vector embedding is expressed as , Represents the first context information and syntactic information of the i-th character in the input sequence The embedded representation of the feature information value vector mapping composed of Represents the second context information and syntactic information of the i-th character in the input sequence The embedded representation of the feature information value vector mapping composed of Indicates the i-th character in the input sequence contextual information and syntactic information Embedded representation of the feature information value vector mapping composed of Step S34: Calculate the feature vector of each syntactic information using the key vector embedding representation and the value vector embedding representation; Use the softmax function to calculate the syntactic information of the i-th character The key vector embedding representation and the feature vector of the i-th character get the weight assigned to each value vector , expressed as: ； Where n is the nth embedding represented by the key vector embedding, is the feature vector of the i-th character in the input sequence, Indicates the i-th character in the input sequence The embedding representation of the key vector mapping of context information; Afterwards, The value vector embedding representation applied to the corresponding combination of contextual information and syntactic information is expressed as: ； in, is the feature vector of the syntactic information of the i-th character in the sentence text, Indicates the i-th character in the input sequence contextual information and syntactic information Embedded representation of the feature information value vector mapping composed of Step S35: fusing the syntactic component feature vector, the part-of-speech tag feature vector and the dependency feature vector to obtain a syntactic integrated information feature vector of the input sequence; Connect the input sequence and syntactic information, and assign different weights to each syntactic information, expressed as: ； In the formula, and is a learnable parameter, is the sigmoid function; Use the softmax function to Calculate the corresponding attention for each syntactic information , expressed as: ； In the formula, Syntactic information about the i-th character in the input sequence The attention , Represents the set of all syntactic information, It is one of the three types of syntactic information; The obtained attention of each syntactic information is weighted and summed with the corresponding syntactic information, expressed as: ； In the formula, is the syntactic integration information of the i-th character in the input sequence. The syntactic information of each input sequence is expressed as , which is the syntactically integrated feature vector.

5. The Chinese named entity recognition method based on sequence interaction and syntax integration according to claim 4 is characterized in that: Step S4, the specific steps are: Using a dynamic gating mechanism, the final fusion feature encoding feature vector is obtained based on the sequence interaction information feature vector and the syntactic integration information feature vector of the input sequence; Step S41: Use a reset function to dynamically generate weights for controlling the sequence interaction information and the syntactic integration information based on the feature vectors of the two types of information, expressed as: ； In the formula, is the feature vector of the i-th character of the input sequence and syntactic integration information The weights obtained, is the sigmoid function, , and is a learnable parameter; Step S42: Use weights To control the proportion of the feature vector of the input sequence and the syntactic integrated information feature vector in the final fusion feature encoding feature vector, expressed as: ； In the formula, For splicing operation, is the feature vector of the i-th character in the final fused feature encoding feature vector of the input sequence. The final fused feature encoding feature vector is expressed as , It is an element-wise multiplication operation.

6. The Chinese named entity recognition method based on sequence interaction and syntax integration according to claim 5 is characterized by: Step S5, the specific steps are: Step S51: Input the final fusion feature encoding feature vector into the conditional random field, and output the entity label sequence of the input sequence, expressed as: ； In the formula, is the conditional probability of the entity tag sequence, is the length of the input sequence, is the potential function, is the predicted label sequence No. tags, is the set of all possible label sequences; Step S52: During the training process, a negative log-likelihood loss function is used. To optimize the parameters of the model, it is expressed as: ； in, is the true label of the input sequence, is the conditional probability of the true label of the input sequence.

Images