A deep learning military weapon entity recognition method integrating relative position features

Abstract

The present invention discloses a deep learning military weapon entity recognition method integrating relative position features. It includes: 1) constructing a Chinese military weapon field data set based on public data. 2) formulating military weapon entity recognition rules in response to the problems existing in entities in public data. 3) Using the data set as the original data for model training, a flat lattice network is used to generate word vectors and extract relative position feature information. 4) Using the training results as input, a multi-head self-attention mechanism structure is added to model the long-distance dependencies in the input text. 5) Using the training results as input, combined with entity recognition rules, a CRF method is used as a label decoder to output a global optimal annotation sequence. The present invention solves the long-distance modeling problem and enhances context feature extraction, making the military weapon named entity recognition task more accurate and efficient.

Description

Technical Field

[0001] The present invention relates to the field of deep learning natural language processing, and in particular to a method for identifying named entities of military weapons using multi-neural network collaboration that integrates relative position features. Background Art

[0002] Named entity recognition is a basic and indispensable research direction in natural language processing. Named entity identification can obtain relevant information from unstructured data and obtain simple and clear structured data, which is convenient for people to efficiently retrieve, manage and analyze data, and is of great significance for processing unstructured data and extracting data information. This patent studies a deep learning named entity recognition method with multi-neural network collaboration.

[0003] Currently, the most popular method for named entity recognition is the BiLSTM-CRF method. This method can obtain the context information of long sentences to a certain extent and improve the recognition efficiency of entity boundaries. At the same time, it combines the CRF method to generate the optimal annotation sequence. However, this method still has great deficiencies and room for improvement in context information acquisition. In addition, it uses a relatively traditional network structure for feature extraction, which has great limitations.

[0004] The FLAT-MHA-CRF method is a named entity recognition model in the field of military weapons that integrates a multi-neural network collaboration with a multi-head self-attention mechanism. The model is a military weapon named entity recognition model (FLAT-MHA-CRF) based on a multi-head self-attention mechanism (Multi-Head Attention Transformer), combined with a flat lattice structure (FLAT) of relative position encoding and a conditional random field (CRF). The model extracts features through a flat lattice structure network of a multi-head self-attention mechanism, converts the entity recognition task into a sequence labeling task through the conditional random field method, and outputs the optimal label sequence to obtain the entity recognition result. Summary of the invention

[0005] Aiming at the problem of military weapon named entity recognition, the present invention proposes a deep learning military weapon entity recognition method integrating relative position features.

[0006] The present invention provides a deep learning military weapon entity recognition method integrating relative position features, which specifically comprises the following steps:

[0007] Step 1: Build a Chinese military weapons dataset based on public data.

[0008] Step 2: Develop military weapon entity identification rules to address issues with entities in public data.

[0009] Step 3: Use the data set constructed in step 1 as the original data for model training, and use the FLAT flat lattice network to generate word vectors and extract relative position feature information.

[0010] Step 4: Use the word vector feature D and the position feature information T of the span interval from the training result of step 3 ij As input, add the Multi-Head Attention Transformer multi-head self-attention mechanism structure to model the long-distance dependencies in the input text;

[0011] Specifically, the calculation formula for the relative position encoding of the span interval fused with the self-attention mechanism is:

[0012]

[0013] in, is a learnable parameter, d model =H×d head , H is the number of self-attention mechanism heads, d head For each head, D xi , D xj It is the word vector feature input to the upper layer.

[0014] The multi-head calculation formula in the Multi-Head Attention Transformer structure is:

[0015] MultiHead(Q,K,V)=Concat(head1,...,head h )W O

[0016] head i＝ Att(QW i Q , KW i K , VW i V )

[0017] in, are learnable parameters.

[0018] The attention mechanism in the Multi-Head Attention Transformer structure realizes direct interaction between characters and arbitrary words. After calculations by the multi-layer Multi-Head Attention Transformer structure, the feature matrix P is finally output. Step 5: Taking the training results of step 4 as input, combined with the entity recognition rules of step 2), the CRF method is used as the label decoder to output the global optimal annotation sequence.

[0019] The present invention uses a deep learning method of multi-neural network collaboration to extract relative position information and add a multi-head self-attention mechanism neural network to achieve the purpose of fast and accurate military weapon entity recognition for a large amount of public data. Experiments on the military weapon named entity recognition task on the Chinese Weapons corpus show that the content of the present invention has achieved certain results in solving the long-distance modeling problem and context feature extraction, as well as the effectiveness and superiority in the named entity recognition task in the field of military weapons. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0021] like Figure 1 As shown, the present invention provides a deep learning military weapon entity recognition method integrating relative position features, which specifically includes the following steps:

[0022] 1) Military weapon data acquisition. Some military-related websites, such as China Military Network and Sina Military Network, have rich named entity information data. The present invention constructs a Chinese military weapon field dataset Chinese weapon based on China Military Network, Sina Military, China Net Weapon Section, and QQ News Weapon Section, and obtains the final Chinese weapon dataset after cleaning the original data text.

[0023] 2) Establishment of military weapon entity identification rules. In response to the problem of entities in public data, the present invention proposes to identify (1) weapon equipment serial numbers and models composed of English letters, dashes, and numbers; (2) weapon codes composed of Chinese characters; (3) weapon equipment series composed of Chinese characters or English letters, dashes, and Chinese characters as weapon entities. For example, "Russia's "Black Hawk" main battle tank is numbered T-80UM2", "Black Hawk" is the Chinese code for this tank, and "T-80UM2" is the tank number. This article identifies both "Black Hawk" and "T-80UM2" as weapon equipment entities.

[0024] The BIOES method is used for annotation to retain more entity features. B indicates the start of entity annotation, I indicates the inside of entity annotation, O indicates irrelevant content, E indicates the end of entity annotation, and S indicates a single character entity. For example, "J-20" is annotated as "J-B-SD, -I-SD, 2I-SD, 0E-SD".

[0025] 3) FLAT relative position encoding feature extraction. Considering the difficulty and particularity of military weapon entity recognition, this paper uses the FLAT flat lattice network layer to complete the generation of word vectors and the extraction of preliminary relative position feature information. The original input data of the model is the Chinese military weapon field dataset established in step 1. After the step 1 dataset is input into the model sentence by sentence, an external dictionary is introduced to perform vocabulary enhancement on the input sentence Si, and the lattice structure with word vector information is obtained using the vocabulary in the dictionary, and it is flattened into a FLAT structure with multiple groups of span intervals. Each span interval contains word information, head position and tail position. Word information is a character or a word. The head position and tail position can indicate the position of word information in the lattice structure. They represent the position index of the first and last characters of the word information in the original sequence. For a character, its head position and tail position are the same. The Flat structure first uses word information with the same head and tail to construct a character sequence, and then uses the head position and tail position of other word information to establish a jump path. Finally, the word vector feature E of the input text is obtained.

[0026] The Flat lattice structure consists of multiple span intervals of different lengths. It can use relative position encoding to encode the interaction between different span intervals. For the two spans xi and xj determined by the head position and tail position in the structure, there are three relationships between them: intersection, inclusion and separation. Use dense vectors to model the three relationships. Let head[i] and tail[i] represent the head position and tail position of span interval xi, and head[j] and tail[j] represent the head position and tail position of span interval xj. Four relative distances are used to represent the relationship between xi and xj, and the specific calculation is:

[0027]

[0028] in represents the distance between the head of xi and the head of xj, represents the distance between the head of xi and the tail of xj, represents the distance between the tail of xi and the head of xj, Represents the distance between the tail of xi and the tail of xj; the final relative position code T of the span interval ij is a nonlinear transformation of the four distances:

[0029]

[0030] Where W r is a learnable parameter, represents the splicing operation, p d The calculation is as follows:

[0031]

[0032] where d is k is the dimension index representing the position encoding.

[0033] The relative position encoding calculated by continuously transforming the head and tail information can not only represent the relationship between the two tags, but also indicate some more detailed information.

[0034] 4) Multi-Head Attention Mechanism Model Training. Taking the training results of step 3 as input, add the Multi-Head Attention Transformer multi-head self-attention mechanism structure to model the long-distance dependency in military weapon named entity recognition. The relative position encoding T of the word vector feature D and the span interval ij Together they serve as the input of the Multi-HeadAttentionTransformer structure.

[0035] The calculation formula for the relative position encoding of the span interval fused with the self-attention mechanism is:

[0036]

[0037] in, is a learnable parameter.

[0038] The multi-head calculation formula in the Multi-Head Attention Transformer structure is:

[0039] MultiHead(Q,K,V)=Concat(head1,...,head h )W O

[0040] head i＝ Att(QW i Q , KW i K , VW i V )

[0041] in, are learnable parameters.

[0042] The attention mechanism in the Multi-Head Attention Transformer structure realizes the direct interaction between characters and arbitrary words. After calculations in the multi-layer Multi-Head Attention Transformer structure, the feature matrix P is finally output.

[0043] 5) Optimal sequence output. CRF conditional random field is an undirected graph model, a conditional probability model for labeling and segmenting ordered data. Taking the training results of step 4 as input, combined with the entity recognition rules of step 2, the CRF method is used as the label decoder at the top of the entire structure to output the predicted label sequence. For the input feature matrix P ij , which represents the input probability of character i to label j; define the transfer matrix F ij , where represents the transition probability from character i to label j; the generated label sequence is defined as; then the feature function is defined as follows:

[0044]

[0045] According to the principle of conditional random field, the conditional probability of the label sequence is:

[0046]

[0047] in, is the normalization factor.

[0048] Maximize the log-likelihood function during model training:

[0049] log(P(y|x))=W ts (x, y)-log(S(x))

[0050] The input feature matrix P is trained through the CRF conditional random field layer to obtain the global optimal label sequence:

[0051] y * = argmax W ts (x, y)

[0052] The embodiments of the present invention described above do not constitute a limitation on the protection scope of the present invention. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A deep learning military weapon entity recognition method integrating relative position features, characterized in that: The method comprises the following steps: 1) Construct a Chinese military weapons dataset based on public data; 2) Formulate rules for identifying military weapon entities in response to problems with entities in public data; 3) Using the data set constructed in step 1) as the original data for model training, the FLAT flat lattice network is used to generate word vectors and extract relative position feature information; specifically: according to the generated word vectors, for the two spans xi and xj determined by the head position and tail position in the word vector, there are three relationships between them: intersection, inclusion and separation; use dense vectors to model the three relationships; let head[i] and tail[i] represent the head position and tail position of the span interval xi, and head[j] and tail[j] represent the head position and tail position of the span interval xj; use four relative distances to represent the relationship between xi and xj, and the specific calculation is: in represents the distance between the head of xi and the head of xj, represents the distance between the head of xi and the tail of xj, represents the distance between the tail of xi and the head of xj, Represents the distance between the tail of xi and the tail of xj; the final relative position code of the span interval is T ij is a nonlinear transformation of the four distances: Where W r is a learnable parameter, represents the splicing operation, p d The calculation is as follows: where d is k is the dimensional index representing the position code, and the relative position code is obtained by continuously transforming the head and tail information; 4) Using the training result word vector feature D of step 3) and the position feature information T of the span interval ij As input, add the Multi-Head Attention Transformer multi-head self-attention mechanism structure to model the long-distance dependencies in the input text; Specifically, the calculation formula for the relative position encoding of the span interval fused with the self-attention mechanism is: in, is a learnable parameter, d model =H×d head , H is the number of self-attention mechanism heads, d head For each head, D xi ,D xj The word vector feature input to the upper layer; The multi-head calculation formula in the Multi-Head Attention Transformer structure is: MultiHead(Q,K,V)=Concat(head1,…,head h )W O head i =That(QW i Q ,KW i K ,VW i V ) in, is a learnable parameter; The attention mechanism in the Multi-Head Attention Transformer structure realizes the direct interaction between characters and arbitrary words; after calculations in the multi-layer Multi-Head Attention Transformer structure, the feature matrix P is finally output; 5) Taking the training result of step 4) as input, combined with the entity recognition rules of step 2), the CRF method is used as the label decoder to output the global optimal label sequence.

2. According to a deep learning military weapon entity recognition method integrating relative position features according to claim 1, it is characterized in that: In step 1), military-related text information from multiple public websites is obtained through crawler technology, and the text is cleaned and filtered and saved in txt format.

3. A deep learning military weapon entity recognition method integrating relative position features according to claim 1, characterized in that The military weapon entity identification rules are as follows: (1) weapon equipment serial number model composed of English letters, dashes, and numbers; (2) weapon code composed of Chinese characters; (3) weapon equipment series composed of Chinese characters or English letters, dashes, and Chinese characters.

4. The deep learning military weapon entity recognition method integrating relative position features according to claim 1 is characterized in that: In step 2), the military weapon entity is annotated using the BIOES method, where B is the beginning of the entity, I is the inside of the entity, E is the end of the entity, S is a single character entity, and O is irrelevant information.

5. The deep learning military weapon entity recognition method integrating relative position features according to claim 1 is characterized in that: For the input feature matrix P ij , which represents the input probability of character i to label j; define the transfer matrix F ij , where represents the transition probability from character i to label j; the generated label sequence is defined as; then the feature function is defined as follows: According to the principle of conditional random field, the conditional probability of the label sequence is: in, is the normalization factor; Maximize the log-likelihood function during model training: log(P(y|x))=W ts (x,y)-log(S(x)) The input feature matrix P is trained through the CRF conditional random field layer to obtain the global optimal label sequence: y * =argmax W ts (x,y)。

Images