Domain short text named entity recognition method fusing domain knowledge and deep learning

Abstract

The application discloses a field short text named entity recognition method fusing field knowledge and deep learning, and relates to the fields of knowledge graph and natural language processing.The application comprises the following steps: clustering new knowledge entities, performing word embedding training by using a global vector word embedding model, generating a field named entity word vector space and a word vector of the new knowledge entities; processing the word vector by the field named entity word vector space, synchronously inputting the word vector and the word vector of the new knowledge entities into two BiLSTM-CRF networks respectively, outputting the results to a full connection network, and forming a field named entity recognition model of double BiLSTM-CRF-full connection network; inputting the new field short text after word segmentation and word vector processing into the double BiLSTM-CRF-full connection network trained and generated, and outputting the recognition result of the named entities in the field short text.The application can accurately and quickly recognize key named entities in field short texts, assist personnel in correctly understanding information, and has great practical application value.

Description

Technical Field

[0001] This invention relates to the fields of knowledge graphs, entity clustering, BiLSTM-CRF networks, and natural language processing, and in particular to a domain-specific short text named entity recognition method that integrates domain knowledge and deep learning, which can solve problems such as insufficient utilization of domain knowledge and low accuracy of domain named entity recognition. Background Technology

[0002] In the military field, the extensive use of short text formats for exchanging information such as social and police reports often results in texts that are not only highly domain-specific but also short in length, unclear in their references, inaccurate terminology, lack of context, inconsistent keyword placement, and widely varying descriptive styles. Therefore, accurately and quickly identifying key named entities in these short texts is crucial for assisting personnel in correctly understanding information and has significant practical application value.

[0003] Currently, the recognition of named entities in short texts in this field is mostly achieved through dictionaries, rules, and methods combined with machine learning. Patent application CN106095749A discloses a deep learning-based text keyword extraction method. This method trains a recurrent neural network model, updates the network parameters using stochastic gradient descent, and after model training, converts a text containing keywords into word vectors, which are then input into the trained recurrent neural network model to generate the keywords for that text. Patent application CN113111172A discloses a method and apparatus for extracting person information from emergency call text based on a deep learning model. It inputs the word vectors corresponding to the segmented words into a classification model based on a deep learning model corresponding to each target person information element identifier in the target person information element identifier set. In response to determining that the obtained classification result is a corresponding person information element, the segmented words are identified as the corresponding person information element. Patent application CN109783812A discloses a Chinese named entity recognition method and apparatus based on a self-attention mechanism, which extracts the private features h of the sentence to be recognized. 1 Common features s 1 ; Calculate the common features s based on the self-attention mechanism 1 Private feature h 1 The corresponding long-distance dependency relationship is used to obtain the fused Chinese named entity recognition data features h″. 1 The fused features h″ are obtained by using conditional random field pairs. 1 To perform Chinese named entity recognition.

[0004] However, these existing methods often suffer from problems such as high workload for manual annotation, insufficient utilization of domain knowledge, and low accuracy in domain named entity recognition. Furthermore, although pre-trained large language models (LLMs) have developed rapidly, and relatively accurate domain models can be quickly obtained using large model fine-tuning techniques, the high computational and storage requirements of LLMs limit their widespread application in domains with limited resources. Summary of the Invention

[0005] In view of this, this invention proposes a domain-specific short text named entity recognition method that integrates domain knowledge and deep learning. This invention selects key entity types and key relationships between key entities based on domain characteristics, performs projection transformation on knowledge entities in the domain knowledge graph, and then clusters them, which can improve the accuracy of knowledge entity clustering calculation.

[0006] The technical solution adopted in this invention is as follows:

[0007] A domain-specific short text named entity recognition method that integrates domain knowledge and deep learning includes the following steps:

[0008] Step 1 involves selecting key knowledge entities and their relationships from existing military domain knowledge graphs, constructing a key knowledge entity association matrix, projecting these key knowledge entities into new knowledge entities, and then clustering these new entities. The specific method for Step 1 is as follows:

[0009] Step 1-1: Select key knowledge entities and key entity relationships between key knowledge entities from existing domain knowledge graphs in the military field.

[0010] Steps 1-2: Based on the entity relationships between knowledge entities, construct a knowledge entity association matrix S, where each element of the matrix represents the degree of association between two knowledge entities. ij , used to represent knowledge entity e in a domain knowledge graph i With knowledge entity e j The degree value of relationships in a knowledge graph;

[0011] Steps 1-3, according to Calculate and generate the knowledge entity association transformation matrix W, where σ is taken as 1 by default;

[0012] Steps 1-4, according to Calculate and generate the diagonal matrix D;

[0013] Steps 1-5, according to the Laplace matrix A=D -1 / 2 WD 1 / 2 Generate the knowledge entity projection transformation matrix A;

[0014] Steps 1-6: Transform the knowledge entity e into a new knowledge entity e' through the projection transformation matrix A, i.e., e' = Ae;

[0015] Steps 1-7: For each new knowledge entity e', construct the association matrix S' of the new knowledge entity set based on the relation degree between the new knowledge entity and other new knowledge entities;

[0016] Steps 1-8: Calculate E using the association matrix S' of the new knowledge entity set. i and E j , of which E i E j Representing the new knowledge entity e respectively i '、e j 'The set of all new knowledge entities that have a relation, |E i | represents set E i The sum of the degrees of each new knowledge entity in the equation, |E j | represents set E j The sum of the degrees of each new knowledge entity in the middle;

[0017] Steps 1-9, according to Calculate the new knowledge entity e i 'and new knowledge entity e j The similarity distance is given by E, where E represents the set of all new knowledge entities, and |E| represents the sum of the degrees of all new knowledge entities. i ∩E j For set E i With set E j The CCP is a set of new knowledge entities, |E i ∩E j | represents set E i ∩E j The sum of the knowledge entity degrees of China and Singapore;

[0018] Steps 1-10: Based on the similarity distance of the new knowledge entities, the K-means clustering method is used to perform clustering calculations on the new knowledge entities, obtaining the clustering results of the new knowledge entities and the new knowledge entity e corresponding to each cluster center. c ′, all new knowledge entities e c ′Constitutes a new set of knowledge entities at the cluster center {e c ′};

[0019] Step 2: Collect short texts in the domain, identify the named entities in them, and then use the JIEBA word segmentation tool to segment the short texts in the domain, and use a stop word dictionary to remove colloquial words and stop words; use the global vector word embedding model GloVe to train word embeddings together with the association matrix S' of the new knowledge entity set to generate the domain named entity word vector space and the new knowledge entity word vectors.

[0020] Step 3: For the named entities that have been identified in the domain short text, generate a set of named entity word vectors for the domain short text according to the temporal relationship in the domain short text, using the domain named entity word vector space;

[0021] Step 4: In the clusters obtained from the new knowledge entity clustering, calculate the new knowledge entity that is most similar to the named entity word vector; the specific method of step 4 is as follows:

[0022] Step 4-1: Select a named entity word vector sequentially from the named entity word vector set and calculate the entity word vector similarity.

[0023]

[0024] Where, r m It is named entity word vectors. It is the set of new knowledge entities at the cluster center {e c A cluster center entity e in '} c 'Corresponding new knowledge entity word vectors;

[0025] Step 4-2: Calculate the maximum value of entity word vector similarity. If the maximum value is less than the threshold If no similar new knowledge entity class is found, proceed to step 5; if the maximum value is greater than the threshold... Then randomly select the r corresponding to one of the maximum values. m , believes that r m Belongs to this cluster;

[0026] Step 4-3: For the clusters determined in Step 4-2, calculate the similarity s(r) between the named entity word vectors and the word vectors of each new knowledge entity in the cluster. m r e ′):

[0027]

[0028] Where, r e These are new knowledge entity word vectors;

[0029] Step 4-4, calculate s(r) m r e’ The maximum value of ) max(s(r) m r e’ If the maximum value is less than the threshold δ, it is considered that no similar new knowledge entity has been found, and the process proceeds to step 5; if the maximum value is greater than the threshold δ, then one of the maximum values ​​corresponding to r is randomly selected. e’ , which serves as the word vector of the most similar new knowledge entity to the named entity;

[0030] Step 4-5: Repeat steps 4-1 to 4-4 to find the corresponding new knowledge entity vector for each named entity in the named entity word vector set, and replace the corresponding named entity word vector in the set with the new knowledge entity word vector to generate the named entity word vector set that replaces the knowledge entity.

[0031] Step 5: Construct two BiLSTM-CRF networks: a knowledge entity replacement BiLSTM-CRF network and a named entity recognition BiLSTM-CRF network. Input the named entity word vector set generated in Steps 4-5 into the knowledge entity replacement BiLSTM-CRF network, and simultaneously input the named entity word vector set generated in Step 3 into the named entity recognition BiLSTM-CRF network.

[0032] Step 6: Input the outputs of the knowledge entity replacement BiLSTM-CRF network and the name entity recognition BiLSTM-CRF network into a fully connected neural network;

[0033] Step 7: Select short texts in the domain in batches, repeat steps 3 to 6, use cross-entropy of named entity recognition as the loss function, and mini-batch stochastic gradient descent with a learning rate of 0.1 as the optimization algorithm to train the parameters of the BiLSTM-CRF network for replacing knowledge entities, the BiLSTM-CRF network for recognizing named entities, and the fully connected neural network, forming the final domain named entity recognition network model that integrates knowledge graph and deep learning with dual BiLSTM-CRF-fully connected neural networks;

[0034] Step 8: Use the domain-specific named entity recognition network model to identify named entities in the domain-specific short text.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] 1. This invention selects key entity types and key relationships between key entities based on the characteristics of the domain, and performs projection transformation on knowledge entities in the domain knowledge graph before clustering, which can improve the accuracy of knowledge entity clustering calculation.

[0037] 2. This invention trains word embeddings using the Global Vector Word Embedding (GloVe) model, which generates a more accurate domain word vector space than the more general Word2vec model.

[0038] 3. This invention compares the word vectors of entities to be identified in short domain texts with the word vectors of knowledge entities in the knowledge entity clustering results, which can improve the search efficiency of similar knowledge entities.

[0039] 4. This invention replaces the set of name entity word vectors to be identified for knowledge entities, incorporates domain knowledge, and improves the accuracy of named entity recognition for short domain texts. Attached Figure Description

[0040] Figure 1 This is a flowchart of the domain word vector space generation process of the present invention, which integrates domain knowledge.

[0041] Figure 2 This is a flowchart of the training process for training two BiLSTM-CRF networks and the parameters of a fully connected network during the training phase of this invention.

[0042] Figure 3 This is a flowchart of domain short text named entity recognition in the application stage of the present invention.

[0043] Figure 4 This is a schematic diagram of the dual BiLSTM-CRF-fully connected domain named entity recognition model that integrates domain knowledge according to the present invention. Detailed Implementation

[0044] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that these examples are only for illustrating the present invention and are not intended to limit the scope of the present invention. After reading this invention, any modifications of the present invention in various equivalent forms by those skilled in the art fall within the scope defined by the appended claims.

[0045] A domain-specific named entity recognition method integrating domain knowledge and deep learning is proposed. This method selects key entity types and key relationship types between key entities from a domain knowledge graph. Through projection transformation of key relationships between key entities, multiple new knowledge entity clusters are formed using K-means clustering. The domain short text is segmented and vectorized to generate named entity word vectors. Similar new knowledge entity word vectors are searched for within the knowledge entity clusters. The set of named entity word vectors is then passed through a BiLSTM-CRF network for named entity recognition. The corresponding new knowledge entity word vectors replace the corresponding named entity word vectors in the named entity word vector set, generating a set of named entity word vectors for the replaced knowledge entities. This set is then simultaneously input into another BiLSTM-CRF network for named entity recognition. Finally, the results of the two named entity recognitions are input into a fully connected neural network layer. After training with batches of domain short texts, a domain named entity recognition network model integrating knowledge graph and deep learning dual BiLSTM-CRF-fully connected layers is formed.

[0046] This method is implemented in three stages: domain word vector space generation, training, and application. Details are as follows:

[0047] (I) Domain Word Vector Space Generation Stage

[0048] like Figure 1 As shown, from the existing domain knowledge graph, key domain knowledge entities and key entity relationships between them are selected. These entities are then projected into new knowledge entities. These new entities are then clustered. The accumulated domain short texts (with identified named entities) and the adjacency matrices of the new knowledge entities are used to train word embeddings using a global vector word embedding model (GloVe). This generates a domain named entity word vector space and word vectors for the new knowledge entities. The specific steps are as follows:

[0049] 101. From the existing domain knowledge graph in the military field, select key knowledge entities and key entity relationships between key knowledge entities. For example, in the knowledge graph of the military field, key knowledge entities mainly include combat subjects, combat targets, equipment allocation, combat behavior, combat intentions, etc., and key relationships between key knowledge entities mainly include affiliation, assignment, allocation, deployment, execution, realization, etc. (Hereinafter, "key knowledge entities" will be referred to as "knowledge entities", and "key entity relationships" will be referred to as "entity relationships").

[0050] 102. Based on the entity relationships between knowledge entities, construct a knowledge entity association matrix S, where the degree of association between two knowledge entities in the matrix is ​​Si. ij For knowledge entity e in the domain knowledge graph i , and knowledge entity e j The degree value of relationships in a knowledge graph;

[0051] 103. According to Generate a knowledge entity association transformation matrix W, where σ can be set to 1 by default;

[0052] 104. According to Generate a diagonal matrix D;

[0053] 105. According to the Laplace matrix A = D -1 / 2 WD 1 / 2 Generate the knowledge entity projection transformation matrix A;

[0054] 106. Transform knowledge entity e into a new knowledge entity e' through projection transformation matrix A, where e' = Ae;

[0055] 107. For each new knowledge entity e', construct the association matrix S' of the new knowledge entity set by the degree of relation between the new knowledge entity and other new knowledge entities;

[0056] 108. Calculate E using the association matrix S' of the new entity set. i and E j , of which E i E j Representing the new knowledge entity e respectively i '、ej 'There exists a set of all new knowledge entities, |E i | represents set E i The sum of the degrees of each new knowledge entity in the equation, |E j | represents set E j The sum of the degrees of each new knowledge entity in the middle;

[0057] 109. According to Calculate the new knowledge entity e i 'and new knowledge entity e j The similarity distance is given by E, where E represents the set of all new knowledge entities, and |E| represents the sum of the degrees of all new knowledge entities. i ∩E j For set E i With set E j The CCP is a set of new knowledge entities, |E i ∩E j | represents set E i ∩E j The sum of the knowledge entity degrees of China and Singapore;

[0058] 110. Based on the knowledge entity similarity distance in steps 101-109, the K-means clustering method is used to perform clustering calculations on the new knowledge entities, obtaining the clustering results of the new entities and the new knowledge entity e corresponding to each cluster center. c ', All new knowledge entities e c 'Constitutes a new set of knowledge entities at the cluster center {e c '};

[0059] 111. The accumulated domain short texts (in which named entities have been identified) are segmented using the JIEBA word segmentation tool. After removing colloquial words and stop words using a stop word dictionary, the association matrix S' of the new knowledge entity set is used together with the global vector word embedding model (GloVe) for word embedding training to generate the domain named entity word vector space and the new knowledge entity word vector.

[0060] (II) Training Phase

[0061] like Figure 2 As shown, accumulated domain-specific short texts (with identified named entities) are processed in batches to generate word vectors using the domain named entity word vector space. These vectors, along with the word vectors of new knowledge entities, are simultaneously input into two BiLSTM-CRF networks. The results are then output together into a fully connected network. After training the parameters of each network, a domain named entity recognition model integrating domain knowledge using a dual BiLSTM-CRF-fully connected network is formed. Figure 4 As shown. The specific steps are as follows:

[0062] 201. Based on the temporal relationship of the identified named entities in the domain short text, generate a set of named entity word vectors for the domain short text using the domain named entity word vector space;

[0063] 202. In the generated cluster of new knowledge entities, calculate the new knowledge entity that is most similar to the word vector of the named entity;

[0064] 203. Select a named entity word vector sequentially from the set of named entity word vectors, according to... Calculate entity word vector similarity, r m It is named entity word vectors. It is the set of new knowledge entities at the cluster center {e c A cluster center entity e in '} c 'Corresponding new knowledge entity word vectors;

[0065] 204. Calculation If the maximum value is less than the given threshold If the value is typically set to 0.6, it is considered that no similar new knowledge entity class has been found, and the process proceeds to step 207. If the maximum value is greater than the given threshold... Furthermore, if multiple values ​​have the same maximum value, and one of them is randomly selected, then it is considered that r is... m Belongs to this cluster;

[0066] 205. According to Calculate the similarity r between the named entity word vectors and the word vectors of each new knowledge entity in the cluster. e’ These are new knowledge entity word vectors;

[0067] 206. Calculate max(s(r) m r e’ If the maximum value is less than the given threshold δ (usually 0.75), it is considered that no similar new knowledge entity has been found, and the process jumps to step 207. If there are multiple identical maximum values, one of them is randomly selected. e’ , is the word vector of the new knowledge entity that is most similar to the named entity;

[0068] 207. Repeat steps 203 to 206 to find the corresponding new knowledge entity vector for each named entity in the named entity word vector set, and replace the corresponding named entity word vector in the set with the corresponding new knowledge entity word vector to generate the named entity word vector set that replaces the knowledge entity.

[0069] 208. Input the generated set of named entity word vectors of the replacement knowledge entities into the BiLSTM-CRF network of the replacement knowledge entities, and simultaneously input the set of named entity word vectors generated in step 201 into the BiLSTM-CRF network of the named entities to be identified.

[0070] 209. Input the outputs of the BiLSTM-CRF network for replacing knowledge entities and the BiLSTM-CRF network for naming entities to be identified into a fully connected neural network.

[0071] 210. Select short texts in the domain in batches, repeat steps 201-209, use the cross-entropy of named entity recognition as the loss function, and the mini-batch stochastic gradient descent with a learning rate of 0.1 as the optimization algorithm. Use PyTorch to train the BiLSTM-CRF network for knowledge entities, the BiLSTM-CRF network for named entities to be recognized, and the fully connected neural network to form the final domain named entity recognition network model that integrates knowledge graph and deep learning dual BiLSTM-CRF-fully connected neural network.

[0072] (III) Application Phase

[0073] like Figure 3 As shown, after word segmentation and word vector generation of short texts from the new domain, these are input into a dual BiLSTM-CRF fully connected network trained with word vectors of new knowledge entities. The network then outputs the recognition results of named entities in the short texts from that domain. The specific steps are as follows:

[0074] 301. Use the JIEBA word segmentation tool to segment the short text in the new field, and then use a stop word dictionary to remove colloquial words and stop words;

[0075] 302. Take the named entities to be identified in the domain short text, and generate a set of named entity word vectors for the domain short text according to the temporal relationship in the domain short text, using the domain named entity word vector space;

[0076] 303. Select one named entity word vector from the set of named entity word vectors to be identified in sequence, and follow the... Calculate entity word vector similarity, r m It is the named entity word vector to be identified. It is the new knowledge entity cluster center and the new knowledge entity set {e c A cluster center entity e in ′} c The corresponding new knowledge entity word vector;

[0077] 304. Calculation If the maximum value is less than the given threshold If the value is typically set to 0.6, it is considered that no similar new knowledge entity class has been found, and the process proceeds to step 307. If the maximum value is greater than the given threshold... Furthermore, if multiple values ​​have the same maximum value, and one of them is randomly selected, then it is considered that r is... m Belongs to this cluster;

[0078] 305. According to Calculate the similarity r between the word vector of the named entity to be identified and the word vector of each new knowledge entity in the cluster. e’ These are new knowledge entity word vectors;

[0079] 306. Calculate max(s(r) m r e’ If the maximum value is less than the given threshold δ (usually 0.6), it is considered that no similar new knowledge entity has been found, and the process jumps to step 307. If there are multiple identical maximum values, one of them, r, is randomly selected. e’ , is the word vector of the new knowledge entity most similar to the named entity to be identified;

[0080] 307. Repeat steps 303-306 to find the corresponding new knowledge entity vector for each named entity to be identified in the set of named entity word vectors to be identified, and replace the corresponding named entity word vector in the set of named entities to be identified with the corresponding new knowledge entity word vector to generate a set of named entity word vectors that replace the knowledge entities.

[0081] 308. Input the generated set of named entity word vectors of the replacement knowledge entities into the BiLSTM-CRF network of the replacement knowledge entities, and simultaneously input the generated set of named entity word vectors of the to-be-recognized named entities into the BiLSTM-CRF network of the to-be-recognized named entities. The output results of the two networks are input into the fully connected neural network, and the recognition results of named entities in the short text of this domain are output.

[0082] This invention can accurately and quickly identify key named entities in short texts in a domain, helping people to correctly understand information, and has great practical application value.

[0083] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present invention.

Claims

1. A domain-specific short text named entity recognition method integrating domain knowledge and deep learning, characterized in that, Includes the following steps: Step 1 involves selecting key knowledge entities and their relationships from existing military domain knowledge graphs, constructing a key knowledge entity association matrix, projecting these key knowledge entities into new knowledge entities, and then clustering these new entities. The specific method for Step 1 is as follows: Step 1-1: Select key knowledge entities and key entity relationships between key knowledge entities from existing domain knowledge graphs in the military field. Steps 1-2: Based on the entity relationships between knowledge entities, construct a knowledge entity association matrix S, where each element of the matrix represents the degree of association between two knowledge entities. ij , used to represent knowledge entity e in a domain knowledge graph i With knowledge entity e j The degree value of relationships in a knowledge graph; Steps 1-3, according to Calculate and generate the knowledge entity association transformation matrix W, where σ is taken as 1 by default; Steps 1-4, according to Calculate and generate the diagonal matrix D; Steps 1-5, according to the Laplace matrix A=D -1 / 2 WD 1 / 2 Generate the knowledge entity projection transformation matrix A; Steps 1-6: Transform the knowledge entity e into a new knowledge entity e' through the projection transformation matrix A, i.e., e' = Ae; Steps 1-7: For each new knowledge entity e', construct the association matrix S' of the new knowledge entity set based on the relation degree between the new knowledge entity and other new knowledge entities; Steps 1-8: Calculate E using the association matrix S' of the new knowledge entity set. i and E j E i E j Representing the new knowledge entity e respectively i '、e j 'The set of all new knowledge entities that have a relation, |E i | represents set E i The sum of the degrees of each new knowledge entity in the equation, |E j | represents set E j The sum of the degrees of each new knowledge entity in the middle; Steps 1-9, according to Computing new knowledge entity e i 'and new knowledge entity e j The similarity distance is given by E, where E represents the set of all new knowledge entities, and |E| represents the sum of the degrees of all new knowledge entities. i ∩E j For set E i With set E j The CCP is a set of new knowledge entities, |E i ∩E j | represents set E i ∩E j The sum of the knowledge entity degrees of China and Singapore; Steps 1-10: Based on the similarity distance of the new knowledge entities, the K-means clustering method is used to perform clustering calculations on the new knowledge entities, obtaining the clustering results of the new knowledge entities and the new knowledge entity e corresponding to each cluster center. c ', All new knowledge entities e c 'Constitutes a new set of knowledge entities at the cluster center {e c '}; Step 2: Collect short texts in the domain, identify the named entities in them, and then use the JIEBA word segmentation tool to segment the short texts in the domain, and use a stop word dictionary to remove colloquial words and stop words; use the global vector word embedding model GloVe to train word embeddings together with the association matrix S' of the new knowledge entity set to generate the domain named entity word vector space and the new knowledge entity word vectors. Step 3: For the named entities that have been identified in the domain short text, generate a set of named entity word vectors for the domain short text according to the temporal relationship in the domain short text, using the domain named entity word vector space; Step 4: In the clusters obtained from the new knowledge entity clustering, calculate the new knowledge entity that is most similar to the named entity word vector; the specific method of step 4 is as follows: Step 4-1: Select a named entity word vector sequentially from the named entity word vector set and calculate the entity word vector similarity. Where, r m It is named entity word vectors. It is the set of new knowledge entities at the cluster center {e c A cluster center entity e in '} c 'Corresponding new knowledge entity word vectors; Step 4-2: Calculate the maximum value of entity word vector similarity. If the maximum value is less than the threshold If no similar new knowledge entity class is found, proceed to step 5; if the maximum value is greater than the threshold... Then randomly select the r corresponding to one of the maximum values. m , believes that r m Belongs to this cluster; Step 4-3: For the clusters determined in Step 4-2, calculate the similarity s(r) between the named entity word vectors and the word vectors of each new knowledge entity in that cluster. m ,r e’ ): Where, r e′ These are new knowledge entity word vectors; Step 4-4, calculate s(r) m ,r e’ The maximum value of ) max(s(r) m ,r e′ If the maximum value is less than the threshold δ, it is considered that no similar new knowledge entity has been found, and the process proceeds to step 5; if the maximum value is greater than the threshold δ, then one of the maximum values ​​corresponding to r is randomly selected. e′ , which serves as the word vector of the most similar new knowledge entity to the named entity; Step 4-5: Repeat steps 4-1 to 4-4 to find the corresponding new knowledge entity vector for each named entity in the named entity word vector set, and replace the corresponding named entity word vector in the set with the new knowledge entity word vector to generate the named entity word vector set that replaces the knowledge entity. Step 5: Construct two BiLSTM-CRF networks: a knowledge entity replacement BiLSTM-CRF network and a named entity recognition BiLSTM-CRF network. Input the named entity word vector set generated in Steps 4-5 into the knowledge entity replacement BiLSTM-CRF network, and simultaneously input the named entity word vector set generated in Step 3 into the named entity recognition BiLSTM-CRF network. Step 6: Input the outputs of the knowledge entity replacement BiLSTM-CRF network and the name entity recognition BiLSTM-CRF network into a fully connected neural network; Step 7: Select short texts in the domain in batches, repeat steps 3 to 6, use cross-entropy of named entity recognition as the loss function, and mini-batch stochastic gradient descent with a learning rate of 0.1 as the optimization algorithm to train the parameters of the BiLSTM-CRF network for replacing knowledge entities, the BiLSTM-CRF network for recognizing named entities, and the fully connected neural network, forming the final domain named entity recognition network model that integrates knowledge graph and deep learning with dual BiLSTM-CRF-fully connected neural networks; Step 8: Use the domain-specific named entity recognition network model to identify named entities in the domain-specific short text.

Images