A Sequence Labeling Method and System Enhanced by Dynamic Knowledge Graph

Abstract

The present invention discloses a sequence annotation method and system enhanced based on a dynamic knowledge graph. The method includes: S1. Mining basic attribute information and related entity information from text information with sparse basic attributes and missing attributes to construct a knowledge graph; S2. Mapping each Token of the input sequence to an entity node of the knowledge graph and extracting a local subgraph; S3. Performing feature extraction through a graph convolutional network to obtain the entity embedding features of each Token; S4. Fusing the original features of each Token with the entity embedding features; S5. Inputting the fused features of each Token into a BiLSTM model to obtain the output features of the input sequence; S6. Inputting the output features of the input sequence into a CRF model to obtain the final comprehensive features of the input sequence. The present invention introduces a GCN model on the basis of BiLSTM and CRF to generate knowledge graph-based features, and fuses the original input features and the features generated by GCN, achieving better performance in sequence annotation in text with sparse basic attributes and missing attributes.

Description

Technical Field

[0001] The present invention relates to technical fields such as knowledge graphs, named entity recognition, deep learning, sequence labeling, data analysis, etc., and particularly relates to a sequence labeling method and system based on dynamic knowledge graph enhancement. Background Art

[0002] There is a large amount of text information in the network, but many text information often has characteristics such as high noise, non-standard format, and variable aliases. This poses a severe challenge to the high-precision recognition and labeling of entity attributes in the text. Accurately labeling these text information to obtain certain attribute values and features corresponding to this text information for subsequent use is the main purpose of sequence labeling. Sequence labeling was originally carried out manually item by item, which is very time-consuming and laborious. Although some semi-automatic tools have been used as assistance in recent years, a lot of manual intervention is still required. Currently, the research on sequence labeling methods for text information mainly focuses on the following aspects:

[0003] 1. The initial work of sequence labeling was mainly completed through manual labeling or through the Hidden Markov Model (HMM). However, there is currently a lack of a method for labeling possible attributes in some texts with sparse basic attributes and scarce attributes.

[0004] 2. The method of combining BiLSTM + CRF for sequence labeling can extract key attribute information to a certain extent, but there are still deficiencies in sequence labeling for scarce attributes in some texts.

[0005] A series of existing solutions have provided theoretical basis and method support for the research on sequence labeling of text information in the clear network. However, due to the weak modeling of long-distance context by these traditional methods, they cannot understand the dependency relationships of sentences in complex text semantics such as in the dark web. Especially when there is a large distance between entities in a long text or sentence, the accuracy of sequence labeling will deteriorate. Although the LSTM model can handle time series data well, they often cannot effectively capture the bidirectional dependency relationships in sentences, and are prone to problems such as gradient disappearance or explosion during training, resulting in unsatisfactory results when processing long text data. Summary of the Invention

[0006] Aiming at the deficiencies in the prior art, the present invention provides a sequence labeling method and system based on dynamic knowledge graph enhancement. Utilizing the characteristics of sparse basic attributes and missing attributes of these text information, a dynamically updatable knowledge graph is introduced. An enhanced feature based on the input feature subgraph is constructed through the GCN model, and then the known BiLSTM + CRF model is used to achieve high-precision sequence labeling of the input text.

[0007] The present invention provides a sequence annotation method enhanced based on a dynamic knowledge graph. The method includes:

[0008] S1. Mine basic attribute information and related entity information from text information with sparse basic attributes and missing attributes, and construct a knowledge graph. The knowledge graph includes a node set and an edge set. The node set includes attribute nodes corresponding to basic attribute information and entity nodes corresponding to related entity information. The edge set includes entity-entity edges and entity-attribute edges.

[0009] S2. Through named entity recognition, map each Token of the input sequence to the entity nodes of the knowledge graph, and extract the local subgraphs related to each entity node.

[0010] S3. Based on the node feature matrix and adjacency matrix of the local subgraph corresponding to each Token, perform feature extraction on the local subgraph corresponding to each Token through a graph convolutional network to obtain the entity embedding features of each Token.

[0011] S4. Fuse the original features of each Token with the entity embedding features to obtain the fused features of each Token. The original features are obtained by performing feature extraction on the original Tokens of the input sequence.

[0012] S5. Input the fused features of each Token into a BiLSTM model to generate the context features of each Token, and obtain the output features of the input sequence.

[0013] S6. Input the output features of the input sequence into a CRF model, calculate the scores of each possible label sequence, find the optimal label sequence with the highest score from the scores of each possible label sequence. Based on the optimal label sequence, transform it through an embedding matrix to obtain an attribute embedding matrix. After fusing the entity embedding features of each Token with the corresponding attribute embedding matrix, and then splicing them with the corresponding original features, the final comprehensive features of the input sequence are obtained.

[0014] Preferably, the basic attribute information includes: application field, product value, information content, information date, information size; the related entity information includes: related countries, related regions, related persons, related institutions.

[0015] Preferably, in step S2, the local subgraph of the entity node corresponding to the Token is expressed as:

[0016] ;

[0017] wherein, is the entity node corresponding to the t th Token The relevant local sub-graph V is the node set of the knowledge graph E is the edge set of the knowledge graph is the node in the node set V in is the edge in the edge set E in i, j, t is a positive integer

[0018] Preferably, in step S3, the propagation rule of the graph convolutional network is:

[0019] ;

[0020] where is the node feature representation of the th layer is the adjacency matrix = + is the adjacency matrix with self-loops added is the diagonal degree matrix is the weight matrix of the th layer, and the initial node feature is usually represented by word embedding or the attribute vector of the entity l is a positive integer

[0021] The high-order feature of the local sub-graph finally generated by the graph convolutional network is represented as:

[0022] ;

[0023] where is the entity embedding feature corresponding to the t th Token t, T is a positive integer

[0024] Preferably, in step S4, assume the input sequence is , and its corresponding attribute sequence is , and the fusion feature representation of each Token is:

[0025] ;

[0026] where is the fusion feature of the t th Token is the original feature of the t th Token is to tThe entity embedding features output after mapping each Token to the knowledge graph and processing through the graph convolutional network.

[0027] Preferably, in step S5, after inputting the fusion features of each Token into the BiLSTM model, the context features of each Token are generated by the following method to obtain the output features of the input sequence:

[0028] Generate the above-context features of each Token through the forward LSTM of the BiLSTM model;

[0029] Generate the below-context features of each Token through the backward LSTM of the BiLSTM model;

[0030] Perform bidirectional concatenation on the above-context features and below-context features of each Token to generate the context features of each Token, and obtain the output features of the input sequence;

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] Among them, is the fusion feature of the t th Token, is the above-context feature of the t th Token, is the above-context feature of the t th Token, is the context feature of the t th Token, is the output feature of the input sequence.

[0036] Preferably, in step S6, the final comprehensive features of the input sequence are obtained by the following method:

[0037] ;

[0038] ;

[0039] ;

[0040] Among them, is the score of each possible label sequence output by the CRF model, is the weight vector corresponding to the attribute , is the context feature of the t th Token, is the entity embedding feature of the t th Token, is the transition score between adjacent attributes, and the optimal label sequence is , is the attribute embedding matrix, is the final comprehensive feature combining attribute information and external knowledge graph information, t, T is a positive integer.

[0041] Preferably, in step S6, through Viterbi algorithm decoding, the optimal label sequence with the highest score is found from the scores of each possible label sequence.

[0042] Based on the same inventive concept, the present invention also provides a sequence annotation system enhanced by a dynamic knowledge graph, and the system includes:

[0043] A knowledge graph construction module, which is used to mine basic attribute information and related entity information from text information with sparse basic attributes and missing attributes, and construct a knowledge graph. Among them, the knowledge graph includes a node set and an edge set. The node set includes attribute nodes corresponding to basic attribute information and entity nodes corresponding to related entity information, and the edge set includes entity-entity edges and entity-attribute edges;

[0044] A local subgraph extraction module, which is used to map each Token of the input sequence to the entity nodes of the knowledge graph through named entity recognition, and extract local subgraphs related to each entity node;

[0045] A feature extraction module, which is used to perform feature extraction on the local subgraph corresponding to each Token through a graph convolutional network based on the node feature matrix and adjacency matrix of the local subgraph corresponding to each Token, and obtain the entity embedding feature of each Token;

[0046] A feature fusion module, which is used to fuse the original feature of each Token with the entity embedding feature to obtain the fusion feature of each Token, where the original feature is obtained by performing feature extraction on the original Token of the input sequence;

[0047] A context feature generation module, which is used to input the fusion feature of each Token into a BiLSTM model to generate the context feature of each Token and obtain the output feature of the input sequence;

[0048] The comprehensive feature output module is used to input the output features of the input sequence into a CRF model, calculate the scores of each possible label sequence, find the optimal label sequence with the highest score from the scores of each possible label sequence, based on the optimal label sequence, transform it through an embedding matrix to obtain an attribute embedding matrix, fuse the entity embedding features of each Token with the corresponding attribute embedding matrix, and then concatenate them with the corresponding original features to obtain the final comprehensive features of the input sequence.

[0049] Preferably, the basic attribute information includes: application field, product value, information content, information date, information size; the relevant entity information includes: relevant countries, relevant regions, relevant persons, relevant institutions.

[0050] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0051] 1. The present invention takes a large amount of illegally trafficked text information in the dark web as the research object. After data cleaning and text tokenization, on the basis of the traditional sequence labeling using BiLSTM model and CRF model, a dynamically updated knowledge graph and a graph convolutional network (GCN) are introduced to enhance feature representation, so as to effectively address the problems of missing dark web text attributes, implicit expressions, entity polysemy, and domain dynamic changes.

[0052] 2. The present invention constructs a dynamically updated knowledge graph according to the basic attribute information and relevant entity information in the text information with sparse basic attributes and missing attributes. By mapping the original input to the knowledge graph, the corresponding knowledge subgraph of the input is obtained, so as to identify and generate some possible attributes of the input text.

[0053] 3. The present invention introduces a GCN model on the basis of the BiLSTM model and the CRF model to generate features based on the knowledge graph, and fuses the original input features and the features generated by the GCN, and achieves good performance in sequence labeling in these texts with sparse basic attributes and missing attributes. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic flowchart of a sequence labeling method based on dynamic knowledge graph enhancement provided by the present invention;

[0055] Figure 2 It is an overall architecture diagram of a sequence labeling method based on dynamic knowledge graph enhancement provided by the present invention;

[0056] Figure 3 It is a schematic structural diagram of a sequence labeling system based on dynamic knowledge graph enhancement provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0058] The following further describes the present invention in detail with reference to the accompanying drawings.

[0059] As Figure 1 shown, an embodiment of the present invention provides a sequence annotation method based on enhanced dynamic knowledge graph, and the method includes:

[0060] S1. Mine basic attribute information and related entity information from text information with sparse basic attributes and missing attributes, and construct a knowledge graph, where the knowledge graph includes a node set and an edge set, the node set includes attribute nodes corresponding to basic attribute information and entity nodes corresponding to related entity information, and the edge set includes entity-entity edges and entity-attribute edges;

[0061] In the embodiments of the present invention, the basic attribute information includes: such as application field, product value, information content, information date, information size, etc. For example, the text may mention the scale of a specific data leak (information size), the field of the traded item (such as the network security field), and the relevant date. The related entity information includes: related countries, related regions, related persons, related institutions, etc. Dark web texts often involve international or cross-regional criminal gangs, financial institutions, or specific personal information. The present invention includes the identification of these attributes as a key point to support subsequent sequence annotation tasks.

[0062] In the dynamically updated knowledge graph, each piece of related entity information / basic attribute information is a network node of this knowledge graph. The edges of the knowledge graph are also divided into two categories: entity-entity edges and entity-attribute edges. If there is a relationship between entities, it means there is an interaction behavior / event between two entity nodes. If there is a relevant relationship between an entity and a basic attribute, it means this entity has this attribute feature.

[0063] S2. Through named entity recognition, map each Token of the input sequence to an entity node of the knowledge graph, and extract a local subgraph related to each entity node;

[0064] In the embodiments of the present invention, in step S2, the local subgraph of the entity node corresponding to the Token is expressed as:

[0065] ;

[0066] where For the entity node corresponding to the t th Token associated local subgraph, V is the node set of the knowledge graph, E is the edge set of the knowledge graph, is the node in the node set V , is the edge in the edge set E , i, j, t is a positive integer.

[0067] The local subgraph includes the first-order neighbor nodes (i.e., the attributes and entities directly associated with this node), and this subgraph can continue to be extended to the second-order neighbors to capture a more extensive context relationship.

[0068] S3. Based on the node feature matrix and adjacency matrix of the local subgraph corresponding to each Token, feature extraction is performed on the local subgraph corresponding to each Token through a graph convolutional network to obtain the entity embedding features of each Token;

[0069] In the embodiment of the present invention, in step S3, the propagation rule of the graph convolutional network is:

[0070] ;

[0071] wherein, is the node feature representation of the th layer, is the adjacency matrix, = + is the adjacency matrix with self-loops added, is the diagonal degree matrix, is the weight matrix of the th layer, and the initial node feature is usually represented by a word embedding or an attribute vector of an entity, l is a positive integer;

[0072] The high-order features of the local subgraph finally generated by the graph convolutional network are represented as:

[0073] ;

[0074] wherein, is the entity embedding feature corresponding to the t th Token, t, T is a positive integer.

[0075] S4. Fuse the original features of each Token with the entity embedding features to obtain the fused features of each Token, where the original features are obtained by extracting features from the original Tokens of the input sequence.

[0076] In the embodiment of the present invention, in step S4, assume that the input sequence is , and its corresponding attribute sequence is . The fused feature of each Token is represented as:

[0077] ;

[0078] where is the fused feature of the t th Token, is the original feature of the t th Token, is the entity embedding feature output after processing the t th Token mapped to the knowledge graph through the graph convolutional network.

[0079] S5. Input the fused features of each Token into the BiLSTM model to generate the context features of each Token, and obtain the output features of the input sequence.

[0080] In the embodiment of the present invention, in step S5, after inputting the fused features of each Token into the BiLSTM model, the context features of each Token are generated through the following method to obtain the output features of the input sequence:

[0081] Generate the above-context features of each Token through the forward LSTM of the BiLSTM model;

[0082] Generate the below-context features of each Token through the backward LSTM of the BiLSTM model;

[0083] Perform bidirectional concatenation on the above-context features and below-context features of each Token to generate the context features of each Token, and obtain the output features of the input sequence;

[0084] ;

[0085] ;

[0086] ;

[0087] ;

[0088] where is the fused feature of the t th Token, is the context feature of the previous text for the t th Token, is the context feature of the previous text for the t th Token, is the context feature of the t th Token, is the output feature of the input sequence.

[0089] Integrate the entity embedding features output by GCN into the existing BiLSTM+CRF sequence labeling scheme. The role of the bidirectional LSTM model (BiLSTM) is to capture the context information of each Token in the input sequence. For each input sequence, BiLSTM generates the context feature representation of each Token through the combination of the forward LSTM and the backward LSTM, while CRF is responsible for modeling the global dependencies between labels, thereby achieving high-precision sequence labeling.

[0090] S6. Input the output feature of the input sequence into the CRF model, calculate the scores of each possible label sequence, find the optimal label sequence with the highest score from the scores of each possible label sequence. Based on the optimal label sequence, obtain the attribute embedding matrix through the embedding matrix transformation. After fusing the entity embedding feature of each Token with the corresponding attribute embedding matrix, then splice it with the corresponding original feature to obtain the final comprehensive feature of the input sequence.

[0091] In the embodiment of the present invention, in step S6, the final comprehensive feature of the input sequence is obtained by the following method:

[0092] ;

[0093] ;

[0094] ;

[0095] wherein, is the score of each possible label sequence output by the CRF model, is the corresponding weight vector, is the context feature of the t th Token, is the t th Token's entity embedding feature, is the transition score between adjacent attributes, and the optimal label sequence is , is the attribute embedding matrix, is the final comprehensive feature that combines attribute information and external knowledge graph information, t, T is a positive integer.

[0096] In the embodiment of the present invention, in step S6, through Viterbi algorithm decoding, the optimal label sequence with the highest score is found from the scores of each possible label sequence. The final comprehensive feature combines the attribute information and the external knowledge graph information, provides richer and more accurate input features for subsequent tasks, and uses it as the input for subsequent model training and classification.

[0097] As Figure 3 shown, the present invention also provides a sequence annotation system based on dynamic knowledge graph enhancement. The system includes:

[0098] A knowledge graph construction module 100, which is used to mine basic attribute information and related entity information from text information with sparse basic attributes and missing attributes, and construct a knowledge graph. The knowledge graph includes a node set and an edge set. The node set includes attribute nodes corresponding to basic attribute information and entity nodes corresponding to related entity information. The edge set includes entity-entity edges and entity-attribute edges;

[0099] A local subgraph extraction module 200, which is used to map each Token of the input sequence to an entity node of the knowledge graph through named entity recognition, and extract a local subgraph related to each entity node;

[0100] A feature extraction module 300, which is used to perform feature extraction on the local subgraph corresponding to each Token through a graph convolutional network based on the node feature matrix and the adjacency matrix of the local subgraph corresponding to each Token, and obtain the entity embedding feature of each Token;

[0101] A feature fusion module 400, which is used to fuse the original feature of each Token with the entity embedding feature to obtain the fusion feature of each Token, where the original feature is obtained by performing feature extraction on the original Token of the input sequence;

[0102] A context feature generation module 500, which is used to input the fusion feature of each Token into a BiLSTM model to generate the context feature of each Token, and obtain the output feature of the input sequence;

[0103] A comprehensive feature output module 600, which is used to input the output feature of the input sequence into a CRF model, calculate the scores of each possible label sequence, find the optimal label sequence with the highest score from the scores of each possible label sequence, and based on the optimal label sequence, obtain an attribute embedding matrix through transformation by an embedding matrix. After fusing the entity embedding feature of each Token with the corresponding attribute embedding matrix, and then splicing it with the corresponding original feature, the final comprehensive feature of the input sequence is obtained.

[0104] Compared with the prior art, the advantages of the present invention are:

[0105] At present, there is no effective attribute annotation system for text information with sparse basic attributes and missing attributes. Therefore, the present invention constructs a dynamically updated knowledge graph based on the basic attribute information and relevant entity information in these texts, and obtains the corresponding knowledge subgraph of the input by mapping the original input to the knowledge graph, so as to identify and generate certain attributes that may exist in the input text.

[0106] Traditionally, the work of sequence annotation mainly relies on manual annotation or implicit Markov model HMM for auxiliary annotation. Even when combined with some semi-automatic tools like the HMM model, a large amount of manual intervention is still required. The annotation process is time-consuming and laborious, and it is also difficult to scale to large-scale data sets, especially the annotation efficiency of text data in complex scenarios such as the dark web and telegram groups is extremely low. This solution introduces the GCN model on the basis of the BiLSTM model and the CRF model to generate features based on the knowledge graph, and fuses the original input features and the features generated by the GCN, and has achieved good performance in sequence annotation in texts with sparse basic attributes and missing attributes.

[0107] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A sequence annotation method enhanced based on a dynamic knowledge graph, characterized in that The method includes: S1. Mining basic attribute information and related entity information from text information with sparse basic attributes and missing attributes, and constructing a knowledge graph, where the knowledge graph includes a node set and an edge set, the node set includes attribute nodes corresponding to basic attribute information and entity nodes corresponding to related entity information, and the edge set includes entity-entity edges and entity-attribute edges; S2. Through named entity recognition, mapping each Token of the input sequence to the entity nodes of the knowledge graph, and extracting local subgraphs related to each entity node; S3. Based on the node feature matrix and adjacency matrix of the local subgraph corresponding to each Token, using a graph convolutional network to extract features from the local subgraph corresponding to each Token, and obtaining the entity embedding features of each Token; the propagation rule of the graph convolutional network is: ； Among them, is the node feature representation of the layer, is the adjacency matrix, = + is the adjacency matrix with self-loops added, is the diagonal degree matrix, is the weight matrix of the layer, and the initial node feature is usually represented by word embeddings or attribute vectors of entities, is a positive integer; S4. Fusing the original features of each Token with the entity embedding features to obtain the fused features of each Token, where the original features are obtained by extracting features from the original Tokens of the input sequence; S5. Inputting the fused features of each Token into a BiLSTM model to generate the context features of each Token, and obtaining the output features of the input sequence; S6. Inputting the output features of the input sequence into a CRF model, calculating the scores of each possible label sequence, finding the optimal label sequence with the highest score from the scores of each possible label sequence, based on the optimal label sequence, transforming it through an embedding matrix to obtain an attribute embedding matrix, fusing the entity embedding features of each Token with the corresponding attribute embedding matrix, and then splicing them with the corresponding original features to obtain the final comprehensive features of the input sequence.

2. The method according to claim 1, wherein, The basic attribute information includes: application field, product value, information content, information date, information size; the related entity information includes: related countries, related regions, related persons, related institutions.

3. The method according to claim 1, wherein In step S2, the local subgraph of the entity node corresponding to the Token is represented as: ； Among them, is the local subgraph t corresponding to the th Token, V is the node set of the knowledge graph, E is the edge set of the knowledge graph, is the node V in the node set, is the edge E in the edge set, i, j, t is a positive integer.

4. The method according to claim 1, characterized in that, In step S3, The high-order features of the local subgraph finally generated by the graph convolutional network are expressed as: ； Among them, is the entity embedding feature corresponding to the t th Token, t, T where t, T is a positive integer.

5. The method according to claim 1, characterized in that In step S4, assume the input sequence is , and its corresponding attribute sequence is . The fusion feature representation of each Token is: ； Among them, is the fusion feature of the t th Token, is the original feature of the t th Token, is the entity embedding feature output after mapping the t th Token to the knowledge graph and processing it through the graph convolutional network.

6. The method according to claim 1, characterized in that, In step S5, after inputting the fused features of each Token into the BiLSTM model, the context features of each Token are generated by the following method to obtain the output features of the input sequence: Generating the context features of each Token through the forward LSTM of the BiLSTM model; Generating the context features of each Token through the backward LSTM of the BiLSTM model; Bi-directionally splicing the context features of each Token and the context features of each Token to generate the context features of each Token, and obtaining the output features of the input sequence; ； ； ； ； Among them, is the fusion feature of the t th Token, is the upstream feature of the t th Token, is the downstream feature of the t th Token, is the context feature of the t th Token, is the output feature of the input sequence.

7. The method according to claim 1, characterized in that, In step S6, the final comprehensive features of the input sequence are obtained by the following method: ； ； ； Among them, is the score of each possible tag sequence output by the CRF model, is the attribute corresponding weight vector, is the context feature of the t th Token, is the entity embedding feature of the t th Token, is the transition score between adjacent attributes, and the optimal tag sequence is , is the attribute embedding matrix, is the original feature of the t th Token, is the final comprehensive feature combining attribute information and external knowledge graph information, t, N is a positive integer.

8. The method according to claim 1, wherein In step S6, decoding through the Viterbi algorithm to find the optimal label sequence with the highest score from the scores of each possible label sequence.

9. A sequence annotation system enhanced based on a dynamic knowledge graph, characterized in that The system includes: A knowledge graph construction module, which is used to mine basic attribute information and related entity information from text information with sparse basic attributes and missing attributes, and construct a knowledge graph. Among them, the knowledge graph includes a node set and an edge set. The node set includes attribute nodes corresponding to basic attribute information and entity nodes corresponding to related entity information. The edge set includes entity-entity edges and entity-attribute edges; A local subgraph extraction module, which is used to map each Token of the input sequence to the entity nodes of the knowledge graph through named entity recognition, and extract the local subgraphs related to each entity node; A feature extraction module, which is used to perform feature extraction on the local subgraph corresponding to each Token through a graph convolutional network based on the node feature matrix and adjacency matrix of the local subgraph corresponding to each Token, and obtain the entity embedding feature of each Token; The propagation rule of the graph convolutional network is: ； Among them, is the node feature representation of the layer, is the adjacency matrix, = + is the adjacency matrix with self-loops added, is the diagonal degree matrix, is the weight matrix of the layer, and the initial node feature is usually represented by word embeddings or the attribute vector of the entity, is a positive integer; A feature fusion module, which is used to fuse the original feature of each Token with the entity embedding feature to obtain the fusion feature of each Token, where the original feature is obtained by performing feature extraction on the original Tokens of the input sequence; A context feature generation module, which is used to input the fusion feature of each Token into a BiLSTM model to generate the context feature of each Token, and obtain the output feature of the input sequence; A comprehensive feature output module, which is used to input the output feature of the input sequence into a CRF model, calculate the scores of each possible label sequence, find the optimal label sequence with the highest score from the scores of each possible label sequence, and based on the optimal label sequence, transform it through an embedding matrix to obtain an attribute embedding matrix. After fusing the entity embedding feature of each Token with the corresponding attribute embedding matrix, and then splicing it with the corresponding original feature, the final comprehensive feature of the input sequence is obtained.

10. The system according to claim 9, wherein The basic attribute information includes: application field, product value, information content, information date, information size; The related entity information includes: related countries, related regions, related people, related institutions.

Images