Knowledge graph construction model based on two stages, construction method thereof and storage medium

Abstract

The invention provides a knowledge graph construction model based on two stages, a construction method thereof and a storage medium, and the method comprises the steps: constructing a knowledge graph by adopting a top-down method and a bottom-up method, firstly, constructing a data pool through the collection and fusion of a multi-source heterogeneous data set, and carrying out the data preprocessing of original data, comprising the steps of missing value filling, noise filtering, standardized conversion and other cleaning operations; then, entity-relation label annotation is carried out on the cleaned data, so that a high-quality annotation data set is provided for subsequent model training; then, a multi-feature fusion named entity recognition model (MacBERT-MultiFeature-Fusion) based on an improved MacBERT structure is provided, and the model can accurately recognize a named entity with a semantic mark from the unstructured text data; thirdly, a novel relation extraction model MacBERT-MaskFUSE is provided, the model is combined with a dynamic mask mechanism, the semantic relation between entities is extracted from the text, and the specific target relation is effectively recognized; and finally, embedding the extracted entities and relationships into a Neo4j graph database, constructing a vertical industry knowledge graph in the form of the graph database, and realizing fusion and query analysis functions of multi-source knowledge. Through the constructed knowledge graph, efficient integration and semantic analysis of the multi-source data can be realized, and the interpretability and availability of the data are improved. The method can be widely applied to scenes of intelligent recommendation, information retrieval, question and answer systems and the like.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph technology, and in particular to a two-stage knowledge graph construction model and a construction method and storage medium thereof. Background Art

[0002] Knowledge graph is a structured data representation that presents entities and their semantic relationships in the form of nodes and edges. Knowledge graphs use nodes to represent entities and edges to represent relationships between entities, thereby building a knowledge network that can be used for reasoning and querying. Knowledge graphs not only record basic information about entities and relationships, but also contain rich semantic information, making them very useful in artificial intelligence and natural language processing tasks. Through knowledge graphs, machines can understand and infer the associations and contexts between different entities, thus having stronger knowledge expression capabilities.

[0003] In the existing technology, knowledge graphs in general fields usually cover a wide range of general information and are suitable for scenarios such as general search and intelligent question and answer. However, in professional fields, general knowledge graphs often cannot meet the complex needs of specific industries. Therefore, expanding the construction of knowledge graphs from general fields to vertical industry fields is an important development direction for deepening the application of knowledge graphs. To this end, it becomes crucial to build knowledge graphs for vertical industries, which can focus on specialized knowledge in specific fields and provide more accurate and in-depth knowledge services through refined entity and relationship models.

[0004] In the general technology field, the objects of knowledge graph construction are mainly web documents, which have relatively consistent structures and mostly public general knowledge. However, in vertical industries, the document types of knowledge sources are more diverse, covering database documents, Office documents, technical reports, industry papers, and even multimedia content (such as video, audio, etc.). The structure and content of these documents are complex and diverse, and highly professional. This difference makes it difficult for the construction method of general field knowledge graphs to be directly applied to vertical industries. In vertical industries, the professional terminology, knowledge depth, and data format of documents vary. When constructing knowledge graphs, it is necessary to combine industry characteristics and specific data sources for targeted processing. Therefore, building a knowledge graph for a vertical industry requires not only flexible parsing and processing capabilities, but also industry knowledge to ensure the professionalism and practicality of the graph. Summary of the invention

[0005] Based on the problem that in professional fields, general knowledge graphs are often difficult to meet the complex needs of specific industries, the present invention provides a two-stage knowledge graph construction model and its construction method and storage medium, which constructs a knowledge graph for the vertical industry by pre-acquiring the target entity types and target attributes of various types of documents in the vertical field, and then classifying the entity types and attributes of the documents to be processed.

[0006] The first aspect of the present invention is a knowledge graph construction model and a construction method thereof, comprising:

[0007] The knowledge graph is constructed using both top-down and bottom-up methods;

[0008] Build a data pool by collecting and fusing multi-source heterogeneous data sets, and preprocess the raw data, including missing value filling, noise filtering, standardization conversion and other cleaning operations;

[0009] Annotate the cleaned data with entity-relationship labels to provide a high-quality labeled dataset for subsequent model training;

[0010] A multi-feature fusion named entity recognition model (MacBERT-MultiFeature-Fusion) based on the improved MacBERT structure is proposed. This model can accurately identify semantically labeled named entities from unstructured text data.

[0011] A new relation extraction model, MacBERT-MaskFUSE, is proposed. This model combines a dynamic mask mechanism to extract semantic relations between entities from text and effectively identify specific target relations.

[0012] The extracted entities and relationships are embedded into the Neo4j graph database, and a vertical industry knowledge graph is constructed in the form of a graph database to realize the fusion and query analysis functions of multi-source knowledge.

[0013] The knowledge graph is constructed using two methods: top-down and bottom-up.

[0014] Starting from the overall knowledge framework from top to bottom, we first define the key concepts, entity types, attributes, and relationship structures in the knowledge graph, and then sort and organize the data according to this global framework so that the data conforms to the pre-designed structure;

[0015] Starting from the actual data from the bottom up, entities and relationships are directly extracted, and a knowledge graph is gradually constructed. The extracted entities and relationships are then clustered, classified, and associated to gradually form the overall structure of the knowledge graph.

[0016] The data pool is constructed by collecting and fusing multi-source heterogeneous data sets, and the raw data is preprocessed, including missing value filling, noise filtering, standardization conversion and other cleaning operations, where:

[0017] Missing value filling refers to replacing missing values ​​in a data set with appropriate values ​​or methods in data processing to ensure data integrity. By filling missing values, data quality can be improved, making the data more complete and consistent, which is beneficial for subsequent data analysis and model training.

[0018] Noise filtering refers to identifying and removing the "noise" part of the data during data processing to improve data quality and model accuracy. Noise usually refers to data that is irrelevant, erroneous, abnormal, or has a significant impact on the analysis results, such as erroneous data, irrelevant information, outliers, and duplicate data.

[0019] Standardization transformation refers to converting data into a uniform scale or distribution to ensure that the magnitude of different features is similar in subsequent analysis or modeling, avoiding the impact of different data scales on model performance.

[0020] The entity-relationship label annotation of the cleaned data is performed, that is, after preprocessing the original data, the entity and relationship labels are finely labeled using doccano software according to the predefined entity and relationship category system to ensure that the data set has consistent and high-quality semantic annotations, thereby providing high-quality annotated corpus for the subsequent deep learning model training.

[0021] The multi-feature fusion named entity recognition model based on the improved MacBERT structure consists of an entity position encoding layer, a MacBERT feature extraction layer, a text encoding layer, a feature fusion layer, a multi-head self-attention mechanism layer and a conditional random field layer, wherein:

[0022] The first layer is the entity position encoding layer. Its function is to help the model capture and represent the position information of each word in the input sequence, improve the model's understanding of the sentence structure, and thus more accurately identify entities. The steps of the entity position encoding layer are:

[0023] Generate positional codes: that is, generate a unique code for each position in the sequence;

[0024] Add position encoding to word embedding: add the generated position information encoding to the word embedding vector of each word;

[0025] Embedding layer passed to the model: The word embedding vector after position encoding is passed as input to the feature extraction layer of the model, so that the model can perceive the position information of the word at each step.

[0026] The second layer is the MacBERT feature extraction layer. The feature extraction layer generates a feature vector after extracting features from the pre-trained model MacBERT, which improves the masking strategy and enhances the context understanding ability to generate high-quality feature representations for each word.

[0027] The third and fourth layers are the text encoding layer and feature fusion layer respectively. The text encoding layer and feature fusion layer dynamically weightedly combine the iterative dilated convolutional neural network (IDCNN) and the bidirectional gated recurrent unit (BiGRU), and normalize the weights through Softmax, so that the model can balance and adjust between local features and long-distance dependent features to adapt to different text input situations and achieve better entity recognition results. The steps of the text encoding layer and feature fusion layer are:

[0028] Get the feature output of IDCNN and BiGRU;

[0029] Calculate the original weighting coefficients;

[0030] Use the Softmax function to normalize the original weight coefficients;

[0031] Weighted fusion of the outputs of IDCNN and BiGRU;

[0032] The fused features are input into the next layer for training and entity recognition.

[0033] The fifth layer is the multi-head self-attention mechanism layer. The multi-head self-attention mechanism layer performs weighted calculations on the input features, so it can further screen and weight the features fused by IDCNN and BiGRU, so that important features get higher weights. Through the multi-head self-attention mechanism, each word can directly establish connections with all other words, focusing on distant words related to it in the sentence, thereby effectively capturing long-distance dependencies; multi-head attention allows the model to learn different feature patterns in different subspaces. For the input feature sequence, different heads can focus on different contextual information, thereby obtaining a richer contextual feature expression. Through the multi-head mechanism, the model can better comprehensively consider the semantic connections between different positions. The steps of the multi-head self-attention mechanism layer are:

[0034] Input embedding layer: converts text sequence into embedding vector;

[0035] Linear transformation: For each input vector, three linear transformations are calculated, namely query, key, and value.

[0036] Calculate attention weights: Calculate the relevance between each position using the dot product between the query and the key;

[0037] Compute weighted feature representations;

[0038] Multiple result concatenation and linear transformation;

[0039] The results are passed to subsequent layers.

[0040] The sixth layer is the conditional random field layer. The conditional random field layer can improve the accuracy of sequence labeling by modeling the dependencies between labels. Unlike the word-by-word labeling method, the conditional random field layer performs global optimization by calculating the labeling score of the entire sequence, so that the model can comprehensively consider the label sequence of the entire sentence to obtain the optimal labeling result; when the model predicts an incorrect label at a single position, the conditional random field layer can use the dependencies between labels to correct this error.

[0041] Among them, the evaluation indicators of the multi-feature fusion named entity recognition model are composed of precision (P), recall (R) and F1 value, which are used to evaluate the accuracy and effect of the model prediction results. The calculation formula is as follows:

[0042]

[0043] Among them, TP (True Positives) is the number of samples that are actually positive and predicted as positive; FP (False Positives) is the number of samples that are actually negative but predicted as positive; FN (False Negatives) is the number of samples that are actually positive but predicted as negative.

[0044] The novel relation extraction model MacBERT-MaskFUSE consists of a feature extraction layer, a mask perception layer, a text encoding layer and an output layer, wherein:

[0045] The first layer is the feature extraction layer. The MacBERT pre-trained model encodes and extracts features from the input sequence to obtain contextual feature representations of each word. These feature representations contain not only the semantic information of each word, but also its relational information in the context, which helps to identify potential entity relationships. After obtaining these features, the model will further process to extract the relationship information between entity pairs in the sentence, extract the representation of the target entity from the encoded sequence features, or obtain the aggregated features of the entity through specific tags. These features will be input into the mask perception layer to further model the relationship between entity pairs.

[0046] The second layer is the mask perception layer. The mask perception layer explicitly marks the position of the target entity in the input feature through position encoding or masking mechanism, which can further filter and process the features from the feature extraction layer, so that the model can focus more on the entities and their relationship information in the sentence. It enables the model to identify the entity pairs in the sentence, helping the model to focus on the part of the sentence related to the entity relationship, ignoring other irrelevant words and information, and can also filter or weight word information outside the target entity to reduce its influence in relationship extraction. Through the masking mechanism, the model can effectively filter out irrelevant context, so that the noise is reduced when the features are aggregated. The steps of the mask perception layer are:

[0047] Get initial features from the feature extraction layer;

[0048] Generate entity location mask;

[0049] Apply mask weights to weight features;

[0050] Context feature screening and integration;

[0051] Feature aggregation.

[0052] The third layer is the text encoding layer. This layer consists of a bidirectional gated recurrent unit (BiGRU) and a text convolutional neural network (TextCNN). The local features extracted by TextCNN are combined with the global context information of BiGRU to enhance the representation ability of the model, thereby improving the effect of relationship extraction. The steps of the text encoding layer are:

[0053] Input the feature information output by the mask perception layer into the BiGRU layer;

[0054] Combine the output of BiGRU with TextCNN to further extract local semantic features;

[0055] Using local features to enhance the representation capabilities of relation extraction models.

[0056] The fourth layer is the output layer. The output layer maps the high-dimensional feature vector passed from the text encoding layer to the relationship category space and generates a score for each relationship category; after Softmax processing, the output layer calculates the probability of each relationship type and finally determines the predicted relationship category based on the maximum probability. The steps of the output layer are:

[0057] Mapping high-dimensional feature vectors to relational category space;

[0058] Linear transformation;

[0059] Softmax processing, calculating the probability of each category;

[0060] Select the relationship category with the highest probability;

[0061] Output the prediction results.

[0062] Among them, the evaluation indicators of the relation extraction model MacBERT-MaskFUSE are composed of precision (P), recall (R) and F1 value, which are used to evaluate the accuracy and effect of the model prediction results. The calculation formula is as follows:

[0063]

[0064] Among them, TP (True Positives) is the number of samples that are actually positive and predicted as positive; FP (False Positives) is the number of samples that are actually negative but predicted as positive; FN (False Negatives) is the number of samples that are actually positive but predicted as negative.

[0065] The extracted entities and relationships are embedded into the Neo4j graph database to build a vertical industry knowledge graph in the form of a graph database. The multi-feature fusion named entity recognition model is used to identify entities in the text and provide a basis for the node construction of the knowledge graph; the relationship extraction model is used to identify and extract the relationships between entities in the text. These relationships will serve as edges in the knowledge graph to connect different entity nodes.

[0066] A second aspect of the present invention provides a knowledge graph storage medium, comprising:

[0067] The computer execution instructions are stored on a computer-readable storage medium, and when the program is running, the device where the computer-readable storage medium is located is controlled to execute the knowledge graph construction method based on named entity recognition and relationship extraction as described above.

[0068] The advantages and beneficial effects of the present invention are as follows:

[0069] After the sequence text is input, the MacBERT pre-training model encodes and extracts features of the input sequence. Compared with the BRTT pre-training model, MacBERT is a pre-training task that is more in line with the characteristics of Chinese. It improves the masking strategy and noise perturbation method in BERT, and significantly improves the performance and generalization ability of the model on Chinese tasks.

[0070] In the named entity recognition task, a multi-feature fusion named entity recognition model (MacBERT-MultiFeature-Fusion) based on the improved MacBERT structure is proposed. The model consists of an entity position encoding layer, a MacBERT feature extraction layer, a text encoding layer, a feature fusion layer, a multi-head self-attention mechanism layer, and a conditional random field layer. In the text encoding layer and the feature fusion layer, the iterative dilated convolutional neural network (IDCNN) and the bidirectional gated recurrent unit (BiGRU) are dynamically weighted and combined, and the weights are normalized by Softmax. The above operation can effectively integrate the advantages of the two networks and alleviate the gradient vanishing and long dependency problems. IDCNN bypasses the gradient vanishing problem through the convolution structure and can quickly extract local and global features. BiGRU uses a gating mechanism to retain long-distance dependency information. At the same time, the features captured by IDCNN and BiGRU each focus on different aspects. IDCNN focuses on spatial structure information, while BiGRU pays more attention to temporal information. Through dynamic weighted combination, multiple features can be better integrated, thereby enhancing the richness and accuracy of semantic expression. The dynamic weighted combination of the two can improve the performance of the model in natural language processing tasks.

[0071] In the relation extraction task, a new relation extraction model MacBERT-MaskFUSE is proposed. The model consists of a feature extraction layer, a mask perception layer, a text encoding layer and an output layer. In the text encoding layer, the bidirectional gated recurrent unit (BiGRU) and the text convolutional neural network (TextCNN) are combined. BiGRU can be used to capture global temporal dependencies, and TextCNN can be used to obtain local feature patterns, thus realizing the fusion of temporal and local features. At the same time, the convolution structure of TextCNN is not affected by the sequence length, which makes up for the deficiency of BiGRU in long text processing, so that the encoding layer can more effectively capture useful features in long sequences. Finally, since the features extracted by BiGRU and TextCNN are different, the combination of the two can avoid the model from being too dependent on a certain feature representation, which helps the model to be more robust in different types of text. During the training process, the combination of BiGRU and TextCNN makes the model more adaptable to various noises, incomplete information, etc., thereby improving the generalization performance. The representation ability of the relation extraction model is enhanced through the input from the previous layer to the next layer.

[0072] The entire process of knowledge graph construction does not require much human involvement, and the rate of corpus processing based on named entity recognition models and relationship extraction models is much higher than manual work, which can enable more vertical field corpora to be identified within a certain period of time, making the final constructed knowledge graph more complete. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A schematic diagram of a usage scenario of the knowledge graph construction method provided by the present invention;

[0074] Figure 2 A schematic diagram of the process flow of the knowledge graph construction method provided by the present invention;

[0075] Figure 3 A named entity recognition model diagram provided by the present invention;

[0076] Figure 4 A relationship extraction model diagram provided by the present invention;

[0077] Figure 5 A schematic diagram of the changes in the knowledge graph question and answer interface in a vertical field provided by the present invention. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described below in combination with the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] The objects of knowledge graph construction in general fields are mostly web documents. For example, when searching for "mobile phone, model" on the terminal, the results returned to the user by the terminal include: web documents for purchasing mobile phones, web documents about mobile phone models mentioned by web users, etc. However, the search results that users may need are what models of mobile phones are divided into, and the functional characteristics of each model of mobile phones, etc. However, the existing technology does not build a knowledge graph for documents in the vertical field of electronic products, so users cannot get accurate results when searching on the web.

[0080] The existing technology has not yet constructed a knowledge graph for documents in vertical fields. The knowledge graphs constructed so far are mainly based on web documents in general fields. In vertical fields, data sources are diverse, such as database documents, Office documents, technical reports, industry papers, and even multimedia content (such as video and audio). The document extraction method of the existing technology is not applicable to these types of documents, and it is impossible to obtain the schema of these types of documents, that is, the entity type and attributes of the document, and thus it is impossible to construct a knowledge graph for these documents. Specifically, the construction of the knowledge graph for a vertical industry is to classify and organize all documents in the industry according to entity types and attributes, so that accurate document information can be returned when users search.

[0081] Figure 1 A schematic diagram of a use scenario of the knowledge graph construction method provided by the present invention, such as Figure 1 As shown, the applicable scenarios of the knowledge graph construction method provided by the present invention include: a knowledge graph construction device and a server. Among them, the server is a server in a vertical field, which contains multiple types of documents in the vertical field. When constructing the knowledge graph of the vertical field, the knowledge graph construction device is required to extract multiple types of different documents from the server and classify them according to the entity types and attributes in the documents.

[0082] Figure 2 This is a schematic diagram of the knowledge graph construction method provided by the present invention. Figure 2 The two-stage knowledge graph construction model and construction method of the embodiment of the present invention specifically include the following steps:

[0083] S11. Build a data pool by collecting and fusing multi-source heterogeneous data sets, and preprocess the raw data, including missing value filling, noise filtering, standardization conversion and other cleaning operations;

[0084] In an embodiment of the present invention, the collection and fusion of multi-source heterogeneous data sets to build a data pool includes obtaining multiple types of document samples in the servers of the corresponding vertical industries, wherein the document samples can be in the form of database documents, Office documents, technical reports, industry papers, and even multimedia content (such as video, audio), etc. Preprocessing the data includes processing and optimizing the original data to ensure the accuracy, consistency and integrity of the data and reduce errors transmitted to the subsequent stages of the knowledge graph.

[0085] For example, to build a vertical domain knowledge graph covering mobile phone models and their functional features, first, it is necessary to systematically collect mobile phone models and related information from various network resources and related documents; then, the collected raw data is preprocessed and cleaned to ensure the quality and availability of the data.

[0086] S12, annotate the cleaned data with entity-relationship labels to provide a high-quality labeled dataset for subsequent model training;

[0087] In the example of the present invention, after data preprocessing of the original data, doccano software is used to annotate the entity-relationship labels in detail based on the predefined entity and relationship category system to ensure that the data set has consistent and high-quality semantic annotations.

[0088] Exemplarily, the entity categories of the vertical domain knowledge graph covering mobile phone models and their functional features may include: mobile phone models, brands, functional features, release time and price range, etc.; relationship categories may include: the relationship between mobile phone signals and brands (such as: iPhone15 belongs to Apple), the relationship between mobile phone models and functional features (such as: iPhone15 supports 5G, iPhone15 has A16 chip), the relationship between mobile phone models and release time, price, and the comparative relationship between mobile phone models and other models (such as: iPhone15 is better than iPhone14), etc.; on this basis, the doccano tool is used to accurately annotate the entities and relationships in the text to support subsequent named entity recognition and relationship extraction tasks.

[0089] S13. A multi-feature fusion named entity recognition model (MacBERT-MultiFeature-Fusion) based on the improved MacBERT structure is proposed. The model can accurately identify semantically labeled named entities from unstructured text data.

[0090] In the present invention, the named entity recognition model is as follows: Figure 3 As shown in the figure, the model consists of an entity position encoding layer, a MacBERT feature extraction layer, a text encoding layer, a feature fusion layer, a multi-head self-attention mechanism layer, and a conditional random field layer.

[0091] The entity position encoding layer represents the positional relationship between each word in the input text and the target entity as a position embedding, enabling the model to more accurately identify the target entity;

[0092] The MacBERT feature extraction layer generates feature vectors after feature extraction using the pre-trained model MacBERT, extracts rich contextual features from the input text, improves the masking strategy and enhances the contextual understanding capability, provides deeper representations for downstream tasks, and generates high-quality feature representations for each word. These feature representations not only contain contextual information, but are also closer to the real semantic relationship.

[0093] The text encoding layer and feature fusion layer dynamically weightedly combine the iterative dilated convolutional neural network (IDCNN) and the bidirectional gated recurrent unit (BiGRU), and normalize the weights through Softmax, so that the model can balance and adjust between local features and long-distance dependent features to adapt to different text input situations and achieve better entity recognition results.

[0094] The multi-head self-attention mechanism layer performs weighted calculation on the input features, and further screens and weights the features fused by IDCNN and BiGRU, so that important features receive higher weights.

[0095] The conditional random field layer globally optimizes the output sequence of the model. This layer can capture label dependencies, improve global prediction consistency, and enhance the utilization of sequence information, thereby improving the accuracy of entity recognition.

[0096] For example, assume that the input text is: "Apple released the new iPhone 15 series, which supports 5G networks and is equipped with an A16 chip."

[0097] Entity position encoding layer: Perform position encoding on the word "iPhone 15" and generate a code related to its relative position, which helps the model recognize the start and end positions of "iPhone 15" as a mobile phone model entity;

[0098] In the MacBERT feature extraction layer. The MacBERT pre-trained model encodes each word in "Apple released a new iPhone 15 series, supporting 5G networks and equipped with A16 chips", so that the model can understand that "Apple" is a brand, "iPhone15" is a model, and "5G" and "A16 chip" are functional features;

[0099] Text encoding layer. When processing "Apple released the new iPhone 15 series", IDCNN can expand the dependency between "Apple" and "iPhone 15" in the sentence, while BiGRU captures the contextual connection between "iPhone 15" and the subsequent functional feature "5G network" through bidirectional loop;

[0100] Feature fusion layer. When fusing the representation of the word "iPhone 15", IDCNN extracts its long-distance dependency to "5G network", while BiGRU extracts its adjacent relationship with "Apple". The fusion layer assigns weights to the features of IDCNN and BiGRU according to semantic importance;

[0101] Multi-head self-attention mechanism layer. When processing "supports 5G network and equipped with A16 chip", multiple attention heads can focus on "5G network" as a feature and "A16 chip" as the processor type, allowing the model to classify these features more accurately;

[0102] Conditional Random Field layer. Assume that the model predicts that the label of "Apple" is brand (B-BRAND), "iPhone 15" is phone model (B-MODEL), and "5G network" and "A16 chip" are functional features (B-FEATURE). The CRF layer will further optimize the rationality of the label, ensuring that "B-MODEL" is followed by "I-MODEL" or other reasonable label sequences, rather than irrelevant labels, thereby improving the accuracy of entity recognition.

[0103] S14. We propose a novel relation extraction model, MacBERT-MaskFUSE, which combines a dynamic masking mechanism to extract semantic relations between entities from text and effectively identify specific target relations.

[0104] In the present invention, the relation extraction model is as follows: Figure 4 As shown in Figure 1, the model consists of a feature extraction layer, a mask perception layer, a text encoding layer, and an output layer.

[0105] Feature extraction layer. After the sequence text is input, the MacBRET pre-trained model encodes and extracts features from the input sequence to obtain the context feature representation of each word. After obtaining these features, the model will further process to extract the relationship information between entity pairs in the sentence and extract the representation of the target entity from the encoded sequence features;

[0106] Mask perception layer. The mask perception layer further filters and processes the features from the feature extraction layer, allowing the model to focus more on the entities in the sentence and their relationship information. The mask perception layer explicitly marks the target entity position in the input feature through position encoding or masking mechanism, allowing the model to identify entity pairs in the sentence, and filter or weight word information other than the target entity to reduce its influence in relationship extraction. Through the masking mechanism, the model can effectively filter out irrelevant context, reducing noise when aggregating features;

[0107] Text encoding layer. This layer consists of a bidirectional gated recurrent unit (BiGRU) and a text convolutional neural network (TextCNN). First, the feature information output by the mask perception layer is input into the BiGRU layer to capture the contextual dependency of the sequence; then, the output of the BiGRU is combined with the TextCNN to further extract local semantic features to enhance the representation ability of the relationship extraction model;

[0108] Output layer. The output layer maps the high-dimensional feature vector passed from the text encoding layer to the relationship category space and generates a score for each relationship category. After Softmax processing, the output layer calculates the probability for each relationship type and finally determines the predicted relationship category based on the maximum probability.

[0109] For example, assume that the input text is: "iPhone 15 belongs to Apple, is equipped with A16 chip, supports 5G function, and has better performance than iPhone 14."

[0110] Feature extraction layer: The model embeds each word, converts the word into a vector representation, and generates preliminary features of the text;

[0111] Mask-aware layer. The mask-aware layer is used to process irrelevant information in the sentence by adding a mask to filter out words that are irrelevant to the target entity. For example, if the model's goal is to identify the relationship between "iPhone 15" and other words, the mask-aware layer will highlight related words such as "belongs to", "Apple", and "equipped with", ignoring interference information. The output features of the mask-aware layer are then passed to the text encoding layer;

[0112] Text encoding layer. First, the features output by the mask perception layer are passed to the BiGRU layer to capture the contextual information of the words in the sentence. BiGRU captures the word information that appears before and after "iPhone 15" through bidirectional processing, so that the model understands the dependency between "iPhone 15" and "belongs to Apple", "equipped with A16 chip" and other words. The output of the BiGRU layer is passed to the TextCNN layer to further extract local semantic features. TextCNN recognizes the structural information of phrases such as "iPhone 15 belongs to Apple" and "iPhone 15 supports 5G" by sliding the convolution kernel, and can capture the relative position and key phrase features between entities;

[0113] Output layer. The extracted features are mapped to relationship labels through a linear layer. This layer will classify the relationship labels, for example, identifying "iPhone 15 belongs to Apple" as a brand relationship, "iPhone 15 supports 5G" as a functional feature relationship, etc. The final output of each input sentence will mark the included relationship type and related entity pairs.

[0114] S15. Embed the extracted entities and relationships into the Neo4j graph database, build a vertical industry knowledge graph in the form of a graph database, and realize the fusion and query analysis functions of multi-source knowledge;

[0115] In the example of the present invention, step S13 and step S14 work together on the text to extract information from natural language and convert it into structured data in the knowledge graph. The task of step S13 is to identify entities in the text and provide a basis for the node construction of the knowledge graph. Step S14 is responsible for identifying the relationships between entities, which will serve as edges in the knowledge graph to connect different entity nodes.

[0116] Exemplarily, a vertical domain knowledge graph covering mobile phone models and their functional characteristics is constructed, which mainly includes information such as mobile phone entities and functional characteristics extracted from data sources, and stores them in Neo4j in the form of a graph database to support query and analysis.

[0117] This paper conducts a model experimental evaluation and makes experimental comparisons between the named entity recognition model and the relationship extraction model. The measurement indicators are the precision, recall and F1 value recognized in the NLP field. The experimental results show that the model adopted in this paper has the best performance.

[0118] Figure 5 A schematic diagram of the change of the knowledge graph question-and-answer interface in a vertical field provided by the present invention, such as Figure 5 As shown in the interface, the search interface is provided with a search box. When the user enters "mobile phone model" in the search box, the interface of the knowledge about "mobile phone model" in the vertical field will be returned to the user. The interface contains the knowledge about "mobile phone model", which can be a document, such as the above-mentioned document 1 and document 2. The user can enter the display page of the corresponding document by clicking the link of document 1 or document 2.

[0119] The present invention also provides a computer-readable storage medium, on which computer execution instructions are stored. When the processor executes the instructions, it implements the method for constructing the knowledge graph in the above embodiment.

[0120] The knowledge graph readable storage medium described in this patent includes but is not limited to hard disks, solid-state drives (SSDs), optical disks (such as CDs and DVDs), tape storage, and cloud-based distributed storage media. The storage medium contains instructions and data structures to maintain and update the knowledge graph structure by executing instructions or reading data to ensure data integrity and consistency.

[0121] This storage medium supports the addition, deletion, modification, and query operations on entities and relationships in the knowledge graph. It can efficiently build and access the knowledge graph through the method described in the patent, and is suitable for applications such as large-scale semantic data management and intelligent search recommendations.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A two-stage knowledge graph construction model and its construction method and storage medium, characterized in that: The two-stage knowledge graph construction model and its construction method include the following steps: S11. It is necessary to build a data pool by collecting and integrating multi-source heterogeneous data sets, and preprocess the original data, including cleaning operations such as missing value filling, noise filtering, and standardization conversion; S12, annotate the cleaned data with entity-relationship labels to provide a high-quality labeled dataset for subsequent model training; S13. A multi-feature fusion named entity recognition model (MacBERT-MultiFeature-Fusion) based on the improved MacBERT structure is proposed. The model can accurately identify semantically labeled named entities from unstructured text data. S14. We propose a novel relation extraction model, MacBERT-MaskFUSE, which combines a dynamic masking mechanism to extract semantic relations between entities from text and effectively identify specific target relations. S15. Embed the extracted entities and relationships into the Neo4j graph database, build a vertical industry knowledge graph in the form of a graph database, and realize the fusion and query analysis functions of multi-source knowledge.

2. The two-stage knowledge graph construction model and its construction method and storage medium according to claim 1 are characterized in that: In the step S11, data cleaning refers to processing and optimizing the original data to ensure the accuracy, consistency and completeness of the data, reduce the error transmission to the subsequent stage of the knowledge graph, and provide a more reliable data foundation for knowledge extraction and graph construction; the missing value filling is to replace the missing values ​​in the data set with appropriate numerical values ​​or methods in data processing to ensure the integrity of the data. By filling the missing values, the data quality can be improved, and the data can be made more complete and consistent, which is conducive to subsequent data analysis and model training; the noise filtering is to identify and remove the "noise" part in the data in data processing to improve the data quality and the accuracy of the model; the noise refers to those data that are irrelevant, erroneous, abnormal or have a great interference with the analysis results, such as: erroneous data, irrelevant information, outliers and duplicate data; the standardization conversion refers to converting the data into a unified scale or distribution, so as to ensure the similarity of the magnitude of different features in subsequent analysis or modeling, and avoid affecting the model performance due to different data scales.

3. The two-stage knowledge graph construction model and its construction method and storage medium according to claim 1 are characterized in that: In step S12, after data preprocessing of the original data, doccano software is used to annotate the entity-relationship labels in detail based on the predefined entity and relationship category system to ensure that the data set has consistent and high-quality semantic annotations, thereby providing high-quality annotated corpus for subsequent deep learning model training.

4. The two-stage knowledge graph construction model and its construction method and storage medium according to claim 1, characterized in that: In the step S13, a multi-feature fusion named entity recognition model (MacBERT-MultiFeature-Fusion) based on an improved MacBERT structure is proposed, which consists of an entity position encoding layer, a MacBERT feature extraction layer, a text encoding layer, a feature fusion layer, a multi-head self-attention mechanism layer and a conditional random field layer.

5. According to claim 4, a multi-feature fusion named entity recognition model (MacBERT-MultiFeature-Fusion) based on an improved MacBERT structure is characterized in that: In step S13, the system performs named entity recognition (NER) on the input text, and its task is to identify entities in the text. The goal of NER is to extract key entities that are valuable for subsequent analysis from a large amount of unstructured text, and provide a basis for the construction of nodes in the knowledge graph.

6. The two-stage knowledge graph construction model and construction method and storage medium according to claim 1, characterized in that: In the step S14, a new relation extraction model MacBERT-MaskFUSE is proposed, which consists of a feature extraction layer, a mask perception layer, a text encoding layer and an output layer.

7. The novel relation extraction model MacBERT-MaskFUSE according to claim 6 is characterized in that: The step S14 is responsible for relationship extraction (RE), and the task of this step is to identify and extract the relationship between various entities in the text. In this process, the system needs to understand the semantic connection between entities and extract their mutual relationships. These relationships will serve as edges in the knowledge graph, connecting different entity nodes.

8. The two-stage knowledge graph construction model and construction method and storage medium according to claim 1, characterized in that: In the step S15, the extracted entities and relationships are embedded into the Neo4j graph database, and a vertical industry knowledge graph is constructed in the form of a graph database to realize the fusion and query analysis functions of multi-source knowledge; step S13 and step S14 work together on the text to extract information in natural language and convert it into structured data in the knowledge graph. Step S13 is responsible for identifying entities, while step S14 is responsible for identifying the relationship between entities. The two complement each other. Step S13 extracts nodes, and step S14 creates connections between nodes.

9. The two-stage knowledge graph construction model and construction method and storage medium according to claim 1, characterized in that: A computer-readable storage medium mainly includes: computer execution instructions are stored on the computer-readable storage medium, and when a processor executes the instructions, the method described in any one of claims 1-8 is implemented.

Images