Knowledge graph construction and intelligent question answering method and device based on deep learning

Abstract

The present invention provides a method and device for constructing a knowledge graph and intelligent question answering based on deep learning. The method comprises: obtaining target text data; extracting information from the target text data to generate entity relationship data, wherein the entity relationship data comprises key data, entity data, and relationship data for indicating the relationship between entities; importing the target text data into a graph database according to the entity relationship data to form a knowledge graph; obtaining the input question statement; classifying the question statement using a question classification model to determine the question features in the question statement, wherein the question features comprise question type and question entity data; retrieving text data matching the question features in the knowledge graph as answer data for the question statement. The present invention improves the efficiency of public text information retrieval and the accuracy of intelligent question answering.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and specifically to a knowledge graph construction and intelligent question-answering method and device based on deep learning. Background Art

[0002] With the continuous release and update of various public texts, text information has exploded, and how to efficiently manage and utilize this information has become an important issue. Traditional information retrieval mainly relies on keyword matching and text retrieval technology, which cannot deeply understand the semantic content of the text.

[0003] At present, common text information processing methods include rule-based text analysis and simple question-answering devices. These methods usually use predefined rules and templates to process text. Although they are simple to implement, they are difficult to deal with complex and changeable professional texts, and the accuracy and completeness of the retrieval results are difficult to guarantee.

[0004] In recent years, knowledge graph technology has been widely used in the field of information processing. Existing knowledge graph construction methods rely on manual annotation, which is inefficient when processing large-scale professional texts and has difficulty in accurately capturing complex semantic relationships in texts.

[0005] Therefore, existing text processing methods are unable to effectively extract and organize key information in professional texts, resulting in low information retrieval efficiency, making it difficult to accurately understand user problems and unable to provide accurate information consulting services. Summary of the invention

[0006] In view of this, the present application provides a method and device for constructing a knowledge graph and intelligent question and answering based on deep learning, which solves the problems of low efficiency in text information retrieval and insufficient accuracy in question and answering in the prior art.

[0007] The present application embodiment provides a knowledge graph construction and intelligent question answering method based on deep learning, including:

[0008] Get target text data;

[0009] Extracting information from the target text data to generate entity relationship data, wherein the entity relationship data includes key data, entity data, and relationship data for indicating the relationship between entities;

[0010] According to the entity relationship data, the target text data is imported into a preset graph database to form a knowledge graph;

[0011] Get the input question statement;

[0012] Using a pre-established question classification model to classify the question sentence, and determining question features in the question sentence, wherein the question features include question type and question entity data;

[0013] Retrieving text data matching the question features in the knowledge graph as answer data for the question statement;

[0014] The step of importing the target text data into a preset graph database according to the entity relationship data to form a knowledge graph includes:

[0015] According to the node type and relationship type of the graph node, a first mask ratio is configured for a node whose node type meets a preset important node condition, and a second mask ratio is configured for a node whose node type meets a preset auxiliary node condition, so as to form mask data, wherein the first mask ratio is lower than the second mask ratio;

[0016] In combination with a preset multi-task learning mechanism, the mask data is used to pre-train a pre-established encoder-decoder structure, wherein the encoder-decoder structure includes a third encoder and a decoder, the third encoder is used to instruct the conversion of the heterogeneous graph data indicated by the knowledge graph into a feature vector, and the decoder is used to instruct the reconstruction of the graph structure with the feature vector;

[0017] Build a classifier based on the pre-trained encoder-decoder structure;

[0018] The classifier is trained using the feature vector and a task feature vector determined by the current task requirement, so that the knowledge graph is optimized using the trained classifier.

[0019] Optionally, the acquiring target text data includes:

[0020] Get the text data to be processed;

[0021] Parsing the text data to be processed and removing interference information in the text data to be processed to obtain cleaned text data to be processed, wherein the interference information includes HTML tags and special characters;

[0022] Performing word segmentation processing on the cleaned text data to be processed, and removing stop words in the cleaned text data to be processed to form a text segmentation result;

[0023] The word frequency statistics are performed on the text segmentation results to generate word vectors, so that the target text data can be obtained based on the word vectors.

[0024] Optionally, the key data includes keywords and document summaries; extracting information from the target text data to generate entity relationship data includes:

[0025] Constructing a word graph network based on the target text data, wherein a node in the word graph network represents a word in the target text data, an edge in the word graph network represents a semantic association between the words, and a weight of the edge represents a co-occurrence number between the words;

[0026] Based on a preset damping coefficient, the TextRank value of each node in the word graph network is iteratively calculated until the TextRank value of the node converges and / or reaches a specified number of iterations, and the word importance score data of the word to which the node belongs is generated according to the TextRank value of the current node;

[0027] Selecting words corresponding to when the word importance score data exceeds a preset word score threshold as keywords of the target text data;

[0028] Dividing the target text data into a plurality of sentences, and determining sentence importance score data of the sentences based on word importance score data of the words in the sentences;

[0029] The sentence corresponding to when the sentence importance score data exceeds a preset sentence score threshold is selected as the document summary of the target text data.

[0030] Optionally, extracting information from the target text data to generate entity relationship data further includes:

[0031] Obtaining a training corpus with multiple layers of annotations, where the annotations are used to indicate entity categories;

[0032] Using the training corpus to train and evaluate a preset first encoder to obtain an entity recognition model, wherein the entity recognition model includes a BERT model with a GlobalPointer layer;

[0033] Using the entity recognition model to recognize the target text data, and obtain entity data indicating a plurality of entities;

[0034] Based on the entity data, determining training data having a relationship identifier, wherein the relationship identifier is used to indicate a type of relationship between the entities;

[0035] Using the training data to train a preset second encoder to obtain a relationship extraction model;

[0036] The target text data and the entity data are input into the relationship extraction model to identify the association relationship between entities based on the entity position data in the entity data and the text sequence in the target text data, and output the relationship data.

[0037] Optionally, importing the target text data into a preset graph database according to the entity relationship data to form a knowledge graph includes:

[0038] Based on a preset graph database, the device memory acting on the knowledge graph is divided into page cache and heap memory according to a specified ratio to form corresponding memory configuration data;

[0039] For the entity attributes corresponding to when the query frequency data is higher than a preset query frequency threshold, an attribute index is constructed, and for the relationship types corresponding to when the relationship frequency data is higher than a preset relationship frequency threshold, a relationship index is constructed, so as to generate an index system by the attribute index and the relationship index;

[0040] Generate a knowledge graph about the target text data according to the entity relationship data, the memory configuration data and the index system;

[0041] Or, the method of importing the target text data into a preset graph database according to the entity relationship data to form a knowledge graph further includes:

[0042] Based on the entity relationship data, a data processing model is constructed, wherein the data processing model includes a decision network and a representation network, wherein the decision network is used to indicate the domain type of the graph node in the knowledge graph, and the representation network is used to indicate that the output result of the decision network updates the embedding representation of the corresponding graph node, wherein each graph node represents an entity, and the edge in the knowledge graph represents the corresponding relationship data;

[0043] Performing reinforcement learning on the data processing model to generate node importance data of the graph nodes;

[0044] According to the node importance data of the graph node, the information importance score of the node information related to the graph node is calculated, and the node information whose information importance score is lower than the preset importance threshold is removed, and then a knowledge graph is generated.

[0045] Optionally, the step of acquiring the target text data further includes:

[0046] By using semantic decomposition, the text data to be processed is converted into a plurality of semantic units containing text information and formulation intentions;

[0047] Performing multi-level information aggregation on the semantic units to generate multiple semantic combinations;

[0048] According to the semantic combination, a plurality of views are constructed, each of the views being associated with a text feature of the text data to be processed, the text feature comprising an object feature, a time sequence feature and an action feature;

[0049] The views are subjected to cross-view information fusion according to the text features to obtain semantic fusion data, so that the target text data is obtained based on the semantic fusion data.

[0050] Optionally, the step of importing the target text data into a preset graph database to form a knowledge graph according to the entity relationship data further includes:

[0051] According to the node type and relationship type of the graph node, a first mask ratio is configured for a node whose node type meets a preset important node condition, and a second mask ratio is configured for a node whose node type meets a preset auxiliary node condition, so as to form mask data, wherein the first mask ratio is lower than the second mask ratio;

[0052] In combination with a preset multi-task learning mechanism, the mask data is used to pre-train a pre-established encoder-decoder structure, wherein the encoder-decoder structure includes a third encoder and a decoder, the third encoder is used to instruct the conversion of the heterogeneous graph data indicated by the knowledge graph into a feature vector, and the decoder is used to instruct the reconstruction of the graph structure with the feature vector;

[0053] Build a classifier based on the pre-trained encoder-decoder structure;

[0054] The classifier is trained using the feature vector and a task feature vector determined by the current task requirement, so that the knowledge graph is optimized using the trained classifier.

[0055] Optionally, the training of the classifier using the feature vector and a task feature vector determined by a current task requirement includes:

[0056] Performing dimensionality reduction processing on the feature vector;

[0057] The reduced-dimensional feature vector and the task feature vector are input into the classifier to calculate the associated weights between the reduced-dimensional feature vector and the task feature vector using a preset attention mechanism to form a feature weight matrix, and based on the feature weight matrix, the reduced-dimensional feature vector and the task feature vector are weightedly combined to obtain a trained classifier.

[0058] The retrieving text data matching the question feature in the knowledge graph includes:

[0059] Obtain a pre-trained large language model and construct a training corpus based on the entity relationship data in the knowledge graph;

[0060] Using the training corpus to fine-tune the large language model in a certain domain, thereby obtaining an intelligent agent model with knowledge graph query capability;

[0061] Inputting the question statement into the intelligent agent model to generate a query plan including multiple sub-query steps;

[0062] Based on the query plan, performing multi-hop path exploration in the knowledge graph, wherein weight data of different paths are calculated based on a graph attention mechanism in each hop exploration;

[0063] According to the weight data, a query path with high confidence is selected for evidence collection, and evidence of multiple query paths is integrated to generate answer data including reasoning chain and evidence support.

[0064] Before extracting information from the target text data, the method further includes:

[0065] Get a pre-trained large language model and build a mixed validation set that includes a general ability test set and a domain ability test set;

[0066] Based on the model capability hierarchical hypothesis, the network structure of the large language model is divided into a basic language understanding layer, a general reasoning layer and a domain professional layer;

[0067] Differentiated tuning strategies are adopted for each layer of the network. By configuring different learning rates and regularization strengths, the bottom layer network maintains basic language understanding capabilities, the middle layer network maintains general reasoning capabilities, and the high layer network obtains domain processing capabilities.

[0068] During the fine-tuning process, the model performance is evaluated based on the mixed validation set, and the learning rate and regularization parameters of each layer are dynamically adjusted according to the evaluation results to generate a preprocessing model with both general and domain capabilities;

[0069] The preprocessing model is used to perform semantic understanding and structural processing on the target text data so as to extract information from the target text data.

[0070] The present application also provides a knowledge graph construction and intelligent question-answering device based on deep learning, including:

[0071] A text acquisition module is used to acquire target text data;

[0072] A data extraction module, used to extract information from the target text data to generate entity relationship data, wherein the entity relationship data includes key data, entity data, and relationship data indicating the relationship between entities;

[0073] A knowledge graph construction module, used to import the target text data into a preset graph database according to the entity relationship data to form a knowledge graph;

[0074] A question acquisition module, used to acquire the input question sentence;

[0075] A question feature extraction module, used to classify the question sentence using a pre-established question classification model, and determine question features in the question sentence, wherein the question features include question type and question entity data;

[0076] A reply module, used to retrieve text data matching the question features in the knowledge graph as answer data for the question statement;

[0077] The knowledge graph construction module is used to import the target text data into a preset graph database according to the entity relationship data to form a knowledge graph, including:

[0078] According to the node type and relationship type of the graph node, a first mask ratio is configured for a node whose node type meets a preset important node condition, and a second mask ratio is configured for a node whose node type meets a preset auxiliary node condition, so as to form mask data, wherein the first mask ratio is lower than the second mask ratio;

[0079] In combination with a preset multi-task learning mechanism, the mask data is used to pre-train a pre-established encoder-decoder structure, wherein the encoder-decoder structure includes a third encoder and a decoder, the third encoder is used to instruct the conversion of the heterogeneous graph data indicated by the knowledge graph into a feature vector, and the decoder is used to instruct the reconstruction of the graph structure with the feature vector;

[0080] Build a classifier based on the pre-trained encoder-decoder structure;

[0081] The classifier is trained using the feature vector and a task feature vector determined by the current task requirement, so that the knowledge graph is optimized using the trained classifier.

[0082] This application has the following technical effects:

[0083] Through deep learning technology, key data, entity data and relational data used to indicate the relationship between entities in the target text data are extracted to realize intelligent processing of text information, improve the accuracy and efficiency of information extraction, and then build a knowledge graph based on the entity relationship data to realize structured representation of text information, realize effective storage and management of complex semantic relationships, and then combine the question classification model to determine the question type and question entity data in the question statement, which can accurately understand user questions and provide accurate question and answer services, so as to retrieve text data matching the question type and question entity data in the knowledge graph as the answer data of the question statement, and can process different types of professional text data, solving the problems of low efficiency of text information retrieval and insufficient accuracy of question and answer in the prior art, and has good scalability and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described are only part of the embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0085] Figure 1 A schematic diagram of the process of constructing a knowledge graph based on deep learning and an intelligent question-answering method provided in an embodiment of the present invention;

[0086] Figure 2 A schematic diagram of a flow chart of an information extraction method provided by an embodiment of the present invention;

[0087] Figure 3 A schematic diagram of a process for constructing a knowledge graph provided by an embodiment of the present invention;

[0088] Figure 4 Schematic diagram of the device architecture for deep learning-based knowledge graph construction and intelligent question-answering method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0089] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0090] Figure 1 A flow chart of a method for constructing a knowledge graph based on deep learning and intelligent question answering is provided in an embodiment of the present invention. The present invention provides a method for constructing a knowledge graph based on deep learning and intelligent question answering, and the method includes steps S1 to S6.

[0091] S1: Obtain target text data.

[0092] In some embodiments, step S1 includes:

[0093] Get the text data to be processed;

[0094] Parsing the text data to be processed and removing interference information in the text data to be processed to obtain cleaned text data to be processed, wherein the interference information includes HTML tags and special characters;

[0095] Performing word segmentation processing on the cleaned text data to be processed, and removing stop words in the cleaned text data to be processed to form a text segmentation result;

[0096] The word frequency statistics are performed on the text segmentation results to generate word vectors, so that the target text data can be obtained based on the word vectors.

[0097] In this embodiment, the text data to be processed are obtained from multiple sources, which may come from official websites, databases, etc. The device establishes a structured document database to store these original text data to ensure the integrity and traceability of the data.

[0098] Then, the obtained text data to be processed is cleaned and preprocessed. First, HTML tags are identified and removed through regular expressions, such as 、 Equal label elements. Secondly, remove special characters, including but not limited to extra spaces, line breaks, tab characters, etc. At the same time, uniformly format numbers, dates, punctuation marks, etc. in the text. The purpose of this step is to obtain pure text data with standardized format and clean content.

[0099] Furthermore, exemplarily, use the jieba tokenizer to tokenize the cleaned text. Among them, the jieba tokenizer has three tokenization modes: exact mode, full mode, and search engine mode. In this embodiment, the exact mode is adopted, which can accurately segment the text into word units suitable for subsequent processing. On the basis of tokenization, stop words are removed, including common words such as "de", "le", "zai", etc. that contribute little to the semantics of the text, thereby reducing the noise in subsequent processing. In this embodiment, a custom stop word list in the professional field is maintained to ensure that the removed stop words do not affect the semantic expression of professional texts.

[0100] Even further, perform word frequency statistical analysis on the tokenized text. Specifically, first count the frequency of each word in the corpus, calculate the TF-IDF value of the word to evaluate the importance of the word. Then, use pre-trained models such as Word2Vec or BERT to map each word to a high-dimensional vector space to generate word vectors. These word vectors not only contain the statistical features of the words but also contain the semantic relationships between the words. Based on these word vectors, the text content can be better understood and processed, laying a foundation for subsequent entity recognition and relationship extraction tasks.

[0101] In addition, during the word vector generation process, this embodiment also considers the context information of the words, uses the sliding window technique to capture the co-occurrence relationship between the words, thereby generating word vectors with stronger semantic expression ability. These word vectors will be used as the basic representation form of the target text data for subsequent deep learning model training and knowledge graph construction.

[0102] In another embodiment, this step S1 further includes:

[0103] Using semantic decomposition, transform the to-be-processed text data into multiple semantic units containing text information and specific intentions;

[0104] Perform multi-level information aggregation on the semantic units to generate multiple semantic combinations;

[0105] According to the semantic combinations, construct multiple views, each view is associated with a text feature of the to-be-processed text data, and the text feature includes object feature, time series feature, and action feature;

[0106] The views are subjected to cross-view information fusion according to the text features to obtain semantic fusion data, so that the target text data is obtained based on the semantic fusion data.

[0107] This embodiment constitutes a complete multi-view semantic processing flow, which uses the multi-view heterogeneous flow graph (MH-Net) method to process public text data. Therefore, this embodiment can better capture the complex semantic relationships and public text intentions in public text.

[0108] In this embodiment, the public text is first semantically decomposed. This process is different from traditional text segmentation, but is a specialized process based on the linguistic features of the public text. This embodiment will identify various semantic markers in the text, including directive terms (such as "must", "should"), conditional expressions (such as "if", "when..."), incentives (such as "reward", "support"), etc. Each semantic unit contains not only basic text information, but also the intention information of the public text maker. This fine-grained semantic decomposition provides a basis for subsequent processing.

[0109] Furthermore, the present embodiment implements multi-level aggregation of semantic units. First, related semantic units are combined into basic semantic units, which may be complete public text measures, execution conditions or supporting objects, etc. Then, these basic units are further organized into larger semantic blocks (i.e., semantic combinations) to form complete public text clauses or regulations. This hierarchical aggregation process maintains the semantic integrity of the public text, while also facilitating subsequent multi-view modeling. Each layer of aggregation takes into account semantic coherence and integrity to ensure that important semantic information is not lost.

[0110] Furthermore, multiple views are constructed based on semantic combination. Each view focuses on different feature dimensions of public text: the object view focuses on the hierarchical relationship between public text objects, reflecting the subject and object of public text implementation; the time sequence view focuses on the time sequence relationship of public text implementation, including the effective time and execution cycle of public text; the action view focuses on the specific execution actions and processes of public text measures. This multi-view design can comprehensively capture the different semantic dimensions of public text and provide rich feature representation for subsequent information fusion.

[0111] Thus, the information of different views is integrated. For example, the concept of flow graph is used to describe the transmission and influence of information. The connection between each view not only represents a simple association relationship, but also includes the direction and intensity of information flow. The importance of different views is balanced through a dynamic weight adjustment mechanism. According to the specific analysis task and context, the weight of each view is dynamically adjusted to ensure that the most relevant information can receive full attention.

[0112] Throughout the process, this embodiment also implements some important optimization mechanisms. The first is the cross-view information verification mechanism, which verifies the accuracy of semantic understanding by comparing information in different views. The second is the incremental update mechanism, which can efficiently update the existing semantic representation when there is new public text to be processed. Finally, this embodiment also establishes a semantic consistency check mechanism to ensure that the fused semantic data maintains coherence and integrity.

[0113] Therefore, the semantic processing method based on multiple views in this embodiment not only improves the accuracy of public text processing, but also enhances the ability to understand the semantics of complex public text. Through multi-dimensional feature representation and fusion, it is possible to better grasp the intrinsic meaning of public text and the intention of the public text maker, providing a richer and more accurate semantic basis for the subsequent knowledge graph construction.

[0114] It should be noted that this embodiment adopts the MH-Net model, that is, different types of intent bits are aggregated into various types of basic units, so as to construct a multi-view traffic graph with different information granularity. Through the multi-view heterogeneous traffic graph, the original text data is converted into a more structured representation, providing a better semantic basis for the subsequent knowledge graph model design. This method can be regarded as an enhancement and optimization of the original text preprocessing method.

[0115] The MH-Net model is described in detail below:

[0116] First, traditional text processing methods often treat text as ordinary text and ignore the unique intention features in the text. In this embodiment, MH-Net is combined with the introduction of the concept of Policy Intent Bits to decompose the text into the smallest semantic units, which contain not only text information but also the intention information of the framer.

[0117] In the data preprocessing stage, MH-Net first performs fine-grained decomposition of the text. This process is different from the traditional word segmentation method, but is a specialized process based on linguistic features. This embodiment will identify various semantic markers in the text, such as directive terms, conditional statements, incentive measures, etc., and convert these markers into intent bits. Each intent bit contains basic semantic information and guidance information.

[0118] Next, MH-Net constructs a multi-level information aggregation structure. At the first level, this embodiment combines related intent bits into basic semantic units. These basic units may be measures, execution conditions, supporting objects, etc. At the second level, these basic units are further organized into larger semantic blocks to form complete clauses or regulations. Through this hierarchical aggregation method, the device can maintain the semantic integrity of the text, while also facilitating subsequent multi-view modeling.

[0119] The core feature of MH-Net is its multi-view flow graph structure. This embodiment constructs multiple view levels from different perspectives, and each view focuses on different features of the text. For example, the measure view focuses on the association relationship between measures, the object view focuses on the hierarchical relationship between objects, and the sequence view focuses on the chronological relationship of implementation. These different views together constitute a multi-dimensional semantic network.

[0120] When establishing associations between views, MH-Net uses the concept of flow graph to describe the transmission and influence of information. The connection between each node not only represents a simple association relationship, but also includes the direction and intensity of information flow. The design of this flow graph enables this embodiment to better capture the causal relationship and influence chain during the execution process.

[0121] In actual operation, MH-Net balances the importance of different views through a dynamic weight adjustment mechanism. This embodiment dynamically adjusts the weights of each view according to the specific analysis task and context to ensure that the most relevant information can receive full attention. This adaptive weight adjustment mechanism greatly improves the flexibility and accuracy of the model.

[0122] Finally, MH-Net also implements a cross-view information fusion mechanism. By designing a specific information transmission and aggregation strategy, this embodiment can transmit and integrate information between different views, thereby obtaining a more comprehensive and accurate semantic representation. This multi-view fusion method not only improves the model's expressiveness, but also enhances its ability to understand complex relationships.

[0123] Through the above design, MH-Net can effectively capture and express the complex semantic relationships in the text, providing a richer and more accurate semantic basis for the subsequent knowledge graph construction. This modeling method based on multi-view heterogeneous traffic graphs not only maintains the semantic integrity of the text, but also provides a more delicate expression of intent, thereby improving the processing effect of the entire device.

[0124] S2: extract information from the target text data to generate entity relationship data, where the entity relationship data includes key data, entity data, and relationship data for indicating the relationship between entities.

[0125] In the present application, step S2 realizes information extraction of target text data and generates an entity relationship data set including key data, entity data and relationship data.

[0126] Optionally, this step S2 adopts a multi-level information extraction architecture. At the first level, the key data of the text is extracted through a deep learning model. The key data includes two important components: keywords and document summaries. Keywords reflect the core themes and important concepts of the text, and document summaries summarize the main content of the text. These key data provide important semantic clues and screening basis for subsequent entity recognition.

[0127] At the second level, this application focuses on the identification and extraction of entity data. Entities are the basic building blocks of knowledge graphs. They can be named entities such as specific names of people, places, and institutions, or they can be professional terms and concepts in the field. This application uses a BERT-based entity recognition model, which has been pre-trained and fine-tuned on large-scale domain data and can accurately identify various entities in the text. In order to improve the recognition accuracy, a domain-specific entity dictionary is also established, and entity recognition is performed by combining dictionary matching and deep learning models.

[0128] At the third level, this application focuses on the extraction of relational data. Relational data describes the semantic associations between entities and is a key link in building a knowledge graph. First, sentences containing multiple entities are identified as candidate relational sentences, and then the relational extraction model is used to analyze the semantic relationships between entities in these sentences. The relational extraction model uses an attention mechanism that can automatically focus on key information in a sentence and accurately capture the relationship type between entities.

[0129] The entire information extraction process is a progressive process, and the output of each level will serve as the input or auxiliary information of the next level. The device also establishes a feedback mechanism to optimize and screen the extraction results through co-occurrence statistics and confidence evaluation of entity relationships. At the same time, this application will save the intermediate results to facilitate later result tracing and model optimization.

[0130] In order to ensure the quality of the extraction results, this application sets up multiple verification mechanisms. First, through rule verification, it ensures that the extracted entities and relationships conform to the predefined pattern; second, through statistical verification, it filters out low-frequency or abnormal entity relationships; finally, through semantic similarity calculation, it ensures that the extracted relationships are semantically reasonable. These verification mechanisms together ensure the accuracy and reliability of entity relationship data.

[0131] Through the above steps, the device finally generates a structured entity relationship dataset, which not only contains the core information of the text, but also retains the complex semantic associations between entities, laying a solid foundation for the subsequent knowledge graph construction.

[0132] In some embodiments, the key data includes keywords and document summaries. Figure 2 As shown, step S2 includes:

[0133] S2.1: constructing a word graph network based on the target text data, wherein a node in the word graph network represents a word in the target text data, an edge in the word graph network represents a semantic association between the words, and a weight of the edge represents the number of co-occurrences between the words;

[0134] S2.2: Based on a preset damping coefficient, iteratively calculate the TextRank value of each node in the word graph network until the TextRank value of the node converges and / or reaches a specified number of iterations, and generate word importance score data for the word to which the node belongs according to the TextRank value of the current node;

[0135] S2.3: Selecting words corresponding to when the word importance score data exceeds a preset word score threshold as keywords of the target text data;

[0136] S2.4: dividing the target text data into a plurality of sentences, and determining sentence importance score data of the sentences according to the word importance score data of the words in the sentences;

[0137] S2.5: Select the sentence corresponding to the sentence whose importance score data exceeds a preset sentence score threshold as the document summary of the target text data.

[0138] In this embodiment, keywords are a set of words that express the core content of a document, and a document summary is a text summary consisting of the most representative sentences. These two parts of data together constitute the core semantic representation of the text.

[0139] In this embodiment, each word in the text is first used as a node in the graph. Then, the edge connection is established based on the co-occurrence relationship of the words in the text. If the distance between two words in the text is less than the preset window size (usually set to 5), an edge is established between the nodes corresponding to the two words. The weight of the edge is determined by the number of co-occurrences of the words. The more co-occurrences, the greater the weight. This co-occurrence-based graph structure can effectively capture the semantic association between words.

[0140] Furthermore, the TextRank algorithm is used to calculate the importance of the node. The calculation formula is as follows:

[0141] TR(Vi) = (1-d) + d * Σ(wji * TR(Vj) / Σwjk)

[0142] Where d is the damping coefficient (usually set to 0.85), wji represents the weight of the edge from node j to node i, TR(Vi) represents the TextRank value of node i, and wjk represents the weight of the edge from node j to node k. This embodiment iterates the calculation until the TextRank values ​​of all nodes converge (the difference between the previous and next two iterations is less than a preset threshold, such as 0.0001) or the maximum number of iterations (such as 100 times) is reached.

[0143] Then, keywords are selected based on the TextRank value of the words. Specifically, all words are sorted in descending order according to the TextRank value, and the top-ranked words are selected as keywords. This embodiment sets an adaptive word scoring threshold, which is related to the document length and word distribution, and usually takes the top 20% of the TextRank value as the threshold. In this way, it can be ensured that the extracted keywords are neither too many nor too few.

[0144] Then, the text is segmented into multiple sentences, and the importance score of each sentence is calculated. The sentence score calculation process includes the following steps:

[0145] Sum the TextRank values ​​of all words in the sentence;

[0146] Considering the position information of sentences, the first and last sentences of a paragraph are appropriately weighted;

[0147] Take sentence length into account and penalize sentences that are too long or too short;

[0148] Consider the similarity between the sentence and the title;

[0149] Set an adaptive sentence scoring threshold, usually selecting the 3-5 sentences with the highest scores to form the summary.

[0150] In order to maintain the coherence of the abstract, this embodiment will sort the selected sentences according to the order of the sentences in the original text.

[0151] Optionally, this embodiment also implements some optimization measures:

[0152] When constructing the word graph, use the word's part-of-speech information to increase the weight of content words (especially nouns and verbs);

[0153] When calculating the TextRank value, the frequency information of the word is taken into account to avoid low-frequency words from getting too high an importance score;

[0154] When generating summaries, avoid selecting sentences with repeated content by calculating cosine similarity;

[0155] Build a dictionary for specialized terms and domain-specific vocabulary and give them higher weight in the calculation process;

[0156] Through the above steps, this embodiment can accurately extract keywords from the text and generate representative document summaries, providing important semantic information for subsequent knowledge graph construction.

[0157] For example, the implementation of the TextRank algorithm in extracting key information from public texts mainly includes three core links. The first step is to build a word graph network. First, the public text is segmented and stop words are removed. Then a sliding window (usually 5 words) is set, and the words in the window are considered to have semantic associations. Based on this, an undirected graph is constructed, in which the nodes are words and the weights of the edges are the number of co-occurrences between words, thus forming a network structure of the relationship between words.

[0158] The second step is to iteratively calculate the TextRank value. An iterative calculation method similar to PageRank is used, and the damping coefficient (generally 0.85) is set. For each node in the network, its TextRank value is calculated by multiple rounds of iterations from the initial value until convergence or the maximum number of iterations is reached. In the calculation process, the importance of a word depends not only on the number of words connected to it, but also on the importance of the connected words, which reflects the recursive characteristics of word importance.

[0159] The third step is to extract keywords and summaries. Based on the calculated TextRank values, select the words with the highest scores as the keywords of the document. At the same time, divide the document into sentences, calculate the score of each sentence (based on the TextRank value of each word in the sentence), and select the sentence with the highest score to form the document summary. In this way, the core content of the public text is summarized.

[0160] In another embodiment, the step S2 further includes:

[0161] Obtaining a training corpus with multiple layers of annotations, where the annotations are used to indicate entity categories;

[0162] Using the training corpus to train and evaluate a preset first encoder to obtain an entity recognition model, wherein the entity recognition model includes a BERT model with a GlobalPointer layer;

[0163] Using the entity recognition model to recognize the target text data, and obtain entity data indicating a plurality of entities;

[0164] Based on the entity data, determining training data having a relationship identifier, wherein the relationship identifier is used to indicate a type of relationship between the entities;

[0165] Using the training data to train a preset second encoder to obtain a relationship extraction model;

[0166] The target text data and the entity data are input into the relationship extraction model to identify the association relationship between entities based on the entity position data in the entity data and the text sequence in the target text data, and output the relationship data.

[0167] In this embodiment, training corpus is first prepared. The training corpus adopts a multi-layer annotation mode, including annotation information at two levels: entity boundaries and entity categories. The annotation adopts the BIO annotation system: B represents the beginning of the entity, I represents the inside of the entity, and O represents a non-entity word. Entity categories include basic categories such as person (PER), place (LOC), organization (ORG), time (TIME), and domain-specific professional terminology categories. In addition, this embodiment also establishes an entity dictionary to assist in annotation and verification.

[0168] Furthermore, this embodiment constructs and trains an entity recognition model. The model uses a neural network architecture based on BERT and combined with the GlobalPointer mechanism. The BERT model has acquired powerful semantic understanding capabilities through pre-training, while the GlobalPointer layer is specifically used to handle the recognition of nested entities. The specific structure of the model is as follows:

[0169] BERT encoding layer: converts input text into context-dependent word vector representations;

[0170] GlobalPointer layer: uses relative position encoding to calculate the probability of each position pair being the head and tail of an entity;

[0171] Multi-head attention mechanism: set independent recognition heads for different entity categories;

[0172] The model training uses the Adam optimizer with a batch size of 32 and a learning rate of 2e-5, and uses the cross entropy loss function. To prevent overfitting, regularization strategies such as dropout (rate of 0.1) and weight decay are introduced.

[0173] Then, the trained model is used to perform entity recognition on the target text. The recognition process not only outputs the entity category, but also records the location information and confidence score of the entity in the text. In this embodiment, a confidence threshold (usually 0.6) is set to retain only high-confidence recognition results.

[0174] Furthermore, this embodiment constructs training data for relationship extraction based on the identified entity data. First, this embodiment finds sentences containing two or more entities in the corpus, and then annotates the relationships between the entities in these sentences. Relationship types include: affiliation, part-whole relationship, time relationship, position relationship, etc. The remote supervision method is used for annotation, and some annotation data is automatically generated using the existing knowledge base, which is then manually verified and supplemented. Thus, the second encoder is trained using the training data to obtain a relationship extraction model. Among them, the second encoder adopts a BERT-based encoder structure and introduces the following innovative designs:

[0175] Entity location awareness mechanism: add entity location encoding in the input layer;

[0176] Multi-task learning framework: simultaneous relationship classification and relationship validity judgment;

[0177] Adversarial training strategy: improve the robustness of the model;

[0178] Attention mechanism: focus on the contextual information between entities.

[0179] Furthermore, the target text data and entity data are input into a relation extraction model to identify the association relationship between entities based on the entity location data in the entity data and the text sequence in the target text data, and output the relation data.

[0180] Specifically, the text and the identified entity information are input into the model (i.e., the relationship extraction model), which makes relationship judgments based on the entity location and context, outputs the relationship type and confidence score, and then post-processes the results. The post-processing includes: relationship validity verification; conflict relationship resolution; transitive closure calculation; relationship rule verification. This embodiment also implements some optimization measures: using entity type constraints to avoid unreasonable relationship extraction; establishing a relationship pattern library to verify the extraction results; implementing an incremental learning mechanism to continuously optimize model performance; and designing a feedback error correction mechanism to allow manual intervention and correction.

[0181] Through the above steps, this embodiment can accurately identify entities in text and effectively extract semantic relationships between entities, providing high-quality basic data for the construction of knowledge graphs. In addition, the modular design of this embodiment is also convenient for optimization and adjustment according to specific application scenarios.

[0182] For example, the Bert-GlobalPointer model is used to deal with the problem of nested entities in public texts. First, in the data annotation and preprocessing stage, the training corpus is manually annotated for predefined entity categories (such as subject, object, time, etc.). Considering the characteristics of entity nesting in public texts, a multi-layer annotation method is adopted to ensure that both inner and outer entities can be correctly labeled. At the same time, the annotated data is converted into the input format required by the model, including preprocessing work such as word segmentation and dictionary establishment.

[0183] Furthermore, the model building phase uses a neural network structure with BERT as the encoder, and adds a GlobalPointer layer on the basis of BERT. The design of GlobalPointer is particularly suitable for dealing with nested entity problems. It can simultaneously identify entities at different levels, avoiding the limitations of traditional sequence labeling methods in dealing with nested entities. The model training uses the cross entropy loss function, uses the AdamW optimizer for parameter optimization, and adopts a learning rate decay strategy.

[0184] During the model training and evaluation phase, batch training is adopted, model performance is monitored through the validation set, and an early stopping strategy is used to prevent overfitting. Model evaluation uses indicators such as precision, recall, and F1 value, with special attention paid to the recognition effect of nested entities. For the recognized entities, post-processing is also required, including entity boundary adjustment, entity type confirmation, etc., to improve recognition accuracy.

[0185] The relationship extraction module is implemented using a remote supervision method combined with a pre-trained language model. The first step is to define the relationship type. According to the characteristics of the field, a series of relationship types are pre-defined, such as "inclusion", "reference", "influence", etc. Through remote supervision, the training data is automatically annotated using the existing knowledge base to reduce the cost of manual annotation. At the same time, a relationship model library is established to verify the rationality of the extracted relationship.

[0186] In terms of model construction, BERT is used as the basic encoder, and the relationship between entity pairs is judged through a special relationship classification layer. The model input includes text sequence and entity position information, and the semantic association between entities is captured through the attention mechanism. In order to handle the situation where a pair of entities may have multiple relationships, multi-label classification is adopted, and the sigmoid activation function is used to output the probability of each relationship type.

[0187] In the process of relation extraction, the directionality of relations must also be considered. The direction of relations is determined by analyzing the semantic roles and contextual information of entity pairs. At the same time, a relation rationality verification mechanism is established, including entity type constraints, relation transitivity checks, etc., to filter out unreasonable relations. Finally, the confidence of the extracted relations is evaluated, and only relations with higher confidence are retained to ensure the quality of the knowledge graph.

[0188] S3: According to the entity relationship data, the target text data is imported into a preset graph database to form a knowledge graph.

[0189] In this application, the knowledge graph construction realizes the conversion of entity relationship data into a queryable and scalable knowledge graph database. This application can use Neo4j as the underlying graph database, and its native graph storage and processing capabilities provide ideal technical support for the construction of the knowledge graph.

[0190] Before importing data, the application first preprocesses and normalizes the entity relationship data. This includes deduplication and merging of entities, normalization of relationships, normalization of attributes, etc. The device checks the unique identifier of the entity to ensure that the same entity from different sources can be correctly merged. For relationship data, the application will unify the directionality of the relationship and ensure the semantic consistency of the relationship.

[0191] In terms of database configuration, key parameters of Neo4j are optimized according to data scale and query requirements. This includes memory allocation, cache strategy, transaction configuration, etc. Especially for large-scale knowledge graphs, this application adopts a batch import strategy to avoid memory overflow and improve import efficiency through parallel processing.

[0192] In order to improve query performance, this application has established a multi-level index system. The first is the entity index, which creates indexes for frequently queried entity attributes; the second is the relationship index, which creates indexes for commonly used relationship types; and the last is the composite index, which is used to optimize multi-condition query scenarios. The index is established based on query log analysis to ensure that the index can cover the most common query patterns.

[0193] During the data import process, the device adopts batch import mode. The specific steps are as follows: entity import, that is, first import all entity nodes to establish the basic structure of the entity; relationship import, that is, establish connections between entities based on relationship data; attribute update, that is, supplement the detailed attribute information of entities and relationships; index construction, that is, create and optimize the necessary index structure.

[0194] In order to ensure the consistency and integrity of the data, this application implements a complete transaction management mechanism. The import of each batch of data is carried out under transaction control. If an error occurs during the import process, this application can perform a rollback operation to ensure the consistency of the database. At the same time, this application will record detailed import logs to facilitate problem location and data recovery.

[0195] After the knowledge graph is formed, this application also carries out a series of optimization and verification work, including: data integrity check, that is, verifying the integrity of entities and relationships; graph structure analysis, that is, evaluating the connectivity and coverage of the knowledge graph; performance testing, that is, performing query performance benchmarking; data consistency verification, that is, ensuring data consistency between different modules.

[0196] In addition, this application also implements an incremental update mechanism for the graph. When new entity relationship data needs to be added, this application can intelligently merge the new and old data to avoid data redundancy and conflict.

[0197] Optionally, the incremental update process in this application includes: preprocessing and verification of new data; comparison and merging with existing data; incremental index update; and graph structure optimization. In order to provide better visualization and management functions, this application integrates a graph visualization module to support interactive exploration and editing of graphs. Administrators can monitor graph status, adjust configuration parameters, and perform maintenance operations through the visualization interface.

[0198] Therefore, this application provides a reliable data foundation for subsequent intelligent question-answering and knowledge services by converting entity relationship data into a structured knowledge graph. The knowledge graph has good scalability, query efficiency and maintainability, and can meet various complex knowledge management and retrieval needs.

[0199] For example, in the field of scientific and technological public texts, first, the core entity categories are divided into the following categories:

[0200] Public text subjects: including departments at all levels, science and technology management agencies and other public text formulation and execution agencies, such as the Ministry of Science and Technology, local science and technology departments, etc.;

[0201] Public text objects: including enterprises, scientific research institutes, colleges and universities, scientific and technological personnel and other public text service objects;

[0202] Public text measures: including specific public text contents such as financial support, tax incentives, talent introduction, and scientific and technological innovation;

[0203] Implementation time: including the start time, end time, transition period and other time nodes of the public text;

[0204] Public text areas: including artificial intelligence, biomedicine, new energy and other specific scientific and technological fields;

[0205] Supporting documents: including implementation details, explanatory notes, supplementary notices and other relevant public text documents.

[0206] Second, entity attribute design, that is, each entity category contains the following core attributes:

[0207] 1) Public text file attributes: Title: the full name of the public text file; Release time: the specific date when the public text is released; Implementation time: the time when the public text is officially implemented; Release agency: the specific department responsible for releasing the public text; Public text level: first-level, second-level, third-level and other administrative levels; Document number: the official number of the public text file; Public text status: whether it is currently valid;

[0208] 2) Attributes of the content of public texts: Scope of application: the region or field to which the public text applies; Support methods: specific measures such as financial support and tax incentives; Support amount: the specific amount or proportion of support; Application conditions: the conditions that must be met to apply for support for the public text; Implementation steps: the specific process and steps for the implementation of the public text.

[0209] Third, entity relationship design, including:

[0210] Types of relationships between entities: inclusion relationship, that is, the relationship between the upper-level public text and the lower-level public text, such as the general outline and implementation details; citation relationship, that is, the mutual citation and reference between public text documents; influence relationship, that is, the effect of public text on specific objects or fields; inheritance relationship, that is, the revision or replacement of the old public text by the new public text; supporting relationship, that is, the relationship between the main public text and the supporting public text; collaborative relationship, that is, the coordination and complementary relationship between different public texts.

[0211] Fourth, knowledge modeling, including:

[0212] 1) Construction of domain vocabulary. The domain vocabulary is constructed according to the following levels: core concept layer, including basic concepts and terms in the field of science and technology public texts; institution name layer, including proper nouns such as departments at all levels and scientific research institutions; public text terminology layer, including common professional terms and expressions in public text documents; time terminology layer, including various time expressions and time units; operation verb layer, including key verbs related to the execution of public texts;

[0213] 2) Knowledge triple structure, including: using the "subject-predicate-object" triple structure to represent knowledge: subject, which can be a public text entity, an institutional entity or other entity types; predicate, which indicates the relationship type between entities; object, which can be a specific attribute value or other related entity; typical triple examples: public text on scientific and technological innovation, issuing agency, Ministry of Science and Technology; public text on innovation vouchers, supporting objects, science and technology-based small and medium-sized enterprises; public text on talent introduction, including housing subsidy measures;

[0214] 3) Entity relationship model: The entity relationship model defines the following relationships that may exist between different types of entities:

[0215] Common text file relational schemas, such as:

[0216] Public text file → Release → Release agency;

[0217] Public Text File → Apply → Public Text Object;

[0218] Public text documents → Regulations → Public text measures;

[0219] Common text measures relational schemas, for example:

[0220] Public text measures → support → support objects;

[0221] Public text measures → involve → support areas;

[0222] Public text measures → execution → implementation steps;

[0223] Temporal relationship patterns, such as:

[0224] Public text file → effective at → effective time;

[0225] Public text file → expires at → expiration time;

[0226] Public Text Measures → Executed On → Execution Time.

[0227] In some embodiments, Figure 3 As shown, step S3 includes:

[0228] S3.1: Based on a preset graph database, divide the device memory acting on the knowledge graph into page cache and heap memory according to a specified ratio to form corresponding memory configuration data;

[0229] S3.2: constructing an attribute index for the entity attribute corresponding to when the query frequency data is higher than a preset query frequency threshold, and constructing a relationship index for the relationship type corresponding to when the relationship frequency data is higher than a preset relationship frequency threshold, so as to generate an index system by the attribute index and the relationship index;

[0230] S3.3: Generate a knowledge graph about the target text data based on the entity relationship data, the memory configuration data and the index system.

[0231] In this embodiment, the entity relationship data is normalized. First, the entity data is standardized, including the normalization of entity names (such as removing extra spaces, unifying upper and lower case), formatting of attribute values ​​(such as unifying date formats, numerical units), and normalization of entity types (ensuring consistency with the predefined ontology architecture). For synonymous entities, this embodiment merges them through entity alignment technology, specifically including: string matching, that is, using algorithms such as edit distance to identify similar entity names; attribute comparison, that is, comparing whether the key attributes of entities are similar; context analysis, that is, considering the relationship environment of the entity; external knowledge base comparison, that is, using authoritative knowledge bases to perform entity disambiguation. The normalization of relationship data includes: relationship type standardization, that is, unifying relationship naming and mapping to a predefined set of relationship types; relationship direction normalization, that is, ensuring that the direction of the relationship complies with predefined rules; relationship weight calculation, that is, setting relationship weights based on frequency of occurrence and credibility; relationship validity verification, that is, filtering out relationships that do not comply with domain rules.

[0232] Furthermore, this embodiment configures the graph database. For the Neo4j graph database, the following configurations are mainly performed:

[0233] Memory configuration, such as setting the heap memory size (usually 70% of the available physical memory), configuring the page cache size (adjusted according to the data scale), adjusting garbage collection parameters, storage configuration, setting the data file storage path, configuring the transaction log location, and optimizing file device parameters; concurrency configuration, such as adjusting the maximum number of concurrent connections, setting the transaction timeout, and configuring the thread pool size; security configuration, such as setting access control policies, configuring user permissions, and enabling encrypted transmission.

[0234] Furthermore, first, a hierarchy of entity types is established, including: top-level types, such as Thing, Object, Event, etc.; middle-level types, such as Person, Organization, Location, etc.; domain-specific types, defined according to specific application scenarios; for each entity type, its attribute mode is defined: required attributes, such as ID, name, creation time, etc.; optional attributes, such as description, label, alias, etc.; constraints, such as attribute value type, value range, etc. Secondly, the relationship model is defined, including: relationship type and its hierarchy; starting entity type constraint of the relationship; attribute definition of the relationship; cardinality constraint of the relationship. Subsequently, a complete indexing strategy is also established, including: creating node indexes for high-frequency query attributes; establishing relationship indexes for important relationship types; designing composite indexes to optimize complex queries; configuring index update strategies. Finally, a schema verification mechanism is implemented, including: data integrity check; schema consistency verification; constraint verification; performance impact assessment.

[0235] Therefore, this embodiment lays a solid foundation for the construction of the knowledge graph and ensures the standardization, consistency and queryability of the data. These tasks directly affect the quality of the knowledge graph and the subsequent application effect, and are the key links of the entire device.

[0236] It should be noted that Neo4j, as a native graph database, is suitable for storing and querying complex relational data. In this embodiment, the configuration and optimization of Neo4j are mainly carried out from the following aspects:

[0237] First, configure the database basic environment. You need to set Neo4j's memory allocation reasonably. Usually, 60% of the total available memory is allocated to Neo4j, of which 40% is for page cache and 20% is for heap memory. This configuration can avoid memory overflow while ensuring database performance. At the same time, you need to adjust the number of concurrent connections to the database and set a reasonable connection pool size based on the expected traffic.

[0238] Secondly, optimize the index configuration. In view of the characteristics of the knowledge graph, a multi-level index system needs to be established. For high-frequency query attributes, such as title, release time, type, etc., establish attribute indexes; for common relationship types, such as "include" and "reference", establish relationship indexes. At the same time, use composite indexes to optimize multi-condition query performance. In order to speed up full-text search, it is also necessary to configure full-text indexes to support fuzzy queries of content.

[0239] Finally, query optimization configuration. By setting the query timeout, prevent complex queries from taking up too many resources. Configure the query plan cache size to improve the response speed of common queries. At the same time, enable Neo4j's query log function to monitor and optimize slow queries.

[0240] The import of knowledge data is a complex engineering process that needs to be carried out in steps and levels. The first step is the import of entity data, which is divided into three stages:

[0241] The first stage is entity preprocessing. The entities extracted from the text are standardized, including the normalization of entity names, the formatting of attribute values, the merging of duplicate entities, etc. A unique entity identifier is established to ensure that the same entity can be correctly identified and associated in different texts. For ambiguous entities, context analysis and manual confirmation are used to eliminate ambiguity.

[0242] The second stage is the batch import of entities. Neo4j's batch import tool is used to import the pre-processed entity data into the database in batches. During the import process, it is necessary to maintain transaction consistency and set an appropriate batch size to avoid a single transaction that is too large and affects performance. At the same time, a breakpoint-resume mechanism is implemented during the import process to ensure the reliability of large-scale data import.

[0243] The third stage is entity data verification. The integrity and accuracy of the imported data are checked through automated scripts, including checking required attributes, verifying the format of attribute values, verifying the rationality of relationships between entities, etc. For data that fails verification, an error report is generated and manually reviewed.

[0244] Importing relational data includes the following steps:

[0245] First, the preprocessing of the relationship needs to verify the existence of the entities at both ends of the relationship to ensure that there is no dangling relationship. For directional relationships, it is necessary to ensure the correctness of the relationship direction. At the same time, it is necessary to process the weight information of the relationship to reflect the importance or confidence of the relationship;

[0246] Then, it is the batch import of relations. A parallel import strategy is adopted to process data of multiple relation types at the same time. During the import process, the integrity constraints of the relations need to be maintained to ensure that no duplicate or conflicting relations are generated. For complex relation networks, special attention needs to be paid to the import order to avoid circular dependencies.

[0247] Finally, the maintenance and update mechanism of the knowledge graph includes: establishing an incremental update mechanism that can timely convert new files into nodes and relationships in the knowledge graph. Implementing version control functions, recording the evolution history of the knowledge graph, and supporting data rollback operations. Designing a data cleaning mechanism to regularly clean up expired or invalid data to maintain the timeliness of the knowledge graph.

[0248] Optionally, set up consistency maintenance for the knowledge graph. When errors are found in entities or relationships, it is necessary to evaluate the scope of the modification and formulate appropriate correction strategies. For important modification operations, it is necessary to record modification logs for easy tracking and auditing. At the same time, regularly conduct quality assessments of the knowledge graph, including assessments of multiple dimensions such as knowledge completeness, accuracy, and timeliness, to ensure that the quality of the knowledge graph continues to improve.

[0249] In another specific embodiment, the step S3 further includes:

[0250] Based on the entity relationship data, a data processing model is constructed, wherein the data processing model includes a decision network and a representation network, wherein the decision network is used to indicate the domain type of the graph node in the knowledge graph, and the representation network is used to indicate that the output result of the decision network updates the embedding representation of the corresponding graph node, wherein each graph node represents an entity, and the edge in the knowledge graph represents the corresponding relationship data;

[0251] Performing reinforcement learning on the data processing model to generate node importance data of the graph nodes;

[0252] According to the node importance data of the graph node, the information importance score of the node information related to the graph node is calculated, and the node information whose information importance score is lower than the preset importance threshold is removed, and then a knowledge graph is generated.

[0253] In this embodiment, for the optimization link in the knowledge graph construction process, the quality of the knowledge graph is improved by combining the decision network and the representation network. It should be noted that this process adopts the decision-based heterogeneous graph attention network (DHGAT) model to optimize and filter the knowledge graph through reinforcement learning methods.

[0254] Specifically, a data processing model consisting of a decision network and a representation network was first constructed. The main task of the decision network is to select the most suitable domain type for each graph node. This process takes into account the local structural characteristics and global semantic information of the graph node. Specifically, the decision network analyzes the attribute information of the graph node, the connection relationship with other graph nodes, and the position in the entire knowledge graph to determine the domain type to which the graph node is most likely to belong. The representation network is responsible for updating the embedded representation of the graph node based on the output results of the decision network. This update fully considers the domain attributes and relationship characteristics of the graph nodes, so that the representation of the graph nodes can more accurately reflect their semantic information in the knowledge graph.

[0255] Furthermore, this embodiment performs reinforcement learning training on the data processing model. During the training process, the model learns how to select the optimal domain type for the graph node by constantly trying different decision strategies. The reward signal design of reinforcement learning takes into account multiple factors, including the accuracy of node classification, the degree of clustering of nodes in the domain, and the overall connectivity of the knowledge graph. Therefore, by constructing and training the model, this embodiment can gradually optimize its decision-making strategy and generate more accurate node importance data. The training process adopts the policy gradient method and introduces the experience replay mechanism to improve learning efficiency.

[0256] Then, for the optimization and filtering stage, this embodiment calculates the information importance score of each graph node based on the node importance data obtained by reinforcement learning. This score comprehensively considers multiple characteristics of the node, including the degree centrality, betweenness centrality, and importance evaluation accumulated during the reinforcement learning process. This embodiment sets an appropriate importance threshold and filters node information with a score below the threshold. This filtering mechanism can effectively remove noise information and redundant data in the knowledge graph, and improve the quality and efficiency of the knowledge graph.

[0257] It should be noted that an important feature of the whole process is its adaptability and interpretability. The decision network can automatically adjust its decision strategy according to the characteristics of different fields, while the representation network can maintain the continuity and consistency of node representation. At the same time, by analyzing the decision process of the model, it is possible to understand why some nodes are considered more important and other nodes are filtered out. This interpretability has important guiding significance for the maintenance and optimization of knowledge graphs.

[0258] Finally, this embodiment also implements an incremental learning mechanism. When new entity relationship data is added to the knowledge graph, the model can quickly evaluate and optimize the new nodes based on existing learning experience, thereby maintaining the dynamic update capability of the knowledge graph. This mechanism ensures that the knowledge graph can maintain a high quality state as the data continues to increase.

[0259] Specifically, this embodiment adopts the DHGAT model. It should be noted that the infrastructure design of the DHGAT model is based on solving the limitations of traditional GNNs in processing heterogeneous graph data. Traditional GNNs often treat all types of relationships equally when processing text. This processing method ignores the differences between different types of relationships, resulting in less than ideal information aggregation effects. By introducing a decision-making mechanism, DHGAT can dynamically select the optimal neighborhood aggregation strategy based on the specific circumstances of the node.

[0260] Secondly, at the data representation level, DHGAT converts text into a heterogeneous graph structure. In this heterogeneous graph, nodes represent various entities in the document, such as subjects, measures, implementation time, etc.; edges represent various types of relationships between entities, such as inclusion relationships, reference relationships, etc. This representation method retains the rich semantic information of the text and also provides a structured basis for subsequent information filtering.

[0261] At the core of the model, DHGAT adopts a dual network structure: a decision network and a representation network. The decision network is responsible for selecting the most suitable neighborhood type for each node. This selection process takes into account the local structural characteristics and global semantic information of the node. The decision network continuously optimizes its decision strategy through reinforcement learning to maximize the effect of information aggregation. The representation network is responsible for updating the embedded representation of the node according to the selection results of the decision network. It uses the attention mechanism to perform weighted aggregation on the selected neighborhood information.

[0262] In actual operation, DHGAT processes heterogeneous graph data layer by layer in an iterative manner. In each layer, the model first uses the decision network to select the optimal neighborhood type for the current node, and then uses the representation network to update the node representation. This process is repeated until the node representation becomes stable. This iterative processing method ensures that the model can fully capture the deep semantic associations in the text.

[0263] In a semi-supervised learning environment, the training process of DHGAT is divided into two stages: first, the basic parameters of the model are trained on labeled data, and then the model is further optimized using unlabeled data. This training strategy allows the model to achieve good performance even with limited labeled data.

[0264] Finally, in terms of redundant information filtering, DHGAT identifies and filters out duplicate or redundant information by analyzing the position and connection relationship of nodes in heterogeneous graphs. The model calculates the importance score of information, retains key information while eliminating secondary information, thereby improving the quality and efficiency of the knowledge graph.

[0265] Through the above design, DHGAT can effectively process complex semantic relationships in text and provide a high-quality information foundation for the construction of knowledge graphs. This decision-based dynamic processing method not only improves the accuracy of information processing, but also enhances the adaptability and interpretability of the model.

[0266] In another embodiment, step S3 may further include:

[0267] According to the node type and relationship type of the graph node, a first mask ratio is configured for a node whose node type meets a preset important node condition, and a second mask ratio is configured for a node whose node type meets a preset auxiliary node condition, so as to form mask data, wherein the first mask ratio is lower than the second mask ratio;

[0268] In combination with a preset multi-task learning mechanism, the mask data is used to pre-train a pre-established encoder-decoder structure, wherein the encoder-decoder structure includes a third encoder and a decoder, the third encoder is used to instruct the conversion of the heterogeneous graph data indicated by the knowledge graph into a feature vector, and the decoder is used to instruct the reconstruction of the graph structure with the feature vector;

[0269] Build a classifier based on the pre-trained encoder-decoder structure;

[0270] The classifier is trained using the feature vector and a task feature vector determined by the current task requirement, so that the knowledge graph is optimized using the trained classifier.

[0271] Therefore, this embodiment constitutes a classifier construction process based on mask pre-training and multi-task learning. This process adopts a differentiated mask strategy and attention mechanism to improve the performance of the model through pre-training and fine-tuning.

[0272] In this embodiment, the device introduces a double-layer masking strategy. For the graph nodes in the knowledge graph, they are classified according to their importance and configured with different mask ratios. Important nodes (such as core public text nodes) are configured with a lower mask ratio (first mask ratio), while auxiliary nodes are configured with a higher mask ratio (second mask ratio). This differentiated masking strategy ensures that the model can pay more attention to the feature learning of important nodes, while enhancing the generalization ability of the model through the high mask ratio of auxiliary nodes. According to practical experience, the first mask ratio is usually set to 15%, while the second mask ratio is set to 30%.

[0273] Furthermore, in combination with a preset multi-task learning mechanism, the mask data is used to pre-train a pre-established encoder-decoder structure, wherein the encoder-decoder structure includes a third encoder and a decoder, the third encoder is used to instruct the conversion of the heterogeneous graph data indicated by the knowledge graph into a feature vector, and the decoder is used to instruct the reconstruction of the graph structure using the feature vector.

[0274] Among them, for the pre-training process of the encoder-decoder structure, the structure adopts the Transformer architecture, in which the third encoder is responsible for converting heterogeneous graph data into feature vectors. This process not only considers the attribute information of the nodes, but also includes the structural information of the graph. The decoder is responsible for reconstructing the graph structure and optimizing the model parameters by minimizing the reconstruction error. The pre-training process adopts a multi-task learning mechanism to perform tasks such as node classification, relationship prediction, and graph structure reconstruction at the same time. This multi-task learning strategy can help the model learn richer feature representations.

[0275] Furthermore, a classifier is constructed based on the pre-trained encoder-decoder structure. This classifier adopts a multi-layer perceptron structure and adds a softmax activation function in the last layer. The design of the classifier takes into account the special needs of public text classification and can handle multi-label classification problems, which is very important for the multi-dimensional classification of public text.

[0276] Based on the above embodiment, the step of training the classifier using the feature vector and the task feature vector determined by the current task requirement includes:

[0277] Performing dimensionality reduction processing on the feature vector;

[0278] The reduced-dimensional feature vector and the task feature vector are input into the classifier to calculate the associated weights between the reduced-dimensional feature vector and the task feature vector using a preset attention mechanism to form a feature weight matrix, and based on the feature weight matrix, the reduced-dimensional feature vector and the task feature vector are weightedly combined to obtain a trained classifier.

[0279] In this embodiment, the t-SNE algorithm can be used for dimensionality reduction. This nonlinear dimensionality reduction method can better maintain the local structural characteristics of the data. In the dimensionality reduction process, this embodiment will dynamically adjust the perplexity parameter to strike a balance between dimensionality reduction effect and computational efficiency. Through dimensionality reduction processing, not only can the computational overhead be reduced, but also the noise impact of the data can be reduced.

[0280] Then, a multi-head attention mechanism is used to calculate the association weight between the reduced feature vector and the task feature vector. The number of heads of the attention mechanism is set according to the specific task requirements, usually 8 or 16. The feature weight matrix obtained by attention calculation reflects the importance relationship between different features. Then, this weight matrix is ​​used to perform a weighted combination of the feature vectors to obtain the final feature representation. The training process uses techniques such as gradient clipping and learning rate warm-up to improve the stability of training.

[0281] The entire training process also includes some important optimization strategies. The first is the use of residual connections, which helps solve the gradient vanishing problem in deep network training. The second is the addition of batch normalization layers to speed up the training process and improve the stability of the model. Finally, an early stopping mechanism is implemented to stop training in time when the performance on the validation set no longer improves to prevent overfitting.

[0282] In one of the embodiments, an innovative heterogeneous graph masked autoencoder (HGMAE) is used to pre-train a large knowledge graph. Subsequently, a specialized classifier model is trained to predict the generalization probability. The classifier uses the cascaded feature vector derived from the pre-trained encoder and the task-specific feature vector of the text. Through pre-training and classification prediction, the quality of the knowledge graph is improved and the accuracy of the question-answering device is enhanced. Specifically, HGMAE pre-trains and optimizes the knowledge graph after the knowledge graph is constructed and before the question-answering device is developed, providing a better semantic basis for subsequent intelligent question-answering.

[0283] First, the HGMAE model adopts a two-stage processing strategy, which fully considers the characteristics and application requirements of knowledge graphs. The heterogeneous graph mask autoencoder in the first stage is dedicated to learning the basic semantic representation of the knowledge graph, while the classifier in the second stage focuses on the prediction ability of specific tasks. This hierarchical design ensures both the versatility of the model and the accuracy of specific tasks.

[0284] In the first stage, HGMAE's pre-training process uses an innovative masking mechanism. Different from traditional masking methods, HGMAE designs differentiated masking strategies based on node types and relationship types. For important entity nodes, a lower masking ratio is used; for auxiliary attribute nodes, a higher masking ratio is used. This adaptive masking strategy can better maintain the core semantic structure of the knowledge graph.

[0285] During pre-training, HGMAE learns the semantic representation of nodes through an encoder-decoder structure. The encoder is responsible for converting the input heterogeneous graph data into a low-dimensional feature vector, which takes into account the type information, structure information, and attribute information of the node. The decoder attempts to reconstruct the original graph structure from these feature vectors, including node attributes and edge relationships. By minimizing the reconstruction error, the model can learn more effective feature representations.

[0286] In the pre-training stage, HGMAE also introduces a multi-task learning mechanism. In addition to the basic graph reconstruction task, the model also performs auxiliary tasks such as node type prediction and relationship type prediction. The introduction of these tasks can help the model learn richer semantic information and improve the quality of feature representation. The model balances the importance of each task through dynamic weight adjustment to ensure the stability of the learning process.

[0287] Entering the second stage, a special classifier is built based on the pre-trained encoder. The main task of this classifier is to predict the generalization probability of knowledge, that is, to evaluate the applicability of a certain knowledge point in different scenarios. The input of the classifier consists of two parts: one is the feature vector extracted from the pre-trained encoder, and the other is the text feature vector extracted according to the specific task requirements.

[0288] The classifier uses a cascade feature fusion method to organically combine feature vectors from two different sources. The device first reduces the dimension of the pre-trained features to remove redundant information, and then fuses them with task-specific features. This fusion method not only retains the general semantic information captured by the pre-trained model, but also optimizes for specific tasks.

[0289] During the training process, the classifier uses a multi-layer perceptron structure and an attention mechanism to handle the associations between different features. By learning the attention weights, it can automatically identify the most important feature combination for the current task. At the same time, regularization technology is introduced to prevent overfitting and ensure the generalization ability of the model.

[0290] Finally, in order to improve the reliability of the prediction, this embodiment also implements an uncertainty estimation mechanism. In addition to outputting the generalization probability, the model also gives a confidence assessment of the prediction. This dual output design can help users better understand and use the prediction results.

[0291] Through the above design, HGMAE can effectively improve the quality and usability of the knowledge graph. The feature representation learned in the pre-training stage provides a good semantic basis for the subsequent question-answering device, while the specialized classifier ensures the accuracy of the device in specific tasks. This pre-training plus fine-tuning mode not only improves the performance of the model, but also enhances the scalability and adaptability of the device.

[0292] S4: Obtain the input question statement.

[0293] In the input link of the intelligent question-and-answer device, the reception and preprocessing of user questions are realized. First of all, a reasonable question input interface needs to be designed to support multiple input methods including text input box and voice input, and to support single-round question-and-answer and multi-round dialogue modes. In order to improve the user experience, this application also implements real-time input response functions, including input prompts and automatic completion features.

[0294] After receiving the question input by the user, this application will perform a series of preprocessing operations. The first is basic text cleaning, removing unnecessary spaces and special characters and other interfering information, and normalizing punctuation. Then, this application will use a synonym dictionary to standardize the words, and replace synonyms and near-synonyms with standard expressions. Finally, the question must be segmented and tagged with parts of speech to prepare for subsequent semantic understanding.

[0295] Question standardization is another important step. This application needs to convert the user's colloquial expressions into standardized expressions, handle possible omissions and references, and ensure that the complete question semantics can be restored. For longer question sentences, this application will perform appropriate segmentation to extract the core questions. At the same time, this application will also identify and mark the keywords and entity words in the questions, which is very important for subsequent question understanding and answer retrieval.

[0296] In a multi-round dialogue scenario, this application also needs to handle context association. This includes saving the historical information of the dialogue, establishing context associations between different questions, processing pronoun resolution to clarify the referent, and maintaining the dialogue state to track the progress of the entire dialogue. These functions are crucial to achieving a coherent dialogue experience.

[0297] Finally, the application will verify the validity of the question. This includes checking the completeness and validity of the question and verifying whether the question is within the processing capabilities. If the question is found to be incomplete or unclear, the application will give corresponding prompt information and provide question reconstruction suggestions when necessary. Through this series of processing, the application can convert the original question entered by the user into a structured and standardized question representation, providing a reliable basis for subsequent question classification and answer retrieval.

[0298] S5: Classify the question sentence using a pre-established question classification model to determine question features in the question sentence, where the question features include question type and question entity data.

[0299] In this application, a pre-established question classification model is used to analyze and classify the input question sentences. Step S5 uses a BERT-based deep learning model to classify questions, determine the question type and extract entity information contained in the question through semantic understanding of the question.

[0300] It should be noted that for the training of the question classification model, the model is trained using a large-scale annotated data set, which contains various types of questions and their corresponding labels. During the training process, the Adam optimizer is used, the learning rate is set to 2e-5, and the learning rate warm-up strategy is used. In order to prevent overfitting, the model also introduces a dropout mechanism, and the dropout rate is set to 0.1. Through multiple rounds of training and verification, the model can accurately identify different types of questions.

[0301] In terms of question type recognition, a variety of question types are predefined, including attribute queries (such as "When was a certain public text published"), relationship queries (such as "Which public texts are related to scientific and technological innovation"), and comparative queries (such as "What is the difference between public text A and public text B"), etc. The question classification model determines which type of question belongs to by analyzing the semantic structure and keyword features of the question. This classification result directly affects the selection of subsequent query strategies.

[0302] For the identification of question entities, a method similar to entity relationship extraction is adopted, using the BERT-GlobalPointer model for processing. This model can accurately identify the entities and their types in the question, such as public text names, organization names, time information, etc.

[0303] It should be noted that this embodiment also implements the contextual understanding capability of question features. In a multi-round dialogue, it is necessary not only to understand the literal meaning of the current question, but also to accurately understand the complete semantics of the question in combination with the previous dialogue history. For example, when a user asks "Are there any other relevant public texts?", it is possible to determine what type of public text "relevant public text" refers to based on the context.

[0304] Finally, this embodiment will conduct a reliability assessment on the identified question features. For classification results with low confidence, this embodiment will initiate a secondary confirmation mechanism, which may clarify the question intent through user interaction. At the same time, this embodiment will integrate the classification results and entity recognition results to form a structured question representation, which contains information in multiple dimensions such as question type, entity information, relationship requirements, etc., to provide accurate retrieval conditions for subsequent knowledge graph queries.

[0305] S6: Retrieve text data matching the question features in the knowledge graph as answer data for the question statement.

[0306] In this application, the question features need to be converted into query statements for the graph database first. This application will generate corresponding Cypher query statements based on different question types. For example, for attribute query questions, the device will construct a statement that directly queries node attributes; for relationship query questions, it will generate a complex statement containing path query.

[0307] During the query execution process, this application adopts multiple optimization strategies. The first is query path optimization, which selects the optimal query path by analyzing the structural characteristics of the knowledge graph. This application considers factors such as node connectivity and relationship weight to avoid performance bottlenecks during the query process. At the same time, this application also uses the previously established index system to ensure that queries can quickly locate relevant nodes and relationships.

[0308] Furthermore, for the screening and sorting of answers, the present application sorts the answers according to the relevance of the query results. The calculation of relevance takes into account multiple factors, including the distance from the query entity, the weight of the relationship, the importance score of the node, etc. In the case of returning multiple possible answers, the device will use a preset sorting algorithm to put the most relevant answer in the front. At the same time, the present application will also consider the timeliness of the answer and give priority to returning the currently valid public text information.

[0309] In order to provide a better user experience, this application implements the assembly and beautification functions of answers. Depending on the type of question, this application will use different answer assembly templates. For simple attribute queries, the attribute value is returned directly; for complex relationship queries, this application will generate structured answers, including relevant contextual information. If the answer involves multiple public text documents, this application will integrate the content, remove duplicate information, and ensure the simplicity and completeness of the answer.

[0310] Optionally, this application also has the ability to explain answers. For each returned answer, this application will generate a corresponding chain of evidence to explain how the answer was obtained from the knowledge graph. This interpretability can not only increase the user's trust in the answer, but also help users better understand the relationship between public texts. If the user is not satisfied with the answer, this application also provides an answer feedback mechanism to allow the user to make further inquiries or clarifications.

[0311] Finally, this application implements a dynamic answer update mechanism. When the information in the knowledge graph is updated, this application can reflect these changes in a timely manner to ensure that the returned answers are always the latest. At the same time, this application will record query logs, including query content, returned results, user feedback and other information, which can be used for subsequent device optimization and answer quality improvement.

[0312] This method based on pre-training and multi-task learning not only improves the performance of the model, but also enhances the interpretability of the model. By analyzing the attention weights, we can understand which features the model pays more attention to when making classification decisions, which has important guiding significance for the practical application of public text classification. At the same time, the modular design of the model also makes it easy to adapt to different public text classification tasks, with good scalability and adaptability.

[0313] In some embodiments, step S6 further includes:

[0314] Obtain a pre-trained large language model and construct a training corpus based on the entity relationship data in the knowledge graph;

[0315] Using the training corpus to fine-tune the large language model in a certain domain, thereby obtaining an intelligent agent model with knowledge graph query capability;

[0316] Inputting the question statement into the intelligent agent model to generate a query plan including multiple sub-query steps;

[0317] Based on the query plan, performing multi-hop path exploration in the knowledge graph, wherein weight data of different paths are calculated based on a graph attention mechanism in each hop exploration;

[0318] According to the weight data, a query path with high confidence is selected for evidence collection, and evidence of multiple query paths is integrated to generate answer data including reasoning chain and evidence support.

[0319] Specifically, in S6, an intelligent agent based on a large language model (LLM) is configured, and the intelligent agent is used to perform advanced search and knowledge exploration in the knowledge graph, including:

[0320] Obtain a pre-trained large language model and fine-tune it based on the domain characteristics of the knowledge graph, so that it has the ability to understand professional terms and complex queries;

[0321] Build a query planner to break down complex problems into multiple subquery steps and determine the order in which queries should be executed;

[0322] Implement a multi-hop reasoning module based on the attention mechanism, which is used to perform multi-step reasoning in the knowledge graph and discover implicit relationships between entities;

[0323] A knowledge synthesis module is established to integrate the results of multiple query paths and generate a complete answer explanation.

[0324] In this embodiment, the specific implementation of the intelligent agent based on LLM includes:

[0325] 1. Domain Adaptation of Large Language Models:

[0326] This embodiment adopts the strategy of pre-training and fine-tuning to achieve domain adaptation of large language models:

[0327] (1) Pre-training stage:

[0328] - Select the base model: GPT-3.5 architecture with 175 billion parameters is used as the base model;

[0329] - Build domain corpus: collect a large amount of public text documents, policy interpretations, expert Q&A and other data;

[0330] - Design pre-training tasks: In addition to the regular Masked Language Model (MLM) task, the following special tasks are also added:

[0331] * Entity relationship prediction: predict the relationship between entities in a sentence;

[0332] * Explanation of professional terms: Explain professional terms according to the context;

[0333] * Policy clause association: determine the association between different policy clauses;

[0334] (2) Fine-tuning stage:

[0335] - Build domain-specific instruction datasets, including the following types:

[0336] * Entity query command: "Find all support policies related to 'technological innovation'"

[0337] * Relational reasoning instructions: "Analyze the possible impact of policy A on industry B"

[0338] * Conditional filtering command: "Find support policies that meet conditions X, Y, and Z"

[0339] - Fine-tuning using Instruction Learning:

[0340] * The learning rate is set to 3e-5;

[0341] * The batch size is 16;

[0342] * The number of training rounds is 3;

[0343] * Use cosine learning rate scheduler;

[0344] 2. Query planner implementation:

[0345] The query planner uses a hierarchical planning architecture, which includes:

[0346] (1) Problem decomposition layer:

[0347] Decomposing complex problems into sub-problems using attention-based sequence-to-sequence models;

[0348] Example breakdown:

[0349] Original question: "Find out the policies supporting the AI ​​industry in the past three years and analyze their impact on startups"

[0350] This breaks down into:

[0351] * Sub-question 1: "Find AI industry policies in the past three years"

[0352] * Sub-question 2: "Screening policy clauses related to start-up enterprises"

[0353] * Sub-question 3: "Analyze the impact of these policies on entrepreneurial enterprises"

[0354] (2) Query optimization layer:

[0355] Build a query dependency graph (QDG, Query Dependency Graph);

[0356] Use heuristic algorithms to determine the optimal query sequence;

[0357] Example query plan:

[0358] Plan {

[0359] step1: {

[0360] query: "MATCH (p:Policy) WHERE p.date>= '2020' AND p.industry = 'AI'RETURN p",

[0361] priority: 1,

[0362] dependencies: []

[0363] },

[0364] step2: {

[0365] query: "MATCH (p)-[:SUPPORTS]->(t:Target) WHERE t.type = 'startup'RETURN p, t",

[0366] priority: 2,

[0367] dependencies: ["step1"]

[0368] },

[0369] step3: {

[0370] query: "MATCH path = (p)-[:INFLUENCES]->(i:Impact)-[:AFFECTS]->(t)RETURN path",

[0371] priority: 3,

[0372] dependencies: ["step2"]

[0373] }

[0374] }

[0375] 3. Implementation of multi-hop reasoning module:

[0376] This embodiment implements a multi-hop reasoning engine based on the attention mechanism:

[0377] (1) Path exploration mechanism:

[0378] - Use Bi-Attention Network to calculate the association strength between entities;

[0379] - Attention calculation formula:

[0380] Attention(Q, K, V) = softmax(QK^T / √d)V

[0381] in:

[0382] Q: the representation of the current query entity;

[0383] K: representation of candidate entity;

[0384] V: value representation of the entity;

[0385] d: vector dimension;

[0386] (2) Reasoning chain generation:

[0387] Example scenario: Analyze the impact of scientific and technological innovation policies on the medical device industry

[0388] Example of inference chain:

[0389] Science and Technology Innovation Policy

[0390] → Provide R&D subsidies [Confidence: 0.92];

[0391] → Reduce the R&D costs of medical device companies [Confidence: 0.87];

[0392] → Promote product innovation [Confidence: 0.85];

[0393] → Improve the competitiveness of the medical device industry [confidence: 0.83];

[0394] 4. Implementation of the knowledge integration module

[0395] The knowledge synthesis module adopts a multi-level results integration mechanism:

[0396] (1) Evidence integration layer:

[0397] - Use evidence aggregator based on Graph Attention Network (GAT);

[0398] - Weight of evidence calculation:

[0399] Python

[0400] evidence_weight = GAT_layer(

[0401] node_features=evidence_embeddings,

[0402] edge_index=evidence_connections,

[0403] attention_weights=attention_scores )

[0405] (2) Answer generation layer:

[0406] - Use a hierarchical decoder to generate structured answers;

[0407] - Example of answer template:

[0408] json

[0409] {

[0410] "query_type": "policy_impact_analysis",

[0411] "target_industry": "Medical Devices",

[0412] "relevant_policies": [...],

[0413] "impact_chains": [...],

[0414] "evidence_summary": "...",

[0415] "confidence_score": 0.89

[0416] }

[0417] Practical application examples:

[0418] User Question:

[0419] "Analysis of the impact of the Next Generation Artificial Intelligence Development Plan on chip manufacturing companies, especially in terms of technological innovation"

[0420] Intelligent agent processing flow:

[0421] 1. Problem understanding and decomposition:

[0422] ```json

[0423] {

[0424] "main_policy": "New Generation Artificial Intelligence Development Plan",

[0425] "target_entity": "Chip manufacturing companies",

[0426] "focus_aspect": "Technology Innovation",

[0427] "query_type": "policy_impact_analysis"

[0428] }

[0429] 2. Query planning:

[0430] ```Python

[0431] query_plan = [

[0432] {"step": 1, "task": "locate_policy", "params": {"policy_name": "New generation artificial intelligence development plan"}},

[0433] {"step": 2, "task": "extract_relevant_clauses", "params": {"industry": "chip manufacturing"}},

[0434] {"step": 3, "task": "analyze_impact_chain", "params": {"aspect": "Technological innovation"}},

[0435] {"step": 4, "task": "synthesize_results"} ]

[0437] 3. Multi-hop reasoning execution:

[0438] Reasoning path 1: policy support → R&D investment → technological breakthrough;

[0439] Reasoning path 2: policy support → talent introduction → innovation capability;

[0440] Reasoning path 3: Policy support → Industry-university-research cooperation → Technology accumulation;

[0441] 4. Results summary:

[0442] ```json

[0443] {

[0444] "policy_analysis": {

[0445] "direct_impacts": [

[0446] {

[0447] "aspect": "R&D support",

[0448] "description": "Provide special funds for AI chip research and development",

[0449] "confidence": 0.95

[0450] },

[0451] {

[0452] "aspect": "Innovation incentives",

[0453] "description": "Establish a technology innovation reward mechanism",

[0454] "confidence": 0.88

[0455] }

[0456] ],

[0457] "indirect_impacts": [

[0458] {

[0459] "chain": "Industry-university-research cooperation → technological innovation → competitiveness improvement",

[0460] "confidence": 0.85

[0461] }

[0462] ],

[0463] "evidence_sources": [

[0464] "Policy Clause 3.2.1",

[0465] "Implementation Rules 5.3",

[0466] "Related Case Analysis" ]

[0468] }

[0469] }

[0470] Through the above technical implementation, the intelligent agent of this embodiment can handle complex policy analysis problems and provide in-depth insights and reliable evidence support. Its reasoning process is transparent and explainable, and the query results are comprehensive and accurate, which greatly enhances the practical value of the knowledge graph.

[0471] Specifically, the workflow of the intelligent agent includes:

[0472] First, receive the user's natural language questions and use the large language model to understand the question intent and constraints;

[0473] Then, the query planner generates a query plan based on the complexity of the problem. For simple problems, a single-step query is performed directly; for complex problems, it is decomposed into multiple sub-queries;

[0474] Then, multi-hop reasoning is performed. In each hop, the agent determines the next query direction based on the current query results and the question context. The attention mechanism is used to assign weights to different query paths and select the most relevant path to continue exploring;

[0475] Finally, the knowledge synthesis module integrates the results of multiple query paths, not only returning direct answers, but also providing reasoning paths and evidence chains to enhance the interpretability of answers.

[0476] The above process is described in detail as follows:

[0477] In the first stage of problem processing, the device first uses a large language model to perform deep semantic understanding of the natural language questions input by the user. This process uses a Transformer-based bidirectional encoder structure to capture long-distance dependencies in the problem through a multi-layer self-attention mechanism. The model performs multiple key tasks in the problem understanding stage: intent recognition classifies questions into different types such as query, analysis, and comparison; slot filling identifies key entities, attributes, and constraints in the problem; and time series understanding processes time qualifiers in the problem to determine the time range of the query. For example, when a user asks "analyze the impact of policies supporting the biopharmaceutical industry on start-ups in the past three years", the model can accurately identify key information such as time range (nearly three years), target industry (biopharmaceuticals), and policy objects (start-ups).

[0478] Based on the understanding of the problem, the query planner begins to build an execution plan. The planner adopts a hierarchical planning architecture, which includes two core levels: problem decomposition and query optimization. In the problem decomposition layer, a graph-based task decomposition algorithm is used to split complex problems into multiple subtasks with dependencies. Each subtask is assigned a priority and execution conditions. In the query optimization layer, the system builds a query dependency graph (QDG) and determines the optimal query execution sequence through a dynamic programming algorithm. For simple problems, such as a single policy query, the system will directly generate an optimized single-step query statement; for complex problems, such as policy impact analysis, the system will generate an execution plan containing multiple query steps and allocate appropriate execution resources to each step.

[0479] When performing multi-hop reasoning, the system implements a path exploration algorithm based on the attention mechanism. In each hop exploration, the system first calculates the attention weights between the current node and its neighboring nodes through the graph attention network (GAT). These weights reflect the importance and relevance of different paths. The calculation of attention weights takes into account multiple factors: the semantic similarity of nodes, the type importance of edges, the success rate of historical query paths, etc. The system uses a gated update mechanism (GRU) to maintain the query state and ensure the coherence of path exploration. In order to improve the efficiency of reasoning, the system also implements a pruning strategy to promptly terminate exploration paths with low confidence or off-topic.

[0480] Finally, the knowledge synthesis module generates the final answer through a multi-level integration mechanism. At the first level, the module uses a graph attention-based evidence aggregator to weightedly combine the evidence obtained from multiple query paths. The calculation of weights takes into account the reliability, relevance, and timeliness of the evidence. At the second level, a hierarchical decoder is used to generate structured answers. The decoder first generates the skeleton structure of the answer and then fills in the specific content. For example, for a policy impact analysis, the answer contains multiple parts such as a policy overview, impact path, and key evidence. The system also implements an adaptive answer generation strategy to adjust the level of detail and presentation of the answer according to the specific needs of the user. To enhance explainability, the system generates decision bases and displays the complete reasoning chain from question to answer, allowing users to understand how the system reaches its conclusions.

[0481] The intelligent agent has the following advantages:

[0482] Able to handle complex problems that require multi-step reasoning, such as "find all relevant industries affected by a policy";

[0483] Possess knowledge reasoning ability and can discover implicit relationships in knowledge graphs;

[0484] Provide explainable answers so users can understand how the answer was arrived at;

[0485] Supports interactive query and can adjust query strategies based on user feedback.

[0486] By introducing LLM-based intelligent agents, the present invention significantly improves the query capability and intelligence level of the knowledge graph, enabling it to better serve complex information needs.

[0487] In addition, in another embodiment, before extracting information from the target text data, the method further includes:

[0488] Get a pre-trained large language model and build a mixed validation set that includes a general ability test set and a domain ability test set;

[0489] Based on the model capability hierarchical hypothesis, the network structure of the large language model is divided into a basic language understanding layer, a general reasoning layer and a domain professional layer;

[0490] Differentiated tuning strategies are adopted for each layer of the network. By configuring different learning rates and regularization strengths, the bottom layer network maintains basic language understanding capabilities, the middle layer network maintains general reasoning capabilities, and the high layer network obtains domain processing capabilities.

[0491] During the fine-tuning process, the model performance is evaluated based on the mixed validation set, and the learning rate and regularization parameters of each layer are dynamically adjusted according to the evaluation results to generate a preprocessing model with both general and domain capabilities;

[0492] The preprocessing model is used to perform semantic understanding and structural processing on the target text data, so as to extract information from the target text data more quickly and efficiently.

[0493] Specifically, in the process of domain adaptation of the large language model, this embodiment innovatively proposes an adaptive stratification and element regularization method to solve the problem that the model may lose general capabilities when optimizing in a specific field. This method is based on the model capability stratification hypothesis and divides the model's capability hierarchy into three levels: basic language understanding layer, general reasoning layer, and domain expertise layer. On this basis, the device implements a refined tuning strategy to ensure that the model maintains its basic capabilities while acquiring domain expertise.

[0494] Specifically, in terms of adaptive stratification, this embodiment first determined the functional characteristics of different layers in the Transformer architecture through large-scale experimental analysis. The study found that the bottom layer is mainly responsible for the extraction of basic language features, the middle layer handles general logical reasoning, and the high layer focuses on the semantic understanding of specific tasks. Based on this finding, the device adopts a differentiated fine-tuning strategy: for the bottom layer (first layer) network, a smaller learning rate (such as 1e-5) is used and knowledge distillation is introduced to ensure the stability of basic language understanding capabilities; for the middle layer (second layer) network, sparse attention mechanism and gradient clipping are used to maintain general reasoning capabilities; for the high-level (third layer) network, a larger learning rate (such as 3e-5) is used and an additional task adaptation layer is introduced to enhance domain-specific capabilities.

[0495] In terms of element regularization, this embodiment innovatively introduces a multi-granularity regularization mechanism. At the parameter level, the device uses weight regularization of different strengths for different functional modules, and controls the degree to which the parameters deviate from the pre-trained values ​​through the L2 regularization term. The regularization strength is adapted to the hierarchical function, with stronger regularization (λ=0.1) used at the bottom layer and weaker regularization (λ=0.01) used at the upper layer. At the representation level, the device implements a contrastive learning framework, which achieves efficient injection of domain knowledge by maximizing the representation similarity of domain-related samples while maintaining the representation distance from general samples.

[0496] In order to dynamically balance general capabilities and domain expertise, this embodiment also designs an adaptive adjustment mechanism. During the training process, the device continuously monitors the performance of the model on general tasks and domain tasks, and ensures balanced performance improvement by dynamically adjusting the learning rate and regularization strength of each layer. Specifically, when it is found that the performance of the model on general tasks has declined, the system will automatically increase the regularization strength of the corresponding layer and slow down the parameter update speed; when the performance improvement of domain tasks stagnates, the regularization constraints of the high level will be reduced accordingly to accelerate the acquisition of professional capabilities.

[0497] In addition, the device also implements a verification feedback mechanism. During the fine-tuning process, the model performance is regularly evaluated on a mixed test set, which includes multiple types of tasks such as general question answering, logical reasoning, and professional consultation. Based on the evaluation results, the system dynamically adjusts the training strategy, including learning rate scheduling, batch size selection, and layer unfreezing order. This refined regulation ensures the balanced development of the model's capabilities, maintaining the ability to handle basic problems while gaining strong professional analysis capabilities.

[0498] Through the above technical innovations, this embodiment successfully achieved efficient domain tuning of large language models. Taking the policy and regulation field as an example, on the professional test set, the model demonstrated excellent policy analysis capabilities, with an accuracy increase of more than 30%; at the same time, on the general question-answering test set, the performance only dropped by less than 2%, effectively solving the "catastrophic forgetting" problem. This tuning method is not only applicable to policy consulting devices, but can also be extended to other professional fields, and has wide application value.

[0499] like Figure 4 As shown, the embodiment of the present application also provides a knowledge graph construction and intelligent question-answering device based on deep learning, and the knowledge graph construction and intelligent question-answering device based on deep learning 400 includes:

[0500] The text acquisition module 401 is used to acquire target text data;

[0501] A data extraction module 402 is used to extract information from the target text data to generate entity relationship data, wherein the entity relationship data includes key data, entity data, and relationship data indicating the relationship between entities;

[0502] A knowledge graph construction module 403 is used to import the target text data into a preset graph database according to the entity relationship data to form a knowledge graph;

[0503] The question acquisition module 404 is used to acquire the input question sentence;

[0504] A question feature extraction module 405 is used to classify the question sentence using a pre-established question classification model, and determine question features in the question sentence, wherein the question features include question type and question entity data;

[0505] The answer module 406 is used to retrieve text data matching the question features in the knowledge graph as answer data for the question statement.

[0506] The device of the embodiments of the present disclosure can execute the method provided by the embodiments of the present disclosure, and the implementation principles are similar. The actions performed by each module in the device of each embodiment of the present disclosure correspond to the steps in the method of each embodiment of the present disclosure. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0507] The above is only an optional implementation method for some implementation scenarios of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the scheme of the present disclosure, other similar implementation methods based on the technical ideas of the present disclosure are also within the protection scope of the embodiments of the present disclosure.

[0508] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A knowledge graph construction and intelligent question answering method based on deep learning, characterized in that: The following steps are involved: Get target text data; Extracting information from the target text data to generate entity relationship data, wherein the entity relationship data includes key data, entity data, and relationship data for indicating the relationship between entities; According to the entity relationship data, the target text data is imported into a preset graph database to form a knowledge graph; Get the input question statement; Using a pre-established question classification model to classify the question sentence, and determining question features in the question sentence, wherein the question features include question type and question entity data; Retrieving text data matching the question features in the knowledge graph as answer data for the question statement; The step of importing the target text data into a preset graph database according to the entity relationship data to form a knowledge graph includes: According to the node type and relationship type of the graph node, a first mask ratio is configured for a node whose node type meets a preset important node condition, and a second mask ratio is configured for a node whose node type meets a preset auxiliary node condition, so as to form mask data, wherein the first mask ratio is lower than the second mask ratio; In combination with a preset multi-task learning mechanism, the mask data is used to pre-train a pre-established encoder-decoder structure, wherein the encoder-decoder structure includes a third encoder and a decoder, the third encoder is used to instruct the conversion of the heterogeneous graph data indicated by the knowledge graph into a feature vector, and the decoder is used to instruct the reconstruction of the graph structure with the feature vector; Build a classifier based on the pre-trained encoder-decoder structure; Using the feature vector and a task feature vector determined by the current task requirement, the classifier is trained so that the knowledge graph is optimized using the trained classifier; In the process of extracting information from the target text data and generating entity relationship data, a multi-level information extraction architecture is adopted: at the first level, the key data of the text is extracted through a deep learning model; at the second level, the focus is on the identification and extraction of entity data; at the third level, the focus is on the extraction of relationship data; the entire information extraction process is a progressive process, and the output of each level serves as the input or auxiliary information of the next level; The second encoder is trained using the training data to obtain a relation extraction model. The second encoder adopts a BERT-based encoder structure and introduces the design: Entity location awareness mechanism: add entity location encoding in the input layer; Multi-task learning framework: simultaneous relationship classification and relationship validity judgment; Adversarial training strategy: improve the robustness of the model; Attention mechanism: focus on the contextual information between entities.

2. The method according to claim 1, characterized in that The step of obtaining target text data comprises: Get the text data to be processed; Parsing the text data to be processed and removing interference information in the text data to be processed to obtain cleaned text data to be processed, wherein the interference information includes HTML tags and special characters; Performing word segmentation processing on the cleaned text data to be processed, and removing stop words in the cleaned text data to be processed to form a text segmentation result; The word frequency statistics are performed on the text segmentation results to generate word vectors, so that the target text data can be obtained based on the word vectors.

3. The method according to claim 2, characterized in that The key data includes keywords and document summaries; extracting information from the target text data to generate entity relationship data includes: Constructing a word graph network based on the target text data, wherein a node in the word graph network represents a word in the target text data, an edge in the word graph network represents a semantic association between the words, and a weight of the edge represents a co-occurrence number between the words; Based on a preset damping coefficient, the TextRank value of each node in the word graph network is iteratively calculated until the TextRank value of the node converges and / or reaches a specified number of iterations, and the word importance score data of the word to which the node belongs is generated according to the TextRank value of the current node; Selecting words corresponding to when the word importance score data exceeds a preset word score threshold as keywords of the target text data; Dividing the target text data into a plurality of sentences, and determining sentence importance score data of the sentences based on word importance score data of the words in the sentences; The sentence corresponding to when the sentence importance score data exceeds a preset sentence score threshold is selected as the document summary of the target text data.

4. The method according to claim 3, characterized in that The extracting information from the target text data to generate entity relationship data also includes: Obtaining a training corpus with multiple layers of annotations, where the annotations are used to indicate entity categories; Using the training corpus to train and evaluate a preset first encoder to obtain an entity recognition model, wherein the entity recognition model includes a BERT model with a GlobalPointer layer; Using the entity recognition model to recognize the target text data, and obtain entity data indicating a plurality of entities; Based on the entity data, determining training data having a relationship identifier, wherein the relationship identifier is used to indicate a type of relationship between the entities; Using the training data to train a preset second encoder to obtain a relationship extraction model; The target text data and the entity data are input into the relationship extraction model to identify the association relationship between entities based on the entity position data in the entity data and the text sequence in the target text data, and output the relationship data.

5. The method according to claim 4, characterized in that The step of importing the target text data into a preset graph database according to the entity relationship data to form a knowledge graph includes: Based on a preset graph database, the device memory acting on the knowledge graph is divided into page cache and heap memory according to a specified ratio to form corresponding memory configuration data; For the entity attributes corresponding to when the query frequency data is higher than a preset query frequency threshold, an attribute index is constructed, and for the relationship types corresponding to when the relationship frequency data is higher than a preset relationship frequency threshold, a relationship index is constructed, so as to generate an index system by the attribute index and the relationship index; Generate a knowledge graph about the target text data according to the entity relationship data, the memory configuration data and the index system; Or, the method of importing the target text data into a preset graph database according to the entity relationship data to form a knowledge graph further includes: Based on the entity relationship data, a data processing model is constructed, wherein the data processing model includes a decision network and a representation network, wherein the decision network is used to indicate the domain type of the graph node in the knowledge graph, and the representation network is used to indicate that the output result of the decision network updates the embedding representation of the corresponding graph node, wherein each graph node represents an entity, and the edge in the knowledge graph represents the corresponding relationship data; Performing reinforcement learning on the data processing model to generate node importance data of the graph nodes; According to the node importance data of the graph node, the information importance score of the node information related to the graph node is calculated, and the node information whose information importance score is lower than the preset importance threshold is removed, and then a knowledge graph is generated.

6. The method according to claim 2, characterized in that The acquiring of target text data further includes: By using semantic decomposition, the text data to be processed is converted into a plurality of semantic units containing text information and formulation intentions; Performing multi-level information aggregation on the semantic units to generate multiple semantic combinations; According to the semantic combination, a plurality of views are constructed, each of the views being associated with a text feature of the text data to be processed, the text feature comprising an object feature, a time sequence feature and an action feature; The views are subjected to cross-view information fusion according to the text features to obtain semantic fusion data, so that the target text data is obtained based on the semantic fusion data.

7. The method according to claim 1, characterized in that The step of training the classifier by using the feature vector and a task feature vector determined by the current task requirement includes: Performing dimensionality reduction processing on the feature vector; The reduced-dimensional feature vector and the task feature vector are input into the classifier to calculate the associated weights between the reduced-dimensional feature vector and the task feature vector using a preset attention mechanism to form a feature weight matrix, and based on the feature weight matrix, the reduced-dimensional feature vector and the task feature vector are weightedly combined to obtain a trained classifier.

8. The method according to claim 1, characterized in that The retrieving text data matching the question feature in the knowledge graph includes: Obtain a pre-trained large language model and construct a training corpus based on the entity relationship data in the knowledge graph; Using the training corpus to fine-tune the large language model in a certain domain, thereby obtaining an intelligent agent model with knowledge graph query capability; Inputting the question statement into the intelligent agent model to generate a query plan including multiple sub-query steps; Based on the query plan, performing multi-hop path exploration in the knowledge graph, wherein weight data of different paths are calculated based on a graph attention mechanism in each hop exploration; According to the weight data, a query path with high confidence is selected for evidence collection, and evidence of multiple query paths is integrated to generate answer data including reasoning chain and evidence support.

9. The method according to claim 1, characterized in that: Before extracting information from the target text data, the method further includes: Get a pre-trained large language model and build a mixed validation set that includes a general ability test set and a domain ability test set; Based on the model capability hierarchical hypothesis, the network structure of the large language model is divided into a basic language understanding layer, a general reasoning layer and a domain professional layer; Differentiated tuning strategies are adopted for each layer of the network. By configuring different learning rates and regularization strengths, the bottom layer network maintains basic language understanding capabilities, the middle layer network maintains general reasoning capabilities, and the high layer network obtains domain processing capabilities. During the fine-tuning process, the model performance is evaluated based on the mixed validation set, and the learning rate and regularization parameters of each layer are dynamically adjusted according to the evaluation results to generate a preprocessing model with both general and domain capabilities; The preprocessing model is used to perform semantic understanding and structural processing on the target text data so as to extract information from the target text data.

10. A knowledge graph construction and intelligent question-answering device based on deep learning, characterized in that: include: A text acquisition module is used to acquire target text data; A data extraction module, used to extract information from the target text data to generate entity relationship data, wherein the entity relationship data includes key data, entity data, and relationship data indicating the relationship between entities; A knowledge graph construction module, used to import the target text data into a preset graph database according to the entity relationship data to form a knowledge graph; A question acquisition module, used to acquire the input question sentence; A question feature extraction module, used to classify the question sentence using a pre-established question classification model, and determine question features in the question sentence, wherein the question features include question type and question entity data; A reply module, used to retrieve text data matching the question features in the knowledge graph as answer data for the question statement; The knowledge graph construction module is used to import the target text data into a preset graph database according to the entity relationship data to form a knowledge graph, including: According to the node type and relationship type of the graph node, a first mask ratio is configured for a node whose node type meets a preset important node condition, and a second mask ratio is configured for a node whose node type meets a preset auxiliary node condition, so as to form mask data, wherein the first mask ratio is lower than the second mask ratio; In combination with a preset multi-task learning mechanism, the mask data is used to pre-train a pre-established encoder-decoder structure, wherein the encoder-decoder structure includes a third encoder and a decoder, the third encoder is used to instruct the conversion of the heterogeneous graph data indicated by the knowledge graph into a feature vector, and the decoder is used to instruct the reconstruction of the graph structure with the feature vector; Build a classifier based on the pre-trained encoder-decoder structure; Using the feature vector and a task feature vector determined by the current task requirement, the classifier is trained so that the knowledge graph is optimized using the trained classifier; In the process of extracting information from the target text data and generating entity relationship data, a multi-level information extraction architecture is adopted: at the first level, the key data of the text is extracted through a deep learning model; at the second level, the focus is on the identification and extraction of entity data; at the third level, the focus is on the extraction of relationship data; the entire information extraction process is a progressive process, and the output of each level serves as the input or auxiliary information of the next level; The second encoder is trained using the training data to obtain a relation extraction model. The second encoder adopts a BERT-based encoder structure and introduces the design: Entity location awareness mechanism: add entity location encoding in the input layer; Multi-task learning framework: simultaneous relationship classification and relationship validity judgment; Adversarial training strategy: improve the robustness of the model; Attention mechanism: focus on the contextual information between entities.

Images