Vertical government affairs big model service method and system based on knowledge retrieval

Abstract

The present invention provides a vertical government affairs big model service method and system based on knowledge retrieval, which relates to the field of big model technology, including performing word segmentation and semantic understanding on the natural language query through a natural language processing model, extracting the query intent, and determining the vertical field to which the natural language query belongs according to the query intent; using an attention mechanism to deeply match the query semantic representation corresponding to the query intent with the knowledge node, and obtaining the top‑K knowledge nodes most relevant to the query semantic representation through a jumping knowledge graph reasoning method; starting from each most relevant knowledge node, a credibility propagation network is constructed in combination with the vertical knowledge graph, and by setting the initial credibility of the most relevant knowledge node, the credibility of each most relevant knowledge node in the propagation process of the credibility propagation network is iteratively calculated, and the most relevant knowledge nodes whose credibility scores exceed a preset credibility threshold are summarized as target reply content.

Description

Technical Field

[0001] The present invention relates to big model technology, and in particular to a vertical government affairs big model service method and system based on knowledge retrieval. Background Art

[0002] The traditional government service model has problems such as single information acquisition channels, rigid service methods, and poor user experience, which makes it difficult to meet the growing personalized and diversified needs. In order to improve the level of government services, it is urgent to introduce advanced artificial intelligence technology, establish an efficient, accurate, and intelligent government question-and-answer system, and provide the public with convenient and high-quality government information query services.

[0003] At present, common question-answering systems mainly use methods based on keyword matching or shallow semantic parsing, which retrieve keywords in questions and search for relevant answers in predefined knowledge bases. Although such methods are simple to implement, they have many limitations: on the one hand, keyword matching lacks a deep understanding of the semantics of the question and is difficult to handle complex long-tail questions; on the other hand, the construction and maintenance costs of predefined knowledge bases are high, the coverage is limited, and they cannot cope with diverse queries in open domains. In addition, the answers generated by traditional methods are usually simple factual statements, lacking contextual association and logical reasoning capabilities, and are difficult to meet users' needs for completeness and explainability of answers. Summary of the invention

[0004] The embodiments of the present invention provide a vertical government affairs big model service method and system based on knowledge retrieval, which can solve the problems in the prior art.

[0005] According to a first aspect of the embodiments of the present invention,

[0006] Provides a vertical government affairs big model service method based on knowledge retrieval, including:

[0007] Obtain a natural language query in the government affairs field to be processed, perform word segmentation and semantic understanding on the natural language query through a natural language processing model that combines an attention mechanism with a bidirectional long short-term memory network, extract the query intent, and determine the vertical field to which the natural language query belongs based on the query intent;

[0008] Retrieve knowledge nodes related to the query intent from the pre-constructed vertical knowledge graph corresponding to the vertical field, use the attention mechanism to deeply match the query semantic representation corresponding to the query intent with the knowledge nodes, and obtain the top-K knowledge nodes most relevant to the query semantic representation through a jumping knowledge graph reasoning method;

[0009] Multiple most relevant knowledge nodes are fused and represented through a graph attention network, and the pre-trained government affairs model corresponding to the vertical field is input. Starting from each most relevant knowledge node, a credibility propagation network is constructed in combination with the vertical knowledge graph. By setting the initial credibility of the most relevant knowledge node, the credibility of each most relevant knowledge node in the propagation process of the credibility propagation network is iteratively calculated. After the credibility propagation network converges, the credibility score of each most relevant knowledge node is obtained, and the most relevant knowledge nodes whose credibility scores exceed the preset credibility threshold are summarized as the target reply content.

[0010] In an optional embodiment,

[0011] Obtain a natural language query in the government affairs field to be processed, perform word segmentation and semantic understanding on the natural language query through a natural language processing model that combines an attention mechanism with a bidirectional long short-term memory network, extract the query intent, and determine the vertical field to which the natural language query belongs based on the query intent, including:

[0012] The natural language query is segmented by a natural language processing model, the natural language query is encoded by a bidirectional long short-term memory network to obtain context information, and the context information is dynamically aggregated by a multi-head attention mechanism to generate a query semantic vector;

[0013] A word joint embedding mechanism is introduced to represent the query semantic vector as a word joint vector concatenated by a character-level embedding vector and a word-level embedding vector. The character-level embedding vector is obtained by fusing the embeddings of each character of the query semantic vector through a feedforward neural network, and the word-level embedding vector is directly obtained through a word embedding matrix.

[0014] A multi-task semantic understanding model is constructed. The word joint vector is used as input. The word joint vector is learned through the encoding layer of the shared bidirectional long short-term memory network. Three tasks are set in parallel: the intent recognition task, which classifies the intent category corresponding to the word joint vector through the attention mechanism and the fully connected network; the slot filling task, which uses the sequence labeling model to extract the key information slots in the word joint vector; the named entity recognition task, which uses the sequence labeling model to identify the entity name and its type in the word joint vector;

[0015] According to the intent recognition result of the intent recognition task, the confidence corresponding to the word joint vector is determined, and according to the corresponding relationship between the confidence and the government affairs vertical field, the confidence corresponding to the word joint vector is mapped to the vertical field to which it belongs;

[0016] For the confidence corresponding to the word joint vector that is lower than the preset confidence threshold, or the confidence that has a one-to-many mapping relationship, the vertical field corresponding to the confidence corresponding to the word joint vector is determined in combination with the results of slot filling and named entity recognition.

[0017] In an optional embodiment,

[0018] Retrieve knowledge nodes related to the query intent from the pre-constructed vertical knowledge graph corresponding to the vertical field, use the attention mechanism to deeply match the query semantic representation corresponding to the query intent with the knowledge nodes, and obtain the top-K knowledge nodes most relevant to the query semantic representation through the jumping knowledge graph reasoning method, including:

[0019] The attention mechanism is used to achieve deep matching between query intent and vertical knowledge graph. The query intent and knowledge nodes in the vertical knowledge graph are spliced ​​as the input of the attention mechanism, and the attention score of the query intent and each knowledge node is calculated through a multi-layer perceptron.

[0020] Perform leap-forward knowledge graph reasoning, starting from the knowledge node corresponding to the query intent, explore on the vertical knowledge graph by combining random walk and attention mechanism, adaptively select the walk path according to the attention score of each knowledge node, and generate multiple reasoning paths corresponding to the query intent;

[0021] Calculate the relevance score of each reasoning path to the query intent, sort the reasoning paths, and select the end nodes of the top-K reasoning paths with the highest relevance scores as the knowledge nodes most relevant to the query intent.

[0022] In an optional embodiment,

[0023] The method further comprises:

[0024] Vectorize the query intent and the knowledge nodes in the vertical knowledge graph, maximize the similarity between the query intent and the relevant knowledge nodes through a triple loss function, minimize the similarity between the query intent and the irrelevant knowledge nodes, and update the vectorized representation of the query intent and the knowledge nodes in the vertical knowledge graph through a gradient descent algorithm;

[0025] The triple loss function is shown in the following formula:

[0026] ;

[0027] in, L represents the triplet loss, D represents the training data set, Indicates query intent QWith positive example node n p and negative nodes n n The triplet composed of u Q The vector representation corresponding to the query intent, e p It represents the vector representation corresponding to the knowledge node in the vertical knowledge graph, and △ represents the preset interval threshold.

[0028] In an optional embodiment,

[0029] The most relevant knowledge nodes are fused and represented through the graph attention network, and the pre-trained government affairs model corresponding to the vertical field is input. Starting from each most relevant knowledge node, the credibility propagation network is constructed in combination with the vertical knowledge graph. By setting the initial credibility of the most relevant knowledge node, the propagation process in the credibility propagation network is iteratively calculated. After convergence, the credibility scores of each most relevant knowledge node are obtained, including:

[0030] The top-K knowledge nodes most relevant to the query intent are obtained through leap-forward knowledge graph reasoning, and the graph attention network is used to fuse the information of the most relevant top-K knowledge nodes to obtain a comprehensive top-K knowledge node fusion representation;

[0031] According to the comprehensive top-K knowledge nodes fusion representation of the link relationship in the vertical knowledge graph, a credibility propagation network is constructed, the initial credibility of the most relevant knowledge nodes is set, and the credibility score of each knowledge node is iteratively updated. The credibility propagation of neighboring knowledge nodes and the initial credibility of the knowledge nodes are comprehensively considered until the credibility propagation network reaches a steady state, and the credibility scores of the top-K knowledge nodes after convergence are obtained.

[0032] In an optional embodiment,

[0033] Update the credibility score of each node in the credibility propagation network in an iterative manner:

[0034] ;

[0035] in, Indicates t+1 Knowledge nodes during round iteration n i The credibility of the neighbor node is represented by λ, which is the balancing factor and controls the weight of the credibility of the neighbor node and the initial credibility. N i Represents knowledge nodes n i The set of neighbor nodes of Nj Represents knowledge nodes n j The set of neighbor nodes of w ji Represents knowledge nodes n j arrive n i The edge weights of w jk Represents knowledge nodes n j arrive n k The edge weights of Indicates t Knowledge nodes during round iteration n j credibility, y i Represents knowledge nodes n i initial credibility.

[0036] In the second aspect of the embodiments of the present application,

[0037] Provides a vertical government affairs big model service system based on knowledge retrieval, including:

[0038] The first unit is used to obtain a natural language query in the government affairs field to be processed, perform word segmentation and semantic understanding on the natural language query through a natural language processing model combining an attention mechanism and a bidirectional long short-term memory network, extract the query intent, and determine the vertical field to which the natural language query belongs according to the query intent;

[0039] The second unit is used to retrieve knowledge nodes related to the query intent from the pre-constructed vertical knowledge graph corresponding to the vertical field, use the attention mechanism to deeply match the query semantic representation corresponding to the query intent with the knowledge nodes, and obtain the top-K knowledge nodes most relevant to the query semantic representation through the jumping knowledge graph reasoning method;

[0040] The third unit is used to fuse and represent multiple most relevant knowledge nodes through a graph attention network, input the pre-trained government affairs model corresponding to the vertical field, start from each most relevant knowledge node, and build a credibility propagation network in combination with the vertical knowledge graph. By setting the initial credibility of the most relevant knowledge node, iteratively calculate the credibility of each most relevant knowledge node in the propagation process of the credibility propagation network, and obtain the credibility score of each most relevant knowledge node after the credibility propagation network converges, and summarize the most relevant knowledge nodes whose credibility scores exceed the preset credibility threshold as the target reply content.

[0041] According to a third aspect of the embodiments of the present invention,

[0042] An electronic device is provided, comprising:

[0043] processor;

[0044] a memory for storing processor-executable instructions;

[0045] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0046] According to a fourth aspect of the embodiments of the present invention,

[0047] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0048] This application uses a natural language processing model that combines an attention mechanism with a bidirectional long short-term memory network to accurately understand the semantics and extract the intent of user queries, ensuring that the question-answering system accurately grasps the user's real needs. At the same time, a jumping knowledge graph reasoning method is adopted to flexibly select the knowledge nodes most relevant to the query intent, ensure the high relevance of the retrieval results, and effectively improve the quality of questions and answers. A graph attention network is introduced to semantically fuse multiple related knowledge nodes, and the neighbor information of the nodes is adaptively aggregated through the attention mechanism. While retaining the graph structure information, high-quality node embedding representations are learned. The fused knowledge representation is input into the pre-trained vertical field government affairs model to generate natural language answers with complete content and logical coherence, thereby improving the user experience.

[0049] A collaborative credibility propagation network is designed to conduct credibility assessment on relevant knowledge nodes, comprehensively considering the initial credibility of the nodes and their propagation influence in the knowledge graph, and obtaining the credibility score of each node through iterative calculation. Nodes with high credibility are prioritized in the answer generation process to ensure the authority and reliability of the output results and improve users' trust in the answers. A jumping knowledge graph reasoning method is adopted to flexibly select different reasoning paths and strategies according to the query intent, avoid searching for redundant and irrelevant nodes, quickly lock highly relevant knowledge nodes, and greatly reduce the computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flowchart of a vertical government affairs big model service method based on knowledge retrieval in an embodiment of the present invention;

[0051] Figure 2 This is a structural diagram of the vertical government affairs big model service system based on knowledge retrieval in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0054] Figure 1 FIG. 1 is a flow chart of a vertical government affairs big model service method based on knowledge retrieval according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0055] S101. Obtain a natural language query in the government affairs field to be processed, perform word segmentation and semantic understanding on the natural language query through a natural language processing model that combines an attention mechanism and a bidirectional long short-term memory network, extract the query intent, and determine the vertical field to which the natural language query belongs based on the query intent;

[0056] S102. Retrieve knowledge nodes related to the query intent from the pre-constructed vertical knowledge graph corresponding to the vertical field, use the attention mechanism to deeply match the query semantic representation corresponding to the query intent with the knowledge nodes, and obtain the top-K knowledge nodes most relevant to the query semantic representation through a jumping knowledge graph reasoning method;

[0057] S103. Multiple most relevant knowledge nodes are fused and represented through a graph attention network, and the pre-trained government affairs model corresponding to the vertical field is input. Starting from each most relevant knowledge node, a credibility propagation network is constructed in combination with the vertical knowledge graph. By setting the initial credibility of the most relevant knowledge node, the credibility of each most relevant knowledge node in the propagation process of the credibility propagation network is iteratively calculated. After the credibility propagation network converges, the credibility score of each most relevant knowledge node is obtained, and the most relevant knowledge nodes whose credibility scores exceed the preset credibility threshold are summarized as the target reply content.

[0058] This application uses a natural language processing model that combines an attention mechanism with a bidirectional long short-term memory network to accurately understand the semantics and extract the intent of user queries, ensuring that the question-answering system accurately grasps the user's real needs. At the same time, a jumping knowledge graph reasoning method is adopted to flexibly select the knowledge nodes most relevant to the query intent, ensure the high relevance of the retrieval results, and effectively improve the quality of questions and answers. A graph attention network is introduced to semantically fuse multiple related knowledge nodes, and the neighbor information of the nodes is adaptively aggregated through the attention mechanism. While retaining the graph structure information, high-quality node embedding representations are learned. The fused knowledge representation is input into the pre-trained vertical field government affairs model to generate natural language answers with complete content and logical coherence, thereby improving the user experience.

[0059] A collaborative credibility propagation network is designed to conduct credibility assessment on relevant knowledge nodes, comprehensively considering the initial credibility of the nodes and their propagation influence in the knowledge graph, and obtaining the credibility score of each node through iterative calculation. Nodes with high credibility are prioritized in the answer generation process to ensure the authority and reliability of the output results and improve users' trust in the answers. A jumping knowledge graph reasoning method is adopted to flexibly select different reasoning paths and strategies according to the query intent, avoid searching for redundant and irrelevant nodes, quickly lock highly relevant knowledge nodes, and greatly reduce the computational complexity.

[0060] In an optional embodiment,

[0061] Obtain a natural language query in the government affairs field to be processed, perform word segmentation and semantic understanding on the natural language query through a natural language processing model that combines an attention mechanism with a bidirectional long short-term memory network, extract the query intent, and determine the vertical field to which the natural language query belongs based on the query intent, including:

[0062] The natural language query is segmented by a natural language processing model, the natural language query is encoded by a bidirectional long short-term memory network to obtain context information, and the context information is dynamically aggregated by a multi-head attention mechanism to generate a query semantic vector;

[0063] A word joint embedding mechanism is introduced to represent the query semantic vector as a word joint vector concatenated by a character-level embedding vector and a word-level embedding vector. The character-level embedding vector is obtained by fusing the embeddings of each character of the query semantic vector through a feedforward neural network, and the word-level embedding vector is directly obtained through a word embedding matrix.

[0064] A multi-task semantic understanding model is constructed. The word joint vector is used as input. The word joint vector is learned through the encoding layer of the shared bidirectional long short-term memory network. Three tasks are set in parallel: the intent recognition task, which classifies the intent category corresponding to the word joint vector through the attention mechanism and the fully connected network; the slot filling task, which uses the sequence labeling model to extract the key information slots in the word joint vector; the named entity recognition task, which uses the sequence labeling model to identify the entity name and its type in the word joint vector;

[0065] According to the intent recognition result of the intent recognition task, the confidence corresponding to the word joint vector is determined, and according to the corresponding relationship between the confidence and the government affairs vertical field, the confidence corresponding to the word joint vector is mapped to the vertical field to which it belongs;

[0066] For the confidence corresponding to the word joint vector that is lower than the preset confidence threshold, or the confidence that has a one-to-many mapping relationship, the vertical field corresponding to the confidence corresponding to the word joint vector is determined in combination with the results of slot filling and named entity recognition.

[0067] Exemplarily, this application proposes a natural language processing model based on the combination of attention mechanism and bidirectional long short-term memory network, which is used to segment and understand the semantics of natural language queries in the government field, extract the query intent, and determine the vertical field to which the query belongs based on the query intent. The core of this method is to achieve in-depth understanding of query semantics and precise division of vertical fields through the word joint embedding mechanism and multi-task semantic understanding model.

[0068] First, the natural language query is segmented and the query text is divided into several independent semantic units. Then, the bidirectional long short-term memory network (Bi-LSTM) is used to encode the query text after segmentation to obtain context information. Specifically, Bi-LSTM consists of a forward LSTM and a backward LSTM. The forward LSTM starts from the starting position of the query text and processes each word in sequence to obtain a forward hidden state sequence; the backward LSTM starts from the end position of the query text and processes each word in reverse order to obtain a reverse hidden state sequence. By concatenating the forward and reverse hidden states, the context representation of each word is obtained.

[0069] After obtaining the contextual representation of the query text, the multi-head attention mechanism is introduced to dynamically aggregate the contextual information and generate a query semantic vector. The multi-head attention mechanism captures the semantic information of the query text from different subspaces by executing multiple attention functions in parallel, thereby enhancing the expressiveness of the model. Specifically, for each attention head, the attention function is used to calculate the correlation weight between each word and other words in the query text, and then the contextual representation of the word is weighted and summed according to the weight to obtain the query semantic vector under the attention head. Finally, the query semantic vectors obtained by all attention heads are concatenated to form the final query semantic vector.

[0070] In order to further enhance the representation ability of the query semantic vector, a word joint embedding mechanism is introduced to represent the query semantic vector as the concatenation of the character-level embedding vector and the word-level embedding vector.

[0071] For the character-level embedding vector, the embeddings of each character in the query semantic vector are fused through a feedforward neural network. Specifically, each character in the query semantic vector is mapped to a character embedding vector of fixed dimension through a character embedding matrix, and then all character embedding vectors are nonlinearly transformed and combined using a feedforward neural network to obtain a character-level query semantic representation.

[0072] For word-level embedding vectors, they are directly obtained through the pre-trained word embedding matrix. Each word in the query semantic vector is mapped to the corresponding word vector as the query semantic representation at the word level. Finally, the character-level embedding vector and the word-level embedding vector are concatenated to form a word-joint vector as a unified representation of the query semantics. The word-joint embedding mechanism integrates the semantic information of characters and vocabulary, providing a more comprehensive and fine-grained query understanding capability.

[0073] After obtaining the word joint vector representation of the query semantics, a multi-task semantic understanding model is constructed to achieve joint learning of intent recognition, slot filling, and named entity recognition. Multi-task learning improves the generalization ability and robustness of the model by sharing the underlying feature representation and utilizing the correlation between different tasks.

[0074] The multi-task semantic understanding model takes word joint vectors as input and learns contextual representations of word joint vectors through the encoding layer of a shared bidirectional long short-term memory network. Above the encoding layer, three tasks are set in parallel: intent recognition task, slot filling task, and named entity recognition task.

[0075] For the intent recognition task, the intent categories corresponding to the word joint vectors are classified through the attention mechanism and the fully connected network. First, the attention mechanism is used to calculate the correlation weight between each position in the word joint vector and the intent category, and then the output of the encoding layer is weighted summed according to the weight to obtain the intent representation vector. Finally, the intent representation vector is classified through the fully connected network and the Softmax function to obtain the probability distribution of the intent category to which the query belongs.

[0076] For the slot filling task, a sequence labeling model is used to extract key information slots in the word joint vector. The output of the encoding layer is learned through a bidirectional LSTM network to obtain the hidden state representation of each position. Then, the conditional random field (CRF) is used to label the hidden state sequence and predict the slot label corresponding to each position to achieve the recognition and extraction of key information slots.

[0077] For the named entity recognition task, the sequence labeling model is also used to identify the entity name and type in the word joint vector. Similar to the slot filling task, the output of the encoding layer is learned using a bidirectional LSTM network to obtain the hidden state representation of each position. Then, CRF is used to label the hidden state sequence and predict the entity label corresponding to each position to achieve entity name and type recognition.

[0078] By sharing the encoding layer and setting three tasks in parallel, the multi-task semantic understanding model can make full use of the complementary information between different tasks and improve the accuracy and completeness of query semantic understanding.

[0079] Based on the results of the intent recognition task, the confidence level corresponding to the word joint vector is determined, and based on the correspondence between the confidence level and the government vertical field, the word joint vector is mapped to the vertical field to which it belongs.

[0080] Specifically, the intent recognition task outputs the probability distribution of the intent category to which the query belongs, selects the intent with the highest probability as the final intent of the query, and uses its probability value as the confidence. The mapping relationship between the government vertical field and the intent category is pre-built, and the vertical field to which the query belongs is determined based on the final intent of the query.

[0081] For queries with confidence values ​​lower than the preset threshold or with a one-to-many mapping relationship, the results of slot filling and named entity recognition are combined to further determine the vertical field to which they belong. The key information slots extracted by the slot filling task and the entity names and types identified by the named entity recognition task are matched with the knowledge base of the government vertical field to find the most relevant vertical field as the final destination of the query.

[0082] In summary, this application proposes a query semantic understanding method based on the combination of attention mechanism and bidirectional long short-term memory network, which realizes accurate understanding and vertical field division of natural language queries in the government field through word joint embedding mechanism and multi-task semantic understanding model. This method integrates character-level and word-level semantic information, uses attention mechanism to dynamically aggregate context information, and improves the accuracy and completeness of semantic understanding through multi-task learning. At the same time, according to the intent recognition results and confidence, combined with the results of slot filling and named entity recognition, the vertical field to which the query belongs is determined, providing important prior information for subsequent knowledge retrieval and question-answer generation.

[0083] In order to quickly and accurately retrieve knowledge nodes related to query intent from massive knowledge graphs, this application proposes a jumping knowledge graph reasoning method. This method uses the attention mechanism to deeply match the query semantic representation with the knowledge nodes, and through a flexible reasoning strategy, obtains the top-K knowledge nodes most relevant to the query, providing high-quality knowledge support for subsequent knowledge fusion and answer generation.

[0084] First, for different government verticals, corresponding domain knowledge graphs are pre-built. The knowledge graph represents entities, their attributes and relationships in a structured form, providing rich background knowledge for intelligent question and answer. Through technologies such as knowledge extraction, knowledge fusion, and ontology construction, entities, attributes and relationships are extracted from authoritative sources such as government websites, public documents, and service guides to form a high-quality, wide-coverage domain knowledge base.

[0085] In the process of knowledge graph construction, ontology construction method is used to define the core concepts, relationship types and concept hierarchical structure of the domain to ensure the standardization and consistency of knowledge organization. At the same time, knowledge fusion technology is used to clean, align and deduplicate multi-source heterogeneous data to improve the quality and reliability of knowledge graph.

[0086] After obtaining the query intent representation in the query semantic understanding stage, the attention mechanism is used to deeply match the query semantic representation with the nodes in the knowledge graph, and the correlation score between the query and each node is calculated.

[0087] Specifically, for each knowledge node, it is mapped into a node embedding vector of fixed dimension through the node embedding matrix. Then, the attention mechanism is used to calculate the attention weight between the query semantic representation and the node embedding vector to characterize the semantic relevance between the query and the node. The node embedding vector is weighted and summed according to the attention weight to obtain the query-related node representation vector.

[0088] Finally, a metric function such as cosine similarity or dot product is used to calculate the correlation score between the query semantic representation and each node representation vector to obtain the matching results between the query and all nodes.

[0089] In order to quickly locate the knowledge nodes most relevant to the query in the massive knowledge graph, a leap reasoning strategy is used for knowledge retrieval. The leap reasoning strategy flexibly selects the next search direction and jump strategy based on the query intent and the relevance of the current node, avoiding exhaustive search in the knowledge graph and improving retrieval efficiency.

[0090] Specifically, starting from the node with the highest match to the query, the adjacent nodes are selectively expanded for exploration according to the predefined jump rules. The jump rules take into account factors such as the type of relationship between nodes, the importance of nodes, and the relevance to the query, and dynamically adjust the breadth and depth of the search. For example, for nodes that are highly relevant to the query, they are preferentially expanded along their key attributes and core relationships; for nodes with weaker relevance, the search depth is reduced to avoid over-exploration.

[0091] In the process of leap reasoning, the termination condition of the search is controlled by setting the relevance threshold and the jump number limit. When the relevance of the explored node to the query is lower than the threshold or the jump number exceeds the limit, the search of the current branch is stopped and the previous node is traced back to continue exploring other branches.

[0092] Through the jump reasoning strategy, a set of knowledge nodes with the highest relevance to the query is obtained. In order to balance the breadth of knowledge coverage and the efficiency of answer generation, the top-K nodes are selected from the reasoning results as the most relevant knowledge nodes.

[0093] Specifically, the inferred knowledge nodes are sorted according to the correlation score between the query and the node, and the K nodes with the highest scores are selected as the top-K knowledge nodes. These nodes are closely related to the query intent and contain the core knowledge required to generate answers, providing important support for subsequent knowledge fusion and answer generation.

[0094] At the same time, in order to improve the diversity and coverage of top-K knowledge nodes, a diversity rearrangement strategy is used to adjust the candidate nodes. On the basis of ensuring relevance, node combinations with complementary attributes and diverse relationships are selected as much as possible to reduce the introduction of redundant knowledge and improve the efficiency of knowledge utilization.

[0095] In summary, this application proposes a jumping knowledge graph reasoning method, which achieves deep matching between queries and knowledge nodes through the attention mechanism, uses a flexible jumping reasoning strategy to quickly locate relevant nodes, and selects the top-K most relevant knowledge nodes to support subsequent knowledge fusion and answer generation. This method overcomes the limitations of traditional knowledge retrieval methods, significantly improves retrieval efficiency while ensuring retrieval accuracy, and lays the foundation for building a real-time, high-quality intelligent question-answering system.

[0096] After obtaining the top-K knowledge nodes related to the query, it is necessary to further semantically fuse these nodes to form a knowledge-enhanced representation of the query. This application proposes a knowledge fusion method based on a graph attention network, which adaptively aggregates node neighbor information through an attention mechanism, learns high-quality node embedding representations while retaining the knowledge graph structure information, and realizes effective knowledge fusion.

[0097] First, a local knowledge subgraph is constructed with the selected top-K knowledge nodes as the core. By traversing the neighbor relationships of these nodes in the original knowledge graph, directly connected entity nodes and relationship edges are included in the subgraph to form a local knowledge structure centered on the top-K nodes. Then, based on the local knowledge subgraph, the Graph Attention Network (GAT) is introduced to perform representation learning and feature fusion on knowledge nodes. The Graph Attention Network adaptively aggregates the neighbor information of nodes through the attention mechanism, while retaining the topological structure of the knowledge graph, and can learn high-quality node embedding representations.

[0098] Specifically, for each node in the subgraph, a node embedding matrix is ​​used to map it into a node feature vector of fixed dimension. Then, the attention weight between the node and its neighboring nodes is calculated through the attention mechanism to characterize the semantic relevance and importance between the nodes. The calculation of the attention weight takes into account the node features, neighboring node features, and semantic information of the connecting edges, and is parameterized by multi-layer perceptron or bilinear function.

[0099] The feature vectors of neighbor nodes are weighted and summed according to the attention weights to obtain the node's neighbor aggregation features. At the same time, in order to retain the node's own feature information, the node features are concatenated or summed with the neighbor aggregation features to form an updated feature representation of the node. Through multiple rounds of graph attention aggregation and feature updates, the representation of the node can integrate the semantic information of multi-hop neighbors and capture the structured knowledge in the knowledge graph. Finally, a high-quality embedded representation of each knowledge node is obtained, which contains the node's own semantic features and contextual structure information.

[0100] After obtaining the graph attention embedding representation of the knowledge node, it is fused with the query semantic representation to generate the knowledge-enhanced representation of the query. The knowledge-enhanced representation integrates the query intent and the semantic information of the relevant knowledge nodes, providing rich knowledge support for subsequent answer generation.

[0101] Specifically, for each top-K knowledge node, its graph attention embedding representation is concatenated or attention fused with the query semantic representation to obtain a node-level knowledge enhancement representation. Attention fusion calculates the semantic relevance between the query and the node through the attention mechanism, dynamically adjusts the weight of the node representation in the query representation, and highlights the knowledge information closely related to the query.

[0102] Then, the knowledge enhancement representations of all top-K nodes are aggregated to generate the overall knowledge enhancement representation of the query. The aggregation method can adopt simple weighted summation or attention pooling methods to adaptively integrate the knowledge information of different nodes to form the knowledge context representation of the query.

[0103] Finally, the knowledge-enhanced representation of the query is used as the input of the subsequent answer generation module to provide a semantic basis for generating knowledge-driven natural language answers. The knowledge-enhanced representation integrates query intent and structured knowledge, contains key information required to generate answers, and helps improve the accuracy, completeness, and interpretability of answers.

[0104] In summary, this application proposes a knowledge fusion method based on graph attention network, which adaptively aggregates the neighbor information of knowledge nodes through the attention mechanism and learns high-quality node embedding representation. On this basis, the knowledge node representation is fused with the query semantic representation to generate a knowledge-enhanced representation of the query, providing rich knowledge support for answer generation. This method makes full use of the structured information of the knowledge graph, while taking into account the query intent and node semantics, achieving a deep fusion of query and knowledge, and improving the knowledge understanding and expression capabilities of the intelligent question-answering system.

[0105] In order to ensure the credibility and authority of the generated answers, this application proposes an answer generation method based on knowledge credibility evaluation. This method introduces a knowledge credibility evaluation mechanism in the answer generation process, comprehensively considers the confidence, semantic relevance and structural importance of knowledge nodes in the knowledge graph, and gives priority to knowledge nodes with high credibility to participate in answer generation, thereby improving the reliability of answers and user trust.

[0106] First, the credibility of the top-K knowledge nodes is evaluated and the credibility score of each node is calculated. The credibility evaluation takes into account multiple factors, including the initial confidence of the node, the semantic relevance to the query, and the structural importance of the node in the knowledge graph.

[0107] For the initial confidence of the node, an initial credibility score is assigned to each node by analyzing the source reliability, extraction confidence, and conflict check of the knowledge node. Source reliability is evaluated based on the extraction source of the knowledge node, and higher confidence is assigned to authoritative and reliable sources; extraction confidence is estimated based on the confidence in the knowledge extraction process, and higher credibility is assigned to nodes with high confidence; conflict check is performed by analyzing the logical conflicts and numerical conflicts between nodes, and higher credibility is assigned to nodes without conflicts or conflict resolution.

[0108] Then, the semantic relevance between the node and the query is considered, and nodes that are more relevant to the query intent are given higher credibility. Semantic relevance can be measured by the attention weight or similarity score calculated in the knowledge fusion stage, which reflects the importance of the node for answer generation.

[0109] In addition, the structural importance of nodes in the knowledge graph is also considered, and nodes located in core positions and connecting important entities are given higher credibility. Structural importance can be measured by graph theory indicators such as node degree centrality, betweenness centrality, PageRank, etc., which reflects the influence and authority of nodes in the knowledge graph. Taking the above factors into consideration, the comprehensive credibility score of each top-K knowledge node is calculated through weighted summation or multi-criteria decision-making methods as a quantitative indicator of node credibility.

[0110] When generating answers, based on the credibility scores of knowledge nodes, nodes with high credibility are selected as the knowledge support for answer generation. By setting a credibility threshold, nodes with low credibility are filtered out to ensure that the knowledge involved in answer generation is highly credible and high-quality knowledge.

[0111] Specifically, the top-K knowledge nodes are sorted according to the credibility scores, and the nodes with the highest credibility are selected as candidate answer nodes. The number of candidate answer nodes can be dynamically adjusted according to factors such as question complexity and knowledge coverage, in order to ensure the knowledge sufficiency of the answer and take into account the efficiency of answer generation.

[0112] For nodes whose credibility is lower than the threshold, different treatments can be performed according to actual needs. One strategy is to directly remove these nodes and only retain highly trusted nodes to participate in answer generation; another strategy is to use them as supplementary knowledge and introduce them as appropriate in the process of generating answers to enrich the content of the answers, while appropriately marking their credibility to prompt users for reference.

[0113] Based on the selected high-confidence candidate answer nodes, combined with the knowledge-enhanced representation of the query, a natural language answer text is generated through a pre-trained sequence-to-sequence model or a generative language model. In the answer generation process, the knowledge nodes serve as additional input information and guide the generation of answers together with the query representation.

[0114] Specifically, the knowledge embedding representation of the candidate answer node is concatenated or fused with the knowledge enhancement representation of the query to form the initial input representation for answer generation. Then, the answer text is autoregressively generated based on the input representation through pre-trained generative models, such as Transformer and GPT. The generative model has learned a lot of language knowledge and generation patterns in the pre-training stage, and can generate fluent and coherent natural language text based on the input information.

[0115] In the process of generating answers, the information of relevant knowledge nodes is dynamically introduced through the attention mechanism to guide and constrain the generation of answers. At the same time, the generation model can also combine prior knowledge such as question type and answer template to control the structure and form of the answer, thereby improving the readability and user acceptance of the answer.

[0116] Finally, the generated answer text is post-processed, such as sentence segmentation, named entity recognition, pronoun resolution, etc., to improve the comprehensibility and completeness of the answer. At the same time, according to the credibility score of the candidate answer node, the credibility of different parts of the answer is annotated to provide users with reference information on the credibility of the answer, thereby enhancing the explainability and transparency of the answer.

[0117] In summary, this application proposes an answer generation method based on knowledge credibility evaluation. Through the knowledge credibility evaluation mechanism, high-credibility knowledge nodes are preferentially selected to participate in answer generation, thereby improving the reliability and authority of the answer. At the same time, the knowledge nodes are integrated with the query representation, and knowledge-driven natural language answers are generated through a pre-trained generation model. The answers are credibly labeled and post-processed to improve the quality of the answers and user trust. This method makes full use of the credibility information in the knowledge graph, and combines advanced natural language generation technology to achieve highly credible, high-quality intelligent question and answer, providing users with reliable decision support.

[0118] In an optional embodiment,

[0119] Retrieve knowledge nodes related to the query intent from the pre-constructed vertical knowledge graph corresponding to the vertical field, use the attention mechanism to deeply match the query semantic representation corresponding to the query intent with the knowledge nodes, and obtain the top-K knowledge nodes most relevant to the query semantic representation through the jumping knowledge graph reasoning method, including:

[0120] The attention mechanism is used to achieve deep matching between query intent and vertical knowledge graph. The query intent and knowledge nodes in the vertical knowledge graph are spliced ​​as the input of the attention mechanism, and the attention score of the query intent and each knowledge node is calculated through a multi-layer perceptron.

[0121] Perform leap-forward knowledge graph reasoning, starting from the knowledge node corresponding to the query intent, explore on the vertical knowledge graph by combining random walk and attention mechanism, adaptively select the walk path according to the attention score of each knowledge node, and generate multiple reasoning paths corresponding to the query intent;

[0122] Calculate the relevance score of each reasoning path to the query intent, sort the reasoning paths, and select the end nodes of the top-K reasoning paths with the highest relevance scores as the knowledge nodes most relevant to the query intent.

[0123] In an optional embodiment,

[0124] The method further comprises:

[0125] Vectorize the query intent and the knowledge nodes in the vertical knowledge graph, maximize the similarity between the query intent and the relevant knowledge nodes through a triple loss function, minimize the similarity between the query intent and the irrelevant knowledge nodes, and update the vectorized representation of the query intent and the knowledge nodes in the vertical knowledge graph through a gradient descent algorithm;

[0126] The triple loss function is shown in the following formula:

[0127] ;

[0128] in, L represents the triplet loss, D represents the training data set, Indicates query intent Q With positive example node n p and negative nodes n n The triplet composed of u Q The vector representation corresponding to the query intent, e p It represents the vector representation corresponding to the knowledge node in the vertical knowledge graph, and △ represents the preset interval threshold.

[0129] For example, in order to achieve efficient and accurate knowledge retrieval, this application proposes a knowledge retrieval method based on attention mechanism and leapfrog reasoning. This method uses the attention mechanism to achieve deep matching between query intent and vertical knowledge graph, explores knowledge nodes related to query intent through leapfrog knowledge graph reasoning, and finally obtains the top-K knowledge nodes most relevant to the query semantic representation.

[0130] First, the query semantic representation corresponding to the query intent is deeply matched with the knowledge nodes in the vertical knowledge graph. Specifically, the query semantic representation and the embedding representation of each knowledge node are concatenated as the input of the attention mechanism.

[0131] Then, the attention score between the query semantic representation and each knowledge node is calculated by a multi-layer perceptron (MLP). The multi-layer perceptron consists of multiple fully connected layers, and through nonlinear transformation and weight parameter learning, the query semantic representation and knowledge node embedding are mapped to the same semantic space to capture the semantic correlation between them.

[0132] The output of the multi-layer perceptron is normalized by Softmax to obtain the attention score of the query semantic representation and each knowledge node, which indicates the relevance between the query intent and the knowledge node. The higher the attention score, the higher the match between the knowledge node and the query intent, and more attention should be paid to it in the subsequent reasoning process.

[0133] After obtaining the query intent and the attention scores of the knowledge nodes, we perform leap-forward knowledge graph reasoning to explore the knowledge nodes related to the query intent. The reasoning process starts from the initial knowledge node corresponding to the query intent and explores the vertical knowledge graph by combining random walks and attention mechanisms.

[0134] Specifically, in each step of reasoning, the softmax function is used to calculate the transition probability of its neighboring nodes based on the attention score of the current knowledge node. The higher the attention score of the node, the higher the transition probability of its neighboring nodes, indicating that the knowledge nodes related to the query intent are more likely to be explored during the reasoning process.

[0135] Then, the neighbor nodes are sampled according to the transition probability to select the next wandering node. The sampling process adopts a weighted random sampling method, using the transition probability as the weight for selection to achieve adaptive wandering path selection.

[0136] Repeat the above process to generate multiple reasoning paths corresponding to the query intent. Each reasoning path represents starting from the initial knowledge node, after a series of jumps and explorations, and finally reaching the target knowledge node related to the query intent. The length of the reasoning path can be dynamically adjusted according to the scale of the knowledge graph and the complexity of the query, ensuring both the depth and breadth of the reasoning and the efficiency of the reasoning.

[0137] After generating the reasoning path, it is necessary to evaluate the relevance of each reasoning path to the query intent and select the most relevant knowledge node as the retrieval result. First, calculate the relevance score of each reasoning path to the query intent. The relevance score takes into account factors such as the attention score of each node on the reasoning path, the semantic similarity between nodes, and the path length.

[0138] Specifically, for each reasoning path, the attention scores of each node on the path are accumulated to obtain the cumulative attention score of the path. At the same time, the semantic similarity between adjacent nodes on the path is calculated, and the semantic coherence of the path is evaluated using metrics such as cosine similarity or Euclidean distance in the knowledge embedding space.

[0139] Taking into account the cumulative attention score, semantic similarity, and path length, the relevance score of each reasoning path is calculated by weighted summation. The higher the relevance score, the stronger the relevance of the reasoning path to the query intent, and the closer the corresponding terminal node is to the query semantics.

[0140] Finally, the relevance scores of all reasoning paths are sorted, and the end nodes of the top-K reasoning paths with the highest scores are selected as the knowledge nodes most relevant to the query intent. These knowledge nodes contain structured knowledge that is closely related to the query semantics, providing important information support for subsequent knowledge fusion and answer generation.

[0141] Through the above steps, the attention mechanism and leap reasoning are used to achieve deep matching of query intent and vertical knowledge graph and efficient retrieval of relevant knowledge nodes. This method overcomes the limitations of traditional knowledge retrieval, can flexibly explore the structure of knowledge graph, capture the deep semantic association between query intent and knowledge nodes, and improve the accuracy and efficiency of knowledge retrieval.

[0142] The knowledge retrieval algorithm based on attention mechanism and leap reasoning is as follows:

[0143] Input: query semantic representation, vertical knowledge graph, reasoning path length, and number of selected top-K knowledge nodes.

[0144] Output: top-K knowledge nodes most relevant to the query semantics.

[0145] Algorithm flow:

[0146] 1. For each knowledge node in the vertical knowledge graph, calculate its attention score with the query semantic representation;

[0147] 2. Starting from the initial knowledge node corresponding to the query semantic representation, perform leap-forward knowledge graph reasoning:

[0148] - According to the attention score of the current knowledge node, calculate the transition probability of its neighboring nodes;

[0149] - Sample neighbor nodes according to the transition probability and select the next wandering node;

[0150] - Repeat the above process until a reasoning path of length is generated;

[0151] 3. Repeat step 2 to generate M different reasoning paths;

[0152] 4. For each reasoning path, calculate its relevance score:

[0153] - Calculate the cumulative attention score of each node on the path;

[0154] - Calculate the semantic similarity between adjacent nodes on the path;

[0155] - Calculate the relevance score by combining the attention accumulation score, semantic similarity and path length;

[0156] 5. Sort the relevance scores of all reasoning paths, and select the end nodes of the top-K reasoning paths with the highest scores as the knowledge nodes most relevant to the query semantics;

[0157] 6. Return the top-K knowledge nodes.

[0158] Through the above algorithm flow, a knowledge retrieval method based on attention mechanism and leap reasoning is realized. This method uses the attention mechanism to capture the semantic relevance between query intent and knowledge nodes, flexibly explores the structure of the knowledge graph through leap reasoning, generates reasoning paths related to query intent, and selects the most relevant knowledge nodes as retrieval results. Compared with traditional knowledge retrieval methods, this method can better mine the deep semantic associations between query intent and knowledge nodes, improve the accuracy and efficiency of knowledge retrieval, and provide high-quality knowledge support for subsequent knowledge fusion and answer generation.

[0159] In an optional embodiment,

[0160] The most relevant knowledge nodes are fused and represented through the graph attention network, and the pre-trained government affairs model corresponding to the vertical field is input. Starting from each most relevant knowledge node, the credibility propagation network is constructed in combination with the vertical knowledge graph. By setting the initial credibility of the most relevant knowledge node, the propagation process in the credibility propagation network is iteratively calculated. After convergence, the credibility scores of each most relevant knowledge node are obtained, including:

[0161] The top-K knowledge nodes most relevant to the query intent are obtained through leap-forward knowledge graph reasoning, and the graph attention network is used to fuse the information of the most relevant top-K knowledge nodes to obtain a comprehensive top-K knowledge node fusion representation;

[0162] According to the comprehensive top-K knowledge nodes fusion representation of the link relationship in the vertical knowledge graph, a credibility propagation network is constructed, the initial credibility of the most relevant knowledge nodes is set, and the credibility score of each knowledge node is iteratively updated. The credibility propagation of neighboring knowledge nodes and the initial credibility of the knowledge nodes are comprehensively considered until the credibility propagation network reaches a steady state, and the credibility scores of the top-K knowledge nodes after convergence are obtained.

[0163] In an optional embodiment,

[0164] Update the credibility score of each node in the credibility propagation network in an iterative manner:

[0165] ;

[0166] in, Indicates t+1 Knowledge nodes during round iteration n i The credibility of the neighbor node is represented by λ, which is the balancing factor and controls the weight of the credibility of the neighbor node and the initial credibility. N i Represents knowledge nodes n i The set of neighbor nodes of N j Represents knowledge nodes n j The set of neighbor nodes of w ji Represents knowledge nodes n j arrive n i The edge weights of w jk Represents knowledge nodes n j arrive n k The edge weights of Indicates t Knowledge nodes during round iteration n j credibility, y i Represents knowledge nodes n i initial credibility.

[0167] Exemplarily, in order to further improve the accuracy and reliability of knowledge retrieval, this application proposes a knowledge fusion method based on graph attention network and credibility propagation. This method uses graph attention network to fuse information of multiple most relevant knowledge nodes to obtain a comprehensive knowledge representation, and calculates the credibility score of each knowledge node by constructing a credibility propagation network, thereby realizing credibility evaluation and optimization of retrieval results.

[0168] After obtaining the top-K knowledge nodes most relevant to the query intent through leap-forward knowledge graph reasoning, it is necessary to fuse the information of these knowledge nodes to obtain a comprehensive knowledge representation. This application uses the Graph Attention Network (GAT) to achieve the fusion of knowledge nodes.

[0169] Graph Attention Network is a deep learning model based on graph structure data, which can effectively capture the structural relationship and semantic information between nodes. It introduces the attention mechanism, assigns different weights to each node, highlights the influence of important nodes, and realizes the adaptive aggregation of node information.

[0170] Specifically, the top-K most relevant knowledge nodes are regarded as the input nodes of the graph attention network, and the initial features of each node are used as the corresponding knowledge embedding representation. Then, through the multi-head attention mechanism, the attention weight between each node and its neighboring nodes is calculated to capture the semantic association and importance between nodes.

[0171] The multi-head attention mechanism calculates multiple sets of attention weights in parallel and extracts the association information between nodes from different semantic subspaces, thus enhancing the expressiveness of the model. Each attention head calculates the attention weights between nodes independently, and then concatenates the results of different heads to obtain the final node representation.

[0172] Through the multi-layer propagation and aggregation of the graph attention network, the feature representation of the node is continuously updated, so that it can integrate the information of neighboring nodes and capture the high-order relationship between nodes. Finally, the fused representation of the top-K knowledge nodes is pooled to obtain a comprehensive top-K knowledge node fusion representation as the input for subsequent credibility propagation and answer generation.

[0173] In order to evaluate the credibility of knowledge nodes, this application proposes a method based on credibility propagation. Credibility propagation is a trust calculation model based on graph structure, which iteratively calculates the credibility score of each node through the link relationship and initial credibility between nodes.

[0174] First, based on the integrated top-K knowledge nodes fusion representation of the link relationship in the vertical knowledge graph, a credibility propagation network is constructed. The credibility propagation network is a directed graph, where nodes represent knowledge nodes, edges represent the link relationship between knowledge nodes, and the weight of the edge represents the confidence or importance of the link.

[0175] Then, the initial credibility of the most relevant knowledge node is set. The initial credibility reflects the relevance of the knowledge node to the query intent and can be calculated based on factors such as the attention score and semantic similarity of the knowledge node. The higher the initial credibility, the higher the match between the knowledge node and the query intent, and the greater the weight should be given in the credibility propagation process.

[0176] Next, the credibility score of each knowledge node is updated iteratively. The credibility propagation process comprehensively considers the credibility propagation of neighboring knowledge nodes and the initial credibility of the knowledge node. Specifically, for each knowledge node, the propagated credibility value is calculated based on the credibility scores of its neighboring nodes and the weight of the link. At the same time, combined with the initial credibility of the knowledge node, the credibility score of the node is updated by weighted summation.

[0177] Repeat the above iterative process until the credibility propagation network reaches a steady state, that is, the credibility score of each node converges. The converged credibility score reflects the confidence and reliability of the knowledge node. The higher the credibility score, the stronger the relevance of the knowledge node to the query intent, and the higher the weight should be given in answer generation.

[0178] Through credibility propagation, not only the direct relevance of knowledge nodes to query intent is considered, but also the link relationship and propagation influence between knowledge nodes, so as to more comprehensively evaluate the credibility of knowledge nodes and improve the accuracy of knowledge fusion and answer generation.

[0179] Through the above algorithm flow, a knowledge fusion method based on graph attention network and credibility propagation is realized. This method uses the graph attention network to fuse information of multiple most relevant knowledge nodes to obtain a comprehensive knowledge representation, and calculates the credibility score of each knowledge node by constructing a credibility propagation network, comprehensively considering the relevance of knowledge nodes to query intent and the link relationship and propagation influence between knowledge nodes. Compared with traditional knowledge fusion methods, this method can more comprehensively evaluate the credibility of knowledge nodes, improve the accuracy of knowledge fusion and answer generation, and provide reliable support for subsequent knowledge applications.

[0180] Figure 2 FIG. 1 is a schematic diagram of the structure of a vertical government affairs big model service system based on knowledge retrieval according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0181] The first unit is used to obtain a natural language query in the government affairs field to be processed, perform word segmentation and semantic understanding on the natural language query through a natural language processing model combining an attention mechanism and a bidirectional long short-term memory network, extract the query intent, and determine the vertical field to which the natural language query belongs according to the query intent;

[0182] The second unit is used to retrieve knowledge nodes related to the query intent from the pre-constructed vertical knowledge graph corresponding to the vertical field, use the attention mechanism to deeply match the query semantic representation corresponding to the query intent with the knowledge nodes, and obtain the top-K knowledge nodes most relevant to the query semantic representation through the jumping knowledge graph reasoning method;

[0183] The third unit is used to fuse and represent multiple most relevant knowledge nodes through a graph attention network, input the pre-trained government affairs model corresponding to the vertical field, start from each most relevant knowledge node, and build a credibility propagation network in combination with the vertical knowledge graph. By setting the initial credibility of the most relevant knowledge node, iteratively calculate the credibility of each most relevant knowledge node in the propagation process of the credibility propagation network, and obtain the credibility score of each most relevant knowledge node after the credibility propagation network converges, and summarize the most relevant knowledge nodes whose credibility scores exceed the preset credibility threshold as the target reply content.

[0184] According to a third aspect of the embodiments of the present invention,

[0185] An electronic device is provided, comprising:

[0186] processor;

[0187] a memory for storing processor-executable instructions;

[0188] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0189] According to a fourth aspect of the embodiments of the present invention,

[0190] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0191] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0192] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vertical government affairs big model service method based on knowledge retrieval, characterized by: include: Obtain a natural language query in the government affairs field to be processed, perform word segmentation and semantic understanding on the natural language query through a natural language processing model that combines an attention mechanism with a bidirectional long short-term memory network, extract the query intent, and determine the vertical field to which the natural language query belongs based on the query intent; Retrieve knowledge nodes related to the query intent from the pre-constructed vertical knowledge graph corresponding to the vertical field, use the attention mechanism to deeply match the query semantic representation corresponding to the query intent with the knowledge nodes, and obtain the top-K knowledge nodes most relevant to the query semantic representation through a jumping knowledge graph reasoning method; A plurality of most relevant knowledge nodes are fused and represented through a graph attention network, and the pre-trained government affairs model corresponding to the vertical field is inputted. Starting from each most relevant knowledge node, a credibility propagation network is constructed in combination with the vertical knowledge graph. By setting the initial credibility of the most relevant knowledge node, the credibility of each most relevant knowledge node in the propagation process of the credibility propagation network is iteratively calculated. After the credibility propagation network converges, the credibility score of each most relevant knowledge node is obtained, and the most relevant knowledge nodes whose credibility scores exceed the preset credibility threshold are summarized as the target reply content; The most relevant knowledge nodes are fused and represented through the graph attention network, and the pre-trained government affairs model corresponding to the vertical field is input. Starting from each most relevant knowledge node, the credibility propagation network is constructed in combination with the vertical knowledge graph. By setting the initial credibility of the most relevant knowledge node, the propagation process in the credibility propagation network is iteratively calculated. After convergence, the credibility scores of each most relevant knowledge node are obtained, including: The top-K knowledge nodes most relevant to the query intent are obtained through leap-forward knowledge graph reasoning, and the graph attention network is used to fuse the information of the most relevant top-K knowledge nodes to obtain a comprehensive top-K knowledge node fusion representation; According to the link relationship in the vertical knowledge graph represented by the comprehensive top-K knowledge node fusion, a credibility propagation network is constructed, the initial credibility of the most relevant knowledge node is set, and the credibility score of each knowledge node is updated iteratively, taking into account the credibility propagation of neighboring knowledge nodes and the initial credibility of the knowledge node, until the credibility propagation network reaches a steady state, and the credibility scores of the top-K knowledge nodes after convergence are obtained; Update the credibility score of each node in the credibility propagation network in an iterative manner: ； in, Indicates t+1 Knowledge nodes during round iteration n i The credibility of the node, λ represents the balancing factor, which controls the weight of the neighbor node credibility and the initial credibility. N i Represents knowledge nodes n i The set of neighbor nodes of N j Represents knowledge nodes n j The set of neighbor nodes of w ji Represents knowledge nodes n j arrive n i The edge weights of w jk Represents knowledge nodes n j arrive n k The edge weights of Indicates t Knowledge nodes during round iteration n j credibility, y i Represents knowledge nodes n i initial credibility.

2. The method according to claim 1, characterized in that Obtain a natural language query in the government affairs field to be processed, perform word segmentation and semantic understanding on the natural language query through a natural language processing model that combines an attention mechanism with a bidirectional long short-term memory network, extract the query intent, and determine the vertical field to which the natural language query belongs based on the query intent, including: The natural language query is segmented by a natural language processing model, the natural language query is encoded by a bidirectional long short-term memory network to obtain context information, and the context information is dynamically aggregated by a multi-head attention mechanism to generate a query semantic vector; A word joint embedding mechanism is introduced to represent the query semantic vector as a word joint vector concatenated by a character-level embedding vector and a word-level embedding vector. The character-level embedding vector is obtained by fusing the embeddings of each character of the query semantic vector through a feedforward neural network, and the word-level embedding vector is directly obtained through a word embedding matrix. A multi-task semantic understanding model is constructed. The word joint vector is used as input. The word joint vector is learned through the encoding layer of the shared bidirectional long short-term memory network. Three tasks are set in parallel: the intent recognition task, which classifies the intent category corresponding to the word joint vector through the attention mechanism and the fully connected network; the slot filling task, which uses the sequence labeling model to extract the key information slots in the word joint vector; the named entity recognition task, which uses the sequence labeling model to identify the entity name and its type in the word joint vector; According to the intent recognition result of the intent recognition task, the confidence corresponding to the word joint vector is determined, and according to the corresponding relationship between the confidence and the government affairs vertical field, the confidence corresponding to the word joint vector is mapped to the vertical field to which it belongs; For the confidence corresponding to the word joint vector that is lower than the preset confidence threshold, or the confidence that has a one-to-many mapping relationship, the vertical field corresponding to the confidence corresponding to the word joint vector is determined in combination with the results of slot filling and named entity recognition.

3. The method according to claim 1, characterized in that Retrieve knowledge nodes related to the query intent from the pre-constructed vertical knowledge graph corresponding to the vertical field, use the attention mechanism to deeply match the query semantic representation corresponding to the query intent with the knowledge nodes, and obtain the top-K knowledge nodes most relevant to the query semantic representation through the jumping knowledge graph reasoning method, including: The attention mechanism is used to achieve deep matching between query intent and vertical knowledge graph. The query intent and knowledge nodes in the vertical knowledge graph are spliced ​​as the input of the attention mechanism, and the attention score of the query intent and each knowledge node is calculated through a multi-layer perceptron. Perform leap-forward knowledge graph reasoning, starting from the knowledge node corresponding to the query intent, explore on the vertical knowledge graph by combining random walk and attention mechanism, adaptively select the walk path according to the attention score of each knowledge node, and generate multiple reasoning paths corresponding to the query intent; Calculate the relevance score of each reasoning path to the query intent, sort the reasoning paths, and select the end nodes of the top-K reasoning paths with the highest relevance scores as the knowledge nodes most relevant to the query intent.

4. The method according to claim 3, characterized in that The method further comprises: Vectorize the query intent and the knowledge nodes in the vertical knowledge graph, maximize the similarity between the query intent and the relevant knowledge nodes through a triple loss function, minimize the similarity between the query intent and the irrelevant knowledge nodes, and update the vectorized representation of the query intent and the knowledge nodes in the vertical knowledge graph through a gradient descent algorithm; The triple loss function is shown in the following formula: ； in, L represents the triplet loss, D represents the training data set, Indicates query intent Q With positive example node n p and negative nodes n n The triplet composed of u Q The vector representation corresponding to the query intent, e p It represents the vector representation corresponding to the knowledge node in the vertical knowledge graph, and △ represents the preset interval threshold.

5. A vertical government affairs big model service system based on knowledge retrieval, used to implement the method described in any one of claims 1 to 4, characterized in that: include: The first unit is used to obtain a natural language query in the government affairs field to be processed, perform word segmentation and semantic understanding on the natural language query through a natural language processing model combining an attention mechanism and a bidirectional long short-term memory network, extract the query intent, and determine the vertical field to which the natural language query belongs according to the query intent; The second unit is used to retrieve knowledge nodes related to the query intent from the pre-constructed vertical knowledge graph corresponding to the vertical field, use the attention mechanism to deeply match the query semantic representation corresponding to the query intent with the knowledge nodes, and obtain the top-K knowledge nodes most relevant to the query semantic representation through the jumping knowledge graph reasoning method; The third unit is used to fuse and represent multiple most relevant knowledge nodes through a graph attention network, input the pre-trained government affairs model corresponding to the vertical field, start from each most relevant knowledge node, and build a credibility propagation network in combination with the vertical knowledge graph. By setting the initial credibility of the most relevant knowledge node, iteratively calculate the credibility of each most relevant knowledge node in the propagation process of the credibility propagation network, and obtain the credibility score of each most relevant knowledge node after the credibility propagation network converges, and summarize the most relevant knowledge nodes whose credibility scores exceed the preset credibility threshold as the target reply content.

6. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 4.

7. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images